School of Computer Science Part of Four New NSF AI Institutes

CMU Contributing to Innovations in Caregiving, Agriculture, Wireless Networks

Thursday, July 29, 2021 - by Aaron Aupperlee

How AI systems interact with a large network of people will be part of CMU’s contribution to the AI Institute for Collaborative Assistance and Responsive Interaction for Networked Groups.

Carnegie Mellon University's School of Computer Science will contribute fundamental and cutting-edge research to a government-led push to bring about life-changing advances through artificial intelligence.  

 The U.S. National Science Foundation today announced a $220 million investment in 11 new Artificial Intelligence Research Institutes. School of Computer Science (SCS) researchers will participate in four of the new institutes.

Reid Simmons, a research professor in the Robotics Institute (RI) and the Computer Science Department (CSD), will lead CMU's involvement in the AI Institute for Collaborative Assistance and Responsive Interaction for Networked Groups (AI-CARING), which seeks to develop AI systems for caregiving environments. George Kantor, a research professor in RI, will lead the university's work in the AI Institute for Resilient Agriculture (AIIRA). Wenzhen Yuan, an assistant professor in the Robotics Institute, will lead CMU's contribution to the Institute for Agricultural AI for Transforming Workforce and Decision Support (AgAID). Both AIIRA and AgAID  focus on AI and robotics in agriculture.  

CMU's College of Engineering is part of a fourth institute, the AI Institute for Future Edge Networks and Distributed Intelligence (AI-EDGE). The institute will focus on designing wireless edge networks that are highly efficient, reliable, robust and secure. The CMU team includes Gauri Joshi, an assistant professor in the Electrical and Computer Engineering Department, as head of the university's efforts; and Ameet Talwalkar, an assistant professor in the Machine Learning Department. Additionally, in 2020 the NSF named CMU a planning institute for developing a National Artificial Intelligence Research Institute in Physics and awarded the university a $500,000 grant. 

Each institute announced today will receive $20 million over five years. The funding will support research efforts at universities across the country, including at CMU, where dozens of SCS faculty, students and staff will be involved. Researchers from the Dietrich College of Humanities and Social Sciences and the Tepper School of Business will also be included in the work. 

The wide reach of these institutes illustrates the role AI will play in the next wave of transformational innovation and as a tool for improving our daily lives. CMU's inclusion in these federal efforts to expand the use of AI underscores the university's depth and breadth of expertise in the field, spanning deep learning, robotics, interaction, ethics and engineering. The university is one of the birthplaces of AI and will continue to push its boundaries through these new initiatives.

AI-CARING will be led by the Georgia Institute of Technology  with partial funding from Amazon and Google. The institute will initially focus on assisting in the care of the elderly by seeking to understand and manage the interaction between humans and AI agents. Researchers will develop methods to teach AI systems to learn a person's needs, preferences and caregiving network, and enable the systems to adapt as those change over time. The tasks will involve not only coordinating a variety of AI agents — like voice-controlled assistants, smart devices or chatbots — but also managing human caregivers ranging from medical and assistance providers to family.

"What we're trying to do is collaborate. We're not replacing caregivers; we're augmenting their capabilities," Simmons said. "The AI has to adapt to the individual and even more so, it has to adapt to the network the individual is in. We're intending that the AI is going to be assisting people over a long time, and while there will be plenty of time for the system to learn, there will also be plenty to learn."

Simmons and his CMU team will bring core AI expertise to the project, as well as years of research and work in multi-agent cooperation, human-computer interaction, AI education and outreach, and ethics. Communication and education are large components of the project, as families, caregivers and those being cared for must understand how and why the AI makes its decisions. The project also emphasizes training the next generation of AI researchers and workers.

CMU will receive about a third of the institute's $20 million in funding over five years. The university's team consists of about a dozen faculty members spanning RI, CSD, the Human-Computer Interaction Institute, Dietrich and Tepper. The team includes Henny Admoni, who researches nonverbal human-robot interaction; Jodi Forlizzi and her work in human-AI interaction; Aaron Steinfeld, who works on assistive technology; Dave Touretzky and his work in teaching AI to K-12 students; Alex London, a professor of ethics and philosophy in Dietrich; and Tepper's Anita Woolley, who studies human-AI teaming and collaboration. 

The institute's CMU ties even extend to its director, Sonia Chernova. She heads the Robot Autonomy and Interactive Learning Lab at Georgia Tech and earned her Ph.D. in computer science from CMU.

AIIRA, pronounced "eye-rah," will be based at Iowa State University and is funded by the U.S. Department of Agriculture's National Institute of Food and Agriculture (USDA-NIFA). The institute aims to use AI to develop digital twins to model plants at an unprecedented scale and with unparalleled precision. The digital modeling will allow for robust simulation and prediction. With the digital twins, for example, scientists will be able to simulate and predict how a crop will perform in climate scenarios that do not yet exist.

CMU, which will receive about a third of this institute's funding, brings its strength in AI, deep learning and robotics to the project. To build the digital twins, scientists will need large amounts of data about plants, soil and other aspects of the environment. The CMU team will design robot systems to gather this data and provide the AI and deep learning tools necessary to model it.

"Imagine a robotic manipulator pushing on a branch. We can record how much force was used to make the branch move and how much the branch moved. Fusing that with other data will help us create a predictive model," Kantor said. "This is going to be a big opportunity to bring intelligent manipulation into the field."

Seven faculty members from RI and the Machine Learning Department make up CMU's contribution to the institute. The challenge for the CMU team is, in part, designing the never-before-used robotic tools that will be required in the field. A robotic manipulator that will interact with crops must be gentle to avoid damaging the plants, so CMU's Nancy Pollard is designing soft manipulators for the project. She will use AI and video capture techniques from her Graphics Lab to build a hand optimized for the task, whether grabbing a leaf or pulling on a stem. Other CMU team members include Oliver Kroemer, who is designing techniques that will allow a robot to insert sensors into a plant's stem and the ground; and Katia Sycara, who studies multi-agent systems and human-robot interaction. Other members of the team will tackle novel imaging techniques, machine learning for decision support and field robotics. 

While the initial work will focus on using robotics and AI to gather and process data, the tools and techniques developed for those tasks could translate to other work on the farm, such as pollinating flowers, which is done by hand in some instances.

"There are people out there in the field with microscope goggles and tweezers peeling the pollen off one flower and transferring it to another. That takes a person a very long time," Kantor said. "To me, the most exciting part is moving into this sort of manipulation."

AgAID will be led by Washington State University and is also funded by the USDA-NIFA. The institute will integrate AI methods into agriculture operations for prediction, decision support, and robotics-enabled agriculture to address complex agricultural challenges. The institute involves farmers, workers, managers and policy makers in the development of these solutions — as well as in AI training and education — to promote equity by increasing the technological skill levels of the next-generation agricultural workforce.

Yuan, who heads CMU's RoboTouch Lab, will bring her research on robotic tactile perception to the institute. CMU will receive about $200,000 in funding to support her work.  Yuan has developed tactile sensors that enable a robot to determine the hardness of an object. She has previously used this technology in her lab to determine if tomatoes, avocados and mangos are ripe, but imagines robots using it in the field.

"Robots can touch the corn or other fruits and determine if they are ripe or not," Yuan said. "I'm excited to take this technology out of the lab and into the field."

For more information, Contact:
Aaron Aupperlee | 412-268-9068 | aaupperlee@cmu.edu