Trustworthy Interaction & Multi-UAV Systems
new! Human Attention Sharing for Failure-Resilent Human-Robot Cooperation
It is challenging for a robot to detect its failures or failure tendency by themselves. In this research, we focus on using human attention to remind robots of its potential failures, in coopertion such as daily assistance and cooperated manufacturing. A novel deep learning architecture will be designed to accurately understand human instructions, and then assist with robot behavior corrections.
Keywords: Attention, Deual Neural Networks, HRI
Publications
new!Human-Guided Robot Behavior Correction Using Stacked Neural Networks #HRI# #AI Transparency#- Under Review
Boyi Song, Yuntao Peng, Ruijiao Luo, Rui Liu*
Keywords: Attention, Deual Neural Networks, HRI
Publications
new!Human-Guided Robot Behavior Correction Using Stacked Neural Networks #HRI# #AI Transparency#- Under Review
Boyi Song, Yuntao Peng, Ruijiao Luo, Rui Liu*
new! Safety-Aware UAV-UGV Teaming for Ground Target Tracking
Ground and aerial cooperation is widely used in cooperated social security and military force deployment. In this research, we focus on safety ensurance for all the manned units involved in the mixed cooperation.
Keywords: Safety, Multi-Agent Deep Reinforcement Learning, Multi-Robot System, UAV-UGV Cooperation
Publications
new!Mixed UAV-UGV Cooperation Using Multi-Agent Reinforcement Learning #multi-robot system##AI Transparency# - Under Review
Qi Fei, Zhexin Shen, Rui Liu*
2019
Keywords: Safety, Multi-Agent Deep Reinforcement Learning, Multi-Robot System, UAV-UGV Cooperation
Publications
new!Mixed UAV-UGV Cooperation Using Multi-Agent Reinforcement Learning #multi-robot system##AI Transparency# - Under Review
Qi Fei, Zhexin Shen, Rui Liu*
2019
new! Dynamic Trust for Human-UAVs Cooperated Emergency Response
(social security, natural disaster response)
(social security, natural disaster response)
Emergency happens in human-UAV cooperation, such as criminal activity tracking and urgent needs for ground assistance. Emergency response usually has high requirements on the motion control of the multi-uav system, by maintaining both the team performance and team behaviors. A dynamic trust modeling can help uav team to flexible adjust the UAV motion behaviors, for improved human-supervisory UAV task performance.
Keywords: Trust, Emergency Response, Multi-Agent Reinforcement Learning, Human-UAV Cooperation
Publications
new!Dynamic Trust Modeling in Human-UAV swarm teaming #multi-robot system##AI Transparency#- Under Review
Yijiang Pang, Rui Liu*
2019
Keywords: Trust, Emergency Response, Multi-Agent Reinforcement Learning, Human-UAV Cooperation
Publications
new!Dynamic Trust Modeling in Human-UAV swarm teaming #multi-robot system##AI Transparency#- Under Review
Yijiang Pang, Rui Liu*
2019
Human Trust for UAV Swarm Deployment in Disaster Search
The deployment of robot swarms is influenced by real-world factors, such as motor issues, sensor failure, and wind disturbances. These factors cause the appearance of faulty robots. In a decentralized swarm, sharing incorrect information from faulty robots will lead to undesired swarm behaviors, such as swarm disconnection and incorrect heading directions. We envision a system where a human operator is exerting supervisory control over a remote swarm by indicating changes in trust to the swarm via a "trust-signal". By correcting faulty behaviors, trust between the human and the swarm is maintained to facilitate human-swarm cooperation.
Keywords: Trust, Multi-agent systems, UAV, Swarm, Multi-Agent Reinforcement Learning
Publications
Trust Repairing in Human-Swarm Teams #trust on UAV#
R. Liu, Z. Cai, M Lewis, K Sycara
IEEE RO-MAN, accepted at July 2019. Final list for Best Conference Paper
NEW! Trust-Aware Behavior Reflection for Robot Swarm Self-Healing #trust on UAV# [PDF]
R. Liu, F Jia, W Luo, M Chandarana, C Nam, M Lewis, K Sycara
International Conference on Autonomous Agents and Multiagent Systems(AAMAS), accepted at Jan 2019. (acceptance rate: 24%)
Keywords: Trust, Multi-agent systems, UAV, Swarm, Multi-Agent Reinforcement Learning
Publications
Trust Repairing in Human-Swarm Teams #trust on UAV#
R. Liu, Z. Cai, M Lewis, K Sycara
IEEE RO-MAN, accepted at July 2019. Final list for Best Conference Paper
NEW! Trust-Aware Behavior Reflection for Robot Swarm Self-Healing #trust on UAV# [PDF]
R. Liu, F Jia, W Luo, M Chandarana, C Nam, M Lewis, K Sycara
International Conference on Autonomous Agents and Multiagent Systems(AAMAS), accepted at Jan 2019. (acceptance rate: 24%)
Cognitive Robot Understanding by Abstracting Human Intents
Intelligent robotic assistance requires a robot to accurately understand human behavior. Many researchers have explored human-object interactions to decode behavior-related information. However, current methods only model probabilistic correlations between objects and activities. Their applications are usually limited to fixed environments and fixed sets of activities. They are unable to deal with variability in the real environments due to the lack of the human-like cognitive reasoning process. To address this urgent problem, we developed an Object Functional Role Perspective method to endow a robot with comprehensive behavior understanding. Instead of using specific objects to identify an activity, our role-based method models the human cognitive process during task performing by analyzing object selection and object interaction. Then activity-related information, such as activity feasibility, likely plan, and urgent need of an activity, is inferred in order to improve a robot’s cognition level for comprehensive behavior understanding. Through a large amount of human behavior observations, this cognitive knowledge is constructed using a Markov random field (MRF) model. Experiments were performed in both real-life scenarios and lab scenarios to evaluate the method’s usefulness. The results demonstrated flexibility and effectiveness of the role-based method for human behavior understanding under variability.
Publications
Understanding human behaviors with an object functional role perspective for robotics [PDF]
R. Liu, X. Zhang
IEEE Transactions on Cognitive and Developmental Systems, vol. 8, no. 2, pp. 115-127, 2015. (IF: 1.952. Robotics:top 40%)
Use context to understand user's implicit intentions in activities of daily living [PDF]
R. Liu, X. Zhang
IEEE International Conference on Mechatronics and Automation (ICMA), Aug. 2014. (Hindex: 18.)
Fuzzy-context-specific intention inference for robotic caregiving [PDF]
R, Liu, X. Zhang
International Journal of Advanced Robotic Systems, vol.13(5), 2016. (IF: 0.987. rank in Robotics: top 69%)
Publications
Understanding human behaviors with an object functional role perspective for robotics [PDF]
R. Liu, X. Zhang
IEEE Transactions on Cognitive and Developmental Systems, vol. 8, no. 2, pp. 115-127, 2015. (IF: 1.952. Robotics:top 40%)
Use context to understand user's implicit intentions in activities of daily living [PDF]
R. Liu, X. Zhang
IEEE International Conference on Mechatronics and Automation (ICMA), Aug. 2014. (Hindex: 18.)
Fuzzy-context-specific intention inference for robotic caregiving [PDF]
R, Liu, X. Zhang
International Journal of Advanced Robotic Systems, vol.13(5), 2016. (IF: 0.987. rank in Robotics: top 69%)
Oral Instruction Interpretation for Human-Robot Cooperated Manufacturing
It is critical for advanced manufacturing machines to autonomously execute a task by following an end-user’s natural language (NL) instructions. However, NL instructions are usually ambiguous and abstract so that the machines may misunderstand and incorrectly execute the task. To address this NL-based human-machine communication problem and enable the machines to appropriately execute tasks by following the end-user’s NL instructions, we developed a Machine-Executable-Plan-Generation (exePlan) method. The exePlan method conducts task-centered semantic analysis to extract task-related information from ambiguous NL instructions. In addition, the method specifies machine execution parameters to generate a machine-executable plan by interpreting abstract NL instructions. To evaluate the exePlan method, an industrial robot Baxter was instructed by NL to perform three types of industrial tasks {“drill a hole”, “clean a spot”, “install a screw”}. The experiment results proved that the exePlan method was effective in generating machine-executable plans from the end-user’s NL instructions. Such a method has the promise to endow a machine with the ability of NL-instructed task execution.
Publications
Generating machine-executable plan from end-user’s natural language instructions [PDF]
R. Liu, X. Zhang
Knowledge-based Systems, DOI:10.1016/j.knosys.2017.10.023, 2017. (IF:4.529 . rank in AI: top 11%)
Natural-language-assisted industrial task execution [PDF]
R. Liu, J. Webb and X. Zhang
ASME International Design Engineering Technical Conferences & Comuters & Information in Engineering Conference (IDETC), Aug 2016.
Publications
Generating machine-executable plan from end-user’s natural language instructions [PDF]
R. Liu, X. Zhang
Knowledge-based Systems, DOI:10.1016/j.knosys.2017.10.023, 2017. (IF:4.529 . rank in AI: top 11%)
Natural-language-assisted industrial task execution [PDF]
R. Liu, J. Webb and X. Zhang
ASME International Design Engineering Technical Conferences & Comuters & Information in Engineering Conference (IDETC), Aug 2016.
Robots Surf Internet for Human-Centered Environment Understanding
Human-centered situation, which describes the surrounding world of a person, indicates his undergoing activities. Understanding of human-centered situations helps an assistive robot with its decision making. Existing methods, such as learning from human demonstration, are economically expensive, time-consuming, and have limited scalability. To address this problem, we developed a web-to-situation (W2S) method with which web natural descriptions are grounded into human-centered situations in a context-specific manner. By comparing the learned knowledge from the web and the survey, we proved that W2S is effective in extracting reliable knowledge in an efficient and low-cost manner. By implementing the W2S-retrieved knowledge in web-collected situations and real life situations, we proved that W2S is effective in situation understanding. Given that the web contains huge amounts of information, W2S is expected to effectively scale up a robot's knowledge.
Publications
Context-specific grounding of web natural descriptions to human-centered situations [PDF]
R. Liu, X. Zhang
Knowledge-based Systems, vol. 111, pp. 1-16, 2016.(IF:4.529 . rank in AI: top 11%)
Context-specific intention awareness through web query in robotic caregiving [PDF]
R. Liu, X. Zhang
IEEE International Conference on Robotics and Automation (ICRA), May 2015. (H-index:105. Acceptance rate: 34.7%)
Final list for Best Conference Paper, Best Student Paper, Best Cognitive Robotics Paper: awarding rate 1.7%
Publications
Context-specific grounding of web natural descriptions to human-centered situations [PDF]
R. Liu, X. Zhang
Knowledge-based Systems, vol. 111, pp. 1-16, 2016.(IF:4.529 . rank in AI: top 11%)
Context-specific intention awareness through web query in robotic caregiving [PDF]
R. Liu, X. Zhang
IEEE International Conference on Robotics and Automation (ICRA), May 2015. (H-index:105. Acceptance rate: 34.7%)
Final list for Best Conference Paper, Best Student Paper, Best Cognitive Robotics Paper: awarding rate 1.7%
Medical AI
Web-video-mining-supported workflow modeling for robotic surgeries
As quality assurance is of strong concern in advanced surgeries, intelligent surgical systems are expected to have knowledge such as the knowledge of the surgical workflow model (SWM) to support their intuitive cooperation with surgeons. For generating a robust and reliable SWM, a large amount of training data is required. However, training data collected by physically recording surgery operations is often limited and data collection is time-consuming and labor-intensive, severely influencing knowledge scalability of the surgical systems. The objective of this research is to solve the knowledge scalability problem in surgical workflow modeling with a low cost and labor efficient way.
Publications
-R. Liu, X. Zhang, H. Zhang, Web-video-mining-supported workflow modeling for robotic surgeries, Artificial Intelligence in Medicine, vol. 74, pp. 9-20, 2016. (IF: 2.142. rank in AI: top 26%) [PDF]
Publications
-R. Liu, X. Zhang, H. Zhang, Web-video-mining-supported workflow modeling for robotic surgeries, Artificial Intelligence in Medicine, vol. 74, pp. 9-20, 2016. (IF: 2.142. rank in AI: top 26%) [PDF]
Metal 3D Printing
By producing mechanical parts directly from computer-aided design (CAD) models in an incremental manner, metal additive manufacturing (metal AM or metal 3D printing) has been broadly used to manufacture products from jet engines, to plane body shells and car gearboxes. However, quality control in metal AM is challenging due to the complex dynamics generated in the interactions among the laser, air, material, part, and mechanical systems inside the 3D printer. Given these dynamics, the quality issues such as big pore sizes, part geometry deformation, material precipitate, small yield strength, etc. are always unavoidable. Therefore, it is critical to investigate the fundamental influence of the printer dynamics on part quality for quality-issue understanding, prediction and avoidance.
Publications
Semantics based part similarity measurement for experiment simplification in additive manufacturing -- Under Review
R. Liu, S. Liu, X. Zhang, A. Stebner, other coauthors will be added
Additive Manufacturing, 2019. (IF:7.00)
A physics-informed machine learning model for in-depth porosity analysis in additive manufacturing -- Under Review
R. Liu, X. Zhang, A. Stebner, other coauthors will be added
Additive Manufacturing, 2019. (IF:7.00)
Considerations for additive manufacturing process optimization [invited by editor]
N. Johnson, R. Liu, D. Drake, B. Meredig, J. Ling, X. Zhang, H. Wang, C. Brice, B. Kappes, A. Stebner
Applied Physics Reviews, 2019. (IF: 13.667.)
Publications
Semantics based part similarity measurement for experiment simplification in additive manufacturing -- Under Review
R. Liu, S. Liu, X. Zhang, A. Stebner, other coauthors will be added
Additive Manufacturing, 2019. (IF:7.00)
A physics-informed machine learning model for in-depth porosity analysis in additive manufacturing -- Under Review
R. Liu, X. Zhang, A. Stebner, other coauthors will be added
Additive Manufacturing, 2019. (IF:7.00)
Considerations for additive manufacturing process optimization [invited by editor]
N. Johnson, R. Liu, D. Drake, B. Meredig, J. Ling, X. Zhang, H. Wang, C. Brice, B. Kappes, A. Stebner
Applied Physics Reviews, 2019. (IF: 13.667.)
New! Physics-Informed AI Modeling for Quality Control in Metal Additive Manufacturing
To control part quality, it is critical to analyze pore generation mechanisms, laying theoretical foundation for future porosity control. Current porosity analysis models use machine setting parameters, such as laser angle and part pose. However, these setting-based models are machine dependent, hence they often do not transfer to analysis of porosity for a different machine. To address the first problem, a physics-informed, data-driven model (PIM), which instead of directly using machine setting parameters to predict porosity levels of printed parts, it first interprets machine settings into physical effects, such as laser energy density and laser radiation pressure. Then, these physical, machine independent effects are used to predict porosity levels according to “pass”, “flag”, “fail” categories instead of focusing on quantitative pore size prediction. With six learning methods’ evaluation, PIM proved to achieve good performances with prediction error of 10∼26%. Finally, poreencouraging influence and pore-suppressing influence were analyzed for quality analysis.
Publications
A physics-informed machine learning model for in-depth porosity analysis in additive manufacturing
R. Liu, S. Liu, X. Zhang
International Journal of Advanced Manufacturing Technologies, Accepted 2021 Jan.
Publications
A physics-informed machine learning model for in-depth porosity analysis in additive manufacturing
R. Liu, S. Liu, X. Zhang
International Journal of Advanced Manufacturing Technologies, Accepted 2021 Jan.
*Last updated at 2019 June.