Loading [MathJax]/extensions/MathZoom.js
Integrating Computer Vision and Natural Language Instruction for Collaborative Robot Human-Robot Interaction | IEEE Conference Publication | IEEE Xplore

Integrating Computer Vision and Natural Language Instruction for Collaborative Robot Human-Robot Interaction


Abstract:

In this paper, we report how we design a human-robot co-working scheme. We believe that natural language is the best way that human and machine can exchange messages. How...Show More

Abstract:

In this paper, we report how we design a human-robot co-working scheme. We believe that natural language is the best way that human and machine can exchange messages. However, understand voice command is not enough, what object is referred in what location is also very import. To better understand a natural language instruction, a robot also require computer vision to resolve the real-world object referencing problem. In this paper we show how to integrate the Google voice recognition result with the YOLO computer vision software with the KINOVA robot arm and achieve a collaborative robot interaction.
Date of Conference: 04-07 November 2020
Date Added to IEEE Xplore: 17 December 2020
ISBN Information:
Conference Location: Hsinchu, Taiwan

I. Introduction

In recent years, the emergence of collaborative robots have the advantage of light-weight, easy to disassemble and install, and with higher safety. The potential of human-robot co-working is very promising. How human and machine work can together has been an important issue. In order to make the human-robot interaction more intelligent, we believe that the natural language can help human and machine exchanges messages and cooperation between each other. The interaction will be carried out through voice and comprehended with the help of computer vision to achieve a better human-robot interaction. In this paper we show how to integrate the natural language understanding with the computer vision for a collaborative robot interaction.

References

References is not available for this document.