Teleoperation permits to control robots from a safe distance while performing tasks in a remote environment. Kinematic differences between the input device and the remotely controlled manipulator or the existence of redundancy in the remote robot may pose challenges in moving intuitively the remote robot as desired by the human operator. Motivated by the above challenges, this work introduces TelePhysicalOperation, a novel teloperation concept, which relies on a virtual physical interaction interface between the human operator and the remote robot in a manner that is equivalent to a “Marionette” based interaction interface. With the proposed approach, the user can virtually “interact” with the remote robot, through the application of virtual forces, which are generated by the operator tracking system and can be then selectively applied to any body part of the remote robot along its kinematic chain. This leads to the remote robot generating motions that comply with the applied virtual forces, thanks to the underlying control architecture. The proposed method permits to command the robot from a distance by exploring the intuitiveness of the “Marionette” based physical interaction with the robot in a virtual/remote manner. The details of the proposed approach are introduced and its effectiveness is demonstrated through a number of experimental trials executed on the CENTAURO, a hybrid leg-wheel platform with an anthropomorphic upper body.

The recent advancements in the development of robotic systems that offer advanced loco-manipulation capabilities have opened new opportunities in the employment of such platforms in various domains. However, despite the increased range of offered capabilities, the collaboration with these robotic platforms to execute tasks through common human–robot interaction interfaces is still an open challenge. In this article, we present a novel human–robot interaction interface that permits to intuitively command and control the manipulation and the locomotion abilities of the robot by exploring a visual servoing guidance method realized with a laser emitter device. By pointing the laser to locations and objects in the environment where the robot is operating, the operator is able to command even highly articulated robots intuitively and efficiently. The detection of the laser projection is performed by a neural network that provides robust and real-time tracking of laser spot. Combined with the responsiveness of the laser detection, a Behavior Trees-based motion planner is employed to reactively select and generate the autonomous robot motions to reach the indicated target. This combination allows the operator to communicate goal locations and paths to follow without requiring prior knowledge of the system, and without worrying about the generation of the potential complex loco-manipulation robot actions. The effectiveness of the proposed interface is demonstrated with the CENTAURO robot, a hybrid leg-wheel platform with an anthropomorphic upper body, exploiting its abilities to accomplish a number of locomotion and manipulation tasks.

Robotics has shown significant potential in assisting people with disabilities to enhance their independence and involvement in daily activities. Indeed, a societal long-term impact is expected in home-care assistance with the deployment of intelligent robotic interfaces. This work presents a human-robot interface developed to help people with upper limbs impairments, such as those affected by stroke injuries, in activities of everyday life. The proposed interface leverages on a visual servoing guidance component, which utilizes an inexpensive but effective laser emitter device. By projecting the laser on a surface within the workspace of the robot, the user is able to guide the robotic manipulator to desired locations, to reach, grasp and manipulate objects. Considering the targeted users, the laser emitter is worn on the head, enabling to intuitively control the robot motions with head movements that point the laser in the environment, which projection is detected with a neural network based perception module. The interface implements two control modalities: the first allows the user to select specific locations directly, commanding the robot to reach those points; the second employs a paper keyboard with buttons that can be virtually pressed by pointing the laser at them. These buttons enable a more direct control of the Cartesian velocity of the end-effector and provides additional functionalities such as commanding the action of the gripper. The proposed interface is evaluated in a series of manipulation tasks involving a 6DOF assistive robot manipulator equipped with 1DOF beak-like gripper. The two interface modalities are combined to successfully accomplish tasks requiring bimanual capacity that is usually affected in people with upper limbs impairments.

In recent years, several robotic end-effectors have been developed and made available in the market. Nevertheless, their adoption in industrial context is still limited due to a burdensome integration, which strongly relies on customized software modules specific for each end-effector. Indeed, to enable the functionalities of these end-effectors, dedicated interfaces must be developed to consider the different end-effector characteristics, like finger kinematics, actuation systems, and communication protocols. To face the challenges described above, we present ROS End-Effector, an open-source framework capable of accommodating a wide range of robotic end-effectors of different grasping capabilities (grasping, pinching, or independent finger dexterity) and hardware characteristics. The ROS End-Effector framework, rather than controlling each end-effector in a different and customized way, allows to mask the physical hardware differences and permits to control the end-effector using a set of high-level grasping primitives automatically extracted. By leveraging on hardware agnostic software modules including hardware abstraction layer (HAL), application programming interfaces (APIs), simulation tools and graphical user interfaces (GUIs), ROS End-Effector effectively facilitates the integration of diverse end-effector devices. The proposed framework capabilities in supporting different robotics end-effectors are demonstrated in both simulated and real hardware experiments using a variety of end-effectors with diverse characteristics, ranging from under-actuated grippers to anthropomorphic robotic hands. Finally, from the user perspective, the manuscript provides a set of examples about the use of the framework showing its flexibility in integrating a new end-effector module.