D2.2 - Mixed-reality stereo vision camera interface

Brief description of functionality/utility:

Streaming video from a stereo camera mounted on the end effector of a robot arm in the remote scene to the Head-MountedDisplay (HMD) for the teleoperator.

  1. The teleoperator can move around and have different viewing angles/positions to maximise their situation awareness. The robot motion is synchronised with the operator’s head motion.
  2. The current implementation is based on a Stereolab ZED camera and Franka Emika Panda arm, however, so long as similar equipment is used, with real-time video stream (camera) and real-time control (robot arm) the same algorithm can be implemented.
  3. The system is currently at TRL 5.