Cybathlon: 6D pose estimation and grasping automation
Overview
During my semester thesis at the Robotic Systems Lab at ETHZ I developed a perception pipeline for the quadrapedal anymal robot. Parts of my pipeline were deplyed at the 2024 Cybathlon, which is a competition where people with different impairments compete in physical challenges with the help of technological aids. I contributed to the "Spice-Up challenge", in which the robot needs to localize a shelf, navigate to it, grab a spice bottle from within the shelf and place the bottle on top of the shelf, as depicted below. Specifically, the robot needs to localize and grasp the target spice bottle indicated by a sign.
Result
As can be seen in the video below the robot is able localize the shelf and the target spice bottle, which allows the robot arm on its back to grasp it. The end effector grap pose is the end result of my perception pipeline. However, I did not implement the motion planning myself.
Bachelor thesis at ETHZ
Overview
I did my bachelor thesis at the Insitute for Neurotechology under Prof. Dr. Mehmet Fatih Yanik. My task was to assemble a robot resembling a small monkey, with height and proportion similar to those of marmoset monkeys. The ulimate goal was and is to let the robot monkey visually demonstrate movements to an audience of real marmoset monkeys, to investigate the imitation learning taking place.
Hardware details
I was given the already fabricated robot model and STL files of the base structure, which was to house the servo motors and on top of which the robot monkey would stand.
I 3D printed this base structure, mounted all servo motors and the robot monkey and spend a lot of time connecting the motors with their respective joints with fisher mans thread. These threads transmit the torque from the motor through the robot body to the target joints.
Next, I setup the power supply circuit according to instructions of my supervisor. The diagram below on the right depicts its concept. As my debut in laser cutting I designed a acrilyc box with CorelDraw to hold the two power supplies (left picture).
Software details
The control architecture I setup to control the robot monkey consists of three steps. First, a user can create 3D waypoints in Rviz. Second, those waypoints are passed to the MoveIt motion planner, which can use a multitude of different inverse kinematics solvers to merge the specified waypoints into a feasible path or warn the user if there is no feasible path. Third, the given path is transmitted over the ROS network from the simulation pc to the raspberry pi, which in turn sends the calculated signals to the servo motors. The following flow chart (right image) illustrates the individual features of the control interface. The left image shows how waypoints can be set in Rviz.
The project report can be found here
Short demo of trajectory creation
Highschool graduation project
Overview
For my highscool graduation project (dt.: Maturarbeit) I assembled the right arm of the humanoid robot InMoov as designed by Gael Langevin. I then used an Arduino Mega to control the elbow, the wrist and the fingers with servo motors. The bigger part of the project consisted of setting up different ways of controlling the arm. First, I used a PS4 Dualshock controller to control some of the fingers. Next, I used an armband which performed EMG (electromyography) to relay the movements of my own fingers and wrist to the robotic arm. Finally I tried to connect an ECG (electroencephalography) headset to the arm, hoping to map readings of the eletrical activity in my brain to movements of the arm, but that didn't work.
Control via PS4 Dualshock Controller
The PS4 Dualshock controller has around 30 buttons/knobs one can control. 14 were mapped to fingers and wrist movements of the hand. The bluetooth signals are first sent to the Raspberry Pi, which unlike the arduino is bluetooth capable out of the box. The Raspberry Pi then forwards the signals to the Arduino via its GPIO (general purpose input output) pins.
Control via EMG (electromyography)
The EMG wristband model is called Myo and was manufactured by Thalmic labs. They sadly stopped producing and supporting it. The wristband is able to to quite accurately determine if the wearers hand is in one of the following states: Pinching, flat-spread, fist, flat-up or flat-down. It can be programmed to record arbitrary hand states, i.e the accompanying electrical patterns measured by the electrodes, over a certain time interval. Sadly that was, and probably still is, to advanced for me. Further information can be obtained by reading my project report.
Sensory Substitution Device
Attempt to build a prototype for a sensory substitution device
Inspired by the work of Dr. David Eagleman, this device was intended to test whether such a
technological aid can help visually impaired patients detect obstacles, or shapes in general,
surrounding them. The plan was that four ultrasonic sensors scan the surrounding of the patient’s head
in four directions and then communicate the specific distance to the patient via four vibrating motors
on the forehead, each placed behind its corresponding sensor. Unfortunately, I completely forgot that the
sonic waves used to measure the distance interfere when more than one sensor is active.
What I did succeed in with this project was getting a glimpse of the thought process behind creating a
sensory aid for visually impaired people. Also, I had some inspiring initial conversations with potential
test persons with whom I talked about their wishes and expectations for such a device.
Click here to see the breadboard layout and
schematics.
Xmas gift distribution system
For Xmas 2019 I wanted to gift my loved ones books, but the delivery got delayed and I feared standing in front of the Xmas tree empty handed. So I did what every sane person would have done. I built a bomb. A bomb that served as a gift-distribution-system. There was a personalised pre-programed text for every family member, which included ordering the respective person to cut through the wire connected to the word that provoked the strongest emotions and thereby contributing to the 'defusing' of the bomb. Navigation could be achieved by pressing the green button.
The 'bomb' would know which wires had already been cut through and adjust its displayed messages accordingly. After everyone had cut through a wire and 'selected' a word, the bomb explained that they had all just chosen which book they would be gifted, with the books all coming from the german author 'Ferdinand von Schirach', whom I highly admire. Funfact: when it was my grandmothers turn, the fridge produced a high pitched noise (due to the accidentally open door) and everyone jumped away from the bomb, already hearing the angels singing^^