The project for my Sophomore Design class at South Dakota School of Mines & Technology. For our semester long project, my group decided to make a wearable device that could translate hand gestures into mouse input for a computer. We nicknamed the device “The BAND” as an acronym of the involved members first names.
Reading Hand Gestures
To read the user’s hand position, we used an electromyography (EMG) sensor attached to the forearm with silver coated fabric and conductive gel. Whenever a muscle flexes in the human body it releases a very small amount of voltage.
The EMG sensor reads this voltage and then amplifies it to be within the analog input range of a microcontroller. When this voltage maintains a value for a certain number of measurements, then the microcontroller knows the user is holding their hand in a certain position.
Measuring Arm Movement
When a specific gesture is detected, pointing two fingers in our case, the microcontroller starts examining values from an additional accelerometer. The accelerometer provides information on the arms linear movement and and rotation. The is then translated into how the mouse should be moved on the screen.
Issues
There were two major issues with the design of the BAND. The first was the conductive gel that was needed to get accurate measurements with the EMG. This gel only allowed for ~20 minutes before it dried up and was no longer conductive. The EMG also required two 9v batteries for amplification. This made the device much larger than would’ve been preferred.
The second issue was the readings from the accelerometer. The chip game with an on board Inertial Measurement Unit (IMU) but the firmware for it was not well documented and was difficult to get running. For this reason the raw values were used and rather than reading linear velocity. This resulted in an unnatural rotation of the arm to move the mouse rather then a swiping motion that is desired.
Next Version
To fix the issue with using conductive gel, I’d like to remake the band using a flex sensor on each finger rather than and EMG. This will give more accurate readings and allow for longer use time. I’ve experimented with some home made fabric flex sensor designs but haven’t found one that I could reliably connect to the microcontroller.
The accelerometer issues can be fixed by spending more time getting the IMU functioning properly. If this took up too much time than creating a better algorithm for the raw values would also work.
The Results
The BAND can be seen being used in the video below. Looking closely, you can see the user having to move their arm in an unnatural way to get the mouse to move. There is also a bug that was eventually fixed in which a single mouse click is performed multiple times.
