Inhaltsverzeichnis

Description

Hier entwickeln Master-Studierende eine Software, um mit einem speziellen Armband (MYO) eine Gestenerkennung für Gebärdensprache zu realisieren.

Gesture recognition with the help of armband sensor.

We started with the task of digit recognition, once the band is worn by subject and a specific digit gesture is made. Once this task is completed, our assumption is the same model can be extended to alphabets and other complex gesture recognition.

We have conducted experiments and collected data of 16 people so far. This data is further used for model generation and analysis for recognition of gestures based on specific feature set in each gesture.

For this we have utilized and analysed 4 classical machine learning approaches(Hidden Markov Models-HMM, Support Vector Machines-SVM, Naive Bayes-NB, K Nearest Neighbor-KNN) and Artificial Neural Networks approach(Long short term memory -LSTM).

Created two sets of training instances
- One with 10 instances per class
- One with 20 instances per class
Evaluated models using the following algorithms,
- HMM - Raw Data
- HMM - Windowed Features
- Naive Bays
- KNN (1 neighbour)
- SVM (Parameters using grid search)
Analysed the accuracy precision, F-Score for all the models in all the folds
Analysed the features and tried to decide which features to eliminate and which features are not significant using.
- Parallel Coordinates
- Andrews Curves

We have also developed and (open sourced) released an application for data visualization and capture for Myo armband. https://github.com/sigvoiced/pewter

Now we aim to wrap up these results via an application that is capable of capturing, analyzing (fixed set of)gestures in real time and classifying them. Once this is done we can give a live demo/presentation of our results so far and continue our work towards more complex and higher order gestures.

Die hier verlinkten weiteren Seiten zu diesem Projekt sind nur für angemeldete SWLab-Teilnehmer lesbar.