2003 IRISH SCIENTIST YEAR BOOK

Home Page

Table of contents

Index by Author

Index by topics

Search


Institute of Technology Carlow
Nigel Whyte
Innovations in new Irish Sign Language Identification System (ISLIS)

Gabriel Sommers, postgraduate research student, in the TV Studios at IT Carlow, demonstrates how the Irish Sign Language Identification System project will work. For example, for those with hearing difficulties entering a service situation like a bank, their sign language can be read by cameras and then translated so that customer service staff can understand and help them.

The ability to recognise specific gestures is essential if computers are to interact naturally with human users. At present this information is unavailable to machines. This relatively new application of computer vision aims to extend the interface between man and machine by the implementation of novel algorithms. We are interested in developing algorithms for Irish Sign Language recognition without the use of external devices, such as data gloves. The goal of ISLIS is to identify complete sentences of signs by analysing stereo video footage of a signer to determine their hand path, posture, orientation and position.

Work to date has resulted in a hand path identification system that analyses movement of a person's hand. Based on an existing Finite State Machine technique, ISLIS has extended the technique to work for three-dimensional hand movements, resulting in an increased ability to identify complex hand paths. Experimental test results have verified the increased capacity for recognition provided by the addition of three-dimensional tracking. Sets of hand paths were created, containing pairs of two-dimensionally similar hand paths. ISLIS and a similar two-dimensional system were then comparison tested. In every instance, the two-dimensional system was unable to differentiate between the two hand paths in each gesture pair. Our three dimensional system successfully recognised each gesture.

Currently under development is a hand posture recogniser. It uses a combination of three-dimensional models and silhouette matching to recognise hand postures. Each of the 36 basic hand shapes used in Irish sign language is modelled. All models can be rotated on any axis, so that a silhouette of the model at the correct orientation is created. This silhouette is then matched against a hand silhouette extracted from the video footage. One of the main downfalls of silhouette matching for articulated objects such as the hand is the loss of hand information due to self-occlusion. However using stereo images we alleviate this problem and therefore increase the likelihood of retrieving a correct match. Preliminary results show this technique can correctly identify 75% of hand postures. Further refinements that we anticipate will significantly improve the posture recognition stage are currently under implementation.

Contact: Nigel Whyte; Tel: 059 91 70455; E-mail: [email protected]