Sign Language Interpreter
Hand gesture interpreter: teach your own gestures, train a CNN, read gestures as words and sentences and speak them. Camera, video or pictures; demo gestures and a trained network included. Python, TensorFlow, OpenCV.
- Price: ₹1,799
- Category: AI / ML
- Technology: Python, TensorFlow / Keras, CNN, OpenCV, Tkinter, Pillow, SQLite, Matplotlib
- Includes: Full source code
- Suitable for: BCA / B.Sc CS, MCA / M.Sc CS, B.E / B.Tech CSE & IT
Project overview
This is a desktop application for Windows with a login and five screens that follow the steps of the method. Hand colour: open a picture or start the camera, hold the hand in the green square, press 'Take the colour' and see the black-and-white mask and the 50 x 50 picture the computer will use (or use the automatic skin colour). Gestures: a list of gestures (each is a word) with their number of pictures; add a gesture, take one picture or many pictures while the camera or a video plays, or add all pictures of a folder; delete a gesture. Train: choose the epochs and train a convolutional neural network in the background with a live accuracy curve; the result shows the test accuracy, the weakest gesture and a confusion matrix. Interpret: play the camera, a video or open a picture; the screen shows the word, the chance of every gesture and the 50 x 50 picture, a gesture held for the chosen number of frames with the chosen confidence becomes a word of the sentence; say the sentence aloud, remove the last word, clear it or save it to the history. History: all saved sentences, a chart of the words used and CSV export. Six demo gestures (fist, 1 to 5 raised fingers, with an example vocabulary: Stop, I, Need, Water, Please, Hello), 12 drawn demo hands, a demo video that spells 'Hello I Need Water Please' and a trained network are included. 26 code tests and 71 screen checks (with a real training run) are included.
Modules
- Hand colour: Takes the colour of the hand from a square on a picture or the camera (hue-saturation histogram), shows the mask and the 50 x 50 picture; automatic skin colour as a fallback.
- Gestures: A list of gestures with their pictures; take one or many pictures from the camera, a video or a folder; add and delete gestures; names are checked.
- Train: Trains a convolutional neural network on the pictures in a background thread with a live accuracy curve; test accuracy, weakest gesture and confusion matrix.
- Interpret: Camera, video or picture; the word with the chance of every gesture; steady gestures become words; a sentence with backspace, clear, speech and save.
- History: Saved sentences with a chart of the words used and CSV export.
- Login: Admin login with a salted hash and a change-password dialog.
Requirements
- Python 3.11
- Internet once, to install the libraries (about 10 minutes, downloads about 600 MB)
- Windows 10 / 11
How to run this project
- Install Python 3.11 from python.org and tick "Add python.exe to PATH" during setup.
- Extract the zip and open the project folder (STUDENT_GUIDE.md explains every step in simple words).
- Double-click `1_INSTALL.bat` once and wait until it says [OK] (about 10 minutes, downloads about 600 MB; it downloads the libraries into a private .venv folder).
- Double-click `2_RUN.bat` to start the project. If it does not start, double-click `CHECK_SETUP.bat` and read what it says.
- Log in with admin / admin123.
- Interpret: press 'Play a video' (demo_signs.avi): the words Hello, I, Need, Water, Please appear one by one; press 'Say it', 'Backspace', 'Save to history'. Open pictures hand_two_a.png and hand_five_a.png to see single predictions. History: show the saved sentence and the words chart; export the CSV.