Netra-Setu: Offline AI Braille Translator

👁️ Introduction: The Vision

According to the WHO, over 2.2 billion people globally have a vision impairment. A major challenge for this demographic is the sheer inaccessibility of standard printed text. While smartphone-based OCR apps exist, they heavily rely on active internet connections (cloud processing) and strictly deliver audio feedback, which does not aid Braille literacy and completely excludes deaf-blind individuals.

Current digital refreshable Braille displays cost thousands of dollars. There is a critical need for an offline, localized, and real-time tactile reading solution that restores true independence without internet dependency.

💡 The Solution: Netra-Setu

Netra-Setu (meaning "Eye-Bridge") is a fully offline, edge-AI assistive device that uses a live camera to read standard printed text and instantly translates it into physical, tactile Braille. By combining the power of Edge-AI with precise electromechanical actuation, it restores reading independence completely offline.

⚙️ How It Works: The Architecture

Netra-Setu leverages a dual-brain processing architecture to separate heavy AI inferencing from real-time hardware control.

1. The Edge-AI Vision Core (MPU)

A live webcam captures video frames of standard printed text. A headless Python-based autonomous vision pipeline processes these frames entirely offline using an Optical Character Recognition (OCR) neural network running on the Linux-based application processor.

Dynamic Tensor Alignment: To guarantee execution stability, a proprietary algorithm intercepts the network's tensor outputs. It auto-scales class matrices to prevent sequence-length mismatch errors and utilizes an auto-healing dictionary engine to dynamically extract exact AI labels directly from compiled model binaries at runtime.

2. Cross-Core RPC Bridge

Detected strings are parsed into individual characters and synchronously transmitted from the Linux AI core down to the real-time microcontroller environment using a high-speed Remote Procedure Call (RPC) protocol.

3. Electromechanical Tactile Actuation (MCU)

Custom C++ firmware on the MCU acts as a software PWM multiplexer engine. It drives a custom 3D-printed, 6-servo electromechanical Braille matrix. The controller actively manages active/rest pulse logic arrays, guaranteeing stable power distribution and eliminating logic-board brownouts during rapid, continuous text actuation. The user physically touches the servo matrix, instantly feeling the raised Braille dots corresponding exactly to the live printed text.

🎥 Project Demonstration

🌍 Societal Impact & Sustainability

Societally, Netra-Setu empowers visually impaired individuals with independent literacy and employability, fulfilling SDG 4 (Quality Education) and SDG 10 (Reduced Inequalities).

Environmentally, it drastically reduces the need to mass-produce physical Braille books (which require significantly more paper and resources than standard books). Furthermore, by utilizing Edge-AI, the system operates on a strict "Privacy by Design" architecture. Absolutely no internet connection is required, meaning no video frames or extracted texts are ever transmitted to the cloud, ensuring absolute data privacy.

🙏 Acknowledgements & Attributions

3D Hardware Base: The 3D-printed Braille cell mechanism (matrix box, pins, and holder) is a modified, custom-edited derivative based on the original mechanical design from the BrailleBot project by Mukesh Sankhla. The STLs were heavily adapted to house the specific servo multiplexing array and custom hardware used in Netra-Setu.