Even with the rise of tools and technologies, mankind hasn’t implemented applications that could help visually impaired people. With the rise of Data Modelling techniques that can be used to infuse “intelligence” even in dumb computers and the ease of accessibility, this “intelligence” can be extended to our Smartphone to help the visually impaired people cope up with their surroundings and get a helping hand in their daily activities. Our Application aims to bridge the gap between them and the visual world by leveraging the power of Deep Learning which can be made accessible even on low-ended devices with a lucid User-Interface that would exactly allow them to better understand the world around.
To know more about the Build Instructions for the App and to better understand the technicalities, check out our Wiki Page.
Our primary purpose behind this project is to leverage and study how Deep Learning Architectures along with easy prototyping tools can help us develop applications that can be easily rendered even on low-end devices. With this Application, we will develop a one-stop solution to allow the Blind or Partially Blind People to better understand the surroundings around them and to be able to cope with the dynamic world ahead of them.
The Minimal Viable Product (MVP) would allow the Users to leverage Image Captioning Architecture to generate a real-time insight into their surroundings while using Natural Language Processing to speak out in a lucid manner. The cornerstone of the Application would be its User Interface which would infuse a lucid experience for the User with its ease of handling and use.
For this project, we will be collaborating on various domains like:
- Data Modelling
- RESTful API Development
- Prototyping Mobile Application using Flutter
- UI/UX Designing
This would be an enriching experience for all of us that are part of this team.
- Every feature, from image labeling to currency detection uses a text-to-speech feature to speak out to the user whatever is detected 🗣️
- Each screen vibrates with different intensity on being opened helping to user navigate. The buttons also have unique vibrations for better accessibility 📳
- We have used a minimum number of buttons, but whichever buttons are there, are of a large size. For instance, the top half of the screen will be one button, and the bottom half, another button so that a user does not need to precisely click on a particular position 🔘
- All features, except the image captioning feature, work completely offline and do not require any internet connection📶
- All offline features work in real-time and do not need any pre-processing time for the models to make predictions so the user can get instant updates🏎️
- To get a deeper understanding and build insturctions for the project please check out our project's Wiki Page.
- Before contributing do go through the Code of Conduct and the Contributors Guidelines.
- If you find any bug in the application, or a feature you think would be nice to have, please open an issue.
- master: This is the default branch of the repository which contains the flutter appliation.
- Backend-CaptionBot: This is the branch with the backend that is currently in use for the image captioning feature.
- experimental-backend: This is an experimental backend we were working on using our own model and training scripts. But the model was too heavy to be hosted on a free server, so we went with our caption bot backend as it is a very light weight and accurate service.
- landing-page: This branch holds the landing website of the project.
Here is a link to the apk for trying it out: https://drive.google.com/file/d/1VjqDbLCVj_YqMb_hoIbc8naaDzEEYeKj/view?usp=sharing
Thanks goes to these wonderful people (emoji key):
 Shambhavi Aggarwal 💻 🤔 |
 Yash Khare 💻 📖 🤔 |
 Harsh Bardhan Mishra 📖 💻 |
This project follows the all-contributors specification. Contributions of any kind welcome!