So I’ve been working on a tool to help people with disabilities browse the web handsfree. Here’s a quick history of the project (links to code, prototypes, etc in description if interested…it’s all buggy still so maybe just watch 
):
Because I’m extremely low income, I walk everywhere. A lot. I live on the outskirts of Portland, OR which is a very eccentric city and I’m always running into oddities (people, places, things). I recently had a thought: “instead of pointing the camera at you, what if it was pointed the other way?”
The idea is that when the camera is pointed at you, you’re web browsing - in other words, you’re getting information from the internet. But if the camera is pointed away from you then you can use the internet to get information about your environment.
So I used tools I had in my apartment to hack a $13 WiFi drone:
And I just managed to get the glasses working! (excuse the poor lighting)
My next steps is to use TensorFlow.js to do basic object classification. There’s no screen (it’s basically just a WiFi camera) but because it’s connected to your device you can hear via your headphones. Everything above (minus the actual stream) is powered by JavaScript through a web browser.
I recently got to chat with the founder of the “Seeing AI” app (he now heads Aira, a smart glasses company for the blind) and I’ve been getting flooded with ideas. Ultimately, this is what I’m building (imagine you use these glasses instead of the phone):
Total cost of materials so far have been $13, but that’s because I ripped apart a drone. Sourcing the materials myself, I should be able to create a DIY parts list for people. And by attaching the components to a glasses arm sleeve, you can retrofit any pair of glasses for less than the cost of a Starbucks coffee (minus shipping)!
These don’t actually have a display, the idea being you would have headphones and it would just dictate to you but you could sign in front of your face to activate commands - sort of like this demo (also done in JavaScript):
Anyways, I’ll update again soon once I actually get the computer vision working on it! Let me know what you think of it, and what use cases you think it could be used for!