**AI is such an upgrade to the accessibility world, and the more I think about it the more it makes sense. ** My first AI project was an AI camera on glasses that would name objects and help visually impaired people find things. It was a webapp that anyone could use and won a few competitions. Cool idea, but to be honest it lacked in user experience. It was difficult to use, lots of buttons to press, and it could be buggy and unresponsive.
This is where LLMs shine. Currently, they can handle multimodal inputs like images and videos, give multimodal outputs like drawing bounding boxes, and most importantly – they can do tasks using tools. But the real game-changer is that they respond in natural language and explain what’s happening. They can potentially even figure out what’s wrong and repair themselves.
LLMs are a new interface to technology, a new way to interact. I don’t believe they’ll replace traditional screens, tapping, clicking and so on, but they’ll take up a significant portion through voice interfaces. When your hands are busy, your phone is out of reach, or your laptop takes too long to open – it’s so nice to just ask an assistant and let it respond or even do something.
Home and phone assistants were already able to do this, but LLMs add a next level of agency. They can process more complex queries, solve problems independently, and respond intelligently.
*For accessibility, this is gold. *
Without needing to swipe, touch, and learn gestures (for blind users, for instance), they can just ask. “When is the bus arriving?” No need to open the fucking “Onay” app – which even healthy people have trouble using – they can just ask an agent to do it and read the results.
No need to traverse hundreds of websites, scrolling and painfully listening to information on every page – an agent can read 10 sources simultaneously and generate a summary better than a human would.
No need to ask a volunteer from BeMyEyes to look at an image, check an outfit, or choose fresh produce – just point the phone camera or open the image. The agent will finally describe what’s in all those images everyone keeps sending to group chats that you can’t see.
What I see coming is agents having full control over computers and other devices. For casual users, it might be cool to show off to colleagues, but for someone with impairments, this is a new and accessible way to interact with technology and do things they couldn’t do before.
AI is changing the world and jobs, algorithms are everywhere – and I’m glad that in some way it just works for good.