Seeing AI

Seeing AI uses the power of artificial intelligence (AI) to describe everything from printed text to photos, and to identify money, nearby people, and things in your cupboard.

It’s a free app created by Microsoft for iPhones, iPads, and Android devices. Seeing AI is available in the UK and beyond – perfect for when you’re abroad – and supports many languages.  

How does Seeing AI work?

Seeing AI uses your camera to provide visual assistance. The different features of Seeing AI are known as “channels”: 

• Short Text: Point your camera at some text, for example a paragraph in a document, prescriptions, menus, or signs, and this channel will read it.
• Document: This channel will ask you to take a picture of a physical document and then read it aloud to you.
• Product: If you need help identifying a product, this channel will scan the barcode or QR code and tell you what it is. It can distinguish between tins in your cupboard or help you find the right item when out shopping.
• Person: Find out how many people are nearby or use this channel when taking a photo to make sure the person is in the centre of the camera and smiling. You can also teach the app to recognise your friends and family.
• Currency: This channel gives an estimated value of money in note form and works with a variety of currencies. If you’re buying something at a cash-only shop, you can check you’re paying with the right amount. Keep in mind that the app can only detect notes, not coins.
• Scene: A channel that gives you a basic description of your environment or an object. Ideal if you need a quick, short description of something, like a picture.
• Colour: This channel will identify the colour of the object in the central finder window. Ideal if you’d like to know the colour of a piece of clothing, make-up, or colours in an image. However, if the object is shiny or in shadow, it may not be accurate. 
• Handwriting: The app asks you to take a picture of something handwritten, and then reads it aloud in this channel. Perfect for reading greeting cards or special boards. 
• Light: This channel is a light meter that uses sound to tell you how bright it is: the higher pitched the sound, the brighter the room is. This can be helpful to know when to draw the curtains, or when your lights are on or off.
• World: The ‘World’ channel uses an audio Augmented Reality experience to help someone understand their environment. It can be used to support and enhance indoor navigation and identify or locate objects. This feature is currently only available on iOS devices.

The camera appears in the centre of the app’s interface, a Help button is in the top right-hand corner, and a burger menu (looks like three stacked lines) featuring options like “Browse Photos” and “Settings” is in the top left of the screen. At the foot of the app, you can choose between tabs – this is where you’ll find the channels.

When you first use a channel, the app provides instructions on how it works, and the “Help” option in the burger menu provides written and video tutorials on how the app works. Seeing AI is fully accessible with Voiceover on iOS and TalkBack on Android to help you navigate the app with a screen reader.

How Seeing AI can help you

Here’s how Seeing AI could be useful on a daily basis:  

• Identify products: From tins of food, to make up, or household items, the Product channel can help you quickly identify or distinguish between items.
• Reading text privately: the Document and Short Text channels enable you to read text in your own time, without having to ask someone else to help you.
• Navigate indoors: (iOS devices only) The World channel can be used indoors using spatial audio cues alongside usual mobility aid. You can either create your own route, create a route with sighted support – someone guiding you on the route for you to complete independently later – or someone can create a route, save it and send to you to use. 

Things to keep in mind when using the Seeing AI app 

• Several of the app’s channels are undergoing improvements - always check the accuracy of the information it provides.
• Similarly, AI itself is never 100% accurate. When the app detects text, it will read it aloud immediately, so make sure you’re with people you trust when using the app to read personal information, or you can use it with headphones.
• To use some channels, you’ll need an active internet connection through WiFi or mobile data.
• The app might struggle to read text if the print quality is poor, or the ink is smudged.
• The app performs better on newer devices.
• To get the most out of this app, you may need to use Voiceover or Talkback to read out text.
• Some of the channels automatically use flash when taking a picture.
• The People channel tries to guess the age of the person in the camera, which might lead to an awkward moment!  

How to download Seeing AI

If you have an iPhone or iPad, you can download Seeing AI for free from the App Store. If you have an Android device, you can download it for free from the Google Play Store.

The app’s icon features a sky blue background with a white, abstract outline of an eye that is also an ‘S’ in the centre.

Apps that are similar to Seeing AI

Seeing AI is a useful app for people with vision impairment, but other smartphone apps are available, such as:

• Envision AI: Envision AI is a free app that uses your smartphone's camera to speak out written information, describe surroundings and objects, and even tell you who's nearby.

• Google Lookout: Google Lookout is designed specifically for people with visual impairment. It uses the camera and other sensors on your Android device to tell you about objects and text nearby. 

• Sullivan+:Sullivan+ is a free app that can detect and read aloud text, describe images, recognise colours and indicate current lighting levels in your surroundings.

You might also like...

Edited and reviewed by Technology Development Lead Tommy Dean on 26/07/2024

Guide Dogs is committed to integrating technology into our service delivery and promoting excellence in accessibility and usability. Our staff undergo comprehensive training in technology and accessibility in collaboration with industry leaders like Apple. We foster knowledge of technology and create content through our Technology Champions, comprising Vision Rehabilitation Specialists and Habilitation Specialists. We review this content periodically to ensure that it is up to date and includes new features and changes with operating systems. We work with our partners to ensure accessibility is at the centre of the design. To find out more, there is more information in our accessibility policy.