Story
When it is easier to show than to describe, I want to add a photo, my camera or my screen to my ask and see what the AI read from it, so I can trust the answer is about the right thing.
Nomvula, setting up a new fibre router at home in Johannesburg, with her phone in 1 hand.
- 01 Setting
Nomvula's new router has 4 lights. 1 is orange and the internet is down.
- 02 Trigger
She doesn't know the light's name, so she takes a photo instead.
- 03 Without it
"Orange usually means something is wrong. Try restarting." She can't tell which light it looked at, and restarting doesn't help.
- 04 With it
"From your photo: 4 lights. WAN is orange." The WAN light is marked on her photo.
- 05 Outcome
She clicks the fibre cable in properly. 2 minutes later the light turns green.
Live demo
This demo follows a script, so it plays the same every time - there is no real AI behind it. Change a setting to see the same moment play out differently.
What happens, step by step
| State | The person sees | The AI is | When | Screen reader |
|---|---|---|---|---|
| 1 | The + offers Photo, Live camera and Share screen, each with its own icon. | Idle. | on tap | - |
| 2 | A photo becomes a chip. Live camera or screen shows an "on" bar with Stop, the whole time. | Idle, or watching while live. | - | “"Photo attached", or "Camera sharing on"” |
| 3 | Words that point at the picture: "Why is this light orange?" | Idle. | - | - |
| 4 | "Looking at your photo" - says which input it is reading. | Reads the image. | 1-3 s | “Looking at your photo” |
| 5 | 1 line on what it read from the image, and that part marked on the image. Then the answer. | Writes. | - | “Generating a response” |
| 6 | People can check the answer against the marked part. | Idle. | when done | “Response complete” |
| 7 | "Camera off · was on for 0:24" stays in the chat. | No longer sees anything. | on Stop, or when the app is left | “Camera sharing off” |
On each device
- phone
- The camera is the main use: photo or live. The on bar sits at the top. The OS indicator (the phone's own camera-on dot) shows too. Screen sharing stops when the screen locks.
- tablet
- As on the phone, plus screen sharing of the app beside it in split view.
- desktop
- Paste or drop a screenshot, or share 1 window - never the whole screen by default. The on bar names what is shared.
When to use it
- The thing is hard to name or describe - a light, a rash, a part, an error on screen.
- A picture holds the facts - a label, a chart, a form, a receipt.
- People are doing something with their hands and can point the camera at it.
- People only need to attach documents to read - files are enough. Use Attachments and context chips instead.
- Speech alone does the job - a spoken ask with no picture. Use Voice input instead.
- The AI must act on the screen, not only look - that needs an agent (an AI that can take actions for you) people can watch. Use Watch mode and takeover instead.
The parts of the pattern
- AThe image in the askThe photo stays in the sent message, so the answer has its source.
- BMarked partThe part of the image the answer is about, outlined.
- CWhat I read1 line in plain words: "From your photo: 4 lights. WAN is orange."
- DThe answerComes after what it read, so people can check the reading first.
Do and Avoid
From your photo: WAN is orange.
An orange WAN light means the router can’t reach your provider.
Orange usually means something is wrong. Try restarting.
● Camera on · Stop
Camera off · was on for 0:24
(camera still on in the background)
Ask anything
Writing and accessibility
- Start the reply with what was read, naming the input: "From your photo: …".
- Name live inputs plainly while on: "Camera on", "Sharing your screen".
- Record when they end: "Camera off · was on for 0:24".
- If the image is unclear, say what is missing and ask for a better 1: "I can't read the label - can you move closer?"
- Give a sent photo the alt text (the words a screen reader says for an image) the AI read from it, so people who use a screen reader get the same reading.
- Announce "Camera sharing on" and "Camera sharing off" in the polite status region (the part of the page a screen reader announces on its own).
- Stop is a real button, reachable at any time, at least 44 px.
- Never make an image the only way to ask - typing and voice always work too.
Seen in real products
- Gemini LiveAndroid · Nov 2026People tap Camera to share it and tap again to stop. Screen sharing shows a "Screen Sharing" card in the notification area with Stop sharing, and both stop when the screen locks.Gemini Apps Help: Talk naturally with Gemini Live ↗
- Gemini Live (beta redesign)Android · Apr 2026The prompt box becomes a pill with a waveform. Camera and screen sharing sit on its left, mute on its right.9to5Google, 19 Apr 2026 ↗
- Gemini AppsWeb · Nov 20261 prompt can carry up to 10 files, including video (up to 5 minutes) and audio (up to 10 minutes), with more on paid plans.Gemini Apps Help: Upload & analyze files ↗
Each example comes from the product’s help pages, release notes or teardowns, with the month it was checked. Products change every month - tell us when 1 is out of date.
Check yourself
Seen this pattern done better, or a product that changed? Press Suggest next to any heading, or tell us about the whole page.