Stop describing the screen and send the screen. Every detail you spend a paragraph describing is a detail you may get wrong, and a model that can see an image resolves layout, wording and error state more accurately than a text description of the same thing.
The failure is not technical, it is habitual. We describe interfaces because we learned to prompt before image input existed. Good multimodal prompt tips all follow the same pattern: attach the image, point at the region, ask one question, then verify against what you can see.
Stop describing what you can show
When you write "a settings page with a left sidebar and a toggle that appears to be off", you have already lost three things. You do not know the toggle's exact label. You cannot tell whether the grey means off or disabled. And your description has frozen the layout at the moment you looked at it, which may not be the moment the person reading your answer is seeing.
Sending the image removes all of that. The model reads the same pixels your user reads, including the parts you would have skipped because they seemed unimportant.
Two habits make this reliable. Crop to the region that matters when the screenshot is large, because a full-page capture spends the model's attention on chrome it does not need. And say what you want done with it, because a model handed an image with no question will describe it, and you already have a description.
Point at the region, then ask one question
The strongest multimodal prompt tips share a shape: a location, a task, a format.
- Location: "in the top right corner", "the second row of the table", "the red banner above the form". Models handle relative positions well once they are named.
- Task: one thing. Extract, compare, diagnose, rewrite, judge. Five tasks in one prompt gets you a shallow answer to each.
- Format: the exact output shape. A list of the visible labels is a different job from a diagnosis, and you should not ask for both.
When the answer is wrong, the first question is whether the image was legible. A cropped screenshot at low zoom produces confident misreads, and the model will not tell you it struggled.
Template: screenshot triage
Here is a screenshot of [the screen, product or page]. The problem is: [one sentence describing what the user is seeing]. Focus on [the region, described by position]. Do three things in order: 1. Transcribe the visible text in that region exactly, including any value in a field or toggle state you can determine. 2. State what the screen indicates about the problem in one sentence. 3. List what you cannot determine from this image alone. Do not guess at anything outside the region. Do not describe the layout, colours or styling unless I ask. Do not suggest a fix yet.
Fill in these details
- Screen: enough context for the model to know what kind of interface this is. One clause.
- Problem: the symptom, in the user's words if you have them. Symptoms are more reliable in the image than diagnoses.
- Region: position plus content type. "The billing card in the left column" beats "the card".
- Transcribe first: transcription is the step that catches a model guessing. If you go straight to diagnosis, plausible errors survive.
- Cannot determine: the line that keeps you from trusting a confident wrong read.
Template: read the UI and propose changes
Screenshot attached: [what it shows]. Target user: [who is using this screen]. List every usability problem you can see, ranked by how much it costs the user to complete the task. For each one give: what the screen shows, what the user has to do next, and the smallest change that would remove the extra step. Only include problems visible in the image. Do not invent screens, states or user goals you cannot see. If you are unsure whether a control is enabled, say unconfirmed rather than choosing.
Fill in these details
- Target user: the skill level of the person on that screen. It changes which problems count as costly.
- Task cost: rank by extra steps or extra confusion, not by visual polish. Interface taste is not usability.
- Smallest change: forces the model to propose an edit rather than a redesign, which is where image-based critique usually goes wrong.
A bug report built from one screenshot
A user reports a blank list. Instead of asking them to describe the filter state, you attach their screenshot and ask a narrow question.
| Step | Prompt | Output you use |
|---|---|---|
| Read | transcription of the filter bar | the active filter value, verbatim |
| Diagnose | what the screen indicates about the blank list | one sentence tying filter to result |
| Confirm | is the empty state copy shown? | yes or unconfirmed |
| Fix | smallest change that removes the confusion | the change to write into the ticket |
The transcription step is what makes this work. A description-based report would have said "some filters applied", and you would still have to ask a follow-up question.
Four checks before you trust a visual read
1. Compare the transcription to the image yourself
Pick two or three lines of transcribed text and check them against the pixels. If they do not match, stop there. Every downstream conclusion depends on the read being right.
2. Look for the unconfirmed label
Models that are unsure often say so, and that label is useful. Where a claim has no label and no visible support in the image, treat it as a guess and verify it yourself.
3. Check the crop boundaries
If the region you care about is half cut off, the model fills the missing part from convention rather than evidence. Re-crop and rerun before concluding anything.
4. Ask about one more region
A second question about a different part of the same screenshot is a cheap consistency test. If the two answers contradict each other, the first read is wrong.
Common multimodal prompt mistakes
- Attaching a full-page screenshot and asking a narrow question. Crop first. Attention spent on page chrome is attention not spent on the question.
- Describing the screen as well as sending it. Now there are two sources of truth and the model has to reconcile them.
- Asking five questions at once. You get five shallow answers and no way to tell which one failed.
- Uploading a scaled-down or compressed image. Small text becomes guesses that look like transcription.
- Treating layout critique as fact. A model can see that spacing is uneven, not that it hurts a specific user.
- Skipping the verification step. Image-capable models are confident. Confidence is not accuracy.
Frequently asked questions
Is a screenshot better than a written description?
For anything visual, yes. A screenshot carries exact wording, state indicators and layout relationships that a description approximates. A written description is still better when the image is low quality, when you are describing something you cannot capture, or when the detail you need is in behaviour rather than appearance.
How do I get an image-capable model to stop describing the screen?
Add the task and the format, and add a line that rules out description. Most chatgpt image upload workflows respond to "transcribe these three fields" far better than to "what do you see".
What if the model gets the text wrong?
Reduce the problem. Crop tighter, enlarge the region, or ask for one field at a time. A wrong read usually means the pixels were too small to read, not that the model ignored you.
Do these prompts work for documents and diagrams, not only interfaces?
Yes. The structure holds: point at the region, ask one question, require a transcription or an explicit uncertainty label. Documents need more attention to small text than interfaces do.
Send the image instead of describing it
Next time a user describes a screen, ask for a screenshot, crop to the region that matters, and run the triage template. For the reasoning that sits above a single image, read system prompt patterns.