About VoiceOver audits
An iOS audit does not simulate a screen reader. It runs your app on an iOS Simulator, or on an iPhone connected to your computer, with the real VoiceOver switched on. An agent operates the app the way a VoiceOver user does. It flicks, touches points on the screen and double-taps, and it judges the app on what VoiceOver says. This page covers how that works and why it is built that way.
Who does what
Section titled “Who does what”Three parties take part in a run.
- The simulator or iPhone runs your app, VoiceOver, and the Binclusive audit host. The host is a UI test. It switches VoiceOver on through Xcode’s test support, makes each gesture it is asked for, and answers with what VoiceOver said and a screenshot of the screen.
- The CLI prepares the phone’s signed host through its owning organization, verifies and
caches the signed pair, installs it, and starts it with pymobiledevice3 on Mac, Windows or
Linux. An admin first connects the Apple team in Settings › iPhones, then the user runs
b8e audit. A server refusal stops the run. On a Mac simulator, the CLI starts the host through Xcode’s test support and also reads VoiceOver’s speech log, so it hears everything VoiceOver said during a move, in order. - The audit service decides what to do next. It asks the CLI for one action at a time and reads the result before it chooses the next.
Xcode’s test support can drive VoiceOver from iOS 27 onwards, and not before. That is why an audit needs a simulator on an iOS 27 runtime or an iPhone on iOS 27. Simulator runs need Xcode 27 on a Mac. Standard phone execution uses the prepared host and pymobiledevice3.
On all three computer platforms, phone runs use ffmpeg to turn the screenshots returned after each action into a frame replay on the run’s page. The replay has no audio or motion between steps. Without ffmpeg, the audit still completes and reports why no replay was saved.
The simulator’s speech log is what makes silence mean something. The host on its own can only report the last thing VoiceOver said. The log carries every utterance, every announcement that did not move the cursor, and the sounds that carry meaning: the boundary sound at the end of a screen, the screen-change sound, and the not-found sound. With the log running, “VoiceOver said nothing” is a fact the audit can report. If the CLI cannot read the log, a result holds only the last thing VoiceOver said, and the tester is told not to treat silence as evidence.
Local Swift development
Section titled “Local Swift development”The explicit local Swift development route, selected by BINCLUSIVE_IOS_HOST_DIR, runs
unpublished host changes from a checkout containing AuditHost.xcodeproj. It builds and starts
the host with xcodebuild, using an Apple account in Xcode, and can export XCTest’s native screen
recording. The local Swift setup
covers this route.
Plan, then test
Section titled “Plan, then test”A journey in the dashboard is plain text, such as “sign in and add an item to the cart”. The audit first turns it into steps a VoiceOver user could follow, with each one tied to a screen and a gesture: “On the Login screen, double-tap ‘Sign in’”, not “Log in”. A journey that names a starting screen can open it through the app’s universal link or URL scheme on a simulator. Standard phone execution requires a tap path instead, because pymobiledevice3 launches an app by its bundle identifier only. Without an entry path, the walk starts from the app’s launch screen.
A tester then walks the steps in order. For each one it does what the step says, listens to every announcement along the way, and records a finding while VoiceOver’s cursor is still on the element at fault. A finding keeps the screenshot from the most recent announcement, so recording it before the next move is what puts the right element in the picture. When a step is done it moves the plan on to the next one.
The tester judges what VoiceOver spoke. With each result it also gets the screen’s accessibility tree and a list of its own moves so far. It uses the tree as a map of what is on screen and where, and never quotes it as evidence. A button drawn with a label that VoiceOver announces only as “button” fails WCAG 4.1.2. The same goes for a change of state that is never spoken. When the log is running and VoiceOver says nothing after an action that should confirm itself, such as saving or adding to a cart, that silence is a 4.1.3 finding.
Before it records that something cannot be reached, the tester tries both ways a VoiceOver user would. It flicks through the screen, then touches the spot where the control should be, such as the bottom edge for a tab bar or the top-left corner for a Back button. Touching is ordinary VoiceOver use when the control was already on screen. If the tester has to go looking for the result of its own action, such as a sheet it opened or an error it caused, the app should have moved VoiceOver’s cursor there or announced it, and the tester records that as a finding.
What the agent can do
Section titled “What the agent can do”The tester has a fixed set of actions, each one something a VoiceOver user does:
- move the cursor to the next or previous item, the one-finger flick;
- move to the next or previous item of one type, such as a heading, a heading level, a button, a link, a text field, an image, a table or a landmark, the search a user runs with the rotor;
- turn the rotor to a setting by name, including a custom rotor your app offers, and flick to its next stop;
- move through text by character, word or line, and delete characters before the insertion point;
- touch one named point on the screen and hear what is under it: the top, the centre, the bottom, or a corner;
- slide one finger from one of those points to another and hear every item it passes;
- double-tap the item the cursor is on;
- type into the field being edited;
- have VoiceOver read the whole screen, or read on from the cursor;
- jump to the first or last item on the screen, the four-finger tap;
- scroll one page up or down, the three-finger flick;
- change an adjustable control, such as a slider or a stepper, by one step;
- list the custom actions of an item VoiceOver says has them, without performing one;
- make the two-finger scrub that dismisses an alert, sheet or menu, or goes back;
- make the magic tap, the two-finger double-tap for the app’s main action;
- deliver a push notification to your app, as its server would, when the journey says the app sends one (on a simulator only);
- hear the last announcement again without moving.
Every action returns what VoiceOver voiced during the move, a screenshot, and the accessibility tree. The tester cannot tap a pixel it read off the screenshot or open a screen by name, and iOS has no system Back key to press. If a user cannot reach something with these gestures, the tester cannot reach it either.
How spoken output is judged
Section titled “How spoken output is judged”A finding’s evidence is what VoiceOver said, word for word, including an item’s hint such as
“Double tap to activate”. Nothing is stripped from iOS evidence. On iOS a hint belongs to the item,
and your app can set its own with accessibilityHint, so the tester checks the hint along with the
label, role and state. This differs from Android, where TalkBack adds its own usage hints and the
audit strips them.
Each finding is checked against the project’s earlier findings to decide whether it is new or one you already have. For an iOS finding, that check compares the words VoiceOver spoke.
What a run changes on the device
Section titled “What a run changes on the device”Before each journey on a simulator, the CLI prepares it:
- it boots the simulator if it is not running, and leaves it running afterwards;
- it turns on VoiceOver’s single-letter Quick Nav keys, which the item-type search uses, and puts each one back the way it found it when the journey ends;
- it quits your app and resets the app’s privacy permissions, so permission prompts appear the way a new user meets them, and the tester can judge them;
- it launches your app again, through its link when the journey starts from one.
On an iPhone the CLI changes less. It checks the phone is unlocked and launches your app again, leaving its permissions and VoiceOver settings as they are. The audit host stays installed on the phone after the run, so the next run starts sooner.
The host then switches VoiceOver on. Standard phone execution turns it off when the journey ends, because pymobiledevice3 cannot read its prior state. A simulator restores the state it read before the journey.
Stopping the CLI doesn’t skip that clean-up, whether you press Ctrl-C, close the terminal, or end it
with kill. The host runs apart from your terminal, so these reach only the CLI. The CLI lets the
host finish the move it is making, then stops it the same way a finished journey does. The host
turns VoiceOver off on a phone, and on a simulator the CLI restores VoiceOver and its Quick Nav
keys before it exits. A second Ctrl-C, or
kill -9, quits at once and skips the rest, which can leave VoiceOver on.
The host also checks before the first action. If VoiceOver cannot be switched on, or your app does
not reach the foreground within 30 seconds, the host reports that, and the run stops before the
tester takes a step, with a message that starts the iOS audit host could not start.
What’s different on a phone
Section titled “What’s different on a phone”An iPhone runs the same host and the same tester as a simulator, but a few things change what the tester hears and can do.
- Speech is sampled, not logged. A Mac can’t read an iPhone’s speech log, so after each move the host listens to VoiceOver itself for a moment, longer after the screen changes, and reports every utterance it heard. That catches the move’s own speech and an announcement that follows it. It can miss an announcement that VoiceOver speaks and replaces in the middle of a gesture. So on a phone, “VoiceOver said nothing” is never treated as evidence on its own. A read-all is heard until VoiceOver has been quiet for six seconds, for up to a minute.
- The rotor finds items by type. On a simulator the host uses VoiceOver’s single-letter Quick Nav keys to move to the next heading or button. A phone has no keyboard, so the host turns the rotor to that item type, the way a user does. The phone’s rotor offers an item only when it is switched on in Settings › Accessibility › VoiceOver › Rotor and applies to what is on screen, so a native screen may not offer Buttons or Text Fields even when they are switched on. The tester then moves with next, previous, jump or explore instead. A heading of one level has no rotor item at all, so the tester moves by heading and reads the level VoiceOver speaks.
- English and Turkish VoiceOver are read. The host recognizes a few of VoiceOver’s own words, such as “adjustable”, “Actions available” and the rotor’s item names, in English and Turkish at once. Your phone’s language decides VoiceOver’s own words, and the voice VoiceOver picks for your app decides how it speaks roles such as “button”. Other languages are untested.
- A double-tap activates VoiceOver’s own cursor. On a phone, VoiceOver activates the item its cursor is on wherever the finger lands, as it does for a user. When nothing in the accessibility tree carries what VoiceOver last said, such as an image VoiceOver described itself, the result says it double-tapped the item under the cursor and quotes that speech, rather than guessing an element’s name.
- No push notifications. The Mac can’t deliver a test notification to an iPhone, so the tester can’t make one arrive.
- No permission reset. The journey meets whatever permissions your app already has on the phone.
- Unlabeled images may be described. VoiceOver on an iPhone can describe an image your app left without a label, such as “a screenshot of a social media post”. That description comes from the phone, not your app, and not every user hears it, so the image still needs a label of its own.
To run an audit, see Run an iOS audit with VoiceOver.