docs/accessibility/browser/how_a11y_works.md - chromium/src - Git at Google

 # How Chrome Accessibility Works

 This document explains the technical details behind Chrome accessibility
 code by starting at a high level and progressively adding more levels of
 detail.

 Please read the accessibility [overview](../overview.md) first.

 [TOC]

 ## Accessibility for a simple (non-browser) application

 As described in the [overview](../overview.md), every platform has its own
 accessibility APIs that are used by both assistive technology and sometimes
 by automation software. To better understand the challenges of accessibility
 support in Chromium, let's first explore what it's like to build an
 accessible application using a standard UI toolkit.

 Examples of standard toolkits would be Win32 UI controls or .NET components
 on Windows, or Cocoa NSViews on macOS, or Android Views via Java or Kotlin.
 When you use such a toolkit to build an application, a lot of accessibility
 comes for free.

 Typically every UI element - including containers and grouping elements - gets
 its own corresponding accessibility element, implementing that platform's
 accessibility API. So if you add a button, checkbox, scroll container, or
 text field to your application, you don't need to do any work to make it
 accessible - the built-in UI toolkit has provided all of that already.

 ![Simple accessibility tree](figures/ax_tree.png)

 ### Changing accessibility properties

 There are some cases where a bit of extra information may be required on the
 part of the app author. If you add an image button, you may need to ensure
 you provide an accessible label ("alt text"). Other common simple modifications
 might include hiding UI elements from accessibility because they're primarily
 decorative, or marking a UI element as an alert to ensure it's announced
 to screen reader users when it appears. However, none of these require
 writing more than a few lines of code.

 #### Windows

 The [dynamic annotation API](https://docs.microsoft.com/en-us/windows/win32/winauto/dynamic-annotation-api)
 lets you change any accessibility property of a UI element represented by an HWND. Here's a tiny code
 snippet showing how you could override the accessible name property (PROPID_ACC_NAME) to be the
 text "Accessible Name":

 ```C++
 CComPtr<IAccPropServices> pAccPropSrv;
 pAccPropSrv.CoCreateInstance(CLSID_AccPropServices);
 COleVariant varName("Accessible Name");
 pAccPropSrv->SetHwndProp(hwnd, OBJID_CLIENT, 0, PROPID_ACC_NAME, varName);
 ```

 #### Android

 On Android, if you don't want to subclass a View you can just set its accessibility delegate
 as an easy way to override one or more accessibility attributes. In this case we override
 the content description to say "Accessible Name".

 ```Java
 view.setAccessibilityDelegate(new AccessibilityDelegate() {
     public void onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfo info) {
         super.onInitializeAccessibilityNodeInfo(host, info);
         info.setContentDescription("Accessible Name");
     }
 });
 ```

 ### Custom controls and subclassing

 When building a custom control, more work is often required. As an example,
 suppose an application is implementing a "cover flow" image picker, such as the
 one found in the original iTunes album picker.  Conceptually the user is picking
 one album from a large list, but visually it's shown as a horizontally scrolling
 list of album cover images that fly by in three dimensions.

 ![Screenshot of a cover flow image picker](images/cover_flow.jpg)

 While this control may be implemented as a single UI element, from the
 perspective of accessibility APIs it would typically be represented as
 a parent element with a "list box" role and children with "option" roles.
 This can't be done using the code snippets above, which are only sufficient
 for overriding some simple accessibility properties. A more complex custom
 control like this requires subclassing.

 To make the CoverFlow accessible, we would need to first need to subclass the
 appropriate accessibility interface, such as IAccessible on Windows or
 AccessibilityDelegate on Android. In addition to setting properties such as the
 name and role (like ComboBox on Windows, or Choice on Android), we'd want to
 override any methods that are used to walk the tree, in order to give this
 object accessible children - one for each option the user can pick.

 ![Figure showing a single CoverFlow control in the UI object tree, and a tree of
 controls in the accessibility tree]( figures/cover_flow.png)

 Each option would be another instance of our subclass of an accessibility
 interface. Each one might need to implement a role, name, and very
 importantly, a bounding box representing the position of that option
 within the view's bounds.

 ## A single-process browser for basic HTML only

 Let's next discuss what's needed in order to make a single-process web
 browser accessible. At the time Chrome was first released, this is
 more or less how other browsers such as Safari and Firefox worked.
 However, let's start with the simplifying assumption that we're dealing
 with basic HTML only, and no CSS or ARIA.

 From the perspective of accessibility APIs, the web contents is one
 big custom control. Modern web browsers do not build their UI using
 platform UI elements.

 *** note
 Years ago, some browsers may have rendered web pages using
 platform UI elements, but this doesn't scale well as most UI frameworks
 get bogged down after thousands of UI elements, whereas many web pages
 have hundreds of thousands of elements.

 In addition, note that web form controls support CSS effects
 including unusual ones like 3-D transformations, whereas most
 platform UI controls don't support, e.g. a 45-degree rotated
 text box.
 ***

 The web contents can be represented as a tree of elements.
 For the moment let's ignore details between the DOM and the
 final rendered layout, and let's just assume that every DOM node
 corresponds perfectly to one node in the accessibility tree.

 Essentially the web browser needs to create one accessible
 object for every DOM node. The accessible object subclasses
 that platform's accessibility interface, and its accessible
 role, name, and other properties can all be computed by
 querying the corresponding DOM node. When DOM nodes are created
 or destroyed, the corresponding accessible objects need to
 be updated accordingly.

 Here's an example of a DOM tree and a corresponding accessibility
 tree. There's a 1:1 correspondence in this example.

 ![Figure showing a tree of DOM nodes for a web page and a corresponding
 tree of accessibility nodes](
 figures/single_process.png)

 Objects in the accessibility tree are sometimes called "wrappers" in this
 design. Each accessible object "wraps" its corresponding DOM element. The DOM
 element contains all of the state, and the accessible object just implements the
 accessibility API based on the DOM element. This design avoids the need for DOM
 code to need to be tightly bound to accessibility code.

 For efficiency, these accessible objects are sometimes built
 lazily. Initially there could just be one accessible object
 for the root of the web contents. If that object is ever
 queried and asked for its children, the accessible objects
 corresponding to the children of the root DOM element could
 be created on-demand, and so on. This is useful because
 many users don't have any features enabled that use
 accessibility APIs, so very little work would be done
 unless they're utilized.

 The overall system so far is pretty simple - the DOM tree is used to
 build the accessibility tree, which is used to communicate with the
 platform accessibility APIs.

 ![System diagram showing a browser consisting of a DOM tree that's used
 to build an accessibility tree, which is used to communicate with
 assistive technology using platform accessibility APIs.](
 figures/single_process_system.png)

 ## A single-process browser with CSS and ARIA

 In the previous example we assumed that there was a 1:1 correspondence
 between a DOM node and an accessible node. Unfortunately that doesn't
 hold true. Here are some examples of where that assumption doesn't work.

 * CSS can have generated content, like list item markers (bullets or numbers)
   or text inserted before or after an element with pseudoselectors
   like ::before and ::after.
 * Some DOM subtrees are never displayed (like &lt;head&gt;) or are hidden using
   display:none
 * Some DOM nodes are hidden using visibility:hidden, but it's possible for
   some of their descendant nodes to be visible
 * aria-hidden can hide a subtree from the accessibility tree even if it's
   displayed visually
 * ARIA role=none or role=presentation hides a node but not its subtree
 * aria-owns can be used to reparent a node within the accessibility tree

 So in practice, the accessibility tree is similar to the DOM tree, but with
 important differences.  Objects in the accessibility tree usually wrap DOM
 nodes, but occasionally they wrap pseudoelements (like list markers or CSS
 generated content) or other layout objects with no corresponding node (such as
 images that are children of pseudoelements).

 Here's an example where the DOM tree has list items and the corresponding
 accessibility tree has nodes for the list item markers:

 ![Figure showing a DOM tree with an ordered list of two items and the
 accessibility tree containing extra nodes for the generated list items](
 figures/generated_content.png)

 Here's a different example where some of the content is excluded from the
 accessibility tree using aria-hidden:

 ![Figure showing a DOM tree with a sidebar with aria-hidden=true on it
 and the corresponding accessibility tree where the sidebar isn't present
 at all](
 figures/hidden_content.png)

 This new system diagram reflects the complexity of the system when
 you consider both HTML, CSS, and ARIA all affecting the accessibility
 tree.

 ![System diagram showing a browser consisting of a DOM tree that along
 with CSS is used to build a Layout tree, then together the DOM and Layout
 trees build an accessibility tree, which is used to communicate with
 assistive technology using platform accessibility APIs.](
 figures/single_process_css_system.png)

 ## A multi-process browser

 In the next section we'll explore how the system diagram needs to change
 in order to support a multi-process browser.

 See [How Chrome Accessibility Works, Part 2](how_a11y_works_2.md)
	# How Chrome Accessibility Works

	This document explains the technical details behind Chrome accessibility
	code by starting at a high level and progressively adding more levels of
	detail.

	Please read the accessibility [overview](../overview.md) first.

	[TOC]

	## Accessibility for a simple (non-browser) application

	As described in the [overview](../overview.md), every platform has its own
	accessibility APIs that are used by both assistive technology and sometimes
	by automation software. To better understand the challenges of accessibility
	support in Chromium, let's first explore what it's like to build an
	accessible application using a standard UI toolkit.

	Examples of standard toolkits would be Win32 UI controls or .NET components
	on Windows, or Cocoa NSViews on macOS, or Android Views via Java or Kotlin.
	When you use such a toolkit to build an application, a lot of accessibility
	comes for free.

	Typically every UI element - including containers and grouping elements - gets
	its own corresponding accessibility element, implementing that platform's
	accessibility API. So if you add a button, checkbox, scroll container, or
	text field to your application, you don't need to do any work to make it
	accessible - the built-in UI toolkit has provided all of that already.

	![Simple accessibility tree](figures/ax_tree.png)

	### Changing accessibility properties

	There are some cases where a bit of extra information may be required on the
	part of the app author. If you add an image button, you may need to ensure
	you provide an accessible label ("alt text"). Other common simple modifications
	might include hiding UI elements from accessibility because they're primarily
	decorative, or marking a UI element as an alert to ensure it's announced
	to screen reader users when it appears. However, none of these require
	writing more than a few lines of code.

	#### Windows

	The [dynamic annotation API](https://docs.microsoft.com/en-us/windows/win32/winauto/dynamic-annotation-api)
	lets you change any accessibility property of a UI element represented by an HWND. Here's a tiny code
	snippet showing how you could override the accessible name property (PROPID_ACC_NAME) to be the
	text "Accessible Name":

	```C++
	CComPtr<IAccPropServices> pAccPropSrv;
	pAccPropSrv.CoCreateInstance(CLSID_AccPropServices);
	COleVariant varName("Accessible Name");
	pAccPropSrv->SetHwndProp(hwnd, OBJID_CLIENT, 0, PROPID_ACC_NAME, varName);
	```

	#### Android

	On Android, if you don't want to subclass a View you can just set its accessibility delegate
	as an easy way to override one or more accessibility attributes. In this case we override
	the content description to say "Accessible Name".

	```Java
	view.setAccessibilityDelegate(new AccessibilityDelegate() {
	public void onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfo info) {
	super.onInitializeAccessibilityNodeInfo(host, info);
	info.setContentDescription("Accessible Name");
	}
	});
	```

	### Custom controls and subclassing

	When building a custom control, more work is often required. As an example,
	suppose an application is implementing a "cover flow" image picker, such as the
	one found in the original iTunes album picker. Conceptually the user is picking
	one album from a large list, but visually it's shown as a horizontally scrolling
	list of album cover images that fly by in three dimensions.

	![Screenshot of a cover flow image picker](images/cover_flow.jpg)

	While this control may be implemented as a single UI element, from the
	perspective of accessibility APIs it would typically be represented as
	a parent element with a "list box" role and children with "option" roles.
	This can't be done using the code snippets above, which are only sufficient
	for overriding some simple accessibility properties. A more complex custom
	control like this requires subclassing.

	To make the CoverFlow accessible, we would need to first need to subclass the
	appropriate accessibility interface, such as IAccessible on Windows or
	AccessibilityDelegate on Android. In addition to setting properties such as the
	name and role (like ComboBox on Windows, or Choice on Android), we'd want to
	override any methods that are used to walk the tree, in order to give this
	object accessible children - one for each option the user can pick.

	![Figure showing a single CoverFlow control in the UI object tree, and a tree of
	controls in the accessibility tree]( figures/cover_flow.png)

	Each option would be another instance of our subclass of an accessibility
	interface. Each one might need to implement a role, name, and very
	importantly, a bounding box representing the position of that option
	within the view's bounds.

	## A single-process browser for basic HTML only

	Let's next discuss what's needed in order to make a single-process web
	browser accessible. At the time Chrome was first released, this is
	more or less how other browsers such as Safari and Firefox worked.
	However, let's start with the simplifying assumption that we're dealing
	with basic HTML only, and no CSS or ARIA.

	From the perspective of accessibility APIs, the web contents is one
	big custom control. Modern web browsers do not build their UI using
	platform UI elements.

	*** note
	Years ago, some browsers may have rendered web pages using
	platform UI elements, but this doesn't scale well as most UI frameworks
	get bogged down after thousands of UI elements, whereas many web pages
	have hundreds of thousands of elements.

	In addition, note that web form controls support CSS effects
	including unusual ones like 3-D transformations, whereas most
	platform UI controls don't support, e.g. a 45-degree rotated
	text box.
	***

	The web contents can be represented as a tree of elements.
	For the moment let's ignore details between the DOM and the
	final rendered layout, and let's just assume that every DOM node
	corresponds perfectly to one node in the accessibility tree.

	Essentially the web browser needs to create one accessible
	object for every DOM node. The accessible object subclasses
	that platform's accessibility interface, and its accessible
	role, name, and other properties can all be computed by
	querying the corresponding DOM node. When DOM nodes are created
	or destroyed, the corresponding accessible objects need to
	be updated accordingly.

	Here's an example of a DOM tree and a corresponding accessibility
	tree. There's a 1:1 correspondence in this example.

	![Figure showing a tree of DOM nodes for a web page and a corresponding
	tree of accessibility nodes](
	figures/single_process.png)

	Objects in the accessibility tree are sometimes called "wrappers" in this
	design. Each accessible object "wraps" its corresponding DOM element. The DOM
	element contains all of the state, and the accessible object just implements the
	accessibility API based on the DOM element. This design avoids the need for DOM
	code to need to be tightly bound to accessibility code.

	For efficiency, these accessible objects are sometimes built
	lazily. Initially there could just be one accessible object
	for the root of the web contents. If that object is ever
	queried and asked for its children, the accessible objects
	corresponding to the children of the root DOM element could
	be created on-demand, and so on. This is useful because
	many users don't have any features enabled that use
	accessibility APIs, so very little work would be done
	unless they're utilized.

	The overall system so far is pretty simple - the DOM tree is used to
	build the accessibility tree, which is used to communicate with the
	platform accessibility APIs.

	![System diagram showing a browser consisting of a DOM tree that's used
	to build an accessibility tree, which is used to communicate with
	assistive technology using platform accessibility APIs.](
	figures/single_process_system.png)

	## A single-process browser with CSS and ARIA

	In the previous example we assumed that there was a 1:1 correspondence
	between a DOM node and an accessible node. Unfortunately that doesn't
	hold true. Here are some examples of where that assumption doesn't work.

	* CSS can have generated content, like list item markers (bullets or numbers)
	or text inserted before or after an element with pseudoselectors
	like ::before and ::after.
	* Some DOM subtrees are never displayed (like <head>) or are hidden using
	display:none
	* Some DOM nodes are hidden using visibility:hidden, but it's possible for
	some of their descendant nodes to be visible
	* aria-hidden can hide a subtree from the accessibility tree even if it's
	displayed visually
	* ARIA role=none or role=presentation hides a node but not its subtree
	* aria-owns can be used to reparent a node within the accessibility tree

	So in practice, the accessibility tree is similar to the DOM tree, but with
	important differences. Objects in the accessibility tree usually wrap DOM
	nodes, but occasionally they wrap pseudoelements (like list markers or CSS
	generated content) or other layout objects with no corresponding node (such as
	images that are children of pseudoelements).

	Here's an example where the DOM tree has list items and the corresponding
	accessibility tree has nodes for the list item markers:

	![Figure showing a DOM tree with an ordered list of two items and the
	accessibility tree containing extra nodes for the generated list items](
	figures/generated_content.png)

	Here's a different example where some of the content is excluded from the
	accessibility tree using aria-hidden:

	![Figure showing a DOM tree with a sidebar with aria-hidden=true on it
	and the corresponding accessibility tree where the sidebar isn't present
	at all](
	figures/hidden_content.png)

	This new system diagram reflects the complexity of the system when
	you consider both HTML, CSS, and ARIA all affecting the accessibility
	tree.

	![System diagram showing a browser consisting of a DOM tree that along
	with CSS is used to build a Layout tree, then together the DOM and Layout
	trees build an accessibility tree, which is used to communicate with
	assistive technology using platform accessibility APIs.](
	figures/single_process_css_system.png)

	## A multi-process browser

	In the next section we'll explore how the system diagram needs to change
	in order to support a multi-process browser.

	See [How Chrome Accessibility Works, Part 2](how_a11y_works_2.md)