Talkback Ordering in Android Jetpack Compose

We Android developers can generally rely on the default TalkBack experience. But sometimes, we need to get our hands dirty and manually override the order in which our UI components are traversed. Android compose semantics properties are the place for developers to change content descriptions and merge elements into announcement groups. With the release of androidx.compose.ui version 1.5.0-beta01, we can now control TalkBack traversal ordering here as well!

Google has been gracious enough to give us two semantic parameters to control announcement ordering:

• SemanticsPropertyReceiver.isTraversalGroup: Boolean: whether a composable node denotes a hierarchy of composables that should be traversed prior to moving on.
• SemanticsPropertyReceiver.traversalIndex: Float: a value to manually control screenreader traversal order. The order is from low to high.

In this article you will learn the basics of how to use these.

Setup

Include androidx.compose.ui:ui:1.5.0-beta01 (or higher) in your module’s build.gradle file.

Traversal Group Usage

If a composable’s semantic node is a traversal group, then its function is to serve as a border in organizing its children. When it is traversed, its entire composable hierarchy should be visited prior to moving on.

Note: LazyLists are traversal groups by default.

The default TalkBack ordering is from left-to-right, top-to-bottom. Here is an example composable whose default TalkBack ordering needs overriding:

Row(...) {
  Column(
    horizontalAlignment = Alignment.CenterHorizontally,
    verticalArrangement = Arrangement.spacedBy(32.dp),
  ) {
    Text(text = "First")
    Text(text = "Second")
  }
  Text(text = "Third")
}

We want the ordering to be: “First”, “Second”, “Third”. However, we observe: “First”, “Third”, “Second”. To solve this we can declare the Column as a traversal group:

Row(...) {
  Column(
    horizontalAlignment = Alignment.CenterHorizontally,
    verticalArrangement = Arrangement.spacedBy(32.dp),
    modifier = Modifier.semantics { isTraversalGroup = true },
  ) {
    Text(text = "First")
    Text(text = "Second")
  }
  Text(text = "Third")
}

Voilà. We can see that now the Column functions as a parent that groups child composables for TalkBack traversal prior to moving on.

Traversal Index Usage

In XML we used android:accessibilityTraversalAfter=”@id/...” and
android:accessibilityTraversalBefore=”@id/...” for explicit TalkBack ordering. In Compose we can now use the traversalIndex. Ordering is from low to high values and the default value is 0f.

Here is an example composable whose Texts we want to be announced in an unorthodox order:

Column(
    ...
    modifier = Modifier.semantics { isTraversalGroup = true },
) {
    Text(text = "First")
    Text(text = "Third")
    Text(text = "Second")
}

The Column is marked as a traversal group in order to guarantee that all of its children are announced prior to moving on. We want to see: “First”, “Second, “Third”, but obviously that will not occur out-of-the-box.

To override this we can do something like:

Column(
    ...
    modifier = Modifier.semantics { isTraversalGroup = true },
) {
    Text(
      text = "First",
      modifier = Modifier.semantics { traversalIndex = 0f },
    )
    Text(
      text = "Third",
      modifier = Modifier.semantics { traversalIndex = 2f },
    )
    Text(
      text = "Second",
      modifier = Modifier.semantics { traversalIndex = 1f },
    )
}

Or even:

Column(
    ...
    modifier = Modifier.semantics { isTraversalGroup = true },
) {
    Text(text = "First")
    Text(
      text = "Third",
      modifier = Modifier.semantics { traversalIndex = 1f },
    )
    Text(text = "Second")
}

Both of these behave the same. The first one’s behavior is explicit and obvious. The second one takes advantage of the default top-down TalkBack ordering and the fact that all semantic nodes default to a traversalIndex of 0f.

This GIF shows the behavior of either of these two solutions:

Merged Descendants Behavior

When a composable has Modifier.semantics(mergeDescendants = true) {...} and is traversed, its content description and those of its children will be announced without further user interaction. In androidx.compose.ui:ui:1.5.0-beta01 there is a bug that causes child traversal specifics to be reverted to the default behavior when a parent composable merges its descendants. In particular, Modifier.clickable() contains Modifier.semantics(mergeDescendants = true){...} and will clear your traversal ordering.

Here is an example:

Column(
  modifier = Modifier.clickable(...),
) {
    Text(
      text = "First",
      modifier = Modifier.semantics { traversalIndex = 0f },
    )
    Text(
      text = "Third",
      modifier = Modifier.semantics { traversalIndex = 2f },
    )
    Text(
      text = "Second",
      modifier = Modifier.semantics { traversalIndex = 1f },
    )
}

If the default TalkBack ordering isn’t working for you in a case like this, you can use Modifier.zIndex(...) to correct the issue.

Column(
  modifier = Modifier.clickable(...),
) {
    Text(
        text = "First",
        modifier = Modifier.zIndex(0f),
    )
    Text(
        text = "Third",
        modifier = Modifier.zIndex(2f),
    )
    Text(
        text = "Second",
        modifier = Modifier.zIndex(1f),
    )
}

However, this work around breaks if you want ordering control between composable containers. For example:

Column(
  modifier = Modifier.clickable(...),
) {
    Column {
        Text(
            text = "First",
           modifier = Modifier.zIndex(0f),
        )
        Text(
            text = "Third",
            modifier = Modifier.zIndex(2f),
        )
    }
    Text(
        text = "Second",
        modifier = Modifier.zIndex(1f),
    )
}

In a case like this, your best option is to explicitly set the full content description using clearAndSetSemantics { contentDescription = "..." } on the parent:

val first = "First"
val second = "Second"
val third = "Third"
Column(
    modifier = Modifier
        .clickable(...)
        .clearAndSetSemantics {
            contentDescription = "$first. $second. $third"
        },
) {
    Column {
        Text(text = "First")
        Text(text = "Third")
    }
    Text(text = "Second")
}

You could also combine child content descriptions like so:

val first = "First"
val second = "Second"
val third = "Third"
Column(
  modifier = Modifier.clickable(...),
) {
    Column {
        Text(text = first)
        Text(
            text = third,
            modifier = Modifier.semantics { invisibleToUser() },
        )
    }

    Text(
        text = second,
        modifier = Modifier.semantics { 
            contentDescription = "$second. $third"
        }
    )
}

Moving Forward

The traversalIndex and isTraversalGroup APIs make TalkBack ordering easy. Take advantage of these tools to give your users the best experience possible!

Lucas traverses indices at Livefront.