The Label Was Wrong So I Switched to Counting. The Count Was Wrong Too

There are many ways to decide "is this button that button" in UI automation. I picked two of them in a row, and both were wrong.

Is the axis your bot uses to identify a screen one that the thing you are identifying can never vary along? Or is it just an axis that happens to be true on the screen in front of you?

What the bot does

A bot drives a phone directly over adb. To switch accounts it first has to get back to the home screen, but a session always ends up on some other screen. So there is a function called _unwind — close a modal if one is up, otherwise go back, repeat up to 8 times until the home tab is visible.

for _ in range(8):
    if find(dump(), is_profile_tab, H):
        return True
    # if modal: tap the close button, else back
    ...

The problem is how you know "if modal."

Attempt 1: the label

I started with labels. A close button will be one of Not now, Close, Skip, Dismiss.

DISMISS = ("Not now", "Close", "Skip", "Dismiss", "Cancel", "Maybe later")

That collapsed fast. Every row in a list also carries content-desc="Dismiss". In a suggested-users list, the "hide this suggestion" button uses that exact label.

So the bot pressed a row button every time. One suggestion disappears, the screen stays put, it burns all 8 tries and fails. An app reset follows, and if that also lands wrong the account switch itself fails. 30 of 76 failures came through this path.

Attempt 2: the count

With labels untrustworthy I looked for another axis. My reasoning went:

A modal's close button is the only one on screen. If there are several, it is not a modal, it is list row buttons.

I had measurements behind it. Follower list: 4. Filter list: 2. Suggestions screen: 8. All two or more.

cands = find_all(root, lambda t, d: (t in DISMISS or d in DISMISS))
if len(cands) == 1:
    tap(*cands[0])      # judged to be a modal
else:
    back()

I even wrote it into the comment: "count is an axis that does not move with renames or localization, so use it instead of a label or resource-id list."

Half right. Count does not move with renames. It moves with scroll position.

Pause — what would you say?

Can a list screen ever show exactly one Dismiss button?

I assumed not. A list has many rows. Three more minutes of thought would have produced the answer.

Attempt 3: I actually opened the failing screen

A few days later another account switch failed. This time the screen dump at the moment of failure had been saved — during the previous fix it bothered me that nobody could say which screen the bot got stuck on, so I had added the capture. That is the only reason this post exists.

Here is what the dump held.

action_bar_title    'Discover people'
row_header_textview 'Suggested for you'
... (6 suggestion rows) ...
row_recommended_hide_icon_button   [1001,2102][1035,2136]  desc='Dismiss'

A suggestions list — with exactly one Dismiss button. Most rows had already flipped to Requested/Following, which removes the hide button, and only one fresh row under "More suggestions" at the very bottom still had it.

Count of one. My code judged it a modal and pressed it. All 8 times. Then again 8 more after the app reset.

The real invariant

The third axis was not allowed to fail. So instead of guessing, I extracted the ancestor chain of every Dismiss candidate across all 15 saved failure screens.

follow_list_container < LinearLayout < ListView#list < ...
LinearLayout < recommended_user_row_content_identifier < RecyclerView#recycler_view < ...

15 out of 15. Every mis-tapped button was a descendant of a ListView or a RecyclerView, without exception.

And that is structure, not coincidence. A row button has to live inside a scrolling container for the list to scroll, and a modal's close button lives inside a dialog, not inside a list. It does not get translated like a label, and it does not shift with scroll like a count.

LIST_CONTAINERS = ("ListView", "RecyclerView")
 
def _modal_dismissers(root):
    parent = {c: p for p in root.iter() for c in p}
    out = []
    for n in root.iter("node"):
        if n.get("clickable") != "true":
            continue
        t, d = label(n)
        if not (t in DISMISS or d in DISMISS):
            continue
        cur, in_list = n, False
        while cur in parent:
            cur = parent[cur]
            if (cur.get("class") or "").rsplit(".", 1)[-1] in LIST_CONTAINERS:
                in_list = True
                break
        if not in_list:
            out.append(center(n.get("bounds")))
    return out

Verification came from the 15 saved screens

I replayed the new logic over all 15.

still taps a row button, out of 15 real screens: 0

Mis-taps 15/15 → 0/15. The opposite regression matters too: if excluding in-list rows also stops real modals from being pressed, that is an over-filter. I synthesized a screen holding both a Not now outside a list and a Dismiss inside one, and pinned down with a test that only the modal gets tapped.

One more thing I learned. Pressing a row button is not merely wasted effort — it is a side effect. It does not just fail to close a modal, it actually deletes a suggestion. Eight iterations means eight of them.

Three checks on your own code

If you have code that drives a UI programmatically:

What axis does your anchor use? Text dies to localization and renames, coordinates die to resolution, counts die to scroll. Structure — ancestry, membership — survives longest.
Are you saving the screen at the moment of failure? Without it, "why did it fail" stays a guess forever. This bug ended on the third attempt because 15 dumps existed; without them there would have been a fourth and a fifth.
Does a wrong tap have side effects? A mis-tap that does nothing and a mis-tap that deletes data are different classes of bug.

The honest part

Twice I wrote "this is the invariant axis" into a comment with confidence, and twice I was wrong. Label, then count.

The only difference this time is that I checked against 15 real captures. Whether ListView membership is truly invariant I do not know either — a bottom sheet containing a list would break it. But in that case the code falls through to back(), and back() closes most dialogs. I spent as much attention on falling to the safe side when wrong as on picking the axis.

Which verdict this one failure counts toward, and which it is excluded from, is written up in I Delayed the Verdict by Two Days and My Bar Got Harder on Its Own.

Open the one anchor line in your own bot. Can you say, in a single sentence, what that axis can shift with?