Md Montaser Hamid
← All publications

"Over-the-Hood" AI Inclusivity Bugs and How 3 AI Product Teams Found and Fixed Them

Anderson, A., Moussaoui, F. A., Noa Guevara, J., Hamid, M. M., & Burnett, M.

Proceedings of the 31st International Conference on Intelligent User Interfaces (ACM IUI) · 2026

Most AI-bias work looks "under the hood" at algorithms and data. This paper looks "over the hood" — at barriers in user-facing AI products that disproportionately exclude users with certain problem-solving approaches. In a field study, 3 AI product teams found 83 instances of 6 AI inclusivity bug types unique to user-facing AI products and fixed 47 of them (57%), using GenderMag-for-AI — our new variant of the GenderMag inclusive design method.

My role: I co-developed GenderMag-for-AI and worked with the AI product teams in the field study reported in the paper.

The problem

Rather than focusing on algorithmic bias, this work investigates inclusivity barriers that live in the user experience of AI products — the parts users actually see and interact with. These "over-the-hood" barriers disproportionately exclude users whose problem-solving styles (e.g., how they process information, their attitude toward risk, their computer self-efficacy) differ from those the product implicitly assumes. Recent research has begun to report such biases — but what do they look like, how prevalent are they, and how can developers find and fix them?

What we did

We conducted a field study with three AI product teams working on real products: Team Game (an explainable-AI game interface with score-based explanations), Team Weather (AI cold-hardiness predictions for agricultural decision support), and Team Farm (an AI-powered irrigation scheduling prototype using soil sensor telemetry). The teams iterated through "find" sessions — GenderMag-driven cognitive walkthroughs with a customized persona — and "fix" sessions on their own products.

To identify the bug types, we qualitatively analyzed the teams' evaluation forms using affinity diagramming until saturation; two authors independently coded 20% of the data with 84% agreement. The teams' work also drove the creation of GenderMag-for-AI, a variant of GenderMag whose "Pre-Action Fork" explicitly walks through two circumstances: when the persona believes the AI and when the persona doubts it.

Key results

  • The teams found 83 AI inclusivity bug instances across 6 types: Interpret AI? ("what does this even mean?", 27 instances), AI: why should I? (19), AI: actionable? ("so what should I DO?", 12), AI input↔output? (9), AI: more info! (9), and AI changes? ("what's changed?", 7).
  • The teams devised fixes for 47 of the 83 bug instances (57%), fixing each bug type at similar rates.
  • Risk-aversion was the teams' most powerful bug-finding lens: for three of the six bug types, risk-averse users were the top-cited reason; comprehensive information processing and lower computer self-efficacy drove most of the rest.
  • GenderMag-for-AI's Pre-Action Fork proved especially effective at detecting AI inclusivity bugs that arise when users doubt the AI.

From the paper

One of the teams' AI products (from the paper's Fig. 1): an AI-powered agricultural tool showing modeled lethal temperatures for a grape variety across a season.
One of the teams' AI products (from the paper's Fig. 1): an AI-powered agricultural tool showing modeled lethal temperatures for a grape variety across a season.
An Interpret AI? bug instance from Team Game (from the paper's Fig. 9): in the Scores Best-to-Worst explanation, it was hard to tell which data series came from which decision point.
An Interpret AI? bug instance from Team Game (from the paper's Fig. 9): in the Scores Best-to-Worst explanation, it was hard to tell which data series came from which decision point.
Team Game's fix (from the paper's Fig. 10): an interactive legend added to the Scores Best-to-Worst explanation to improve its interpretability.
Team Game's fix (from the paper's Fig. 10): an interactive legend added to the Scores Best-to-Worst explanation to improve its interpretability.
The paper's Table 3: AI inclusivity bug instances found and fixed, by type. The teams fixed 47 of 83 instances (57%), at similar rates across bug types.
The paper's Table 3: AI inclusivity bug instances found and fixed, by type. The teams fixed 47 of 83 instances (57%), at similar rates across bug types.
Cite: Anderson, A., Moussaoui, F. A., Noa Guevara, J., Hamid, M. M., & Burnett, M. (2026). "Over-the-Hood" AI Inclusivity Bugs and How 3 AI Product Teams Found and Fixed Them. Proceedings of the 31st International Conference on Intelligent User Interfaces (ACM IUI). doi:10.1145/3742413.3789068