Fairness Issues in AI Systems that Augment Sensory Abilities

Leah Findlater; Margot Hanley; Shiri Azenkot; Steven Goodman; Yuhang Zhao

REVIEW 1 major objections 3 minor 24 references

Fairness Issues in AI Systems that Augment Sensory Abilities

T0 review · 1 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For AI that augments the senses, fairness means accessible data, visible choices, and real privacy, because the system delivers information others already sense directly.

desk verdict A clear, well-scoped position paper that frames three fairness challenge areas for AI sensory augmentation; the uniqueness claim is asserted more than shown, but the agenda is useful and deserves a serious referee. read the letter →

arxiv 1908.07333 v1 pith:CJUMR7BX submitted 2019-08-16 cs.CY cs.HC

Leah Findlater , Steven Goodman , Yuhang Zhao , Shiri Azenkot , Margot Hanley This is my paper · ORCID

classification cs.CYcs.HC

keywords AIfairnessaccessibilityassistivetechnologysensoryaugmentationmodelexplainabilitydataprivacyalways-onsensing

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper stakes out a distinct fairness problem for AI systems that augment sensory abilities—object recognition and scene description for blind users, sound awareness for d/Deaf users—because these systems deliver information that non-disabled people already obtain through their own senses. That goal of equal access, the authors argue, makes fairness issues unavoidable: the data and models can be inaccessible to the people they serve, the system must make ethically loaded choices about which sensory details to convey and how, and always-on sensing creates privacy risks for the user and for everyone around them. The paper matters because these tools are meant to close a disabling gap, yet the same AI decisions can bias, mislead, or expose the people they are designed to help. It is a research agenda rather than a solution, mapping where accessibility and AI research would need to join forces.

What carries the argument

The central object is the equal-access gap: the difference between sensory information a non-disabled person gathers directly and the AI-mediated rendering of that same information offered to a user with a sensory disability. The paper uses this gap as the mechanism that generates its three fairness issue families. Because the user cannot inspect the raw input, the model and its explanations become inaccessible; because a system must decide what to convey from an overwhelming sensory stream, every implementation encodes value choices; and because capturing that stream requires continuous sensing, privacy is structurally at risk. The equal-access gap is what makes fairness in these systems look different from fairness in credit scoring or hiring.

What would settle it

A systematic comparison of AI assistive tools for non-sensory disabilities—for example, predictive communication or text simplification for cognitive disabilities—that documents the same data-inaccessibility, content-curation, and privacy issues would falsify the paper's uniqueness claim; so would a study showing blind users can reliably verify model outputs through current accessible channels.

Watch

Extended reading notes

Core claim

The paper's central claim is that the fairness challenges of AI-based sensory augmentation are unique, not just special cases of generic AI fairness. In an assistive sensing setting, the user cannot directly see or hear the input the model consumes, so standard transparency mechanisms like visual saliency maps fail; the AI's choice of what to report—an important sound, a stranger's face, a label for age or gender—embeds human decisions that are amplified and hidden; and the always-on microphones and cameras required for the system to work expose both the primary user's intimate data and the privacy of bystanders. The authors do not resolve these problems; they argue that the equal-access goal is precisely what triggers them, and they call for studying the full decision-making pipeline, accessible model explanation and personalization, and policy that balances privacy with assistive use.

Load-bearing premise

The load-bearing premise is that the three challenge families are unique to sensory-augmenting AI, a claim the paper asserts through examples rather than demonstrating through systematic comparison with other AI fairness contexts.

Editorial extensions

If this is right

Fairness evaluation for assistive sensing systems must include whether users can inspect, personalize, and verify model output, not just whether predictions are accurate across groups.
Designers cannot avoid the content-curation question: choosing a default set of sounds or faces to report still makes a fairness-relevant decision.
Privacy protections for these systems must treat bystanders and conversation partners as affected parties, with consent and notice designed for them.
The right to explanation needs to be adapted to accessible modalities—sound, touch, simplified language—rather than satisfied by visual explanations alone.
Assistive use may need legal exceptions parallel to service-animal rules, allowing facial recognition or audio capture where general bans apply.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the equal-access framing generalizes, AI aids for cognitive or motor disabilities may exhibit structurally similar fairness issues, which would blur the boundary the paper draws between sensory and other assistive AI.
The decision-making pipeline resembles content curation in mainstream recommender and moderation systems, but with a higher safety floor: a wrong sensory description does not just mis-recommend, it misinforms the user's model of the world.
A testable extension would give users confidence or uncertainty information in their own modality and measure whether trust calibration improves relative to current text-based or visual confidence displays.
The privacy analysis points toward a research program on an assistive-purpose standard for always-on sensing, analogous to service-animal exceptions, and its effect on bystander acceptance.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You can skip the deep skepticism here. This is a workshop position paper that does what position papers should: it names a problem space, organizes it, and points at research directions. The real contribution is the framing itself—three families of fairness challenges for AI assistive sensing: data and model accessibility, decision-making about what to convey, and privacy for the user and bystanders. That framing is clear and well-referenced, and it comes from authors with direct experience in the area, which shows in the specificity of the examples.

The intersection of AI fairness and accessibility, specifically for sensory augmentation, is under-discussed. The paper synthesizes known problems (explainability, personalized models, bystander privacy) into this context and draws attention to how "equal access" creates distinct tensions—the face-recognition-for-blind-users case, for instance, where an equal-access argument conflicts with bias and consent. That is a genuinely useful lens.

The soft spot is the "unique" claim. The paper asserts rather than demonstrates that these issues are unique to sensory-augmenting AI. Inaccessible data and models is a version of explainability for lay users; deciding what to convey resembles content curation; bystander privacy with wearables is well-trodden. The paper never defines what would make a fairness issue unique or systematically compares with other AI fairness contexts. This weakens the framing as a distinct research area, but it does not undermine the individual challenges or the proposed directions. For a workshop paper, this is acceptable.

There is no empirical result, no formal derivation, no dataset. It is an argument and an agenda. If you read it as a scientific result, you will be disappointed; read it as a research briefing and it is solid.

For a reader working on AI fairness or accessibility, it is worth a look. For peer review, I would send it out: it is coherent, relevant, and the framing gap is fixable with some comparative analysis. Accept it as a workshop paper and encourage the authors to extend it with a more rigorous uniqueness analysis for a journal version.

Referee Report

1 major / 3 minor

Summary. This position paper argues that AI systems that augment sensory abilities—such as object recognition for blind users and sound awareness for d/Deaf users—raise fairness challenges distinct from those of other AI applications. The authors' central claim is that because these systems provide information already available to non-disabled people, the goal of equal access creates unique fairness issues. They identify three families of such issues: (1) accessibility of data and models, including explainability, personalization, and the user's ability to verify recognition results; (2) ethical decisions about what sensory information to convey, illustrated by facial recognition and sound-filtering systems; and (3) privacy for both the primary user and third parties, covering always-on sensing, bystander attitudes, and legal/policy implications. The paper concludes with directions for collaboration between accessibility and AI/ML researchers, such as datasheets for datasets, accessible model explanations, and 'assistive use' exceptions in policy.

Significance. If the framing holds, the paper charts a valuable research agenda at the intersection of AI fairness and accessibility. Its strengths are concrete, grounded examples from existing systems (VizWiz, Seeing AI, sound awareness tools) and references to empirical studies that lend weight to each challenge. The paper is a workshop-style position contribution, not a formal technical result, so its primary value is agenda-setting. The main weakness is that the 'uniqueness' claim is asserted rather than demonstrated; however, the specific challenges and research directions remain useful even if they are reframed as especially salient instances of broader AI fairness problems. This is a constructive and timely contribution for the ASSETS/AI-fairness community.

major comments (1)

[Introduction and Conclusion] The central claim that equal access 'raises unique fairness issues for AI-enabled assistive technology' is not supported by a definition of 'unique' or by a systematic comparison with fairness challenges in other AI contexts. For example, 'inaccessible data and models' is closely related to explainability for lay users, 'deciding what information to convey' resembles content curation and editorial discretion, and 'privacy for others' is an established issue for wearable cameras. Without a criterion that distinguishes these challenges from their general counterparts, the framing as a distinct research area is asserted rather than demonstrated. I suggest adding a sentence that defines uniqueness (e.g., challenges arising specifically because the user lacks independent access to the sensed modality), or softening the claim to 'particularly salient' or 'understudied' issues. The examples and research directions do not depend on strict uniqueness, so this is a local revision rather than a fundamental flaw.

minor comments (3)

[Data and Model (In)accessibility] The sentence 'the data used by AI-enabled assistive technologies is inherently not accessible to its primary users' is a bit loose: it is the sensory data (images, audio) that is inaccessible, not necessarily the dataset or model in the usual machine-learning sense. Clarifying this would avoid confusion for readers unfamiliar with assistive technology.
[Decision-Making in AI-Based Sensing] The discussion of the EU 'right to explanation' refers to it as coming into force in 2018, but the scope and existence of such a right under the GDPR is debated in the legal literature. A brief acknowledgment of this debate (e.g., citing Wachter et al. 2017) would make the paper more precise, though this does not affect the main argument.
[Individual and Societal Privacy Issues] The example 'a Deaf person hears a knock on the door' uses 'hears' loosely; 'perceives' or 'is aware of' would be more accurate and respectful of the diversity of d/Deaf experiences. This is a wording choice, not a substantive issue.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a self-contained position paper whose claims rest on external studies, policy references, and illustrative examples rather than on fitted inputs or self-citation chains.

full rationale

This is a workshop position paper that makes no quantitative predictions, fits no parameters, and derives no formal results. Its central claim is that AI-enabled sensory augmentation raises distinctive fairness challenges because such systems provide information that is already available to non-disabled people. That claim is an argument from the goal of equal access, not a mathematical or empirical derivation, so there is no input-output relation that could collapse into itself. The three challenge families discussed—data and model accessibility, decision-making about what sensory information to convey, and individual/societal privacy—are each supported by external empirical work and policy documents: Kacorri et al. on personalized object recognition, Ahmed et al. on bystander privacy, Profita et al. on the social acceptability of head-mounted displays, Gurari et al. on the VizWiz-Priv dataset, and the EU right-to-explanation discourse, among others. The authors do cite their own prior work in two places (a sound awareness user-needs study [8] and a study on the acceptability of head-mounted display use [17]), but these citations are illustrative examples of existing assistive sensing contexts, not load-bearing premises that justify the paper's central claim. The weakest point is the assertion that the fairness issues are 'unique'; the paper does not systematically compare them against fairness challenges in other AI contexts. That is a framing or scope weakness, not circularity: the paper's proposed research directions remain substantive even if the challenges are not strictly unique. No equations are present, no parameter is fitted and then renamed as a prediction, and no uniqueness theorem from the authors' prior work is invoked to forbid alternatives. Accordingly, the derivation chain is self-contained and the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper makes no quantitative claims, so there are no free parameters. Its argument rests on domain assumptions about the value of equal access and the accessibility of data to users, which are stated in the introduction and data accessibility sections. No invented entities are introduced.

assumptions (3)

domain assumption AI-enabled assistive technologies for sensory augmentation are increasingly being deployed and will continue to use AI/ML methods.
The introduction states this as the motivating premise, e.g., 'An increasing number of AI-enabled assistive technologies leverage advances...'. The relevance of the paper depends on this trend.
domain assumption Equal access to sensory information for disabled users is a desirable goal that should guide design.
The paper treats 'equal access' as a value premise, saying 'This goal of equal access raises unique fairness issues' without justifying this goal itself.
domain assumption Users with sensory disabilities cannot independently assess the accuracy of AI outputs because the data is inherently inaccessible to them.
This is argued in the 'Data and Model (In)Accessibility' section with examples like visual explanations being inaccessible to blind users. If this premise were false, part of the fairness problem would dissolve.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Fairness Issues in AI Systems that Augment Sensory Abilities." pith.science (2026). https://pith.science/paper/CJUMR7BX

@misc{pith2026190807333,
  author       = {Pith},
  title        = {Pith review of: Fairness Issues in AI Systems that Augment Sensory Abilities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CJUMR7BX}},
  note         = {Machine review of arXiv:1908.07333}
}

read the original abstract

Systems that augment sensory abilities are increasingly employing AI and machine learning (ML) approaches, with applications ranging from object recognition and scene description tools for blind users to sound awareness tools for d/Deaf users. However, unlike many other AI-enabled technologies, these systems provide information that is already available to non-disabled people. In this paper, we discuss unique AI fairness challenges that arise in this context, including accessibility issues with data and models, ethical implications in deciding what sensory information to convey to the user, and privacy concerns both for the primary user and for others.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 23 canonical work pages

[1]

Ahmed, T., Kapadia, A., Potluri, V., & Swaminathan, M. 2018. Up to a limit?: Privacy concerns of bystanders and their willingness to share additional information with visually impaired users of assistive technologies. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(3), 89

work page 2018
[2]

Aira. 2018. Retrieved July 2, 2019 from https://aira.io/

work page 2018
[3]

P., Jayant, C., Ji, H., et al

Bigham, J. P., Jayant, C., Ji, H., et al. 2010. VizWiz: nearly real-time answers to visual questions. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, 333-342. ACM

work page 2010
[4]

Buolamwini, J., & Gebru, T. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of the Conference on Fairness, Accountability and Transparency, 77-91

work page 2018
[5]

and Kovaleski, S.F

Conger, K, Fausset, R. and Kovaleski, S.F. 2019, May

work page 2019
[6]

T., Peterson, S

Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. 2003. The role of trust in automation reliance. International Journal of Human- Computer Studies, 58(6), 697-718

work page 2003
[7]

Fiannaca, A., Apostolopoulous, I., & Folmer, E. 2014. Headlock: a wearable navigation aid that helps blind cane users traverse large open spaces. Proceedings of the 16th international ACM SIGACCESS Conference on Computers & Accessibility, 19-26. ACM

work page 2014
[8]

Findlater, L., Chinh, B., Jain, D., Froehlich, J., Kushalnagar, R., & Lin, A. C. 2019. Deaf and hard-of- hearing individuals’ preferences for wearable and mobile sound awareness technologies. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), paper 46. ACM

work page 2019

Show all 24 references

[9]

W., Wallach, H., Daumé III, H., & Crawford, K

Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. 2018. Datasheets for datasets. arXiv preprint arXiv:1803.09010

2018 arXiv
[10]

right to explanation

Goodman, B., & Flaxman, S. 2017. European Union regulations on algorithmic decision-making and a “right to explanation”. AI Magazine, 38(3), 50-57

2017
[11]

Gurari, D., Li, Q., Lin, C., Zhao, Y., Guo, A., Stangl, A., & Bigham, J. P. 2019. VizWiz-Priv: a dataset for recognizing the presence and purpose of private visual information in images taken by blind people. Proceedings of the IEEE Conference on Computer Vision and Pattern Re...

2019
[12]

Kacorri, H. 2017. Teachable machines for accessibility. ACM SIGACCESS Accessibility and Computing, (119), 10-18

2017
[13]

M., Bigham, J

Kacorri, H., Kitani, K. M., Bigham, J. P., & Asakawa, C. 2017. People with visual impairment training personal object recognizers: Feasibility and challenges. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 5839-5849. ACM

2017
[14]

New York Times

San Francisco bans facial recognition technology. New York Times. Retrieved July 1, 2019 from https://www.nytimes.com/2019/05/14/us/facial- recognition-ban-san-francisco.html

2019
[15]

L., Morris, M

MacLeod, H., Bennett, C. L., Morris, M. R., & Cutrell, E. 2017. Understanding blind people's experiences with computer-generated captions of social media images. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 5988-5999. ACM

2017
[16]

Microsoft. (2019). Seeing AI. Retrieved July 2, 2019 from https://www.microsoft.com/en-us/ai/seeing-ai

2019
[17]

H., Hsi, M

Peng, Y. H., Hsi, M. W., Taele, et al. 2018. SpeechBubbles: Enhancing Captioning Experiences for Deaf and Hard-of-Hearing People in Group Conversations. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, paper 293. ACM

2018
[18]

Profita, H., Albaghli, R., Findlater, L., Jaeger, P., & Kane, S. K. (2016, May). The AT effect: how disability affects the perceived social acceptability of head- mounted display use. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 4884-4895. ACM

2016
[19]

T., Singh, S., & Guestrin, C

Ribeiro, M. T., Singh, S., & Guestrin, C. 2016. Why should I trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 1135-1144. ACM

2016
[20]

Sicong, L., Zimu, Z., Junzhao, D., Longfei, S., Han, J., & Wang, X. 2017. Ubiear: Bringing location- independent sound awareness to the hard-of-hearing people with smartphones. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1(2), 17

2017
[21]

US Department of Justice. 2011. ADA Requirements:

2011
[22]

Retrieved July 1, 2019 from https://www.ada.gov/service_animals_2010.htm

Service Animals. Retrieved July 1, 2019 from https://www.ada.gov/service_animals_2010.htm

2019
[23]

Zhao, Y., Szpiro, S., Knighten, J., & Azenkot, S. 2016. CueSee: exploring visual cues for people with low vision to facilitate a visual search task. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 73-84. ACM

2016
[24]

Zhao, Y., Wu, S., Reynolds, L., & Azenkot, S. 2018. A face recognition application for people with visual impairments: understanding use beyond the lab. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, paper 215. ACM

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Ahmed, T., Kapadia, A., Potluri, V., & Swaminathan, M. 2018. Up to a limit?: Privacy concerns of bystanders and their willingness to share additional information with visually impaired users of assistive technologies. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(3), 89

work page 2018

[2] [2]

Aira. 2018. Retrieved July 2, 2019 from https://aira.io/

work page 2018

[3] [3]

P., Jayant, C., Ji, H., et al

Bigham, J. P., Jayant, C., Ji, H., et al. 2010. VizWiz: nearly real-time answers to visual questions. In Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, 333-342. ACM

work page 2010

[4] [4]

Buolamwini, J., & Gebru, T. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of the Conference on Fairness, Accountability and Transparency, 77-91

work page 2018

[5] [5]

and Kovaleski, S.F

Conger, K, Fausset, R. and Kovaleski, S.F. 2019, May

work page 2019

[6] [6]

T., Peterson, S

Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. 2003. The role of trust in automation reliance. International Journal of Human- Computer Studies, 58(6), 697-718

work page 2003

[7] [7]

Fiannaca, A., Apostolopoulous, I., & Folmer, E. 2014. Headlock: a wearable navigation aid that helps blind cane users traverse large open spaces. Proceedings of the 16th international ACM SIGACCESS Conference on Computers & Accessibility, 19-26. ACM

work page 2014

[8] [8]

Findlater, L., Chinh, B., Jain, D., Froehlich, J., Kushalnagar, R., & Lin, A. C. 2019. Deaf and hard-of- hearing individuals’ preferences for wearable and mobile sound awareness technologies. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), paper 46. ACM

work page 2019