2023

FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines

Barker, M., Kallina, E., Ashok, D., Collins, K., Casovan, A., Weller, A., Talwalkar, A., Chen, V., & Bhatt, U.

Abstract

As machine learning (ML) pipelines affect an increasing array of stakeholders, there is a growing need for documenting how input from stakeholders is recorded and incorporated. We propose FeedbackLogs, addenda to existing documentation of ML pipelines, to track the input of multiple stakeholders. Each log records important details about the feedback collection process, the feedback itself, and how the feedback is used to update the ML pipeline. In this paper, we introduce and formalise a process for collecting a FeedbackLog. We also provide concrete use cases where FeedbackLogs can be employed as evidence for algorithmic auditing and as a tool to record updates based on stakeholder feedback.

BibTeX

				
					@inproceedings{10.1145/3617694.3623239,
author = {Barker, Matthew and Kallina, Emma and Ashok, Dhananjay and Collins, Katherine and Casovan, Ashley and Weller, Adrian and Talwalkar, Ameet and Chen, Valerie and Bhatt, Umang},
title = {FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines},
year = {2023},
isbn = {9798400703812},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3617694.3623239},
doi = {10.1145/3617694.3623239},
abstract = {As machine learning (ML) pipelines affect an increasing array of stakeholders, there is a growing need for documenting how input from stakeholders is recorded and incorporated. We propose FeedbackLogs, addenda to existing documentation of ML pipelines, to track the input of multiple stakeholders. Each log records important details about the feedback collection process, the feedback itself, and how the feedback is used to update the ML pipeline. In this paper, we introduce and formalise a process for collecting a FeedbackLog. We also provide concrete use cases where FeedbackLogs can be employed as evidence for algorithmic auditing and as a tool to record updates based on stakeholder feedback.},
booktitle = {Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization},
articleno = {19},
numpages = {15},
location = {Boston, MA, USA},
series = {EAAMO '23}
}

Copy the above and convert to citation here

APA Reference

Barker, M., Kallina, E., Ashok, D., Collins, K., Casovan, A., Weller, A., Talwalkar, A., Chen, V., & Bhatt, U. (2023). FeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. Association for Computing Machinery.

DOI

10.1145/3617694.3623239

Cyber-human Lab Contributors

Emma Kallina

Emma is driven by a desire to design technology that enhances human well-being – beyond human performance. She started her PhD at the Institute...