Enhancing augmented reality with machine learning for hands-on origami training

Abstract

This research explores integrating augmented reality (AR) with machine learning (ML) to enhance hands-on skill acquisition through origami folding. We developed an AR system using the YOLOv8 model to provide real-time feedback and automatic validation of each folding step, offering step-by-step guidance to users. A novel approach to training dataset preparation was introduced, which improves the accuracy of detecting and assessing origami folding stages. In a formative user study involving 16 participants tasked with folding multiple origami models, the results revealed that while the ML-driven feedback increased task completion times, it also made participants feel more confident throughout the folding process. However, they also reported that the feedback system added cognitive load, slowing their progress, though it provided valuable guidance. These findings suggest that while ML-supported AR systems can enhance the user experience, further optimization is required to streamline the feedback process and improve efficiency in complex manual tasks.

BibTeX

				
					@article{lysakowski_enhancing_2025,
	title = {Enhancing augmented reality with machine learning for hands-on origami training},
	volume = {6},
	issn = {2673-4192},
	url = {https://www.frontiersin.org/journals/virtual-reality/articles/10.3389/frvir.2025.1499830/full},
	doi = {10.3389/frvir.2025.1499830},
	language = {English},
	urldate = {2025-01-27},
	journal = {Front. Virtual Real.},
	author = {Łysakowski, Mikołaj and Gapsa, Jakub and Lyu, Chenxu and Bohné, Thomas and Tadeja, Sławomir Konrad and Skrzypczyński, Piotr},
	month = jan,
	year = {2025},
	note = {Publisher: Frontiers},
	keywords = {assembly task, augmented reality, edge computing, education, machine learning},
}
				
			
APA Reference

Łysakowski M, Gapsa J, Lyu C, Bohné T, Tadeja SK and Skrzypczyński P (2025) Enhancing augmented reality with machine learning for hands-on origami training. Front. Virtual Real. 6:1499830. doi: 10.3389/frvir.2025.1499830

Cyber-human Lab Contributors

Dr Sławomir Tadeja

Dr Slawomir K. Tadeja is a Postdoctoral Associate with the Department of Mechanical Engineering at the Massachusetts Institute of Technology (MIT). Here, he works...

Dr Thomas Bohné

Thomas Bohné is the founder and head of the Cyber-Human Lab at the University of Cambridge’s Department of Engineering. He is also leading research...