Comparison of AI-assisted OSM mapping in fAIr environment and manual mapping in JOSM editor -

Our study investigates the efficiency (mapping speed), effectiveness (mapping quality), and the most common errors of AI-assisted mapping in OSM through user testing of the fAIr environment (https://fair.hotosm.org/) and JOSM editor. By analysing mapper performance, interaction behaviour, and resulting spatial data, the research contributes empirical evidence on how editor design influences human–AI collaboration in volunteer mapping. More broadly, the study addresses the ongoing transition of OSM from primarily crowdsourced production to hybrid geospatial workflows and provides insights for the design of future AI-assisted mapping tools. The experiment took place over four sessions between 7 October and 11 October 2024. The experiment was organised, and the results were analysed as part of two theses by students Jan Horák and Hugo Krotil from the Department of Geography, Faculty of Science, Masaryk University. The authors of this paper were the supervisor and consultants of these theses. The authors organised an experiment, validated the data, and ensured the accuracy of analyses and the interpretation. They also trained participants, prepared fine-tuned models, and divided the localities into simple and complex ones. The organisation of the experiment was consulted with fAIr developers Omran Najjar, Ksithij Sharma, and their colleague Anna Zanchetta, who analysed the impact of locality difficulty. They participated in training participants, preparation of fine-tuned models, dividing the localities into simple and complex ones, and extracting data from the fAIr database after the experiment. The growing use of GeoAI in volunteered geographic information is reshaping Digital Earth workflows through collaboration between people and machine learning models. OpenStreetMap is central to this ecosystem, yet evidence on how AI-assisted editors affect behavior, data quality, and mapping performance remains limited. We present an evaluation of fAIr, an open, locally trainable building-mapping environment developed by the Humanitarian OpenStreetMap Team. In a controlled experiment with twenty-six participants of varied experience, we compared fAIr with JOSM, the standard editor for manual mapping. We analyzed mapping efficiency, effectiveness, error types, and the use of model parameters across simple and complex localities. Manual mapping in JOSM was superior in speed and accuracy, driven by strong results from experienced contributors. fAIr narrowed differences between beginners and experienced mappers but produced characteristic AI-related errors, most often merging several buildings into a single outline. Participants adjusted only some parameters of fAIr. Overall, fAIr offers a fast, accessible workflow for simple structures, but its wider applicability depends on improving model accuracy and reducing systematic segmentation errors. These findings informed the design of trustworthy AI-assisted mapping within the Digital Earth paradigm and were used to develop a new version of fAIr.