Content area
Augmented Reality (AR) technology is gaining relevance in spatial applications, offering interactive ways for users to explore and navigate destinations by overlaying digital information onto real-world environments. As this technology advances, AR maps present both new possibilities and unique challenges in cartographic communication. Despite their growing presence, a structured, user-centered framework for evaluating the usability and effectiveness of AR maps remains underdeveloped. This paper proposes a theoretical foundation for assessing AR maps through a user-centered lens, emphasizing cognitive, perceptual, and interactional dimensions of map use. Drawing from existing usability evaluation models, human-computer interaction principles and cartographic theory, the framework outlines methods for capturing user experience, including taskbased assessments, heuristic evaluation, and subjective usability metrics. This framework is designed to guide future studies and can be adapted across various domains-from navigation to education. It highlights the importance of understanding how users interpret and interact with spatial information in immersive environments. However, ensuring their usability remains a challenge, particularly due to the novelty of AR interaction and the lack of established evaluation guidelines tailored to spatial applications. This paper presents a user-centered theoretical framework for evaluating the usability of AR maps within the fields of human-computer interaction (HCI) and cartography.
ABSTRACT
Augmented Reality (AR) technology is gaining relevance in spatial applications, offering interactive ways for users to explore and navigate destinations by overlaying digital information onto real-world environments. As this technology advances, AR maps present both new possibilities and unique challenges in cartographic communication. Despite their growing presence, a structured, user-centered framework for evaluating the usability and effectiveness of AR maps remains underdeveloped. This paper proposes a theoretical foundation for assessing AR maps through a user-centered lens, emphasizing cognitive, perceptual, and interactional dimensions of map use. Drawing from existing usability evaluation models, human-computer interaction principles and cartographic theory, the framework outlines methods for capturing user experience, including taskbased assessments, heuristic evaluation, and subjective usability metrics. This framework is designed to guide future studies and can be adapted across various domains-from navigation to education. It highlights the importance of understanding how users interpret and interact with spatial information in immersive environments. However, ensuring their usability remains a challenge, particularly due to the novelty of AR interaction and the lack of established evaluation guidelines tailored to spatial applications. This paper presents a user-centered theoretical framework for evaluating the usability of AR maps within the fields of human-computer interaction (HCI) and cartography.
Keywords: Augmented Reality maps, User-centered evaluation, Usability assessment, Tourism technology, Human-computer interaction
INTRODUCTION
Maps are a form of communication, a way of comprehending, articulating, and constructing the human world. Cartographic visualizations are usually given as printed or digital maps, visualized through the screens or printed on paper. Map users can interact intuitively and directly with a paper map in a natural and familiar way. However, technological advances made significant changes in the way we communicate and access information, and printed maps are found to offer poor interactive tools [1] that modern users need. Further advance in hardware and software allowed developers to transfer to one of the most sophisticated services: Augmented Reality (AR).
Among the different types of maps, touristic maps play a crucial role in supporting tourism activities worldwide [2]. They are specifically designed to help tourists organize their activities and fulfil their informational needs. Touristic maps represent a distinct and complex category within cartographic design, requiring special consideration of the principles and processes involved. The tourism sector is increasingly looking for new ways of visitor engagement through the latest technological innovations. While AR is increasingly being implemented across many industry sectors, an open question remains over the impacts of this new technology within the touristic cartography context. As more tourists are using mobile devices, the potential benefits of using AR for the enhancement of the tourist experience are increasing [3]. Nevertheless, it is necessary to carefully explore the potential of investing in new technologies, as visitor satisfaction 1s closely linked to the destination's overall success. User experience developed as an important area Within human-computer interaction and involves aspects such as usability, usefulness and emotional impacts. However, the topic of user experience lacks theory and empirical research, particularly in the context of tourism.
This paper proposes a theoretical foundation for assessing AR maps through a usercentered lens, emphasizing cognitive, perceptual, and interactional dimensions of map use. Drawing from existing usability evaluation models, human-computer interaction principles and cartographic theory, the framework outlines methods for capturing user experience, including task-based assessments, heuristic evaluation, and subjective usability metrics.
AUGMENTED REALITY MAPS
Augmented Reality (AR) 1s a technology that enhances the perception of the real environment by adding virtual information. An AR system supplements the real world with virtual objects that appear to coexist in the same space - the real world. Therefore, it enhances the user's interaction and experience in both real and virtual environments [4]. AR evolved into augmented virtual environment (AVE) and augmented geographic reality (AGR) by joining the concept of geography to the "virtuality continuum" [5]. AGR should be categorized into augmented reality environments (AREs) and augmented maps (AMs), based on the two ways in which users cognize geographic environments, i.e., based on field experiences and maps, respectively. In AREs, visual information is primarily added to objects in the real world to realize functions such as navigation and illustration. AMs, the main objects of this study, are maps on which multiple types of geographic information are superimposed to enhance cartographic information transfer and users' spatial cognitive ability (Figure 1).
The first developed AMs were examined as a collaboration interactive tool [4], which led to the birth of the concept of mobile AMs [5]. Morrison et al. [6] created the first mobile AM system based on map features (MapLens) and they were the first ones that exhibited collaborative properties of AMs superior to those of electronic maps outside laboratory conditions. From a software design perspective, Paelke and Sester [7] integrated paper maps with functions like location data updating and positioning. However, their objectives were not focused on cartographic principles. De Almeida Pereira et al. [8] were the first ones that examined the contribution of AR system for map reading and improving users' understanding of spatial data. Their prototype included a map overlay over a paper map-alike marker, which implies that virtual content is seen as a whole, and not as an analogue map extension.
The majority of AR research to-date has been technology-driven, and this line of research is well-justified; functioning technology plays an important role in any AR experience [5]. In terms of spatial references, the main focus was on wayfinding activities. Navigation aids nowadays can efficiently guide users to their destinations and are supposed to help the spatial knowledge acquisition. However, common users tend to ignore the idea to "learn" if they can simply follow the instructions without thinking, and they may have difficulties remembering the routes, which implies that these navigation aids may damage spatial knowledge acquisition, especially considering the tourists needs to navigate through the unknown areas.
AR views are substantially different from traditional graphical user interfaces developed for desktop systems. Delivery of information is difficult because of various technical (e.g. screen size, patchy connectivity, short battery life) and contextual (e.g. changes in weather and lightning conditions) challenges. In addition, in contrast to standard graphical user interfaces, AR combines both physical (real-world) and computergenerated virtual information. This novel interface questions the applicability of the established cartographic design principles. What this means is that there is still little knowledge how content and graphical design decisions impact the effectiveness and efficiency of users. The main reason for this is that there is still little understanding with respect to the user requirements that have to be fulfilled in order to ensure efficient work with AR in general, and AMs in particular [5].
USER CENTERED DESIGN IN AUGMENTED REALITY
Cartographic products have evolved drastically over time, yet their primary function remains the same: communication of spatial information. Nowadays, in technologically driven world, the possibilities and limitations related to screen displays significantly impact on how maps can be presented to the user, and how the information is perceived and used [9]. These changes have placed greater emphasis on understanding the processes and methods of effective communication of spatial information [10]. Contemporary cartography focuses not only on the visual representation of geographic data, but also on how relevant information is shaped, delivered, and cognitively processed by users [9].
Buchner et al. [11] and Amorim and Schmidt [12] examined cognitive load from AR technology literature. Buchner et al. [11] stated that "AR serves as a supportive technology that might reduce cognitive load and thus assists the performance of different tasks". Thus, the process of AM creation needs to be based on particularly conscious choices as well-thought-out decisions [12]. Despite advancements in AR mapping products, most of the AM products do not focus on communication processes involving users, which can lead to potential design and effectiveness issues. Consequently, there is a rising need to understand the map user, how they interpret and process the visual information on the maps.
User design studies have been emphasized as a framework for generating substantial cartographic contributions [13]. This aligns with the agenda set by the International Cartographic Association (ICA), which stresses the importance of user-centered design (UCD) studies in enhancing the quality and impact of cartographic products. Usability Engineering (UE) and User Centered Design (UCD) are established themes in the software development domain. UCD involves the user in the development stages to enhance the usability of the final product. By involving the user in the production process, the effectiveness of the product - or its quality towards the user - improves significantly. ISO Standard 9241 defines usability as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. In usability testing, as stated in Vanicek and Popelka [14], effectiveness addresses the accuracy and completeness of answers (e.g., the success rate of questions answered), efficiency addresses how answers are achieved (e.g., the time taken to complete a task), and satisfaction focuses on participants' attitude and comfort. The most common method for usability assessment is through user testing. Within the field of cartography, UCD involves incorporating user feedback throughout the design process to ensure that the final product is more effective and user-friendly [13]. This iterative approach relies on a combination of methods to continuously assess and refine designs using real users feedback, rather than relying on hypothetical assumptions. In both academic research and practical applications, the focus of UCD studies is primarily on the process itself.
PROPOSED EVALUATION FRAMEWORK
This research aims to adopt a novel approach incorporating major concepts from previous studies within cartography and visual analytics fields. Based on recommendations into formal and repeatable UCD processes, formative evaluation is prioritized over summative evaluation; therefore, the evaluation methods will be suitably applied, following Roth [13] categories of expert-based, theory-based and user-based evaluation methods. Following this, research will be conducted in several phases (Figure 2), displayed bellow.
The initial phase involves a comprehensive review of the existing cartographic visualizations within a certain domain, in this case touristic maps. Critical analysis of existing cartographic representations results in the basis for designing a new cartographic product [15]. This process uses general cartographic rules and guidelines for displaying spatial objects and the principles of cartographic communication. Each cartographer employs a unique method, depicting objects or phenomena with simplified signs associated with real objects in nature. This involves subjective decisions to add or subtract, suppress, or emphasize certain properties [9].
The second phase focuses on a needs assessment study to determine the user needs from the touristic products. Participants with diverse user characteristics (gender, age, level of experience, background knowledge) are considered. An online survey method is employed to gain insights into user needs and preferences for touristic sights, based on the analysis from the previous phase [15].
The third phase concerns the conceptual development design of the new product itself. User inputs from the needs assessment study will be applied to the design of the AM, emphasizing the most recognizable cartographic elements to differentiate cartographic symbols in AR environments. The special attention is on the generalization processes and the cartographic load on the product.
In the fourth phase, the developed product will undergo an evaluation and refinement by domain experts. Experienced cartographers will be interviewed to provide feedback, suggestions, and guidelines for improving the visualization of individual signs. This qualitative research phase aims to test initial prototype designs.
The final phase involves usability evaluation using think aloud method and summarizes the results based on the participant responses. The think-aloud method, well-integrated in psychological research, involves participants verbalizing every thought during the task performance [16]. The sequence of verbalized thoughts from working memory is the same as in the cognitive process that is performed without any thinking aloud. Since the general public is the intended user of this kind of a map, the tasks have been designed so that the person without any expertise can solve them. The participants will be recruited through online platforms, universities, and local communities, ensuring a balanced representation of the target user group. Since their answers should not be affected by their prior knowledge, tested cartographic displays will be fictional, but based on the real environment. During the usability evaluation, participants will be video and audio recorded in laboratory conditions. Participants will be analyzed following the methodologies described by [16], among others. These approaches will result in a comprehensive evaluation of the produced AM.
Satisfaction is usually evaluated through a simple questionnaire at the end of a usability test, either in the form of a single ease question (SEQ) or system usability scale (SUS), which can also be used as a validation input [14].
The combination of quantitative and qualitative research is a common characteristic of the user-centered design process, whereby qualitative research is implemented to test the first prototype designs (phases 1-4) and more quantitative research to evaluate the final designs (last phase) [16].
DISCUSSION
User-Centered Design (UCD) studies represent an epistemological shift in how cartographers generate knowledge about maps and visualizations. Rather than relying on simplified, controlled experiments, UCD recruits real users in realistic contexts for insights that are more realistic. While AR technology has advanced significantly in terms of technical advancement, systematic methodological frameworks for user-centered AR map design remain underdeveloped. The proposed framework addresses this gap by providing structured guidance for what is currently an ad hoc process in many AR development contexts. To our knowledge, this paper introduces the first multi-stage, usercentered framework tailored specifically to AR map design and assessment.
Heuristic evaluation and needs assessment (also task analysis or work domain analysis) is an initial characterization of the target user needs and other relevant use context. In practice, a needs assessment formalizes user personas (i.e., generic descriptions of different kinds of target users) and use case scenarios. It also answers a number of questions about the design context, such as user goals, currently met and unmet user needs, user backgrounds and abilities, and the users" technological infrastructure and environmental setting. In this case 1t has been applied to the tourism industry, but it does not have to be the case in other applications. An online survey method 1s proposed to gain insights into user needs and preferences in order to reach higher number of potential users.
Conceptual design then translates feedback from the needs assessment into design features for the proposed product. Sometimes described as a requirements analysis, conceptual design enumerates the functional requirements of the product, listing all necessary geospatial data, map representations and user interactions as well as nonfunctional requirements such as accessibility, customization, and interoperability. Considering the technology applied, 1t shows the most valuable part of the framework, since the technology adapts to the user needs and can be of great potential.
Conceptual design 1s followed by prototyping. Following a user-centered approach, target users and other stakeholders evaluate prototypes of all degrees of fidelity on their prospective usability and utility. Formative evaluation of early prototypes identifies missing functionality, considers potential bottlenecks in navigation and map use and reveals potential usability issues before investing heavily in production. User evaluations draw from a wide range of empirical methods. The integration of quantitative metrics with rich qualitative data ensures a holistic understanding of how target users perceive, interpret, and interact with the augmented spatial information.
Together, these five phases form an iterative cycle, and by following user feedback at every stage, this framework moves beyond one-off evaluations and toward a sustained, evidence-based approach to AR map design. Future work should empirically validate this framework across diverse application domains, explore its adaptability to different AR platforms, and investigate cross-cultural and accessibility considerations. Ultimately, this user-centered methodology promises to guide the creation of AR maps that are not only technologically sophisticated but also intuitively aligned with how people process and use spatial information in immersive environments.
CONCLUSION
This paper has offered a structured, user-centered theoretical framework for evaluating augmented reality maps. By articulating five sequential phases-cartographic review, needs assessment, conceptual design, expert refinement, and user-centered usability testing-we provide a transparent roadmap for future researchers and practitioners aiming to assess AMs" cognitive, perceptual, and interactional dimensions. Our integration of heuristic evaluation, task-based assessment, think-aloud protocols, and subjective usability metrics underscores the framework's flexibility across diverse spatial contexts.
While this work remains theoretical, it draws on two prior studies: one exploring AR in tourism from a cartographic perspective [3], and another applying participatory map evaluation with national park visitors [15], to ground our approach in real-world cartographic challenges. The framework's modular design permits adaptation to various AR platforms and use cases, from field navigation to educational overlays.
Looking forward, empirical validation is essential. Pilot studies deploying this framework can test its effectiveness in uncovering usability bottlenecks and inform refinements. Moreover, future research should examine cross-cultural differences in AM perception, assess accessibility considerations, and explore interactive visualizations for real-time user feedback. AMs can enhance static visual representation and serve as an extension of traditional cartography and map visualization. The focus is on augmentation of physical maps with useful and interesting information. Paper maps have a large static surface, and AR can provide a see-through lens without forcing the user to watch map data only through the small mobile display, ultimately leading to AMs that are not only technologically robust but also intuitively aligned with how people perceive and interact with spatial information.
REFERENCES
[1] Аза! К, Kondo T, Kobayashi H, Mizuki A (2008) A Geographic Surface Browsing Tool Using Map-Based Augmented Reality. In: 2008 International Conference Visualisation. IEEE, pp 93-98
[2] Ozogul G, Baran GG (2016) Accessible tourism: the golden key in the future for the specialized travel agencies. Journal of Tourism Futures 2:79-87. https://doi.org/10.1108/JTF-03-2015-0005
[3] Cibilié I, Posloncec-Petric У, Tomini К (2021) Implementing Augmented reality in Tourism. In: Proceedings of the ICA. pp 1-5, 10.5194/ica-proc-4-21-2021
[4] Bobrich J, Otto S (2002) AUGMENTED MAPS. International Archives of Photogrammetry Remote Sensing and Spatial Information Sciences 4:502-505
[5] Cheng Y, Zhu G, Yang C, et al (2022) Characteristics of augmented map research from a cartographic perspective. Cartogr Geogr Inf Sci 49:426-442. https://doi.org/10.1080/15230406.2022.2059571
[6] Morrison A, Oulasvirta A, Peltonen P, et al (2009) Like bees around the hive. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, pp 1889-1898
[7] Paelke V, Sester M (2010) Augmented paper maps: Exploring the design space of a mixed reality system. ISPRS Journal of Photogrammetry and Remote Sensing 65:256- 265. https://doi.org/10.1016/j.isprsjprs.2009.05.006
[8] de Almeida Pereira GH, Stock K, Stamato Delazari L, Centeno JAS (2017) Augmented Reality and Maps: New Possibilities for Engaging with Geographic Data. Cartogr J 54:313-321. https://doi.org/10.1080/00087041.2017.1411417
[9] MacEachren AM (1995) How maps work: representation, visualization, and design. The Guilford Press, New York
[10] Robinson AC, Kettunen P, Delazari L, Coltekin A (2023) New directions for the state of the art and science in Cartography. International Journal of Cartography 9:143- 149. https://doi.org/10.1080/23729333.2023.2216334
[11] Buchner J, Buntins K, Kerres M (2021) A systematic map of research characteristics in studies on augmented reality and cognitive load. Computers and Education Open 2:100036. https://doi.org/10.1016/j.caeo.2021.100036
[12] Amorim FR, Schmidt MAR (2021) Classification of Dynamic Cartographic Symbols applied to Augmented Reality (AR) Systems. Proceedings of the ICA 4:1-8. https://doi.org/10.5194/ica-proc-4-8-2021
[13] Roth R (2019) How do user-centered design studies contribute to cartography? Geografie 124:133-161. https://doi.org/10.37040/geografie2019124020133
[14] Vanicek T, Popelka S (2023) The Think-Aloud Method for Evaluating the Usability of a Regional Atlas. ISPRS Int J Geoinf 12. https://doi.org/10.3390/jg112030095
[15] Cibilié I, Poslonéec-Petrié V (2025) Evaluating Cartographic Communication in Croatian National Parks: User Perceptions of Pictograms. ISPRS Int J Geoinf 14:41. https://doi.org/10.3390/jg114020041
[16] Elzakker У (2004) The use of maps in the exploration of geographic data. PhD Thesis, Utrecht University, Netherlands. https://dspace.library.uu.nl/handle/1874/1235
Copyright Surveying Geology & Mining Ecology Management (SGEM) 2025