An Experiment to Discover Usability Guidelines for Designing Mobile Tourist Apps

Abstract

Tourist apps can be very useful for sightseeing, and this is one of the reasons that makes them so numerous in app stores. Good usability can make the difference when choosing an app, from the user’s point of view, so this study is aimed at analyzing and discovering common usability issues in apps for tourism. This paper presents a research study with two objectives: analyzing the most common usability problems in mobile apps for tourism and proposing recommendations for improving the usability of those apps. The research process firstly identifies the main functionalities that tourist guides should have, by filtering the results obtained through the store. This was followed by a two-part experiment. The first part was a Keystroke Level Modelling analysis, where the number of taps needed to perform each main functionality in each tourist guide app was registered. Secondly, a heuristic evaluation was carried out on the best rated apps in the previous stage. Four usability experts tested the apps in real devices, by performing several tasks with a smartphone, and evaluated them according to some mobile heuristics. After this process, the most frequent usability problems in tourist guide apps were found, which allowed to discover recommendations for designing and developing mobile tourist guides. One of the recommendations is that tourist guide apps should provide at least the following functionalities: listing POIs (Points of Interest), showing information of a POI, selecting language, showing a map, working offline, and showing a tourist’s current location. Other recommendations proposed are, for example, showing relevant and complete information of POIs, such as opening hours and admission fees; showing more than one POI on the map and the distance to the tourist; and avoiding showing at the same time commercial information and tourist information.

This work is licensed under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.

1. Introduction

Mobile devices have become an essential tool in our daily lives, in both the professional and personal fields. As an example, the number of connections through these devices has increased in recent years, from 5,290 million in 2010 to 8,160 million in 2018 [1]. Mobile devices include many technological devices, such as smartphones, tablets, or smartwatches, although the most popular are smartphones. A smartphone is a mobile phone that offers more advanced computing ability and connectivity than a basic mobile phone [2]. Smartphones have evolved, and they have now stronger input capabilities, larger screens, reliable and unlimited Internet access, and powerful location awareness [3], and they have different characteristics from other devices such as PCs (Personal Computers) or tablets [4]. Due to the above, most of smartphone developers such as Apple™, Google™, or BlackBerry™ provide open platforms for mobile application (app) developers [5], where not only smartphone developers can create apps but also individual developers and organizations can do it. As a result, the number of commercial and noncommercial apps in those platforms (usually called “markets”) has hugely increased [6].

Travel apps are very important within the markets because smartphones have rapidly become a powerful tool for tourists. Smartphones allow tourists to search for information on accommodation, places where to eat, or locate the best Points of Interest (POIs) to visit. In addition, they can show maps, which provide information on how to get efficiently to a place [6]. Tourist mobility, together with the different capabilities of mobile devices, is fundamental for management and research in the field of tourism [7]. Since there are so many apps for tourism, users have too many apps to choose from, and they usually download the apps to test them, but around 26 percent of apps are used just once [8]. One of the causes is because many of them have a bad usability, since a good usability is essential for an app to be successful [9]. On the other hand, tourist guides seem to be the most common apps for tourism [2, 10], so we decided to focus our study on this type of apps. For this reason, the present study discovers and analyzes the usability issues existing in current tourist guide apps. Furthermore, the contribution of this paper is some usability recommendations, which could be used by designers and developers of those apps. By applying these recommendations, they will be able to make their apps more usable than the others, in order to make them widely adopted and not used only once. Therefore, more usable apps would result in an improvement of satisfaction and user experience of tourists [11].

The rest of the paper is structured as follows: Background (Section 2) shows related works to this paper, Section 3 explains both the objectives and the process carried out, Section 4 shows the results obtained and Discussion (Section 5) compares the results of this paper with the results found in other previous studies. Finally, Conclusion (Section 6) summarizes the process, results, and recommendations found to make mobile tourist guides more usable.

2. Background

Smartphones have been rapidly been adopted as a tourism travel tool, thanks to their advanced computing capabilities and ubiquity [6]. In addition to basic tourism information, mobile apps offer dynamic information to tourists, such as their position, service locations, distances between locations, and social and marketing information related to specific locations [12]. Knowing this, tourism companies are developing mobile applications with the aim of increasing customer satisfaction and improving the marketing and sale success of their tourism services [12].

However, it is necessary to combine good aesthetics with good usability for an app to have a good acceptance in a short period of time [9]. For instance, in a study [10], 19.7% of apps analyzed did not have an appropriate title, which is a bad usability practice and can disorient the users, because they usually have different apps installed in their smartphones and they may not be able to identify it after installation. Furthermore, in this study, the authors indicated that many apps were not easily found in the apps store, only after a long search process, in which tourists are unlikely to carry out, so some of the apps analyzed could be described as invisible to the user.

In addition to good usability, it is important that tourist apps have various functionalities. Scolari et al. [10] analyzed 66 tourist apps and discovered that each one included 3 functionalities on average. The most common functionalities found were the following: destination resource guide, geolocation, agenda information, augmented reality, image gallery, sharing comments, audio guide, promotions and discounts, rating places, and video guide. On an emotional level, it would be important for tourist apps to include the following functionalities: geolocation for navigation, POIs, and positioning; APIs and access to map services and other open data; and local storage for saving maps and selected content for offline use [12]. Not and Venturini [13] also agreed that maps are in general essential in a mobile setting. Not in vain, 98% of tourist apps have mapping services [9].

On the other hand, it is essential that the apps work offline, especially when abroad. A study showed that only 20% of tourist apps analyzed met this requirement [9]. Therefore, providing maps/geolocation in tourist apps is essential, as well as the possibility of using the app offline. In this sense, it is recommended to show more than one POI at a time on the map, especially for hotels, itineraries, and sports [13].

Another interesting point found in the literature is that travelers give more credibility to the opinions of other travelers [9], but nevertheless, only 22% of apps analyzed by Scolari et al. [10] allow generating content by the user (comments and photos). Hence, it is recommended that tourist apps allow exchanging information between tourists.

Finally, apps help globalization; however, too many developers wrongly think locally [9]: 60% of tourist apps are only available in one language [10], so they are of little use for international tourists who come from countries with different languages. Then, tourist apps should be available at least in two languages.

Tourist apps can be divided into the following groups [10]: those that allow or help the user to carry out tasks (e.g., finding a restaurant or scheduling a walk through the historic center of a city); those whose main objective is to provide information (about the destination, places to visit, events, agenda, etc.); and the rest which are mixed or hybrid, because they allow actions to be carried out, but they also provide information.

More specifically, there is a study [9] that classifies tourist apps into the following types: maps and guides, transport, hotels, destination guides, recommendations, and others. For their part, Wang et al. [2] established the following classification (ordered by a number of apps in Apple Store): flight information manager, destination guides, online travel agency, facilitator, attractions guides, entertainment, food finder, language assistant, local transportation, augmented reality, currency converter, and tip calculator. However, many of these are sometimes included as functionalities of a tourist app, not as independent apps. These functionalities can improve tourists’ travel experience and add value to the general communication strategy of the destination [10]. In the two aforementioned classifications, only two types coincided: destination guides and (local) transportation. These were the two types initially valued to be included in our study. Destination guides are apps for providing various information about a particular place, such as New York City; and local transportation are apps for searching and reserving local transportations such as buses, subways, and taxi [2].

On the other hand, Ahmad et al. [14] discovered, after an exhaustive literature review, that there were only six usability guidelines proposed for tourist guide apps, obtained from three studies. This agrees with Azizi et al. [15], who affirm that there are not enough usability studies on tourist guides.

Therefore, we decided to focus our study on mobile tourist guides, for two reasons: (1) tourist guides are more tourism-specific than transport, since transport can be used for many other different reasons; and (2) there are not enough studies on usability guidelines on tourist guide apps.

3. Material and Methods

Apps for tourism can be divided into three groups according to the three stages of tourism consumption, i.e., depending on when they are used: preconsumption, consumption, and postconsumption [16]. During the preconsumption stage, tourists use information for planning, destination decision-making, transactions, and anticipation. In the consumption stage, information is used for connection, navigation, short-term decision-making, and on-site transaction. Finally, during the postconsumption stage, information is used for sharing, documentation, and reexperiencing. Most of the apps for tourism are focused on the consumption stage [9], and most of travel apps (55 percent) are purchased within three days: while travelers are sitting at the airport waiting for the plane to take off or while they are at the destination [17]. Therefore, this study is focused on the consumption stage. Tablets and PCs are preferred by tourists at the preconsumption stage, while smartphones are preferred during the consumption stage [9]. Thus, since this research is focused on the consumption stage, our study was performed with smartphones.

Of all the markets available for mobile apps, we decided to focus our study on Google Play for Android™ because this is the most used mobile operating system, and it is expected to continue increasing its use in the future [18]. Furthermore, in Android, there is no previous control to the publication of apps, as it exists in other platforms (e.g., in Apple Store™), so the quality and usability of these apps are expected to be lower, which is in turn better for detecting common usability issues.

The present research study has the following objectives:

(i) Objective 1: study, analyze, and classify the most common usability issues in mobile apps for tourism.

(ii) Objective 2: based on the results obtained in Objective 1, propose recommendations and guidelines for improving effectiveness, efficiency, and satisfaction (i.e., usability, according to its definition in ISO [19]: “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”) of these apps.

To evaluate the usability of apps for tourism, the protocol proposed by Martin et al. [20] was followed. This protocol was selected because it is especially designed for studying usability in mobile devices, and it has already been successfully applied in other similar studies for spreadsheet [21] and diabetes [22] apps. Therefore, the steps carried out and based on the protocol were the following (Figure 1):

(Step 1) Identifying potentially relevant apps: there are too many types of apps for tourism, so firstly, we had to specify in which kind the study would be carried out. As aforementioned, tourist guides are the most common apps for tourism ([10]; [2]), so we decided to focus our study on this type of apps. Therefore, a search was performed in the Google Play for Android™ to identify potentially relevant apps for tourism, using the “tourist guide” term.

(Step 2) Deleting lite versions of the apps: some of the apps that appear as a result of step 1 are trial and lite apps; i.e., they provide limited functionalities of the full app or provide access to the full app only for a limited time. These apps were deleted and excluded for the next steps, so only full apps were used. To do the selection, the description of the app in Google Play helped.

(Step 3) Identifying the main functionalities and excluding those apps that do not meet them: the main functionalities that a mobile tourist guide should meet were identified, and then the apps that did not meet them were excluded for the next steps. This can be taken as a measure of effectiveness because only the apps that allow performing the required tasks were taken into account (the main functionalities were then used as tasks in Step 5). Based on the literature, we decided that the main functionalities should be

(F1) Listing POIs (Points of Interest): the first problem which tourists face in an unfamiliar place is what to do [23], and a tourist guide should enable visitors to explore a city in their own way [24], so we concluded that showing a list of POIs is a basic functionality that tourist guides should provide.

(F2) Showing information of a POI: tourists have to manage when they do different activities [23], and the tourist guide should enable visitors to learn about the city in their own way [24], so at least information of each POI should be provided.

(F3) Selecting language: one of the benefits of mobile devices compared to generic online services is personalization, which allows additional customization according to other personal variables [13]. This personalization may include selecting the language of the app, since this is one of the most common issues in apps for tourism. For instance, 60 percent of Spanish apps for tourism are available only in one language [10], which makes them useless for foreign people who do not understand the language; and other studies showed that only 40 percent of the apps for tourism analyzed are available in several languages [9]. Therefore, having multilanguage support (minimum two languages) is required to have a global impact.

(F4) Showing a map: tourists should be able to find where POIs are around the city, which prevents them to spend too much time travelling between places when some attractions are close together [23]. The best way to do this is by showing a map, whose interaction in smartphones allows in turn to easily and intuitively zoom in and zoom out geographical areas [13].

(F5) Working offline: tourists usually have to pay extra costs for Internet access when they are not in their country, so if they use apps for tourism abroad they will probably have to pay excessive extra costs for using an app if it cannot run offline. Therefore, apps for tourism should be available offline [9].

(F6) Showing current location: one of the benefits of mobile devices when compared to other traditional and digital media is that they are able to know the location of the user ([25]; [13]). Furthermore, Cheverst et al. [24] point that the information presented by the apps for tourism should be tailored to their environmental context, so showing the location of the user is essential to take advantage of smartphones

(Step 4) Identifying the secondary functionalities: existing secondary functionalities (those additional to the main functionalities) were identified in the remaining apps. To do this, all the remaining apps were installed and tested in an Android device.

(Step 5) Performing a KLM (Keystroke Level Modelling) [26] analysis and a heuristic evaluation [27]: there are recent studies where these methods are still used (for instance, Lee et al. [28] used KLM, and Jeddi et al. [29] recently performed heuristic evaluation in their study). The KLM method was performed to measure efficiency of each app by counting the number of taps on the screen of the device to carry out each task. Apps from the top 3 highest score levels were selected to perform the heuristic evaluation, whose objective was to identify some usability issues by involving four expert evaluators (the number of experts is between three and five, as suggested by Nielsen [30, 31]). They tested the apps in real devices and evaluated them according to the mobile heuristics proposed by Bertini et al. [32], since researchers can use sets of specific usability heuristics to evaluate applications that have specific domain features [33]. The tasks the experts had to perform were decided based on the main functionalities. As shown in Table 1, five tasks correspond to the main functionalities, but functionality #5 (working offline) was divided into two tasks: T5a and T5b. This was decided because it was detected that there were two ways to achieve task #5, which were not both included in all apps and which required a different number of taps during the KLM. Dividing this task into two subtasks made it more reliable when comparing the number of taps.

[figure omitted; refer to PDF]

Table 1

Tasks performed by experts during the heuristic evaluation.

T1	Try to show a list of Points of Interest (POIs). Please note that the POIs can be showed in a list or on a map.
T2	Try to show the information of a POI.
T3	Try to change the language of the app (e.g., into English or Spanish).
T4	Try to show a map of the city.
T5a	Try to download the content for further viewing offline.
T5b	Try to enable the offline mode, if available.
T6	Try to show the current location of the user.

4. Results

In this section, the results obtained in each of the aforementioned steps are explained.

Firstly, the search was performed in the Google Play for Android™ and a total of 250 apps were found (Step 1). From these apps, two were discarded in Step 2 because they were lite versions of other full apps. However, while performing the second step, we realized that there were six more apps for tourism that were not found in the first step. Two out of six were the corresponding full version of those lite versions discarded, and the other four were discovered by checking the initial list: one of the apps in Step 1 allowed buying different guides, another one indicated that there is an app with improved content, another one indicated that it was an old version, and finally one of the apps contained a link to an offline version. All those apps were not detected in the first step because Google Play shows a maximum of 250 apps in one search, so there were many more tourist guide apps that did not appear with the search term used. Then two apps were removed, and the six new apps were added to the list, so 254 apps remained for the next step. Figure 2 shows how many apps were discarded and how many remained at each step.

[figure omitted; refer to PDF]

During Step 3, all remaining apps were installed and tested on a Samsung mobile device in order to check how many of the main functionalities they met. It is noteworthy that some of the tourist guide apps were hard to be found on the device after they were installed because their icon and/or name was different from those shown in Google Play. Table 2 shows the number of apps meeting the main functionalities, and Table 3 shows the number of apps meeting a different amount of functionalities. Only the 15 apps that met all the functionalities remained for the next step.

Table 2

Number of apps meeting the main functionalities.

Functionality	Number of apps meeting the functionality
F1	209
F2	209
F3	60
F4	192
F5	113
F6	167

Table 3

Number of apps meeting a different amount of functionalities.

Number of main functionalities met	Number of apps meeting those functionalities
6	15
5	113
4	49
3	29
2	5
1	2
0	41

In Step 4, all available functionalities (not only the main ones) were analyzed for each app. These functionalities were additional to the main ones, and the most common was “showing information about the city or the country” (14 out of 15 apps met this functionality). The second most common functionalities were two (met by 12 out of 15 apps): “showing predefined routes or trips” and “searching POIs.” The third most common functionalities were also two (10 out of 15 apps): “rating a POI” and “writing and/or reading user reviews.” Finally, the fourth most common functionality was allowing a currency converter (met by 8 out of 15 apps). There were 28 more functionalities that only a few apps (7 or less) met. To name a few, selecting a POI as a favourite, booking hotels, login, filtering results/POIs, creating itineraries, booking tours and activities, renting a car, marking POIs as visited, or showing POIs around the user’s position.

Later, in Step 5, a KLM analysis was performed for each app by counting the number of interactions (taps) needed to carry out each task. In cases where an action was not required, the number of taps was 0. The app is better the smaller the number of interactions. The idea is to keep apps from the top 3 highest score levels at this point to be further analyzed in the heuristic evaluation. One of the 15 apps did not properly work (to be sure, it was also tested in two more different devices), so it could not be assessed in this step. Therefore, the other 14 apps were analyzed. As seen in Table 4, there were four apps that required only eight taps to perform the six functionalities. However, the “Turkey offline Map Guide News” and “Rome Offline Map Guide Hotels” apps were similar to “Osaka Offline Map Guide Flight.” The only difference was the destination place, so we decided to keep only the last one for the next step. Since there are really two apps with eight taps, we decided to take them into account as only one case, and then we kept those apps for the next step plus the app with nine taps (Prague Minos Guide) and the app with ten taps (New York Travel Guide-mTrip). So four apps remained for the last step.

Table 4

KLM results for the 14 apps analyzed.

App name	T1	T2	T3	T4	T5a	T5b	T6	Total
Osaka Offline Map Guide Flight	1	2	3	1	0	0	1	8
Turkey offline Map Guide News	1	2	3	1	0	0	1	8
Rome Offline Map Guide Hotels	1	2	3	1	0	0	1	8
Paris Offline Map for Tourists	1	2	3	1	0	0	1	8
Prague Minos Guide	0	2	3	0	1	2	1	9
New York Travel Guide-mTrip	1	2	4	1	0	0	2	10
Kurdistan Tourism Travel Guide	1	2	4	1	1	0	2	11
Tripoli Guide	1	2	4	1	1	0	2	11
Official Madrid Guide	2	3	3	1	0	0	2	11
Belgrade Info Guide	2	3	4	2	2	0	5	18
Tijarafe: Tourist Guide	3	4	5	4	0	4	5	25
Travel guide/City Tour Guide	3	5	5	3	5	3	4	28
TouristEye-Travel Guide	5	6	4	5	4	0	6	30
PocketGuide Audio Travel Guide	4	5	1	4	5	4	9	32
Average	1.86	3.00	3.50	1.86	1.36	0.93	3.00
Median	1.00	2.00	3.50	1.00	0.50	0.00	2.00

Finally, a heuristic evaluation was performed by four experts. They tested the four remaining apps in a real smartphone and evaluated them according to the Nielsen’s Severity Ranking Scale (SRS) [30, 31]. They had to evaluate if each heuristic was met or not in each app, by discovering usability problems, which had to be scored by taking into account their severity. To do this, they had to fill one template per app, which contained all heuristics and some fields to rate each heuristic according to the SRS. The template also had an “evidence and comment” field for each heuristic, where the experts could justify the score given to each heuristic. Finally, when all experts had finished their analysis, they put the results in common, and the average of all responses was calculated. The mobile heuristics used were as follows: A—visibility of system status and losability/findability of the mobile device; B—match between system and the real world; C—consistency and mapping; D—good ergonomics and minimalist design; E—ease of input, screen readability, and glanceability; F—flexibility, efficiency of use, and personalization; G—aesthetic, privacy, and social conventions; and H—realistic error management [32]. The average results of the experts’ scores for each app and heuristic are shown in Table 5. The SRS rates from 0 (“I don’t agree that this is a usability problem at all”) to 4 (“Usability catastrophe: imperative to fix this before product can be released”), so the higher the score, the worse usability.

Table 5

Results (mean of experts’ scores) of the heuristic evaluation.

App name/heuristics	A	B	C	D	E	F	G	H	Total
Paris Offline Map for Tourists	0.19	2.67	1.75	2.13	1.35	1.00	1.88	0.33	11.29
Osaka Offline Map Guide Flight	0.19	2.58	1.50	2.13	1.65	1.13	2.00	0.50	11.67
Prague Minos Guide	0.38	0.67	0.63	0.38	0.90	1.00	0.88	0.17	4.98
New York Travel Guide-mTrip	0.25	0.58	0.50	0.63	0.75	0.25	0.38	0.25	3.58
Average	0.25	1.62	1.09	1.31	1.16	0.84	1.28	0.31

In general, no major problems were found related to heuristic A. This suggests that most of mobile tourist guide apps provide a good visibility of system status and findability of the mobile device.

Heuristic B (match between system and the real world) was the worst scored in general, and major usability problems were found. For instance, some apps included two different options, such as “attractions” and “POIs,” which experts considered confusing for tourists because they could not know the difference between them. Therefore, information and option titles of tourist guides should be in a language that tourists can easily understand, without ambiguities. Another common issue related to this heuristic was that sometimes information of POIs is not clear and/or is not enough (for instance, only the type and name are shown). Therefore, information and a detailed description of POIs should always exist, such as opening hours or admission fees. Furthermore, maps usually show POIs but not their type, which is very useful for tourists. Some guides even do not allow getting additional information about a certain POI shown on the map, which is frustrating for the tourist. Therefore, the maps of tourist guides should show the type of POIs and allow displaying more information about them, e.g., opening a new window with detailed information. Another common issue related to maps in tourist guides is that font size of street names is not big enough to be read, and it is not increased when zoomed in (Figure 3), so tourists may be lost when trying to find a specific street.

[figure omitted; refer to PDF]

One of the main issues related to heuristic C (consistency and mapping) is that some maps only show one POI at the same time and the distance to them is never shown (Figure 4), which is a problem for tourists who want to know what POIs are close to them. Therefore, showing more than one POI on the map is recommendable, as well as their distances to the tourist. In addition, some tourist guides show full screen ads, which is something that tourists may find uncomfortable. If ads are needed to fund the app, then it is advisable to provide a paid version of the app without ads.

[figure omitted; refer to PDF]

Design (heuristic D) is the second worst rated issue in general. Several tourist guide apps look like webpages (Figure 5) or have buttons coloured differently (Figure 6) and different font sizes for no apparent reason, etc. Therefore, screens should be consistent and well-designed, and fonts and colours should be adequately chosen in order to have a good look and feel.

[figure omitted; refer to PDF]

On the other hand, ease of input (heuristic E) is usually a weak point of tourist guide apps. Surprisingly, many guides do not allow searching POIs or do allow it but after filtering by category, which makes impossible to search a POI when the tourist does not know the category (see example in Figure 7). Therefore, an efficient way to search POIs should be provided, i.e., not only just showing them in a list but also searching by name or type. Navigation around the guides sometimes is not intuitive because the back button moves to the previous option in the menu instead of to the previous screen, which disorients the user. Furthermore, the buttons are sometimes too small to be clicked. Hence, it is recommended to provide the back button with the default and predictable behaviour, and the buttons should be big enough to facilitate clicking. The tourist may not be able to distinguish between commercial information and tourist information, which makes using the app annoying. Tourist guides should clearly separate those types of information.

[figure omitted; refer to PDF]

Regarding heuristic F, personalization is not generally an issue, but minor problems have been identified in some tourist guides, such as currency and units of length which cannot be chosen/changed. It would be desirable that tourist guides provide personalization for units in the app, due to the fact that tourists are from different countries. Minor issues have also been identified about the efficiency to input data. For example, the soft keyboard sometimes overlaps input fields and the tourist has to click twice on the text fields in order to use the soft keyboard. App developers should try to be careful when designing the screens in order to avoid these issues.

In some tourist guides, the design did not look good, according to the experts’ answers. The main issue was that there was no consistency, there were many colours (e.g., rainbow buttons), and there was no consistency among different screens (some developed in a native way and others looking like a webpage). This means the tourist loses the context, so the design should be consistent and not causing user disorientation.

Finally, only isolated problems were found related to heuristic H (realistic error management), such as confusing error messages or unexpected app closings.

At this point, Objective 1 (study, analyze, and classify the most common usability issues in mobile apps for tourism) and Objective 2 (propose recommendations and guidelines for improving effectiveness, efficiency and satisfaction of these apps) have been met.

5. Discussion

This study analyzes and classifies the most common usability issues in mobile apps for tourism. Results of KLM analysis and heuristic evaluation of common apps and functionalities return a set of usability recommendations for designing and developing mobile tourist guides that can improve effectiveness, efficiency, and satisfaction.

Our paper presents a research study different from those found in the literature since it included all the tourist guide apps that were found after a search on the market and not only one app, or top 20 or top 100, as other studies did [2, 9, 13, 24]. On the other hand, several studies only included apps for domestic tourism [6] or Spanish apps for tourism [10], but our work included all types of tourist guide apps, since tourism is global.

Our results support some of the results obtained in other studies. First, we discovered that many tourist guides could not be found by tourists because they have not a descriptive name or a good description. This supports the results presented by Scolari et al. [10], since they obtained that only 13 out of the 66 apps analyzed had an appropriate title and many apps were not easily found but after a too long search process. This issue has also been pointed in another study [9], which explained that Spanish apps for tourism have a lack of visibility.

Our recommendation about showing relevant and well-structured information of POIs also supports the results obtained by Scolari et al. [10] because they found that only 12% of the apps for tourism have a good information about accommodation and calendar of events.

Recommendations about aesthetics are also a recurrent issue, since it is essential and, when combined with a good usability, it is necessary for the users to have a good reception of the app in a short period of time [9]. In view of the results, we suggest to choose adequate colours and fonts, trying to keep a consistent design and well-designed screens.

On the other hand, an efficient search for POIs has been found in this research to be desirable and advisable, which supports the results of other studies [13], in which search methods such as searching by product categories have been suggested when using mobile devices.

We have recommended to show more than one POI at a time on a map, and this has also been recommended by Not and Venturini [13], who said that displaying many results on a map may be of particular help for hotels, itineraries, and sports.

Finally, some of the recommendations found in this paper have not been yet proposed in any other study. For instance, we found that the name of the guide and its icon should be the same both in the market (in this case, Google Play) and when installed on the device. As far as the authors know, some general recommendations here proposed have not been previously proposed in any other study. Some of these recommendations are avoiding ambiguity, information, and options that are unnecessary for the tourist; providing the back button with the default and predictable behaviour; designing the buttons with a big enough size to facilitate clicking; and facilitating input data. Furthermore, some specific guidelines have also not been previously found, such as allowing personalization of units in the tourist guide (e.g., currency and units of length), showing on a map the basic information of POIs or providing a mechanism to distinguish graphically their type.

As a summary, from the whole process, the following general recommendations were obtained:

(i) The name of tourist guides should be descriptive. Since Google Play only shows maximum 250 results in a search, it is essential to give a descriptive name and a good description to our app, because otherwise it will not be found by potential users (tourists)

(ii) Tourist guide apps should provide at least the following functionalities: listing POIs, showing information of a POI, selecting language, showing a map, working offline, and showing a tourist’s current location

(iii) The name of the app for tourism and its launch icon should be the same in Google Play and when installed on the device. Sometimes, the tourist may not find the app installed on the device because the name and/or the icon is different from that shown in Google Play

Furthermore, the following specific recommendations were obtained from the heuristic evaluation:

(i) Avoid confusion and ambiguity: both information and the names of the options in a menu of apps for tourism should be easily understood by tourists, without ambiguity (consistency is important). For instance, if a menu contains two options named as “POI” and “Attractions,” the tourist may not be able to know what each option will show, i.e., what is the difference between “POI” and “Attraction”

(ii) Show relevant and well-structured information: information of POIs should be always present, not just their name, containing information and a description about it. Furthermore, if there are reviews and/or user opinions, they should also be shown. Avoid showing this information in an unstructured manner

(iii) Show on a map the basic and needed information of POIs, distinguishing graphically their type (museum, monument, etc.). Include a mechanism to show detailed information for a given POI

(iv) Show more than just one POI on the map, as well as their distances to the tourist, because this is valuable information for tourists

(v) Avoid showing information or options that are unnecessary for the user. For instance, avoid full screen ads whenever possible. If ads are included, then a paid version of the app should be available for tourists to use the app without ads. Avoid showing two or more options with the same functionality

(vi) Choose adequately colours and fonts: avoid using too many different colours with no sense and try to keep screens consistent and well-designed. Fonts should be consistent, and font size should be big enough to be read

(vii) Avoid mixing commercial information (about restaurants, hotels, etc.) with tourist information (museums, monuments, etc.). This information must be clearly differentiated

(viii) Provide the back button with the default and predictable behaviour to do not disorient the tourist while using the app

(ix) Make the buttons of the tourist guide app big enough to facilitate clicking

(x) Take into account that tourists are from different countries and provide personalization of units (such as currency or distance measurements)

(xi) Design carefully the screens of the tourist guide in order to provide efficient input of data; e.g., the keyboard should not overlap input fields, and the tourist should not have to click twice on an input text field to be able to use the keyboard

(xii) The design should be good and consistent. For instance, if the tourist uses options that are external to the app (such as search for car rental) and a new window is opened, he/she should feel like continuing using the app, i.e., not losing the context. Special care has to be put in the design to avoid the tourist gets lost while using the app

(xiii) An efficient search for monuments and/or attractions should be provided, not only just showing them in a list but also the tourist must be able to search by name or type

This paper has a limitation due to the usage of the protocol: mobile apps are continuously changing in markets, so the applications analyzed with this protocol depend on the time the process is performed. However, it is very useful to get an idea of the general usability issues that tourist guide apps usually have. Furthermore, another limitation is that main functionalities have been obtained from literature, but it would be interesting to perform a survey of markets and taking into account online ratings for tourist apps. Finally, this study does not analyze accessibility aspects for users with special needs, which is a very important aspect.

6. Conclusion

In this paper, we have followed a protocol that has already been successfully used in other studies to discover usability issues in many different fields, but in this case, it was applied to mobile tourist guides. Firstly, potentially relevant apps were found through a search performed in Google Play for Android™ and then lite versions of those apps were removed. Next, the main functionalities that tourist guides should have were identified, as well as the secondary functionalities. Finally, a KLM analysis and a heuristic evaluation were performed to discover the most frequent usability issues in tourist guide apps in order to get recommendations for designing and developing mobile tourist guides.

As a result, this work provides some recommendations or guidelines for developers and designers of apps for tourism. Meeting these recommendations will increase the usability of their apps, which in turn will increase their use, and as a result, it is expected that the tourism will also increase.

As a future work, we are planning to identify new main functionalities by taking into account the more downloaded or rated apps, or by performing a survey of markets. We would also like to continue experimenting with other usability heuristics, based on the Nielsen’s heuristics [34–36], and others specific for mobile devices [37–39]. Developing a tourist guide following the recommendations presented in this paper is also planned, then performing an analysis with real users to compare the usability of the new app with those published in Google Play.

Acknowledgments

We would like to acknowledge the support by the Junta de Comunidades de Castilla-La Mancha for funding the first author with a postdoctoral training grant for carrying out this work.

References

[1] ITU, "The key 2005-2018 ICT Data for the world: Mobile-cellular subscriptions," 2018. March 2019, https://www.itu.int/en/ITU-D/Statistics/Documents/statistics/2018/ITU_Key_2005-2018_ICT_data_with%20LDCs_rev27Nov2018.xls

[2] D. Wang, S. Park, D. R. Fesenmaier, "An examination of information services and smartphone applications," Paper Presented at the 16th Annual Graduate Student Research Conference in Hospitality and Tourism, .

[3] R. Want, "When cell phones become computers," IEEE Pervasive Computing, vol. 8 no. 2,DOI: 10.1109/mprv.2009.40, 2009.

[4] E. Garcia-Lopez, A. Garcia-Cabot, L. de-Marcos, "An experiment with content distribution methods in touchscreen mobile devices," Applied Ergonomics, vol. 50, pp. 79-86, DOI: 10.1016/j.apergo.2015.02.009, 2015.

[5] M. A. Cusumano, "Platforms and services," Communications of the ACM, vol. 53 no. 10, pp. 22-24, DOI: 10.1145/1831407.1831418, 2010.

[6] J. E. Dickinson, K. Ghali, T. Cherrett, C. Speed, N. Davies, S. Norgate, "Tourism and the smartphone app: capabilities, emerging practice and scope in the travel domain," Current Issues in Tourism, vol. 17 no. 1, pp. 84-101, DOI: 10.1080/13683500.2012.718323, 2014.

[7] W. Zheng, R. Zhou, Z. Zhang, Y. Zhong, S. Wang, Z. Wei, H. Ji, "Understanding the tourist mobility using GPS: how similar are the tourists?," Tourism Management, vol. 71, pp. 54-66, DOI: 10.1016/j.tourman.2018.09.019, 2019.

[8] B. Leger, "26% of mobile apps downloaded in 2010 were used just once," 2011. July 2015, http://info.localytics.com/blog/first-impressions-26-percent-of-apps-downloaded-used-just-once

[9] Seggitur, Estudio de Mercado de Apps Turísticas, 2013.

[10] C. A. Scolari, M. Guerrero, M. López, J. Fernández, Aplicaciones Móviles de los Destinos Turísticos Españoles, Proyecto CODETUR, 2013.

[11] Y. H. Kim, D. J. Kim, K. Wachter, "A study of mobile user engagement (MoEN): engagement motivations, perceived value, satisfaction, and continued engagement intention," Decision Support Systems, vol. 56, pp. 361-370, DOI: 10.1016/j.dss.2013.07.002, 2013.

[12] A. Dirin, T. H. Laine, A. Alamäki, "Managing emotional requirements in a context-aware mobile application for tourists," International Journal of Interactive Mobile Technologies (iJIM), vol. 12 no. 2,DOI: 10.3991/ijim.v12i2.7933, 2018.

[13] E. Not, A. Venturini, "Discovering functional requirements and usability problems for a mobile tourism guide through context-based log analysis," Information and Communication Technologies in Tourism 2013, pp. 12-23, 2013.

[14] N. Ahmad, A. Rextin, U. E. Kulsoom, "Perspectives on usability guidelines for smartphone applications: an empirical investigation and systematic literature review," Information and Software Technology, vol. 94, pp. 130-149, DOI: 10.1016/j.infsof.2017.10.005, 2018.

[15] A. A. K. Azizi, I. N. F. M. Fuad, M. Abdullah, M. N. A. Aziz, "Design and evaluation of a mobile user Interface usability," Idealogy, vol. 3 no. 2, pp. 253-262, 2018.

[16] U. Gretzel, D. R. Fesenmaier, "Information technology: shaping the past, present and future of tourism," The SAGE Handbook of Tourism Studies, pp. 558-580, 2009.

[17] I. Mickaiel, "Mobile the new black for travel," 2011. July 2015, http://www.zdnet.com.au/mobile-the-new-black-for-travel-339321469.htm

[18] IDC, "Smartphone OS market share," 2019. July 2015, https://www.idc.com/promo/smartphone-market-share/os

[19] ISO, 9241-11. Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs), 1998.

[20] C. Martin, D. Flood, R. Harrison, "A protocol for evaluating mobile applications," Information Systems Research and Exploring Social Artifacts: Approaches and Methodologies: Approaches and Methodologies, pp. 398-414, 2012.

[21] D. Flood, R. Harrison, C. Martin, K. McDavid, "A systematic evaluation of mobile spreadsheet apps," Paper presented at the IADIS International Conference Interfaces and Human Computer Interaction, .

[22] E. Garcia, C. Martin, A. Garcia, R. Harrison, D. Flood, "Systematic analysis of mobile diabetes management applications on different platforms," Paper Presented at the 7th Conference on Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society: Information Quality in e-Health, .

[23] B. Brown, M. Chalmers, "Tourism and mobile technology," Paper Presented at the Eighth European Conference on Computer Supported Cooperative Work, .

[24] K. Cheverst, N. Davies, K. Mitchell, A. Friday, "Experiences of developing and deploying a context-aware tourist guide: the GUIDE project," Paper Presented at the 6th Annual International Conference on Mobile Computing and Networking, .

[25] G. Marsala, Aplicaciones Móviles para el Sector Turístico, 2012.

[26] S. K. Card, A. Newell, T. P. Moran, The Psychology of Human-Computer Interaction, 1983.

[27] J. Nielsen, R. Molich, "Heuristic evaluation of user interfaces," Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90,DOI: 10.1145/97243.97281, .

[28] S. C. Lee, S. H. Yoon, Y. G. Ji, "Modeling task completion time of in-vehicle information systems while driving with keystroke level modeling," International Journal of Industrial Ergonomics, vol. 72, pp. 252-260, DOI: 10.1016/j.ergon.2019.06.001, 2019.

[29] F. R. Jeddi, E. Nabovati, R. Bigham, R. Farrahi, "Usability evaluation of a comprehensive national health information system: a heuristic evaluation," Informatics in Medicine Unlocked, vol. 19, article 100332,DOI: 10.1016/j.imu.2020.100332, 2020.

[30] J. Nielsen, "How to conduct a heuristic evaluation," 1995. http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation

[31] J. Nielsen, "Severity ratings for usability problems," Papers and Essays, vol. 54, 1995.

[32] E. Bertini, T. Catarci, A. Dix, S. Gabrielli, S. Kimani, G. Santucci, "Appropriating heuristic evaluation for mobile computing," International Journal of Mobile Human Computer Interaction, vol. 1 no. 1, pp. 20-41, DOI: 10.4018/jmhci.2009010102, 2009.

[33] D. Quiñones, C. Rusu, "Applying a methodology to develop user eXperience heuristics," Computer Standards & Interfaces, vol. 66,DOI: 10.1016/j.csi.2019.04.004, 2019.

[34] L. Kuparinen, J. Silvennoinen, H. Isomäki, "Introducing usability heuristics for mobile map applications," Paper Presented at the Proceedings of the 26th International Cartographic Conference, .

[35] F. Martinsa, C. Figueiredoa, R. Raposoa, P. Beçaa, "Evaluating the usability of a mobile tourist guide: the mesh-t outdoor mobile application," Paper presented at the proceeding of: ENTER 2013 conference on information and communication Technologies in Tourism, .

[36] M. K. Othman, F. Mahudin, C. H. Ahaguk, M. F. A. Rahman, "Mobile guide technologies (smartphone apps): Collaborative Heuristic Evaluation (CHE) with expert and novice users," 2014 3rd International Conference on User Science and Engineering (i-USEr),DOI: 10.1109/iuser.2014.7002708, .

[37] R. P. da Costa, E. D. Canedo, R. T. de Sousa, R. de Oliveira Albuquerque, L. J. Garcia Villalba, "Set of usability heuristics for quality assessment of mobile applications on smartphones," IEEE Access, vol. 7, pp. 116145-116161, DOI: 10.1109/ACCESS.2019.2910778, 2019.

[38] M. A. D. Dourado, Usability Heuristics for Mobile Applications a Systematic Review, 2018.

[39] G. Joyce, M. Lilley, T. Barker, A. Jefferies, "Mobile application usability: heuristic evaluation and evaluation of heuristics," Advances in Human Factors, Software, and Systems Engineering, pp. 77-86, 2016.