Content area
The importance of considering people with impairments’ needs in the early stage of web application design increases with technological advances. Human–computer interaction researchers have developed many applications and tools to improve inclusive web design and alleviate struggles related to the accessibility of the web. Recent Arabic web accessibility studies have shown that the Arabian countries are falling behind western countries in many aspects of web accessibility. One of these aspects is the lack of tools and applications that assist web developers in designing inclusive websites and web applications. The purpose of this research is to implement a web tool, a previously suggested solution, to improve Arabic web accessibility and examine the effects of using an Arabic web accessibility tool as a solution to enhance Arabic web accessibility. An Arabic web accessibility testing tool called AATT was developed, tested, and improved. This research will report the final analysis and results of using the improved version of AATT. Thirty-one web developer participants took part in this study. Twenty-two of the survey respondents evaluated the tool while nine used it to improve some websites. The nine participants were chosen from different organizations in Saudi Arabia. The participants are web developers who manage organizations’ websites: two universities, a hospital, four charities, four cultural content, and one personal website. Some of these organizations’ websites are administrated by one or more web developers. One of the web developers administers our charity websites. So, twelve websites’ homepages were tested and improved using AATT. The findings show that AATT enhanced the accessibility of Arabic websites, provided user satisfaction, and improved web accessibility awareness.
Introduction
The importance and the need for web accessibility improvement has become an increasingly important subject for researchers and disability advocates in recent years. Several web accessibility guidelines have been developed for web developers to follow while creating accessible web content, websites, web applications, and tools. This means people with disabilities can access the web and benefit from all information and services on the web. Web Content Accessibility Guidelines (WCAG), Sect. 508 Standards of Rehabilitation Act, The International Organization for Standardization Standards, The Accessible Rich Internet Applications (WAI-ARIA) Guidelines, and Authoring Tool Accessibility Guidelines are some of the web accessibility guidelines that support creating accessible web content. In addition, several web accessibility evaluation tools have been developed to assess the quality of web content, applications, and sites. These tools assist web developers in understanding and solving accessibility issues. The guidelines and evaluation tools are mainly developed and being used by western countries. Arabic web accessibility, on the other hand, has not been given much attention.
Arabic web accessibility studies have been conducted to assess the accessibility of different types of websites. Research has determined that inaccessibility in Arabic websites is often a result of a lack of awareness of web accessibility guidelines and accessibility policies. These findings demonstrate the necessity of developing an inclusive web (Abanumy et al. 2005, Abu Shawar 2015; Al-Faries et al. 2013; Al-Khalifa 2010; Alayed et al. 2016, AlMeraj et al. 2020; Liginlal and Al-Muftah 2017; Masood Rana et al. 2011). Besides the issue of web accessibility awareness, there is a need to create evaluation tools, guidelines, and authoring tools in the Arabic language. A recommendation that has been echoed by Saleem (2018) showed the importance of developing Arabic guidelines to support Arabic web accessibility (Saleem, 2018). Although many well-developed tools evaluate websites using different accessibility guidelines, none of the tools is designed for use in Arabic, which may prevent Arabic developers from applying web accessibility features. In other words, accessibility issues in Arabic websites may be caused by a lack of access to web accessibility tools that help Arabic developers understand and solve accessibility issues related to inclusive design. A research study presented by Alnahari et al. (2021) evaluated 54 Arabic websites in the Arabian Gulf countries and showed the need for Arabic accessibility improvement. They also provided some recommendations and solutions. To improve Arabic web accessibility, Alnahari and Chakraborty (2019) developed an initial version of the tool as one possible solution to the problem, and this paper has made improvements to it. The usability study was conducted to understand the tool’s effectiveness and to gather user requirements. The results of the initial version are encouraging; however, some improvements are needed to meet the users’ expectations of the new tool. The preliminary assessment of the tool was used to adjust the latest version of the tool based on the users’ requirements. Changes to the tool’s interface were made to improve the feel and look. Arabic accessibility error messages were edited and enhanced to assist the understanding of accessibility issues better. The translation was checked and reviewed by some Arabic-speaking colleagues. In this paper, we present the analysis of using an Arabic accessibility tool and its effects on Arabic web accessibility, user satisfaction, and awareness of web accessibility.
Literature review
According to Alayed et al. (2016) and Beseiso et al. 2010), Arabic is one of the most widely spoken languages in the world, with more than 200 million speakers in about twenty-four countries. The exact number of people with disabilities in Arabic nations is unknown; however, the estimated disability prevalence rate is between 10% (Kan et al. 2011) and 15% (Hadidi and Al Khateeb 2015). The number of people with disabilities in Saudi Arabia is around 720,000, constituting 4% of the total population (Akram and Bt 2017; Al-Gain et al. 2002).
Despite these numbers, an estimated 90% of websites and 98% of government sites are inaccessible to users with disabilities (Weber 2016) because these sites were mainly designed for neurotypical users. Alnahari and Chakraborty 2019) stated that the number of universities in Saudi Arabia had increased recently, yet there are no clear accessibility guidelines. Masood Rana et al. (2011) showed that 80% of Saudi universities had low accessibility requirements and did not meet the minimum standard accessibility guidelines. The study found that web developers and policymakers in Saudi Arabia lack the knowledge and the understanding of WCAG. One study aimed to evaluate education websites conducted by AlMeraj et al. (2020). The research evaluated the accessibility of 41 higher education web pages using automated tools and found that none of the education websites met WCAG 2.0 (A). The researchers argue that there is a pressing need to solve accessibility issues in Kuwait’s education websites to assist individuals with disabilities (AlMeraj et al. 2020).
To understand the reasons behind this lack of accessibility and usability of e-government sites, Abanumy et al. (2005) surveyed governmental website developers in Saudi and Oman. They found that 70% of governmental web developers believed that inaccessibility is caused by a lack of awareness of website accessibility guidelines; however, 65% of the respondents identified the problem as the non-existence of accessibility policies in these countries mentioned above. Saleem (2018) conducted a study to evaluate e-government websites’ accessibility in Kuwait, United Arab Emirates, and Qatar and showed that none of the websites met the minimum requirements of WCAG 2.0. The author concluded that one of the reasons for the web accessibility issue was the Arabic language barrier. The lack of Arabic web context and the Arabic accessibility guidelines can make it difficult for Arab web developers to build accessible websites (Saleem 2018). Arabic accessibility guidelines with enough examples could help web practitioners understand and raise awareness of web accessibility.
A study by Liginlal and Al-Muftah 2017) examined the state of web accessibility of Qatar’s government website. The study results demonstrate the need for creating awareness between key stakeholders and executives to improve the policy framework related to web accessibility. In Oman, the policy suggests improving government websites’ accessibility and e-services within these websites. According to Omari (2013), government websites in Oman and Saudi Arabia still need to be improved to allow all users access — including those with disabilities. A recent study was conducted by Alnahari et al. (2021) to evaluate the homepages of 54 websites in the six Arabic Gulf countries, including three government websites, three university websites, and three healthcare websites from each representative country. The websites were chosen from the region’s top-ranking universities, government, and healthcare facilities. The results showed that some websites have fewer accessibility issues, while others have a significant number of known problems. Almost every website tested had issues with text alternatives for non-text content. Other common web accessibility problems included navigation of the interface, making the content easier for users to see and hear, separating foreground from background, and creating content that can be presented in different ways. These errors cause issues for people with disabilities trying to access a website directly or through assistive technologies.
One of the major reasons that Arabic websites remain less accessible is that website designers often lack access to tools that can help them improve accessibility. Considering the range of tools (detailed below) that assist western web designers in meeting content accessibility guidelines, these tools utilize WCAG and Sect. 508 accessibility guidelines. They help designers evaluate websites and web applications and highlight accessibility issues that may prevent people with disabilities from accessing and using those websites. Moreover, they are particularly useful to help novice designers test their websites for accessibility issues. Keep in mind, though, that most of these tools use the English language and that none of the evaluation tools has been developed in Arabic. This makes it more difficult for Arabic-language designers to meet accessibility guidelines. However, most of these tools use the English language and none of the evaluation tools has been developed in Arabic.
Some accessibility tools provide automated evaluation and recommendations that assist with conforming to web content accessibility guidelines. They also provide easy access to a set of specific checking capabilities such as the web accessibility toolbar. For example, authoring tools, such as Dreamweaver, offer tools that can be used for visualizing accessibility issues, such as tools that attempt to show how a webpage appears to color blind people. Other tools, such as Lynx browser, were not designed for accessibility but can be used to check for specific accessibility issues. This browser allows a developer to view the linearized version of a webpage so they can better understand how the page will be read for blind people via a screen reader (Petrie et al. 2006). Pearson et al. (2011) developed a similar tool to assist novice auditors in the process of accessibility evaluation of websites. The tool uses a structured walkthrough approach to guide novice auditors through several checks to implement web accessibility guidelines and perform a comprehensive accessibility evaluation (Pearson et al. 2011). Herramhof et al. (2006) developed a test case management tool for accessibility testing. The EU-funded project BenToWeb aims to create test suites for Web Content Accessibility Guidelines 2.0. One of the tools developed by the team is called Parsifal which is a desktop application that allows developers to edit text description files. The test description files control an Extensible Markup Language (XML) layer that contains descriptive information about certain test cases. The other tool that was developed by the team is called Amfortas (Rangin 2006). The International Center for Disability Resources in cooperation with Hi-Software developed an automated testing tool called Cynthia Says is an example of the automated testing tools that have been developed by the International Center for Disability Resources in cooperation with Hi-Software. This tool evaluates a web page and generates a report based on Sect. 508 standards and WCAG guidelines (Benavídez et al. 2006). HERA 2.0 is an improvement of the beta version of HERA.
Tenon is another accessibility tool that provides flexibility for users. The tool can evaluate a single webpage or a group of web pages and report the results. The guidelines that are used by Tenon include WCAG 1.0, WCAG 2.0, Sect. 508, and US federal procurement standards. Tenon’s licenses are commercial or enterprise. Tenon also allows the user to check a website for free using the demo version. Total Validator is another tool that provides accessibility evaluation. The tool evaluates a website for accessibility issues, CSS issues, broken link issues, HTML and XHTML coding issues, and spelling issues. The guidelines are the same as those used by Tenon. Total Validator can automatically check a single webpage and a group of web pages. The license is available for free with limited services and is also available for commercial use (Bigby 2019a, b). These are just a few examples of the tools that have been developed. Many other tools are available on the web.
The guidelines that are used in the above-mentioned tools were developed by the World Wide Web Consortium (W3C). These guidelines replaced the older versions of the WCAG guidelines (WCAG 1.0). WCAG 2.0 was published in 2006 and consists of four principles, 12 guidelines, testable statements within each guideline, and sufficient advisory techniques. The principles define the basics of web accessibility, which are perceivable, operable, understandable, and robust. Under each principle, there is a list of guidelines that web developers should follow to make content accessible to people with disabilities. The guidelines provide a framework that helps web developers understand accessibility objectives and implement appropriate techniques. The principles of WCAG 2.0 focus on understanding users (Bigby 2018). Perceivably, this first principle assumes that not every individual has the same sense when using the web. Therefore, a web developer must consider making the content perceivable by all people with different cognitive abilities. The second principle, being operable, emphasizes that a website must allow the user to navigate the content of the site not only with a mouse but also through a keyboard, allowing people who have motor impairments access to the content of the website. Websites must allow all people with different kinds of disabilities to access their content either directly or through assistive technologies, as discussed above. The third principle is that websites must be understandable. This principle is vital in making the first two previous principles work. Understandable websites include simple instructions, transparent terms, and straightforward functions. The fourth principle is that websites must be robust, meaning that a website must meet the recognized standard, such as using clean HTML and cascading style sheet (CSS) coding and allowing third parties to process websites using assistive technologies (Bigby 2018).
Several website evaluation studies have measured the usability of prominent Arabic-language websites by applying criteria mentioned above. Research by Al-Khalifa 2010) and Al-Faries et al. (2013) evaluated Saudi e-government sites and demonstrated the lack of accessibility requirements on those websites. Al-Khalifa (2012) assessed the accessibility of the top Saudi government services using web accessibility guidelines. The study aimed to identify violations for each guideline per principle. The most violated guideline was ensuring that a text alternative is provided for all non-text content, which relates to the principle of perceivability. Another frequently violated guideline was to ensure that all functionalities are accessible from a keyboard. This guideline relates to operable principles. Several other violations related to understandable and robust principles were found to pose significant accessibility problems in Saudi government websites (Al-Khalifa, 2012). The following are WCAG principles and checkpoints.
Perceivable
Provide text alternatives for any non-text content so that it can be changed into other forms people need, such as large print, Braille, speech, symbols, or simpler language.
Provide alternatives for time-based media.
Create content that can be presented in different ways (for example, simpler layout) without losing information or structure.
Make it easier for users to see and hear content, including separating foreground from the background.
Operable
Make all functionality available from a keyboard.
Provide users enough time to read and use the content.
Do not design content in a way that is known to cause seizures.
Provide ways to help users navigate, find content, and determine where they are.
Understandable
Make text content readable and understandable.
Make web pages appear and operate in predictable ways.
Help users avoid and correct mistakes.
Robust
Maximize compatibility with current and future user agents, including assistive technologies.
These guidelines have since been used to create the WCAG 2.1 Guidelines. These guidelines include the same principles, guidelines, and success criteria of WCAG 2.0, but WCAG 2.1 extends WCAG 2.0 with 17 additional success criteria (Bigby 2018). The additional success criteria support mobile accessibility, as well as supporting people with low vision, and cognitive or learning disabilities (Henry 2021). In order for Arabic website designers to meet accessibility goals, it will be necessary for them to have access to a tool that can help them identify these issues and use appropriate techniques to fix them.
Methodology and research design
The authors of this study created a tool called the Arabic accessibility testing tool (AATT) to help Arabic web designers meet accessibility guidelines. Because usability was an essential part of the interface design, it was developed in ways to help the user navigate the system easily. This tool was designed to provide Arabic web designers the support with accessibility issues they may have on their websites. This tool was also designed to provide users with recommendations and examples to improve their understanding of accessibility issues and troubleshooting abilities. During its testing, the tool was made available in both Arabic and English. The Web Content Accessibility Guideline WCAG 2.0 (A) was translated into Arabic, and the translation was reviewed multiple times by the author and Arabic-speaking colleagues to ensure the quality of the translation. The Arabic tool used web service. Because many accessibility tools provide web services that can be used in different applications, AChecker’s web service was used in this research. AChecker was an accessibility testing tool that provided website accessibility evaluations in different languages but not in Arabic. AChecker also provided a public API web service to allow developers to integrate accessibility review directly into their applications (Gay et al. 2010). The tool was shut down in 2021 after the end of the study in May 2021. A web service is a service that allows applications to make remote procedure calls (RPCs). Web services receive requests to perform certain tasks and return responses over the HTTP protocol, usually in XML format (Abascal et al. 2004b, a). AATT stored all the accessibility checks found in WCAG 2.0 level one guidelines in both Arabic and English. The API service used in this study provided the web accessibility assessment result in the English language in XML format based on WCAG 2.0 level one. Once the application received the assessment from the web service, it would extract, parse, and compare the accessibility review to the checks that are stored in the application. The application would then provide the Arabic equivalent to the English evaluation. The tool was tested and evaluated by Arabic-speaking web developers. Fig. 1 shows the interface of the tool in English. Figs. 2 and 3 show the interface of the tool in Arabic. Thirty-one web developer participants evaluated the tool, and nine of them improved websites by solving accessibility issues generated from AATT. The study was conducted online. Participants were asked to complete the survey and analysis questionnaires electronically using Qualtrics software.
Fig. 1 [Images not available. See PDF.]
The interface of the new version of the AATT in English
Fig. 2 [Images not available. See PDF.]
The interface of the new version of AATT in Arabic
Fig. 3 [Images not available. See PDF.]
AATT output after testing a website’s page
Thirty-one adult Arab web developers participated in the study. The level of web development experience in the participants was self-reported as either intermediate, advanced, or expert. Arab web developers were asked to participate in the study to evaluate the Arabic accessibility tool and use it to make changes to their websites. Twenty-two participants were recruited through online computer science groups where Arab computer scientists share their experiences. A public message requesting volunteer participation in the study was sent to all members of these groups. The participants agreed and provided their consent to participate in the study. They evaluated the tool by filling out the surveys and provided their opinions on the tool. At the same time, nine of the participants were chosen from different sectors in Saudi Arabia. They were selected to provide an evaluation on the tool, in addition to using AATT to improve their websites. The demographic information of the web developers who only evaluated the websites was not considered. Participants were only required to have some level of experience with web development, while the demographic information for the web developers who evaluated and made changes to the websites was considered as they provided a further contribution to the study. All nine participants were males with four individuals between 25 and 35 years old, two between 36 and 45 years old, two between 46 and 60 years old, and one web developer older than 60. Four of the responding web developers held bachelor’s degrees and five held master’s degrees. Two participants rated their web development experience as intermediate, three rated themselves as advanced, and four rated themselves as experts.
Evaluating a website using AATT
The Integrated Definition Method (IDEF0) diagram (see Fig. 4) explains the processes that AATT follow to generate the results on the interface:
A user enters a URL of a website to get an accessibility review of the site.
The URL is then validated for correctness before proceeding to the next step; otherwise, the system returns an error message to the user.
If the URL is valid, the system sends the URL to a web service.
The API service evaluates a website and returns an XML file that contains a list of all identified accessibility issues.
The application imports and stores the XML file.
The application parses the data in the file and extracts the accessibility review into different categories.
The application produces all the information into Arabic.
The accessibility review report is then shown on the interface of AATT in both Arabic and English.
Fig. 4 [Images not available. See PDF.]
The process of evaluating a website using AATT
Research hypotheses
In this study, a list of questions was given to the participants who modified their websites using AATT in order to measure their level of awareness of web accessibility. We hypothesized the following:
H01: Web accessibility of the selected websites will be different after applying the AATT tool.
H02: Creating an Arabic accessibility tool improves user satisfaction.
H03: Creating an Arabic accessibility tool improves web developer’s awareness of web accessibility.
Participants use the tool to evaluate websites, and some of them were asked to fix accessibility issues found on the sites. To address our first hypothesis, the participants evaluated the objective of AATT. They were primarily asked to provide screenshots of the webpage before and after solving the accessibility issues the AATT identified.
Because user satisfaction is a critical aspect of usability testing, our second hypothesis aimed to determine whether developers were satisfied with the developed AATT tool. This was measured using a list of open-ended, opinion-based questions.
Lastly, web developers’ knowledge and awareness of accessibility issues were evaluated using the third hypothesis. This hypothesis was based upon the work of Sloan (2006) who developed accessibility audits for web developers to understand the impacts of audits. The study aimed to assist web developers in adopting web accessibility techniques while also improving developers’ knowledge about the importance of web accessibility for people with disabilities. Their findings showed a positive correlation between fewer years of experience and the impact of the audits on the awareness-raising. Another study conducted by Antonelli et al. (2018) was designed to examine knowledge and awareness of web accessibility in Brazilian web developers. The study found that the web developers who participated in the exploratory research had never developed accessible websites. As a result, the authors believe it is essential to create policies to improve web accessibility and raise awareness of inclusive web design (Antonelli et al., 2018).
Tasks and procedure
Participants were asked to provide informed consent using an online form that also served as an online pre-test questionnaire. They were then asked to assess the tool by copying and pasting a link for an Arabic website, reading the information and accessibility evaluation provided by the tool, and then filling out a post-test survey. The pre-test questionnaire allowed the researcher to collect demographic information about the participants, while the post-testing questionnaire provided insight and feedback regarding user satisfaction, web accessibility awareness, and the importance of AATT. The questionnaires were developed in both English and Arabic to allow participants who spoke only Arabic to participate in the study.
Nine of the participants were asked to evaluate AATT in addition to solving the accessibility problems (i.e., known problems and likely problems) generated from the tool. Twelve web pages were evaluated using AATT, and then a list of accessibility issues was returned. The participants were asked to solve as many problems as they could, and all of them were able to solve most of the issues. Table 1 shows the web pages’ URLs and number of errors before and after using the tool.
Table 1. Number of errors before and after applying AATT for 12 webpages
Website | Before corrections | After corrections |
|---|---|---|
1 | 12 | 0 |
2 | 5 | 0 |
3 | 9 | 0 |
4 | 3 | 0 |
5 | 2 | 0 |
6 | 3 | 0 |
7 | 15 | 10 |
8 | 4 | 3 |
9 | 2 | 0 |
10 | 6 | 0 |
11 | 3 | 0 |
Analysis and results
A within-group study design was adopted to measure the importance of the Arabic accessibility tool and the impact of the tool on improving the web accessibility of some selected websites. We examined factors such as user satisfaction and awareness to answer the hypotheses. The resulting data were analyzed using SPSS. Non-parametric tests were used due to the small sample size. Sample normality was established prior to inferential statistical analysis, and both the paired t-test and the Wilcoxon signed-rank test were used. Categorical variables, frequencies, counts, and percentages were calculated for descriptive statistics. Qualtrics was used for qualitative analysis.
To study the impact of the Arabic accessibility testing tool on web accessibility, 12 websites were tested using the tool, and the necessary modifications were made on these websites based on the provided suggestions. The number of errors before and after using the tool was calculated. Table 2 shows that the mean number of errors before using the tool was six and which was reduced to one after the modifications. The range of errors before using the tool was 13 (range: 2–15), and the range after using the tool was 10 (range: 0–10). As shown in Fig. 5.
Table 2. Descriptive analysis for the number of errors before using AAT
N | 12 |
|---|---|
Mean | 6.08 |
Trimmed mean 5% | 5.8148 |
Median | 4.5 |
IQR | 6 |
Std. dev | 4.25 |
Std. err | 1.23 |
Upper 95% CI mean | 3.38 |
Lower 95% CI mean | 8.79 |
Shapiro–Wilk W test | P-value = 0.07 |
Fig. 5 [Images not available. See PDF.]
The numbers of errors before and after using AATT
To test the first hypothesis, data were assessed for normality using the Shapiro–Wilk test. Table 2 shows that the pre-test data were normally distributed (P-value = 0.07) as was the difference between the number of errors before and after using the AATT (P-value = 0.23) as shown in Table 3. In contrast, the number of errors following the use of the tool was not normally distributed (P-value < 0.001). Given that the difference between the number of errors prior to tool implementation and the number of errors after tool implementation was normally distributed, a paired t-test was used to evaluate the first hypothesis as shown in Table 5. The test in Table 2 and Table 4 also shows a significant difference of five errors in the mean number of errors before and after using AATT (P-value < 0.001 and 95% CI: 2.8 and 7.2) as shown in Table 4. Thus, we concluded that there were significant differences in web accessibility of the selected websites after applying AATT (Table 5).
Table 3. Descriptive analysis for the difference between the number of errors before and after using AAT
N | 12 |
|---|---|
Mean | 1.08 |
Trimmed mean 5% | 0.6481 |
Median | 0 |
IQR | 0 |
Std. dev | 2.94 |
Std. err | 0.85 |
Upper 95% CI mean | − 0.78 |
Lower 95% CI mean | 2.95 |
Shapiro–Wilk W test | P-value = 0.00 |
Table 4. Descriptive analysis for the number of errors before using AAT
N | 12 |
|---|---|
Mean | 5 |
Trimmed mean 5% | 4.83 |
Median | 4 |
IQR | 5 |
Std. dev | 3.41 |
Std. err | 0.98 |
Upper 95% CI mean | 2.83 |
Lower 95% CI mean | 7.17 |
Shapiro–Wilk W test | P-value = 0.144 |
Table 5. Paired t-test
Sample | Sample size | Paired t-test value | P-value |
|---|---|---|---|
Difference = before − after | 12 | 5.078 | 0.001 |
Hypothesis two: AATT usability and web developer satisfaction
To assess the quality of the tool, a total of 31 responses were collected from a post-testing questionnaire. Among 31 responders, nine (29%) modified their websites based on the suggestions provided by AATT, and 22 (71%) only evaluated the tool without implementing any modifications. Each participant responded to a survey that consisted of 26 questions; some were ranked on a five-point Likert scale (5 = “Strongly Agree”; 1 = “Strongly Disagree”), while the remaining questions were open-ended.
To determine whether the AATT improved user satisfaction, nine variables were used to test the null hypothesis that the AATT is not useful in improving user satisfaction. Items that measured user satisfaction on a Likert scale included the following statements:
The interface of the tool is easy to navigate.
The tool was easy to use.
The Arabic instructions on the interface were easy to follow and understand.
The Arabic instructions helped me complete the tasks.
The Arabic accessibility issues were clear and understandable.
I recommend the tool to people who are interested in web accessibility.
I think AATT will make a difference in improving Arabic web accessibility.
I think the tool interface looks good.
I think that accessibility errors are easy to solve.
Cronbach’s α was calculated to ensure that all nine items measure the same concept (user satisfaction) and measure the scale’s internal consistency. Cronbach’s alpha coefficient value was 0.81, indicating that the measure had good internal consistency for the proposed scale.
Analysis began by calculating the descriptive statistics for each survey question. The overall satisfaction was very high, with more than 90% of participants indicating that they were satisfied with the tool (satisfaction was defined as responding with “Somewhat Agree” or “Strongly Agree” to questions). Of the total number of respondents, 28 web developers (90.3%) agreed that the tool’s interface was easy to navigate, and 29 (93.6%) found it easy to use. When assessing the quality of the Arabic translation, 29 (93.6%) indicated they were satisfied with translation quality of both the tool’s main interface and the instructions on the interface. Additionally, 26 (83.9%) participants reported that the tool was clear and understandable indicating that the quality of the translation was high while also highlighting aspects of the tool that would benefit from improvement. Further, 22 (71%) participants agreed that the tool inference was aesthetically pleasing, with 7 (23%) indicating that they disagreed.
Overall, 30 web developers (96.8%) indicated that they would recommend the tool to other people interested in web accessibility and believed that the AATT would improve Arabic web accessibility. See Table 6 for a complete overview of the descriptive statistics for these nine items.
Table 6. Overview of the descriptive statistics for nine questions
Q# | Strongly agree N (%) | Somewhat agree N (%) | Neither agree nor disagree N (%) | Somewhat disagree N (%) | Strongly disagree N (%) | Mean (SD) |
|---|---|---|---|---|---|---|
Q1 | 20 (64.52) | 8 (25.81) | 1 (3.23) | 2 (6.45) | 0 (0) | 4.5 (0.84) |
Q2 | 24 (77.42) | 5 (16.13) | 2 (6.45) | 0 (0) | 0 (0) | 4.7 (0.59) |
Q3 | 19 (61.29) | 10 (32.26) | 1 (3.23) | 1 (3.23) | 0 (0) | 4.5 (0.72) |
Q4 | 19 (61.29) | 10 (32.26) | 1 (3.23) | 1 (3.23) | 0 (0) | 4.5 (0.72) |
Q5 | 15 (48.39) | 11 (35.48) | 3 (9.68) | 1 (3.23) | 1 (3.23) | 4.2 (0.99) |
Q6 | 18 (58.06) | 12 (38.71) | 1 (3.23) | 0 (0) | 0 (0) | 4.5 (0.57) |
Q7 | 20 (64.52) | 10 (32.25) | 1 (3.23) | 0 (0) | 0 (0) | 4.6 (0.56) |
Q8 | 13 (41.94) | 9 (29.03) | 2 (6.45) | 3 (9.86) | 4 (12.90) | 3.8 (1.43) |
Q9 | 11 (35.48) | 20 (64.52) | 0 (0) | 0 (0) | 0 (0) | 4.3 (0.88) |
To better understand the degree of the users’ reported satisfaction, the agreement scale was divided into 5-point Likert scale intervals with each level having a value range of 0.8, with values beginning at one and ending at five, as recommended by Pimentel (2010). Values between 1.0 and 1.80 reflected strong disagreement with the statement; 1.81 and 3.60 indicated somewhat disagreement; 2.61 and 3.40 indicated a neutral opinion; 3.41 and 4.20 indicated some agreement; and 4.21 to 5 indicated a strong agreement. Based on these recommendations, the value of 3.41was considered the threshold for satisfaction as it indicates the lowest level of positive agreement (see Table 7). Refer to Pimentel (2010) for a thorough overview of threshold values.
Table 7. Five-Likert point used in this study
Description | Likert scale | Interval |
|---|---|---|
Strongly disagree | 1 | 1.0–1.80 |
Somewhat disagree | 2 | 1.81–2.60 |
Neither agree nor disagree | 3 | 2.61–3.40 |
Somewhat agree | 4 | 3.41–4.20 |
Strongly agree | 5 | 4.21–5.00 |
Overall, the data indicated high user satisfaction with the mean above 3.40 in all items shown in Table 6. Participants reported being most satisfied with the tool’s user-friendliness with a mean score of 4.7. The remaining questions were also rated highly, representing high satisfaction with the interface, ease of use, and clarity of the Arabic instructions. Participants were also highly likely to report that they would recommend the tool to others and that they felt that the tool would improve web accessibility for Arabic speakers. There was a slight decrease in mean satisfaction for questions five and eight; however, mean scores were still above the minimum threshold of satisfaction. The following graphs are divided into two parts based on participants’ actions. The first two bars represent the web developers who only evaluated the tool, while the second shows answer frequency for web developers who evaluated the tool and modified the websites. Fig. 6 shows the level of the agreement for each item.
Fig. 6 [Images not available. See PDF.]
The frequencies of the satisfaction levels with the AATT
Table 6 shows that the median satisfaction values for all items were significantly higher than the satisfaction threshold of 3.40. A non-parametric test was used to test the second hypothesis because of the small sample size. Using a Wilcoxon signed-rank test, the results show that participants were generally satisfied with the tool’s interface and the difference is statistically significant (P-value < 0.001). The same is also applicable for the degree of agreement to the statement that the tool was easy to navigate, where the difference between the median scale of agreement and the threshold is statistically significant (P-value < 0.001). Participants also expressed a high level of satisfaction regarding the Arabic instructions and the ease to understand and follow; this question also had a statistically significant degree of satisfaction beyond the threshold (P-value < 0.001). Respondents also expressed their interest in the tool by demonstrating a great willingness to recommend it to other web developers who are interested in web accessibility (P-value < 0.001) and agreeing on its importance in improving Arabic web accessibility (P-value < 0.001). The respondents had a lower level of satisfaction in terms of the interface look (P-value > 0.001) in the Wilcoxon signed-rank test and (P-value = 0.014 < 0.025) (see Table 8).
Table 8. P-values for one-tailed Wilcoxon signed-rank test
Question | One-tailed Wilcoxon signed-rank test P-value Ha: Median — 3.40 > 0 |
|---|---|
Q1 | 0.000 |
Q2 | 0.000 |
Q3 | 0.000 |
Q4 | 0.000 |
Q5 | 0.000 |
Q6 | 0.000 |
Q7 | 0.000 |
Q8 | 0.04 |
Q9 | 0.000 |
The mean score for the entire measure was calculated for each respondent. It was determined that the data violated the assumption of normality based on a Shapiro–Wilk test (P-value < 0.05). A Wilcoxon signed-rank test indicated that the overall mean level of satisfaction was significantly higher than the threshold value of 3.40. Also, to study the difference between the mean of the overall satisfaction degree and the threshold value of 3.40, the one-sample t-test was conducted, and it also showed that the mean overall satisfaction degree was 4.4 and it is statistically significantly above 3.40 (P-value < 0.000 and 95% CI between 3.80 and 4.70) as shown in Table 9. Based on this analysis, we can conclude that the AATT is effective in improving user satisfaction.
Table 9. Overall satisfaction and one-sample t-test P-value
Overall satisfaction mean | Maximum | Minimum | Std. deviation | Std. error | One-sample t-test P-value |
|---|---|---|---|---|---|
4.40 | 4.70 | 3.80 | 0.269 | 0.089 | 0.001 |
Hypothesis three: AATT and awareness of web accessibility
To assess the impact of the Arabic accessibility testing tool on web accessibility awareness, nine web developers were asked to respond to a questionnaire. Some of the questions asked participants to report demographic information to gather information on the participant’s age, gender, level of education, and level of experience in web development; the remaining were attitude, “Yes/No,” and multiple-choice questions.
Most participants (n = 7; 77.8%) indicated that they had a good understanding of web accessibility. However, when participants were asked to rate the applicability of a statement expressing a desire to learn more about web accessibility, all participants indicated some level of agreement by selecting either “Somewhat Agree” (n = 2) or “Strongly Agree” (n = 7). When participants were asked whether the organizations they worked for had plans to improve accessibility, most answered “No” (n = 6; 66.7%).
After using the AATT and applying the recommended changes to the websites, the participants were asked three questions to assess the perceived benefit of the tool. All respondents strongly agreed that the AATT gave them a better understanding of web accessibility issues on their employer’s website. Most participants (n = 8; 88.9%) felt that the tool would bring more awareness to web accessibility issues for people with disabilities, with only one participant indicating that they were unsure.
Using the tool also encouraged developers to learn more about web accessibility issues and better understand their role in improving websites’ accessibility. Eight participants (88.9%) strongly agreed, and one participant (11.1%) held a neutral attitude.
A Pearson’s correlation coefficient was calculated to determine if there was a relationship between how important participants thought the AATT was for improving accessibility and their motivation to learn more about web accessibility. A significant correlation was found (P < 0.025), such that those who thought the AATT was important for improving accessibility were more likely to report higher levels of motivation to learn more about web accessibility.
Finally, to determine whether using the AATT improved participants’ awareness of web accessibility, responses to the following questions were considered:
Q1. AATT gave me a good idea about the accessibility issues in the organization’s website.
Q2. AATT made me more aware of the importance of the issues of website accessibility to users with disabilities.
Q3. AATT motivated me to learn more about web accessibility issues and the role I can play in improving the accessibility of websites.
Internal consistency of these items was established by calculating Cronbach’s α. For these items, Cronbach’s α = 0.93, indicating that the items were highly consistent with one another. Therefore, three variables were used to determine whether using the AATT improved developers’ awareness of web accessibility. A Wilcoxon signed-rank test was conducted, where the null value was equal to 3.40, the satisfaction threshold.
Table 10 shows that the average level of agreement for each item was 4.7 or higher, with the mean level of agreement over all items being 4.8 (SD = 0.4). The Wilcoxon signed-rank test in Table 10 shows that the median level of agreement for each question was significantly higher than the satisfaction threshold of 3.40 (P-value < 0.025), and similar results were found when the overall level of agreement was considered. Given this, we concluded that the AATT was effective in improving developers’ understanding of web accessibility (Fig. 7).
Table 10. Descriptive statistics for items regarding the importance of the AATT
Q# | Strongly agree N (%) | Somewhat agree N (%) | Neither agree nor disagree N (%) | Somewhat disagree N (%) | Strongly disagree N (%) | Mean (SD) |
|---|---|---|---|---|---|---|
Q1 | 9 (100) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 5 (0) |
Q2 | 7 (77.78) | 1 (11.1) | 1 (11.1) | 0 (0) | 0 (0) | 4.7 (0.71) |
Q3 | 8 (88.89) | 1 (11.1) | 0 (0) | 0 (0) | 0 (0) | 4.8 (0.41) |
Fig. 7 [Images not available. See PDF.]
AATT improved web accessibility awareness
Discussion
Our first hypothesis, that the AATT would improve web accessibility for Arabic web developers, was supported. Most web developers solved the accessibility issues found on the provided webpages within less than 2 h. Most of the error messages found on these websites were related to a lack of alternatives for different forms of media. The most common issue was that images, forms, and text were missing alternative text or labels.
These results showed that the AATT helped Arab web developers test their websites, find accessibility issues, and improve their websites. Two tests were used to test the first hypothesis. Based on this test, it was concluded that using the AATT improved Arabic web accessibility.
Our second hypothesis, that the AATT would improve user satisfaction, was also supported. The tool’s usability and user satisfaction were measured using nine items rated by participants on a five-point Likert scale. Based upon the calculated Cronbach’s α value for these nine items, it was determined that the internal consistency was high. The one-tailed Wilcoxon signed-rank test, a non-parametric test equivalent to one-sample t-test, was used to measure satisfaction. Based on recommendations from Pimentel (2010), the satisfaction threshold value was set to equal 3.40. If the mean of the variable is above 3.41, we can conclude that this is an indication of the positivity of the responses and vice versa. All items had a mean value over this threshold, most of the mean values indicated high levels of user satisfaction (Table 10). To determine whether participants were satisfied with the tool, a one-sample t-test was conducted, with the results indicating that, generally, participants were satisfied. It was therefore concluded that the AATT had a high level of usability and that users were generally satisfied with the tool (Tables 11 and 12).
Table 11. P-values for one-tailed Wilcoxon signed-rank test
Question | One-sided Wilcoxon test P-value Ha: Median — 3.40 > 0 |
|---|---|
Q1 | 0.0015 |
Q2 | 0.0035 |
Q3 | 0.003 |
Table 12. Descriptive statistics for level of overall awareness
Overall mean agreement | Maximum | Minimum | Std. deviation | Std. error mean | One-sample t-test P-value |
|---|---|---|---|---|---|
4.80 | 5 | 4.70 | 0.152 | 0.088 | 0.002 |
These findings highlight the need to have user-friendly and language-specific tools that Arabic-speaking web developers can easily use for website accessibility assessment. The data showed that the participants liked the tool as it was easy to use, understand, and navigate. Overall, participants stated that they would recommend this tool to other Arab web developers and that they felt the tool was useful for improving Arabic web accessibility.
Our third and final hypothesis was that the AATT would improve web developers’ awareness of web accessibility.
Three questions were used to measure the level of awareness of web accessibility and evaluate the third hypothesis. The Cronbach’s α value indicated that the internal consistency of the three items was high. Also, the one-tailed Wilcoxon signed-rank test was used to determine whether each item’s mean was significantly different from the threshold value of 3.40. Items had a mean value between 4.7 and 5, indicating that participants generally believed that the AATT improved web developers’ awareness of web accessibility. Additionally, the overall level of agreement with statements related to developers’ awareness of web accessibility was also significantly higher than the satisfaction threshold.
These findings emphasize that the third hypothesis reflects the need to create web accessibility awareness between countries, organizations, and web developers. To effectively address accessibility issues, developers must understand why and how these web accessibility changes benefit people with disabilities.
Limitations
The research was conducted to understand the reasons behind the lack of web accessibility and to explore some solutions to enhance and improve Arabic web accessibility. The limitations of this research can be summarized as follows:
Although the results of the study highlighted the importance of having an Arabic accessibility testing tool to provide support to web developers in creating accessible web content, other solutions were not explored in this study. This research focused on only one solution for improving web accessibility. Other solutions need to be explored, experienced, and tested to adequately address accessibility issues.
It is worth mentioning that the study’s sample size is small and limited to one country in the Gulf Region, while expanding the study to include more participants in various Arabic-speaking countries might provide greater insight into the issue and the proposed solution.
As the researcher was not in the country where the experiment took place, the data collected was through online, self-report questionnaires. Other data collection methods, such as focus groups, case studies, and interviews, were not used in this study.
Conclusions
This research presented an improved version of the Arabic web accessibility tool. The tool was evaluated by thirty-one web development participants and used by nine participants to modify web pages. Our findings showed that the tool positively impacted the developers’ satisfaction with the tool and their awareness of web accessibility issues by involving people with disabilities in the assessment and improvement of Arabic web accessibility. Different data collection methods such as interviews, focus groups, and case studies need to be utilized to better understand the state of Arabic web accessibility. The reported results can help web developers create better Arabic accessibility tools that serve web developers to build accessible websites. Future work should focus on developing an Arabic accessibility testing tool that can be integrated into web applications and websites so that it can perform daily automated checks for any accessibility violations and notify the website owner. Moreover, developing customized applications suitable for certain types of impairments may allow better access to information and services on the web. Therefore, future research should also focus on developing Arabic semantic web accessibility applications to provide specified services based on the specific disability of the user. Furthermore, future research should include web accessibility for mobile phone devices in addition to the development of customized applications for the elderly and individuals with disabilities.
Declarations
Conflict of interest
The authors declare no competing interests.
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
Abanumy, A; Al-Badi, A; Mayhew, P. e-Government website accessibility: in-depth evaluation of Saudi Arabia and Oman. Electron J e-Government; 2005; 3,
Abascal, J; Arrue, M; Fajardo, I; Garay, N; Toms, J. The use of guidelines to automatically verify web accessibility. Univ Access Inf Soc; 2004; 3,
Abu Shawar, B. Evaluating web accessibility of educational websites. Int J Emerg Technol Learn (IJET); 2015; 10,
Al-Khalifa, HS. The accessibility of Saudi Arabia government web sites: an exploratory study. Univ Access Inf Soc; 2010; 11,
AlMeraj, Z; Boujarwah, F; Alhuwail, D; Qadri, R. Evaluating the accessibility of higher education institution websites in the State of Kuwait: empirical evidence. Univ Access Inf Soc; 2020; 20,
Alnahari, MM; Chakraborty, JM. Web accessibility research and practice: the need for Arabic web accessibility improvement. Int J Art Culture Des Technol IJACDT; 2021; 10,
Beseiso, M; Ahmad, AR; Ismail, R. A survey of Arabic language support in semantic web. Int J Comput Appl; 2010; 9,
El-Soud, MA; Hassan, AE; Kandil, MS; Shohieb, SM. A proposed web-based framework for e-learning and dictionary system for deaf Arab students. IJECS; 2010; 2828, 106401.
Hadidi, MS; Al Khateeb, JM. Special education in Arab countries: current challenges. Int J Disabil Dev Educ; 2015; 62,
Martín, Y-S; Yelmo, JC. Guidance for the development of accessibility evaluation tools following the unified software development process. Procedia Comp Sci; 2014; 27, pp. 302-311. [DOI: https://dx.doi.org/10.1016/j.procs.2014.02.033]
Omari, A. Technology adoption in the Arabian Gulf countries: the case of e-government. Int J Comp Sci, Eng Inf Technol; 2013; 3,
Weber, AS. Web-based learning in Qatar and the GCC States. SSRN Electron J; 2016; [DOI: https://dx.doi.org/10.2139/ssrn.2825912]
Alnahari, M., Chakraborty, J. (2019). Preliminary findings from a study of an Arabic accessibility tool checker. Lect Notes Comp Sci, 201–210. https://doi.org/10.1007/978-3-030-23560-4_15
Al-Faries, A., Al-Khalifa, H. S., Al-Razgan, M. S., & Al-Duwais, M. (2013). Evaluating the accessibility and usabilityf top Saudi e-government services. Proceedings of the 7th International Conference on Theory and Practice of Electronic Governance. https://doi.org/10.1145/2591888.2591898
Alayed, A., Wald, M., and Draffan, E. A. (2016). Developing a framework for localised web accessibility guidelines for university websites in Saudi Arabia. Lecture Notes in Computer Science, 243–250. https://doi.org/10.1007/978-3-319-41264-1_33
Antonelli, H. L., Rodrigues, S. S., Watanabe, W. M., & de Mattos Fortes, R. P. (2018, June). A survey on accessibility awareness of Brazilian web developers. In Proceedings of the 8th international conference on software development and technologies for enhancing accessibility and fighting info-exclusion (pp. 71–79).
Al-Gain, S. (1970, January 1). Issues and obstacles in disability research in Saudi Arabia. Semantic Scholar. https://www.semanticscholar.org/paper/ISSUES-AND-OBSTACLES-IN-DISABILITY-RESEARCH-IN-Al-Gain-Alabdulwahab/52c1af9f72be80535195f823bef41c93fdc7b8a2.
Akram, M., & Bt, R. (2017). A systematic literature review to determine the web accessibility issues in Saudi Arabian university and government websites for disable people. International Journal of Advanced ComputerScience and Applications, 8(6). https://doi.org/10.14569/ijacsa.2017.080642
Abascal, J., Arrue, M., Fajardo, I., Garay, N., & Toms, J. (2004b). The use of guidelines to automatically verify web accessibility. Universal Access in the Information
Benavídez, C., Fuertes, J. L., Gutiérrez, E., & Martínez, L. (2006). Semi-automatic evaluation of web accessibility with HERA 2.0. Lecture Notes in Computer Science, 199–206. https://doi.org/10.1007/11788713_30
Bigby, G. (2018). WCAG: Web content accessibility guidelines explained. https://dynomapper.com/blog/27-accessibility-testing/273-wcag-the-web-content-accessibility-guidelines-explained.
Bigby, G. (2019a). Top 25 awesome accessibility testing tools for websites. Create Sitemaps - Sitemap Generator - Visual Sitemap Generator. https://dynomapper.com/blog/27-accessibility-testing/246-top-25-awesome-accessibility-testing-tools-for-websites.
Bigby, G. (2019b). What is web accessibility? DYNO Mapper. https://dynomapper.com/blog/27-accessibility-testing/555-what-is-web-accessibility.
Gay, G., & Li, C. Q. (2010, April). AChecker: open, interactive, customizable, web accessibility checking. In Proceedings of the 2010 international cross disciplinary conference on web accessibility (W4A) (pp. 1–2).
Herramhof, S., Petrie, H., Strobbe, C., Vlachogiannis, E., Weimann, K., Weber, G., & Velasco, C. A. (2006). Test case management tools for accessibility testing. Lect Notes Comp Sci 215–222. https://doi.org/10.1007/11788713_32
Henry, S. L. (2021, January 6). W3C accessibility standards overview. Web Accessibility Initiative (WAI). http://www.w3.org/WAI/standards-guidelines/#wcag. WorldWideWeb Consortium.Web Accessibility Initiative. Web Content Accessibility Guidelines.
Kan Boon Hock, S., & Lafi, S. M. (2011). Assistive communication technologies for augmentative communication in Arab countries: research issues. UNITAR e-Journal, 7(1).
Liginlal, D., & Al-Muftah, D. (2017). Web accessibility as a barrier to successful digital ..https://core.ac.uk/download/pdf/301372250.pdf.
Masood Rana, M., Fakrudeen, M., & Rana, U. (2011). Evaluating web accessibility of university web sites in the Kingdom of Saudi Arabia. International Journal of Technology, Knowledge & Society, 7(3).
Miesenberger, K., Klaus, J., Wolfgang, Z., Karshmer, A., Al-Wabil, A., Zaphiris, Wilson, S. (2006). Web design for dyslexics: accessibility of Arabic content. In, Computers Helping People with Special Needs (p. 817).
Petrie, H., & Kheir, O. (2007). The relationship between accessibility and usability of websites. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ‘07. https://doi.org/10.1145/1240624.1240688
Pearson, E., Bailey, C., & Green, S. (2011). A tool to support the web accessibility evaluation process for novices. Proceedings of the 16th annual joint conference on innovation and technology in computer science education - ITiCSE ‘11. https://doi.org/10.1145/1999747.1999758
Pearson, E., Bailey, C., & Green, S. (2011). A tool to support the web accessibility evaluation process for novices. Proceedings of the 16th annual joint conference on innovation and technology in computer science education - ITiCSE ‘11. https://doi.org/10.1145/1999747.1999758
Sloan, D. (2006). The effectiveness of the web accessibility audit as a motivational and educational tool in inclusive web design (Doctoral dissertation, School of Computing).
Saleem, M. (2018). Arabic web accessibility guidelines. Proceedings of the 15th International Web for All Conference. https://doi.org/10.1145/3192714.3196315
© This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply 2022.