Content area
This study examines digital public services within the European Union, focusing on digital inclusion and the barriers preventing widespread e-government adoption. A significant knowledge gap exists in understanding the interaction between citizens and digital public services, particularly for vulnerable groups, and in evaluating the impact of regional and sector-specific factors. The research employs a mixed-methods approach, including a systematic review based on PRISMA methodology, a document analysis of the digital and programs of the European Union and the Member States, 33 semi-structured expert interviews, and a survey targeting Spanish intermediate leaders, to explore digital divides in access, skills, and outcomes. Key findings reveal that barriers such as limited digital skills, access to technology, and inadequate service design disproportionately affect the elderly, low-income populations, and those with minimal education. Eighteen recommendations have been proposed, associated to five indicators of the model proposed to evaluate the key dimensions of e- government adoption and digital inclusion: internet access, digital skills, user interaction, social drivers and barriers, and outcomes. The recommendations include, among others, enhancing digital skills through tailored training programs, involving the third sector, simplifying service interfaces, and promoting a hybrid service model combining digital and face-to-face options. The study also highlights gaps in addressing civic participation as part of e- government, suggesting it as a focus for future research. These findings underscore the need for user-centred, context-specific policies to ensure equitable access to digital public services and reduce digital exclusion.
Introduction
E-government can improve citizenry interaction with the public administration through more efficient and accountable delivery of online public services (United Nations, 2020). However, the rapid pace of digital government poses an urgent challenge: ensuring that no one is left behind. This requires an integrated approach to developing inclusive programs and policies that address the digital divide and the unique needs of diverse population groups (Hardill and O’Sullivan, 2018; United Nations, 2022).
The success of the universal transition towards the adoption of digital services has been intrinsically linked to the availability of material access to infrastructure, such as internet connectivity and up-to-date technological devices, which constitutes the first digital divide to overcome. Subsequently, as this divide has been mitigated in more developed countries, the focus has shifted to the second digital divide, concerning the possession of the digital skills required to use the Internet (Riggins and Dewan, 2005; Pejic et al. 2013; Salemink et al. 2017). Overcoming this second divide is crucial for effectively addressing the third-level digital divide, which involves achieving meaningful opportunities, outcomes and benefits through the use of ICTs, such as improving education, health or socio-economic situation (Van Deursen and Helsper, 2015; Ragnedda, 2016; Scheerder et al. 2017), as well as fostering the ability to innovate and create products and services in the digital economy (Esteban-Navarro et al. 2020). The third digital divide highlights that technologies must not only be accessible and understandable but also capable of creating value and driving equitable transformation in their outcomes. In the context of e-government services, the digital divide can be influenced by all three levels of disparity. These include the lack of access or skills (first and second digital divides) or the inability to interact effectively with public administration due to the inherent complexity of electronic services designed for administrative procedures and participation in e-government processes (third digital divide).
The European Commission has advanced various strategies, projects, and policy initiatives to drive digital transformation. Europe’s Digital Decade: digital targets for 2030 outlines the Commission’s vision for Europe’s digital transition by 2030, as detailed in a set of official documents published between 2021 and 2023 (European Commission, 2024a). To operationalize this vision, the European Commission proposed a 2030 Digital Compass for the European Union’s digital decade, structured around four intervention areas: digital skills, digital infrastructures, business digital transformation and public services digitalization. In the first area, which is related to the second and third digital divides, the EU has established an ambitious goal for 2030: ensuring that 80% of its citizens acquire at least basic digital skills (European Commission, 2021).
Assessing the state of development and utilization of digital public services across countries, particularly the associated digital skills, is essential for the effective design and evaluation of public policies. There are several international frameworks with sets of indicators. The National Interoperability Framework Observatory (NIFO) is a monitoring tool of the European Commission that gathers information about the state- of-play of digital public administrations across the EU Member States and associated countries (National Interoperability Framework Observatory, 2022). The OECD Digital Government Index (DGI) compares the digital government progress of the 29 OECD Members and four key partner countries (OECD, 2020) through indicators related to design, data-driven, platform, open by default, user-driven, and proactivity. The E-Government Development Index (EGDI), provided by the United Nations E-Government Survey 2022 (United Nations, 2022), that delivers a numerical ranking of e-government development among the UN Member States calculated through three indexes: Online Service Index (OSI), Telecommunication Infrastructure Index (TII) and Human Capital Index (HCI). However, the key report on digital transformation in the European Union is the Digital Economy and Society Index (DESI), published by the European Commission, which monitors digital progress in EU states since 2014. In 2021, DESI underwent a methodological restructuring to align with the EU’s strategic digital transformation objectives and offers a European and national-level analysis, using 33 indicators to measure the progress of the European digital targets, grouped in four principal dimensions: human capital, connectivity, integration of digital technology, and digital public services (European Commission, 2022a). This recalibration directly corresponds to the intervention areas delineated in the Digital Compass 2030: skills, infrastructures, business and government. DESI is now integrated into the State of the Digital Decade report (European Commission, 2024b).
The DESI indicator “Digital public services for citizens” measures “what extent to services or information concerning services for citizens is provided online and via a government portal” (European Commission, 2022a). This indicator has an overall score of 79.4 for the EU with data from 2023: Malta (100), Estonia (95.8), and Luxembourg (94.7) lead this category, while Croatia (67.1), Poland (64), and Romania (52.1) rank the lowest (European Commission, 2024b). The indicator “e-Government users” is “the percentage of individuals who used the Internet in the last 12 months to interact with the public authorities” (European Commission, 2022a). DESI 2024 shows that 75% of European internet users interacted with public administration online. Denmark (98.6%), Finland (97.6%), and Sweden (96.4%) have the higher results in e-government users.
Instead, Germany (62.1%), Bulgaria (35.3%) and Romania (24.6%) have the lowest percentages (European Commission, 2024b). With regard to digital skills, 55.56% of individuals aged 16–74 years in the EU possess at least basic competences, according to DESI 2024. This represents a percentage of citizens still far from the 80% target set for 2030. The Netherlands (82.7%), Finland (82%) and Ireland (72.9%) are the frontrunners in this indicator, whereas countries such as Poland (44.3%), Bulgaria (35.5%) and Romania (27.7%) lag behind (European Commission, 2024b).
Improving digital competences for digital transformation is recognized as a top priority of the European Policy Agenda, as highlighted in DigComp 2.2 (European Commission et al. 2022). The Digital Competence Framework for Citizens (DigComp) defines the five areas of digital competence: information and data literacy, communication and collaboration, digital content creation, safety and problem solving. It measures 21 indicators, outlines the knowledge, skills and attitudes associated with each one, and establishes eight proficiency levels for each competence, divided into four stages: foundation, intermediate, advanced and highly specialised. The DigComp mentions the use of digital public services throughout four of the five competence areas, particularly the competence 2.3 “Engaging citizenship through digital technologies” (Table 1).
Table 1. Use of digital public services in DigComp 2.2.
Competence area | Competence | Examples |
|---|---|---|
1. Information and data literacy. | 1.3 Managing data, information and digital content. | Knows how to interact with dynamic data visualisation and can manipulate dynamic graphs of interest. |
2. Communication and collaboration. | 2.3 Engaging citizenship through digital technologies. | Knows that a secure electronic identification, enables citizens to increase safety when using online services provided by the government or by the private sector. |
3. Digital content creation. | 3.1 Developing digital content. | Knows how to create digital content to support one’s own ideas and opinions. |
4. Safety. | 4.1 Protecting devices. | Knows about measures to protect devices and prevent others from having access to all data. |
4.2 Protecting personal data and privacy. | Knows how to use digital certificates acquired from certifying authorities |
Source: Digital Competence Framework for Citizens (European Commission et al. 2022).
DESI calculates two indicators—basic and above basic digital skills—based on the areas and indicators of DigComp tool. The results help policymakers to formulate policies and to plan education and training initiatives to improve digital competences among specific European target groups. Additionally, DigComp 2.2 identifies four key strategies and policies as relevant for enhancing digital skills of all European citizens: The European Skills Agenda, the Digital Education Action Plan, the Digital Compass, and the European Pillar of Social Rights and its action plan.
Finally, the e-Government Benchmark, a report by Capgemini, Sogeti, IDC and Politecnico di Milano for the European Commission Directorate General for Communications Networks, Content and Technology, measures the availability and the quality of the public digital services of 35 European countries (the 27 EU Member States, Iceland, Norway, Switzerland, Albania, Montenegro, North Macedonia, Serbia, and Türkiye). It compares how governments deliver digital public services (Capgemini et al. 2023). The overall e-Government maturity score is the average of the user centricity, transparency, key enablers, and cross-border services dimension scores. The EU27 overall performance averages at 70 points. According to the e-Government benchmark, Malta (96 points), Estonia (92), Luxembourg (89) and Iceland (88) achieve the highest overall performance. Albania (45 points), Montenegro (41) and North Macedonia (35) struggle to keep up with the frontrunners (Capgemini et al. 2023).
In summary, the digital transformation process of public services is often evaluated by public bodies from a delivery-focused perspective, frequently neglecting the citizen’s viewpoint in the development of digital strategies and programs. As highlighted in official reports, these evaluations often reduce the complex realities of European countries to a limited set of indicators, which can lead to overly simplistic interpretations. Furthermore, there is a notable lack of official studies adopting a qualitative perspective—such as satisfaction, public perception, trust, or the inclusiveness of digital services—that adequately considers users’ experiences, especially when identifying vulnerable groups. Consequently, these reports often fail to account for the social, economic, and cultural contexts unique to each country, particularly those with uneven levels of digital development and specific socioeconomic challenges. Few countries incorporate measures or formulate policy decisions based on input from vulnerable populations (United Nations, 2022).
Literature review
Addressing Inequalities in the use of digital public services
In contrast, the academic literature reveals that while the expansion of e-government enhances service delivery, it simultaneously risks exacerbating social inequalities through the widening of digital divides. As governments worldwide increasingly implement digital public services and citizens must use them without an offline choice, initial concerns about the potential impacts of the digital divide persist over time (Belanger and Carter, 2009; Hall and Owens, 2011; Zhao et al. 2014; Beaunoyer et al. 2020). Despite these concerns, many public administrations remain focused on infrastructure development, and studies on e-government adoption often regard digital public services as generic phenomenon, ignoring the context and diversity of those services (Lindgren et al. 2019). Nevertheless, it should not be ignored that while e- government offers significant benefits, it also presents risks by deepening the digital divide, further disadvantaging already marginalized populations and reinforcing mechanisms of social inequality (Taewoon and Sigit, 2011; Marien and Prodnik, 2014; Van Deursen and Helsper, 2018; Ragnedda et al. 2022). As stated by the United Nations (2022): the new face of inequality is digital.
The study of digital public services and their utilization by the citizens is a significant area of research within the academic literature e-government (Kumar et al. 2017; Rana et al. 2017; Ammenwerth, 2019; Li and Shang, 2020; Soong et al. 2020). Extant studies highlight the influence of sociodemographic variables on access to and utilization of digital public services (Belanger and Carter, 2009; Taipale, 2013; Serrano Cinca et al. 2018; Adamczyk and Betlej, 2021) such as age (Choudrie et al. 2013), education level (Medina Molina et al. 2012; Zheng and Schachter, 2017; Pérez-Morote et al. 2020; Mesa, 2023), living area (Inkinen et al. 2018), gender (Taipale, 2013) and income (Gerpott and Ahmadi, 2016; Chohan and Hu, 2022). Vulnerable groups, particularly older adults, individuals with lower educational attainment, and those from low-income households, are disproportionately affected and face significant barriers that perpetuate exclusion cycles (Belanger and Carter, 2009; Ragnedda et al. 2022).
Other studies also highlight the importance of factors more closely related to the user experience of digital services, such as trust and security (Camilleri, 2019; Ejdys et al. 2019; Sundberg, 2019; Pérez-Morote et al. 2020; Botric and Bozic, 2021) or usability and quality of the service (Scheerder et al. 2017; Barrera-Barrera et al. 2019; Mensah, 2019; Alruwaie et al. 2020; Frohlich et al. 2020; Taejun et al. 2020; Heponiemi et al. 2022). Additionally, some studies establish a more specific connection between the level of digital skills and the use of these services (Van Deursen and van Dijk, 2010; Ebbers et al. 2016; Rodríguez Hevía et al. 2020; Boksova et al. 2021; Heponiemi et al. 2022).
These issues underscore a pressing need for targeted interventions aimed at ensuring equitable access for the population. Evidence (Van Deursen and Helsper, 2015; Dobrolyubova, 2021; Lněnička and Máchová, 2022) indicates that a techno-determinist approach focusing on infrastructure provision, which prioritizes infrastructure provision and technological advancements over social and cultural contexts, fails to accommodate individuals lacking digital skills, particularly those in vulnerable circumstances. The European Commission is aware of the fact that a low level of digital skills may deepen the digital divide as more services are offered only online (European Commission, 2023). Digital skills are widely recognized as essential for navigating digital public services, yet a large proportion of the population lacks even basic competencies, limiting their ability to benefit from these services and perpetuating cycles of exclusion (European Commission, 2022a). The Digital Skills and Jobs Coalition considers four broad groups to tackle the digital skills gap: digital skills for all, digital skills for the labour force, digital skills for ICT professionals and digital skills for education (European Commission, 2022b).
If the digital divide is understood as a multi-dimensional concept—encompassing differences in access, usage, and the effective exploitation of online services by citizens—public policies must address the various factors contributing to exclusion and inequality. While research on the digital divide is extensive, studies specifically examining its implications in the context of implementing e-government programs from a user-centered perspective remain scarce. Additionally, there is a notable lack of qualitative research aimed at identifying the conditioning factors that influence the uptake of digital public services. Quantitative studies dominate, offering useful metrics but failing to address qualitative barriers and enablers from marginalized users’ perspectives (Lindgren et al. 2019). It should also be noted that, in general, there is a lack of studies that consider both user and provider perspectives in empirical e- government research (Wirtz and Daiser, 2018).
Research gap
A key yet underexplored issue in the literature is the role of digital competencies in shaping e-government adoption, either by enabling access or perpetuating barriers. Moreover, while technology adoption models and multivariate approaches have been applied to e-government use research, there is a critical need to integrate the perspectives of users, experts, and, also, intermediate leaders (individuals working in local positions and closely connected to the population) to develop more inclusive digital strategies and bridge this digital divide in the use of digital public services. In particular, the role of local intermediaries, such as civil servants and community leaders, remains underexplored, despite they are increasingly recognized as essential for the success of digital inclusion policies and initiatives (Centeno, 2021).
Research questions. To address this research gap, the following three research questions will be posed:
RQ 1: What strategies, programs and actions promote EU public authorities in the field of digital skills are taking to decrease digital exclusion?
RQ 2: What are the conditioning factors that are involved in the digital interaction between citizens and digital public services?
RQ 3: What measures and actions propose the community of experts in digital inclusion and the intermediate leaders to reduce inequalities in the use of digital public services, with a particular focus on digital competences?
Objective
The general objective of this research is to propose a set of common and applicable measures and recommendations to improve the interaction of citizens with e- government, considering their digital competences to avoid risks of exclusion, with special attention to vulnerable groups.
Methods
To this end, a concurrent triangulation design, integrating both qualitative and quantitative techniques, was employed to corroborate findings and provide a more comprehensive understanding of the various potential interpretations of a social reality (Gibson, 2016), hereby the digital divide and the use of digital public services. In short, a systematic review based on the PRISMA methodology, a document analysis of e- government policies of the EU and the member countries, 33 in-depth semi-structured interviews with experts and a survey conducted among intermediate leaders were carried out. The European Union as a whole was chosen as the field of study for several key reasons. First, the EU is a global leader in the provision of digital public services and has established significant strategic frameworks, such as the European Digital Agenda and the e-Government Action Plan. Additionally, the relative homogeneity in e- government development among Member States allows for meaningful comparisons and regionally applicable conclusions. Finally, European policies emphasize digital inclusion and digital competences as strategic pillars, directly aligning with the objective of this research.
Research techniques
First, a systematic review was conducted, based on the PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analysis) methodology (Page et al. 2021) to identify studies about the interaction of citizens with digital public services. Before conducting the systematic review, a protocol was written following the PRISMA-P) guidelines (Morte-Nadal and Esteban-Navarro, 2022). The preparation of a protocol is an essential component of the systematic review process because reduces arbitrary in decision making when extracting and using data from primary research (Moher et al. 2015). The publication of the protocol increases the transparency, the clarity, and the traceability of the process in social sciences. The inclusion and exclusion criteria are detailed in Table 2. A total of 66 articles (see annex A) were selected and analysed for extract risk factors or barriers which cause digital exclusion, enabling factors which cause digital inclusion, public measures to promote digital inclusion, and proposals and recommendations to end the e-government divide.
Table 2. Inclusion and exclusion criteria of the systematic review.
Inclusion criteria | Exclusion criteria |
|---|---|
Articles published in peer-reviewed journals | Studies which only focus on the first digital divide of e-government |
Years of publication between 2011 and 2024 | |
Studies located into a European Union Member State | |
Studies written in English, French, or Spanish |
Next, a document analysis of the national digital strategies and plans of the European Union (EU) Member States (NUT1) was undertaken to identify the objectives and measures aimed at enhancing the basic digital skills of the EU population. Document analysis involves the collection, selection according to relevant criteria, inductive or deductive categorization, and interpretation of texts such as reports, policies, regulations and newspapers to draw insights that are organised into major themes, and the answer to the research questions (Gross, 2018; Kayesa and Shung-King, 2021). This technique is often used with other research methods to ensure triangulation (Armstrong, 2021). It allows to test theoretical issues with the purpose of providing knowledge, increases a researcher’s understanding of particular phenomena, a representation of facts or informs practical actions (Krippendorff, 2018). There are no guidelines for data analysis, since it is a flexible technique that depends on the research problem (Elo and Kyngäs, 2008).
The documents were collected from the Digital Skills and Jobs Platform (https://digital-skills-jobs.europa.eu) about the National strategies of the Member States of the EU and in web portals of national authorities in e-government. The inclusion and exclusion criteria are specified in Table 3. Nineteen out of the 28 countries (EU 27 and the United Kingdom) meet the inclusion criteria, contributing a total of 25 documents for analysis (see annex B). Three documents were selected from Germany, while Ireland, Portugal, Slovakia, and Spain each provided two documents. The countries excluded from the analysis were Belgium, Czech Republic, Greece, Hungary, Latvia, Lithuania, Luxembourg, Poland, and Sweden. The results of the analysis were compared with the DESI 2022 Country Reporting (European Commission, 2022a).
Table 3. Inclusion and exclusion criteria of the document analysis.
Inclusion criteria | Exclusion criteria |
|---|---|
National strategies included in the Digital Skills and Jobs Platform | Strategies that give a superficial treatment of digital skills. |
Year of publication between 2013 and 2022 | Digital skills strategies focused only on the education system. |
Strategies written or translated in Spanish, English or French | Digital skills strategies focused only on the labour force. |
Only digital strategies in force that dedicate some of its content to the enhancement of the basic digital skills of the general population will be regarded. | Digital skills strategies focused only on the upskilling of ICT specialists. |
All publication formats |
Likewise, an in-depth analysis of Spanish regional e-government and digital skills strategies (NUTS2) was carried out using the same document analysis technique (see annex C). This regional analysis at NUTS2 level was conducted considering that most European countries have a decentralized territorial structure and the significance of regional policies in e-government deployment. Spain was selected as a representative case study due to its position as a mid-ranking EU country in terms of e-government users (15th out of 27) and at least basic digital skills (6th out of 27) indicators in DESI 2022 report (European Commission, 2022a), as well as the availability of strategic documents across most of its regions.
As the next step, 33 in-depth semi-structured interviews with experts in digital public services, digital inclusion and digital competences were carried out online between November 2022 and April 2023. The objectives were to compare their perspectives with the results of the systematic review, to delve into what is being done currently by governments, and to collate what measures related to digital inclusion could be done in public policies. The systematizing expert interview aims to gather actionable knowledge by focusing on both technical expertise in a high specific field and processual knowledge derived from practical experience and the institutional context of actions (Döringer, 2021). Participants were recruited through snowball sampling, as experts provided recommendations of other experts who could offer further insights. To reduce information gaps and personal biases, experts were selected from both inside (decision- makers such as civil servants and government advisors) and outside (analysts such as researchers, with a majority representation, and some specialized journalists in e- government). The principle of the saturation point was used to delimitate the sample, when the introduction of new respondents in a professional category and an expertise area does not collect new knowledge as they tend to repeat the information.
The experts were selected by purposive sampling and from a variety of European countries, with a higher proportion of Spanish participants (63% of the sample) to ensure data that could contrast the analysis of Spanish regional strategies and the data collected from Spanish intermediate leaders. Additionally, the sample included two Latin American academics who have conducted a significant part of their careers in Europe. Gender balance was sought among the interviewees: 18 (55.5%) were male and 15 (45.5%) were females.
Each interview was transcribed verbatim (to consult transcriptions see section Data Availability) and coded with Atlas.ti. The total hours analysed was 21:07:29. The average duration of the interviews was 38 min 25 s. The minimum duration was 12:57 and the maximum duration was 1:56:04. The codification method to identify the interviewees during the analysis and synthesis was: interviewee correlative number according to the date of the interview, professional category (R = researcher; N = non- researcher), country of their actual workplace (following the international standard code ISO 3166-1 alpha2) and years of experience in their actual job. If they worked for the European Commission, the country code was designated as “EU”. The decision was made to group the professional category into researcher/non-researcher intentionally, as the sample is biased towards researchers, comprising 66% of the sample (see Table 4).
Table 4. Profile of the experts interviewed.
Code | Name abbreviation | Professional category/occupation | Gender | Country of workplace | Self-perception of expertise area | Expe- rience | Date |
|---|---|---|---|---|---|---|---|
1RES30 | AM | Researcher | Female | Spain | Political Science and Administration | 30 y. | 14/11/22 |
2RFI3 | MM | Researcher | Female | Finland | Digital inequality | 3 y. | 16/11/22 |
3RES33 | PM | Researcher | Male | Spain | Digital inequality | 33 y. | 22/11/22 |
4NES6 | JS | Civil servant | Male | Spain | Digital public services design | 6 y. | 29/11/22 |
5RNL20 | AD | Researcher | Male | The Netherlands | Digital divide and digital inclusion | 20 y. | 01/12/22 |
6RNL3 | EL | Researcher | Female | The Netherlands | Digital skills | 3 y. | 01/12/22 |
7RPL15 | JP | Researcher | Male | Poland | Digital skills | 15 y. | 09/12/22 |
8NES5 | GP | Third sector | Female | Spain | Digital skills | 5 y. | 15/12/22 |
9RFI11 | RH | Researcher | Female | Finland | Digital inclusion | 11 y. | 15/12/22 |
10NEU12 | GM | Government Advisor | Male | EU | Digital governance | 12 y. | 12/01/23 |
11NFI31 | OK | Government Advisor | Male | Finland | Digital governance | 31 y. | 13/01/23 |
12REU21 | CC | Researcher | Female | EU | Digital skills | 21 y. | 18/01/23 |
13NES2 | AI | Civil servant | Male | Spain | Digital governance | 2 y. | 18/01/23 |
14RBE10 | IM | Researcher | Female | Belgium | Digital inequality | 10 y. | 24/01/23 |
15RGB14 | EH | Researcher | Female | United Kingdom | Digital inclusion | 14 y. | 24/01/23 |
16RES19 | RS | Researcher | Female | Spain | Education | 19 y. | 06/02/23 |
17NES15 | AC | Third sector | Male | Spain | Digital skills | 15 y. | 07/02/23 |
18RES5 | CL | Researcher | Female | Spain | Communication and vulnerability | 5 y. | 07/02/23 |
19RES15 | MO | Researcher | Male | Spain | Journalism | 15 y. | 07/02/23 |
20RCO15 | MB | Researcher | Male | Colombia | Communication | 15 y. | 08/02/23 |
21RES10 | EC | Researcher | Female | Spain | Journalism | 10 y. | 10/02/23 |
22RES12 | JAG | Researcher | Male | Spain | Communication | 12 y. | 13/02/23 |
23RGB10 | MR | Researcher | Male | United Kindgom | Digital divide | 10 y. | 17/02/23 |
24RES2 | SC | Researcher | Female | Spain | Digital skills | 2 y. | 03/03/23 |
25NES10 | SR | Civil servant (librarian) | Female | Spain | Information Science | 10 y. | 10/03/23 |
26NES5 | JM | Civil servant | Male | Spain | Digital public services | 5 y. | 15/03/23 |
27NES8 | JL | Civil servant (archivist) | Male | Spain | Digital public services | 8 y. | 16/03/23 |
28RES20 | SO | Researcher | Female | Spain | Educommuni-cation | 20 y. | 20/03/23 |
29NES35 | FG | Journalist | Male | Spain | Journalism | 35 y. | 21/03/23 |
30RAR20 | FA | Researcher | Male | Argentina | Communication | 20 y. | 23/03/23 |
31RES35 | JF | Researcher | Male | Spain | Educommuni-cation | 35 y. | 28/03/23 |
32RES34 | JAG | Researcher | Male | Spain | Digital public services | 34 y. | 29/03/23 |
33NES2 | JG | Chief in a Communication Department | Female | Spain | Digital public services and service design | 2 y. | 26/04/23 |
In addition, as a fourth technique, a survey of 602 Spanish intermediate leaders was conducted using an online panel. This survey addresses two gaps in academic literature: the need for regional studies on digital competences, and the lack of research on the role of intermediate leaders in bridging the digital divide. The data were collected from 30 March to 11 April 2023 using an online panel through Bilendi, a market research company. The sample included intermediate leaders in Spain by proportional quota sampling based on sex and age ranges (groups of 10 years from 18 to 60). The response rate was 89%. We excluded respondents that were out of quota. The results of the questionnaire were organised into Nielsen Regions. The questionnaire comprised six sections: sociodemographic questions, factors influencing the use of digital public services, perceptions of the difficulty associated with certain tasks (such as obtaining a digital certificate or contacting a civil servant), satisfaction levels with the design and operation of various digital public services, identification of vulnerable population groups, and suggestions for enhancing citizens’ digital skills for utilizing digital public services (see annex D). Likert scales were employed for all questions except for sociodemographic queries.
Finally, the analysis of data obtained through these diverse techniques were integrated and triangulated into five key themes: the state of digital public services in the EU, vulnerable groups, drivers, barriers, and measures.
Model
The methodological process designed for evaluating the key dimensions of e- government adoption and digital inclusion relies on a model grounded in well- established theories and frameworks. Specifically, the Technology Acceptance Model (TAM), the Unified Theory of Acceptance and Use of Technology (UTAUT), the Digital Competence Framework (DigComp 2.2) and the construct of digital divide, serve as the foundation. The Technology Acceptance Model (TAM), developed by Davis (1989), identifies two key determinants of technology adoption: perceived usefulness and perceived ease of use. Perceived usefulness reflects the degree to which individuals believe that using a particular system enhances their performance, while perceived ease of use addresses the effort required to interact with the system. In the context of e-government, these factors are critical for understanding how citizens evaluate the utility and accessibility of digital public services, shaping their willingness to adopt these technologies. Building on TAM, the Unified Theory of Acceptance and Use of Technology (UTAUT) adds further dimensions to explain technology adoption (Williams et al. 2015). UTAUT incorporates factors such as social influence, which highlights the role of societal and peer expectations, facilitating conditions, which focus on the resources and support available to use the technology effectively, and behavioural intention (Ammenwerth, 2019; Soong et al. 2020); which have been applied to e-government (Camilleri, 2019; Rodríguez Hevía et al. 2020). These dimensions are essential for analysing how external influences and individual motivations interact to drive or hinder the adoption of e-government services.
To address the critical role of digital skills, the model incorporates the categorization of digital competence proposed by the Digital Competence Framework DigComp 2.2 (European Commission et al. 2022), ensuring that the analysis accounts for the disparities in digital skills that may act as barriers to accessing and utilizing e- government services, particularly among vulnerable groups. The conceptual framework also draws on the construct related to the three levels of digital divide: access, usage and exploitation, exposed above.
By integrating these theoretical approaches, the conceptual framework provides a robust basis for analysing e-government adoption through a structured lens that focuses the interaction between citizens and digital public services, identifying areas where inequalities emerge and how they can be mitigated. The model balances three core components: user-centric factors, such as motivation, digital skills and trust, with system-level considerations, such as platform accessibility and quality, and societal influences, such as policy environments and cultural attitudes. The indicators used in the analysis of data were selected based on their ability to capture those three key components and their relevance to the research objectives: internet access, digital skills, user interaction—focused on user experience, trust, and the ability to navigate digital public services effectively—, social drivers and barriers, and outcomes, which evaluate the extent to which users derive meaningful benefits from digital public services. These indicators were drawn from widely recognized benchmarks, including the DESI (European Commission, 2022a) and the e-Government Benchmark (Capgemini et al. 2023), ensuring comparability and consistency across the European Union.
By addressing technological, individual, and societal dimensions of digital public services, this model provides a multidimensional perspective on digital inclusion and the factors influencing the effective adoption of e-government. Thus, the model adopted, by focusing on digital competences for assessing the usability and effectiveness of digital services, complements the perspective provided by the Information System Success Model (DeLone and McLean, 2003), which emphasizes attributes such as system quality, information quality, service quality, usage intentions, user satisfaction, and net system benefits, but does not account for the importance of digital skills when dealing with systems designed for all citizens.
Results
The systematic review reveals that policies primarily focus on digital inclusion rather than e-government, with quantitative studies dominating the analysis and influencing the findings. Sectoral analysis, such as education or healthcare, was unfeasible since most studies addressed general e-government services, and no studies focused on education. Even among qualitative and mixed-methods studies, only a few identified barriers and enabling factors in interactions with e-government services.
The studies underscore a persistent social gap: those most in need of digital public services often fail to access them, necessitating policies to prevent social and digital exclusion. Barriers are largely socio-demographic, with digital skills emerging as a critical determinant for adoption. Key barriers and enabling factors influencing citizens’ interactions with e-government were identified, highlighting education as a crucial element to bridge the digital divide. Education fosters better engagement with public services and empowers citizens to achieve positive outcomes.
A significant number of studies suggest that national strategies must address the digital competence levels of vulnerable groups, incorporating socio-demographic variables like age, education, and income to define these groups and create targeted strategies and communication campaigns. Despite public institutions emphasizing digital service design, strategies addressing citizens’ digital skills are underexplored. Measures to enhance these skills and promote digital inclusion are vital to improving citizen engagement with e-government services. In short, the review highlights the need to identify necessary skills, address barriers, and develop inclusive programs to ensure equitable use of digital public services, moving beyond the limitations of predominantly quantitative approaches.
In turn, European public documents about e-government reveal that all EU member states operate within the framework of the EU Digital Skills and Jobs Coalition, focusing on skills for all citizens, the labour force, education, and digital experts. Most strategies address the four groups in their objectives and measures, emphasizing education through specific digital skills courses and awareness campaigns.
Collaboration with local communities, public spaces, civil organizations, and proximity agents is often highlighted. Measures for vulnerable groups closely resemble those for the general population, with added attention to identifying their specific needs. The third sector is seen as fundamental, requiring more support and training for mediating agents. Recent policies (e.g., Malta, Slovenia, Slovakia) demonstrate increased concern for ensuring that no one is left behind in the digitization process. However, there is a lack of strategies targeting individuals unwilling or unable to use digital services.
Digital inclusion features prominently in recent strategies, aligning with United Nations recommendations.
In Spain, strategies similarly align with the EU framework but place greater emphasis on rural infrastructure, digital skills for civil servants, and women as a vulnerable group. Spanish regions prioritize analysing the digital divide and skills levels, but awareness campaigns are only present in a few strategies. Additionally, Spanish regions stress creating digital skills certificates and providing equipment and internet access to vulnerable populations, reflecting the challenges of rural depopulation. Both EU and Spanish strategies rely heavily on quantitative data, with limited evaluation of measures or inclusion of third-sector roles. Only a few European countries (Ireland, Portugal, Malta) recognize the importance of broader stakeholder involvement involving all stakeholders in designing digital public services, and only one Spanish region (Aragon) has a defined strategy for this purpose.
Both academic literature and strategies agree that basic digital skills are essential for engaging with digital public services, with proposed measures including information campaigns and tailored training programs for vulnerable groups. The analysis of barriers to using digital public services is limited, with the lack of digital skills being the most frequently mentioned, while sociodemographic factors appear primarily in defining vulnerable groups. Unemployment and distrust in government are additional factors that warrant further exploration, as unemployed individuals are identified as a vulnerable group lacking digital skills, aligning with academic findings.
The analysis of expert interviews highlights several critical aspects of digital public services (for items of the interviews see Annex E, and for a comprehensive analysis with the transcription of key statements see Annex F). Overall, these services are perceived as complex and inaccessible due to unintuitive interfaces, lack of user-centric designs, and technical jargon, underscoring the need for simplified, proactive systems. Key barriers include sociodemographic factors such as age, education, and income, compounded by low digital and administrative literacy, limited technology access, and usability flaws. Conversely, drivers like digital literacy, trust in public administration, and motivation can enhance engagement, with suggestions for awareness campaigns and simplified processes. The analysis underscores the importance of addressing the digital skills gap, particularly for basic and medium-level competences, with the DigComp framework proposed as a structure for evolving training programs.
The persistent “e-government divide” reflects inadequate service designs and limited stakeholder involvement, risking further exclusion of vulnerable populations without targeted interventions. Vulnerable groups, including the elderly, individuals with disabilities, and low-income households, face unique challenges, necessitating customized strategies, local support networks, and inclusive policies. Experts recommend a multistakeholder user-centric approach to service design, emphasizing interoperability, face-to-face alternatives, and lifelong digital skills training to ensure equity. This study highlights the complexity of integrating digital inclusion into e- government and calls for collaborative, evidence-based strategies to promote accessible and inclusive digital public services.
Finally, the survey of Spanish intermediate leaders highlights that privacy, security, the simplification of procedures, and language accessibility are perceived as the most significant barriers to the use of digital public services. Intermediate leaders emphasize the importance of face-to-face channels and personalized support, advocating for a hybrid model to complement digital services. Vulnerable groups, including the elderly, those with limited internet experience, and the homeless, face significant barriers, particularly regarding internet access and user-friendly designs. Digital skills and lifelong training are identified as essential measures to bridge the first and second digital divides. Internet access, whether through local centers or financial aid, is also a prioritized measure, while the creation of a digital skills certificate is less valued.
Awareness campaigns are not highly regarded, though third-sector representatives suggest increased visibility is needed. Respondents express dissatisfaction with the current design of digital public services, citing trust and accessibility as critical areas for improvement. Contacting civil servants is viewed as the most challenging task, yet support from civil servants is highly appreciated. Comparisons between regions reveal slight variations. Overall, the findings indicate that Spanish strategies must prioritize internet access, digital skills training, and personalized support to promote digital inclusion. Collaboration with public and private sectors and greater emphasis on user- centric designs are also recommended for future improvements.
Discussion
Overall state of digital public services in the European Union
Currently, digital public services are not user-friendly for the majority of citizens. The interaction with these services often resulted in a negative experience for interviewees, shaping their overall perception of public administration. Additionally, there is a perceived disengagement between public administration and the citizenry.
The existence of a e-government divide was especially remarked by the experts. This digital divide affects particularly to the most vulnerable groups, not only because of their socioeconomic status (e.g, Serrano Cinca et al. 2018) but also, because their attitude is different from less vulnerable groups (15RGB14). However, it appears that digital vulnerability is a condition that can affect anyone at any stage of life, given the evolving nature of the social, economic, and technological context of society. An aspect related to vulnerable groups and design of digital public services that appeared only in the interviews with civil servants and government consultant was that, typically, participants in these tests are individuals who have already used digital public services. Moreover, they are generally more motivated to engage with digital applications.
Consequently, there may be a bias in the participants conducting these user experience tests (experts 4NES6, 13NES2, 15RGB14 and 26NES5).
Vulnerable groups
Certain groups of the population may require increased attention from governments: “Vulnerable groups may face more challenges in accessing and using digital services” (23RGB10). The most frequently mentioned vulnerable groups identified across the systematic review, digital strategies, interviews, and the survey included the elderly (e.g., Abad et al. 2017), individuals with a low level of education (e.g., Gerpott and Ahmadi, 2016; Laenens et al. 2018), those with low socio-economic status (e.g., Choudrie et al. 2013; Siren and Knudsen, 2017), and individuals with low digital skill (e.g., Al-Muwil et al. 2019; Boksova et al. 2021). Additionally, people with disabilities (e.g., Republic of Bulgaria, 2020; Government of Italy, 2020; Pethig et al. 2021), residents of rural areas (e.g., Government of Navarra, 2021; Government of Madrid, 2022), and the unemployed (e.g., Andersen et al. 2019; Government of Portugal, 2020), are mentioned as vulnerable groups in second place.
Users facing physical, mental, or cognitive challenges may encounter difficulties using these services and may require ongoing assistance (Ministry of Economic Affairs and Digital Transformation, 2022). However, intermediate leaders assigned a higher level of importance to privacy and data security (M = 4.34) and the language used by public administration (M = 4.26) compared to sociodemographic and socioeconomic factors.
It appears that sociodemographic determinants play a significant role in influencing digital vulnerability. Factors associated with social exclusion, such as poverty, being outside the labour force, and low socioeconomic status, seem to contribute to digital vulnerability when using digital public services.
Conditioning factors: drivers
Throughout all the analyses, three factors consistently were highly mentioned or rated: having access to the internet, possessing a high level of education, and having at least basic digital skills are deemed necessary for effectively using digital public services.
The importance of having internet access was particularly emphasized in the digital strategies when addressing vulnerable groups, highlighting them as the segments most at risk of lacking access to the Internet and technological devices (Federal Government of Germany, 2018; Department of the Taoiseach, 2022; Ministry of Investment, Regional Development, and Informatisation of the Slovak Republic, 2022). The intermediate leaders rated over 4 to “having access to the internet” (M = 4.24) and the experts also stressed the importance of access in terms of addressing the first digital divide.
Having a high level of education is closely tied to socioeconomic factors and having advanced digital skills. Education significantly influences how people utilize the internet, the type of job they have, and their income level, according the experts. This relationship has been extensively studied in the systematic review (Inkinen et al. 2018; Barrera-Barrera et al. 2019). Additionally, education facilitates a better understanding of the language used by public administration. Digital strategies considered the education system and lifelong learning as key components for fostering equal opportunities in society and promoting future employment growth (Government of Madrid, 2022; Republic of Slovenia, 2023), a perspective shared by both experts and intermediate leaders (M = 4.14). Understanding the benefits of using digital public services was also identified as a driver factor: success is achieved when the real benefits of using specific services become evident (10NEU12).
According to the literature (e.g., Bacache-Beauvallet et al. 2011; Ebbers et al. 2016), the experts, the digital strategies (Government of Portugal, 2017; Junta of Andalusia, 2022) and the intermediate leaders (M = 4), it is necessary to have at least basic skills to use digital public services autonomously. The development of digital skills in general will allow people to have skills to face the digital public administration. Support from family or civil servants is deemed essential (e.g., Government of the Netherlands, 2021; 9RFI11) especially for the most vulnerable groups. Support from civil servants was rated as one of the most important conditioning factors by the intermediate leaders (M = 4.04).
Regarding factors related to the public administration, trust and security and privacy of data, simplification of administrative procedures, and making digital public services more user-friendly emerged consistently in the systematic review, digital strategies, interviews, and the survey. Across all analyses, it was evident that trust in public administration is often linked to the security and privacy of data (e.g., Government of the United Kingdom, 2022; Government of Malta, 2022), serving as a key factor in increasing citizens’ willingness to use digital public services. Everything related to security, particularly in the sense of providing a feeling of control, is emphasized (27NES8).
Simplification of administrative procedures and making digital public services more user-friendly was also included in the digital strategy of Portugal (Government of Portugal, 2020), as well as in the systematic review (Abad et al. 2017; Klich, 2021) and highlighted by the intermediate leaders (M = 4.26). This design could be enhanced by making them more accessible, interoperable, secure, inclusive, efficient, and user- friendly (Government of the United Kingdom, 2022). The design process should have the users at the centre (Government of Aragon, 2022). Therefore, it should be tested by all types of users (15RGB14).
Conditioning factors: barriers
Lack of skills is identified as an obstacle to the development of the country (Government of Italy, 2020) and is one of the barriers related to digital exclusion (Laenens et al. 2018; Government of the United Kingdom, 2022). However, the systematic review (e.g., Taipale, 2013) and the experts also identified sociodemographic and socioeconomic factors as significant barriers: the economic status is perceived as the most important barrier because determines the level of studies. and this level generally determines the purchasing power (25NES10).
The design of services was consistently mentioned throughout the systematic review (Viñarás et al. 2017), public policies (Government of Aragon, 2022), and interviews, and it was rated as very important in the survey (M = 4). This design should be improved by making it more citizen-centric and accessible. Nevertheless, one of the most frequently mentioned barriers for the experts was the complexity of public administration and the lack of awareness about the problems users face when using digital public services. The idea that public administration has the duty to offer a public service and that technology is there to serve the citizen was reiterated by the experts.
The administration should prioritize placing the citizen at the centre; however, public administration often positions itself at the centre, forcing citizens to adapt to its structure (27NES8). Additionally, concerns were raised about the lack of accessibility for people with disabilities (Republic of Bulgaria, 2020) even though this is required by both European and National regulations.
Similarly, the importance of the language used by the public administration as a barrier was highly emphasized by the experts and rated highly by the intermediate leaders (M = 426). The technical language is discouraging not only for migrants but also for those who have a low level of literacy (23RGB10). However, this aspect was scarcely mentioned in the digital strategies (Government of the Netherlands, 2021) and in the results of the systematic review.
A concept that emerged during expert interviews was the lack of administrative literacy among the population (1RES30 and 25NES10). This literacy is related to the level of education. It also has to do with the type of job individuals have, their previous experience with public administration, and age, as older people tend to face fewer difficulties understanding these services even if they lack technological skills.
In the survey, one of the most challenging tasks for the intermediate leaders was contacting a public employee (M = 3.69). This lack of clarity and the absence of support for the user when they are using a digital public service create feelings of helplessness (31RES35).
Measures
The most cited measures throughout all the analyses for the improvement of the digital skills of the population were having household access to the internet, training in digital skills, and creating information points at a local level. The proposed measures dedicated to improving the digital skills of the entire population were very similar to those for vulnerable groups since they are related to training in digital skills with the support of local centres and mediating agents.
The digital strategies (Federal Government of Germany, 2018; Regional Government of Castile and Leon, 2023), the experts (3RES33 and 11NFI31), and the intermediate leaders considered that access to the Internet is still essential. Therefore, ensuring that citizens have quality access to the Internet (M = 4.17) and affordable connectivity (M = 4.26) and providing financial support for the acquisition of technological devices (M = 3.90) should be a priority.
The creation of IT training programs to upskills the population were essential for the systematic review (e.g., Camilleri, 2019), the digital strategies (e.g., Government of the Netherlands, 2021; Region of Murcia, 2022) the experts (7RPL15) and the intermediate leaders (M = 3.92). This training should be incorporated from the early stages of education, following the DigComp framework (24RES2). The experts considered that a basic level of digital skills is needed, so the content of these programs should be based on the Digital Competence Framework for Citizens (DigComp) and focus on practical activities related to the use of digital public services (12REU21). These programs should promote lifelong learning and be adapted to the needs of the target group.
Besides, these programs should be first targeted to the most vulnerable groups, though they should be offered to all citizens since digital vulnerability can appear at any moment in life (5RNL20 and 14RBE10). It is more recommended to conduct these training sessions face-to-face, particularly for the most vulnerable groups lacking digital skills (28RES20).
For the digital strategies (e.g., Government of the United Kingdom, 2022), the experts, and intermediate leaders, these training sessions should take place in local centres, such as public libraries, or local organizations, because actions must be done locally, where the vulnerable groups live (12REU21). The digital strategies place greater emphasis on establishing digital competence centres where local agents can provide technical access, advice, and support (Government of Spain, 2021). However, it should be noticed that the intermediate leaders assessed the use of local centres more highly (M = 3.94) than the creation of digital competence centres (M = 3.67). Thus, it seems that the value of these centres is precisely their geographical proximity and the possibility of weaving social support networks. Grassroots level digital support could very well be the way to help people keep up with digital technologies (9RFI11). For the digital strategies (Government of Romania, 2014; Government of Malta, 2022), experts (10NEU12 and 17NES15), and intermediate leaders (M = 3.42), collaboration with the private sector, entities, associations, and particularly the third sector, is key to tackle digital divide gaps. According to the intermediate leaders, governments should support the third sector (M = 3.79).
However, governments should continue to maintain face-to-face channels (e.g. Regional Government of Castile and Leon, 2023) for individuals facing difficulties and for citizens who have encountered issues during the digital process. Some issues can still be resolved over the phone (3RES33). In fact, it seemed that the experts were more concerned with improving face-to-face services than with improving digitization. When people have a problem with a digital public service, there must be an immediate access system where they can resolve the problem, because, in the end, digital public services are a relationship tool between the citizen and the administration (19RES15). Therefore, offering multiple channels and facilitating face-to-face care and assistance is significant to improve the digital inclusion of the use of these services, as there will always be population groups that will not be able to use these services online (26NES5).
Some experts have emphasized that, just as in policies aimed at enhancing the digital skills of the population, performance indicators for digital public services should be established (7RPL15, 11NFI31, 18RES5)., and according to the intermediate leaders, periodic evaluations with feedback from users should be conducted (M = 3.80).
Proposal of recommendations
The discussion of findings leads to the proposal of 18 actionable recommendations aimed at enhancing digital inclusion in e-government implementation, particularly for vulnerable groups, while offering policymakers essential insights for developing inclusive public policies.
Digital vulnerability will not disappear with the generational reveal since there are socioeconomic determinants inherent in society, so public policies must be aimed at reducing and mitigating existing social inequalities.
Ensuring and supporting universal access to the internet and updated technological devices. Cost of the use of the internet should be affordable.
Implement the DigComp framework (European Commission et al. 2022) as the standard in the measurement of digital competences and design of information and media literacy programs.
At least the basic digital skills defined in the DigComp framework (foundation and intermediate proficiency levels) are necessary to interact autonomously with digital public services.
Education in digital competences since the early stages of life. Information and media literacy are also necessary to provide citizens for a more critical view and to promote lifelong learning due to the constant changes in technology. Content related to basic public digital services should be included.
Training in digital skills for all the population but focusing more on the most local vulnerable groups. This training should be customized to their needs and take place in their own communities.
The third sector and the intermediate leaders are essential in the training process so more investment and support to these agents should be promoted.
Support from family, friends and volunteers is essential for the most vulnerable groups. Collaboration with the third sector and private organisations would create a network of economic, technological and informational support.
Transforming local public centres related with education and culture into a network of RICT centres specialized in the training and education on digital competences.
Promoting user experience studies of digital public services taking into account all the stakeholders with a special focus on the most vulnerable.
Digital public services should improve to make them simpler and more user- friendly. They should follow the accessibility standard for websites. Simplification of administrative procedures, multidisciplinary work groups and a multi-stakeholder approach is recommended.
Civil servants should also be trained in digital skills and user experience because this change is not only of infrastructure but of the idiosyncrasy of the public administration.
Omnichannel services and reinforcement of technical support when users have a problem with a service. This support must be given in person, by telephone and by email. Not all public services have to be offered exclusively online to have a digitally inclusive society.
Awareness and communication campaigns to all the citizens and civil servants should be launched to make this transition easier and to increase citizen trust through transparent and understandable information about digital public services and fostering the improvement of digital skills.
Regarding public policies about the inclusive use of digital public services: designing performance indicators, investing in digital inclusion, more control over the financing of indicatives to the upskilling of the population, periodic assessment of these policies and initiatives, and collecting feedback from the users for continuous improvement.
Financing more studies about the digital competences of the population, moving away from quantitative data and collaborating with all the stakeholders. It is a complex problem that is also a social problem, and that requires taking all the methodological techniques to extract the most complete information possible
Conducting specific research to determine vulnerability factors in a particular population to design customised digital competence programs rather than applying general programs to pre-defined vulnerable groups.
Do not rely exclusively on benchmarks about the delivery and use of digital public services as a way of saying that the problem has been solved.
The association between each proposed recommendation and the corresponding indicators in the model used to evaluate the key dimensions of e-government adoption and digital inclusion is presented in Table 5.
Table 5. Association between proposals and indicators.
Indicator | Proposal |
|---|---|
Internet access | 2, 9, 11, 13, 18 |
Digital skills | 3, 4, 5, 6, 9, 12, 16 |
User interaction | 4, 5, 10, 11, 12, 13, 14 |
Social drivers and barriers | 1, 2, 6, 7, 8, 9, 11, 14, 15, 17 |
Outcomes (benefits) | 1, 7, 8, 10, 15, 16, 17, 18 |
Conclusion
This research highlights that digital competences play a crucial role in digital inclusion, emphasizing that citizens must acquire at least basic digital skills to use digital public services inclusively, regardless of service design. The digital divide in digital public services is closely linked to the digital divide in internet usage. Furthermore, this divide is mostly influenced by sociodemographic and socioeconomic factors. The vulnerable groups face the most significant barriers in accessing these services. Therefore, it appears that as long as social inequalities persist, the digital divide will not disappear.
Besides, digital vulnerability can affect anyone at different life stages, necessitating essential changes in both public administration and the educational system to bridge this gap.
Although in many cases, citizens have an obligation to use these services, this gap can manifest itself in receiving more and better information from the government, in applying for benefits or subsidies, or in the personal autonomy of certain citizens when they must interact with the government online, requiring others to complete the process on their behalf. Another aspect of inequality within this gap could be observed in participation in civic or political affairs. The social and legal consequences of the digital divide are that digital citizenship is a right that is not being exercised. Particularly in democratic welfare states, when asking for social benefits, only those who know that these benefits exists and that know how to ask for them will receive them.
The public administration is generally unaware of the difficulties users face when utilizing online services. The digital strategies of member countries often idealize what services should be like, but there is a need to shift focus towards analysing their current state, going beyond mere compliance with international rankings and reports. Therefore, without a thorough understanding of specific problems, the correct solutions will not be formulated.
A concept that emerged during this research is that of “intermediate leaders”. While originally intended to be middle managers in education, intermediate leaders can play a significant role in promoting digital inclusion. They can provide the support needed to reduce these structural inequalities and can also serve as a valuable source of information for future qualitative research. While the sample of experts is relatively large and diverse enough to provide insights into the phenomenon as a whole, it may be necessary in the future to explore the experiences of specific sectors, such as the third sector or civil servants. Education since the early stages of the education system were one of the most recommended measures. Therefore, future research focusing on digital skills and training from the early stages of education would be valuable to undertake.
Additionally, it would be essential to allocate time and resources for directly soliciting feedback from users of digital public services to validate the results presented in this research.
It seems that most citizens lack “administrative literacy”, and this can have multiple causes related to the level of education, type of job, previous experience with these services, or attitude towards public administration. An idea that emerged during the interviews was whether it is necessary for all key public services to be offered 100% online or if it should depend on the nature of the service and the type of user it serves.
Making public administration more accessible to citizens by simplifying contact channels with civil servants is crucial. While the digitization of public services brings numerous benefits, one way to ensure a fully inclusive administration where no one is left behind might be to continue offering face-to-face services as well. In conclusion, online channels are more appropriate for obtaining information, but when it comes to addressing individual issues or making arrangements, face-to-face interactions have proven to be more effective.
The collaboration of all stakeholders is essential in addressing this problem, and it is not solely the responsibility of users, but also of the government to ensure that administration is accessible and usable for all. Further research is needed on the role of institutional communication in e-government and how an effective communication strategy for digital public services could enhance their adoption.
Finally, we want to address three limitations of this research.
First limitation is that the recommendations proposed do not apply equally to all individuals, to all services provided by the government or to different administrative levels (local, regional, national and European). These recommendations are intended as general guidelines, recognizing that certain measures may be more relevant or effective in specific environments. As such, the applicability of some recommendations will depend on the particular context, such as regional characteristics, sectoral needs, or target populations. This underscores the need for further regional or sector-specific studies to validate and adapt the conclusions drawn here. Conducting such studies would ensure that the recommendations are tailored to the unique challenges and opportunities within different environments, thereby enhancing their effectiveness and relevance. This would also allow for prioritizing the implementation of each proposal, based on the needs and resources of each context.
Second, the research has focused solely on one dimension of the digital divide in e- government: the interaction with digital public services. However, the second dimension of the concept, which involves public participation in political and civic decision-making processes, also deserves attention. This includes activities such as e- voting, participation in online forums, and engagement in public consultations. While these forms of participation are integral to e-government and closely tied to issues of digital exclusion—particularly the risk of being excluded from democratic structures and institutions—they have been set aside in this study and are suggested as areas for future research.
And third, education from the early stages of the educational system is one of the most recommended measures. However, this research did not focus on the school-age population nor on the potential content related to public administration that could be taught in the educational system of the countries during the compulsory stages. Given the broad and complex nature of this system, it was not feasible to address it in this study. Furthermore, it would be essential to allocate time and resources to obtain feedback from users on their experience with digital public services.
Author contributions
At all stages of the research, both authors contributed jointly.
Data availability
The datasets generated during and/or analyses during the current study are available from the corresponding author on reasonable request.
Competing interests
The authors declare no competing interests.
Ethical approval
This research, as a PhD thesis project, was approved by the Ethic Committee of the Doctoral School of the University of Zaragoza on October 16th, 2019. It was supervised by the Academic Commission of the Information and Communication Doctoral Program of the University of Zaragoza. The approval, with number: 091/2024, covers all aspects of the study. This research complies with ethical principles including the highest standards of research integrity as set out in the ALLEA European Code of Conduct for Research Integrity, as well as applicable international and national law, including the Charter of Fundamental Rights of the European Union and the European Convention on Human Rights and its Supplementary Protocols. All procedures adhered to the tenets of the Declaration of Helsinki.
Informed consent
This research does not collect personal data, but only experiences, opinions and evaluations without the identification of individuals, including expert participants in the studies (interviewees) and participants in online questionnaires. Interview participants were informed orally about the study’s aim and scope, how the data would be used, and their right to withdraw from the study at any time. Since all the interviews were conducted online, they were asked for permission to voice record the interview. An informed consent statement was presented in the introduction of the online survey without providing an informed written consent form. For all participants, consent covered participation, data use, and publication. All participants were fully informed that their anonymity was assured, why the research was being conducted and how their data would be utilized. They were also informed about their right to withdraw from the research at any point. There were no indications from any participants expressing a wish to discontinue their participation prematurely.
Supplementary information
The online version contains supplementary material available at https://doi.org/10.1057/s41599-025-04576-7.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
Abad Alcalá, L; Llorente-Barroso, C; Sánchez-Valle, M; Viñarás-Abad, M; Pretel-Jiménez, M. Electronic government and online tasks: towards the autonomy and empowerment of senior citizens. EPI; 2017; 26,
Adamczyk, M; Betlej, A. Social determinants of digital exclusion in an ageing society. The case of Poland. Entrep, sustain issues; 2021; 8,
Al-Muwil, A; Weerakkody, V; El-haddadeh, R; Dwivedi, Y. Balancing Digital-By-Default with Inclusion: A Study of Factors Influencing E-Inclusion in the UK. Inf Syst Front; 2019; 21, pp. 635-659. [DOI: https://dx.doi.org/10.1007/s10796-019-09914-0]
Alruwaie M, El-Haddadehb R, Weerakkody V (2020). Citizens’ continuous use of eGovernment services: The role of self-efficacy, outcome expectations and satisfaction. Gov. Inf. Q 37 (3). https://doi.org/10.1016/j.giq.2020.101485
Ammenwerth, E. Technology Acceptance Models in Health Informatics: TAM and UTAUT. Stud Health Technol Inf; 2019; 263, pp. 64-71. [DOI: https://dx.doi.org/10.3233/SHTI190111]
Andersen KN, Nielsen JA, Kim S (2019) Use, cost, and digital divide in online public health care: lessons from Denmark. ransform. Gov.: People Process Policy 13(2). https://doi.org/10.1108/TG-06-2018-004
Armstrong C (2021). Key Methods Used in Qualitative Document Analysis (December 29, 2021). SSRNhttps://doi.org/10.2139/ssrn.3996213
Bacache-Beauvallet M, Bounie D, François A (2011) Existe-t-il une fracture numérique dans l'usage de l'administration en ligne? Rev Econ (62):215–235. https://doi.org/10.3917/reco.622.0215
Barrera-Barrera R, Rey-Moreno M, Medina-Molina C (2019). Factores explicativos de la preferencia y uso de la administración electrónica en España. Revista de Administración Pública 53 (2). https://doi.org/10.1590/0034-761220170391
Beaunoyer, E; Dupéré, S; Guitton, MJ. COVID-19 and digital inequalities: Reciprocal impacts and mitigation strategies. Comput Hum Behav; 2020; 111, [DOI: https://dx.doi.org/10.1016/j.chb.2020.106424] 106424.
Belanger, F; Carter, L. The Impact of the Digital Divide on E-Government Use. Commun ACM; 2009; 52,
Boksova, J; Boksa, M; Horak, J; Pavlica, K; Strouhal, J; Saroch, S. E-Government Services and the Digital Divide: A Quantitative Analysis of the Digital Divide between the General Public and Internet Users. J Telecommun Digit Econ; 2021; 9,
Botric V, Bozic L (2021) The digital divide and E-government in European economies. Economic Research-Ekonomska Istraživanja. https://doi.org/10.1080/1331677X.2020.1863828
Camilleri MA (2019). The online users’ perceptions toward electronic government services. J. Inf. Commun. Ethics Soc. https://doi.org/10.2139/ssrn.3476543
Capgemini, Sogeti, IDC, Politecnico di Milano (2023). eGovernment Benchmark 2023: Connecting Digital Governments. Insight report. Publications Office of the European Union. https://doi.org/10.2759/474056
Centeno C (2021). La brecha digital: ¿qué es, a quiénes afecta y cómo? Posibles actuaciones. In Aliaga E et al. (ed). Reimaginando el presente para reducir las brechas digitales. Fundación Esplai. https://cutt.ly/fwHkE0Td
Chohan, SR; Hu, G. Strengthening digital inclusion through e-government: cohesive ICT training programs to intensify digital competency. Inf Technol Dev; 2022; 28,
Choudrie, J; Ghinea, G; Songonuga, VN. Silver Surfers, E-government and the Digital Divide: An Exploratory Study of UK Local Authority Websites and Older Citizens. Interact Comput; 2013; 25,
Davis, FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q; 1989; 13,
DeLone, WH; McLean, ER. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. J Manag Inf Syst; 2003; 19,
Department of the Taoiseach (2022) Harnessing Digital-The Digital Ireland Framework. Government of Ireland. https://assets.gov.ie/214584/fa3161da-aa9d-4b11-b160-9cac3a6f6148.pdf
Dobrolyubova E (2021). Measuring Outcomes of Digital Transformation in Public Administration: Literature Review and Possible Steps Forward. The NISPAcee J. Public Adm. Policy 14(1). https://doi.org/10.2478/nispa-2021-0003
Döringer, S. The problem-centred expert interview. Combining qualitative interviewing approaches for investigating implicit expert knowledge. Int J Soc Res Methodol; 2021; 24,
Ebbers, WE; Marloes, GMJ; van Deursen, AJAM. Impact of the digital divide on e-government: Expanding from channel choice to channel usage. Gov Inf Q; 2016; 33,
Ejdys J, Ginevicius R, Rozsa Z, Janoskova K (2019). The role of perceived risk and security level in building trust in e-government solutions. IM, 22 (3). https://doi.org/10.15240/tul/001/2019-3-014
Elo, S; Kyngäs, H. The qualitative content analysis process. J Adv Nurs; 2008; 62,
Esteban-Navarro, MA; García-Madurga, MA; Morte-Nadal, T; Nogales-Bocio, AI. The Rural Digital Divide in the Face of the COVID-19 Pandemic in Europe— Recommendations from a Scoping Review. Informatics; 2020; 7,
European Commission (2021). 2030 Digital Compass: the European way for the Digital Decade. https://cutt.ly/FeNVI0Wi
European Commission (2022a). Digital Economy and Society Index (DESI) 2022. Thematic chapters. https://ec.europa.eu/newsroom/dae/redirection/document/88764
European Commission (2022b). Digital skills and jobs coalition. https://digital-strategy.ec.europa.eu/en/policies/digital-skills-coalition
European Commission (2023). 2023 Report on the state of the Digital Decade. https://cutt.ly/OeNVOpdg
European Commission (2024a). Europe’s Digital Decade: digital targets for 2030. https://cutt.ly/VeNVcHM5
European Commission (2024b). State of the Digital Decade 2024 report. https://digital-strategy.ec.europa.eu/en/factpages/state-digital-decade-2024-report
European Commission, Joint Research Centre, Vuorikari R, Kluzer S, Punie Y (2022). DigComp 2.2, The Digital Competence framework for citizens: with new examples of knowledge, skills and attitudes. Publications Office of the European Union. https://data.europa.eu/doi/10.2760/115376
Federal Government of Germany (2018). Shaping Digitalisation. https://cutt.ly/UNtwCxE
Frohlich K, Nieminen M, Pinomaa A (2020). Factors influencing the adoption of m- government: Perspectives from a namibian marginalised community. AFRICOMM 2019. https://doi.org/10.1007/978-3-030-41593-8_17
Gerpott TJ, Ahmadi N (2016). Use levels of electronic government services among German citizens: An empirical analysis of objective household and personal predictors. TGPPP 10 (4). https://doi.org/10.1108/TG-05-2016-0025
Gibson, CB. Elaboration, Generalization, Triangulation, and Interpretation. Organ Res Methods; 2016; 20,
Government of Aragon (2022) Plan de Servicios Digitales de Aragón. https://www.aragon.es/tramites/plan-servicios-digitales-aragon
Government of Italy (2020) National Strategy for Digital Skills. Minister for Technological Innovation and Digitalization. https://repubblicadigitale.innovazione.gov.it/it/le-azioni/#documenti
Government of Madrid (2022) Plan de Capacitación Digital para los ciudadanos de la Comunidad de Madrid 2022-2025. https://cutt.ly/BwHkI9uh
Government of Malta (2022) National eSkills Strategy 2022-2025. Ministry of Economy, European Funds and Lands. https://cutt.ly/1wHkDjEg
Government of Navarra (2021) Plan de Inclusión y Capacitación Digital 2021-2025. https://cutt.ly/MwHkOwLT
Government of Portugal (2017) National Digital Competences Initiative e.2030, Portugal INCoDe.2030. https://www.incode2030.gov.pt/en/
Government of Portugal (2020) Action Plan for Digital Transition. https://cutt.ly/awHkOoql
Government of Romania (2014) National Strategy on Digital Agenda for Romania. Ministry of Communications and Information Society. https://cutt.ly/owDxlO8I
Government of Spain (2021) National Plan for Digital Skills. https://portal.mineco.gob.es/RecursosNoticia/mineco/prensa/noticias/2021/210127_np_digital.pdf
Government of the Netherlands (2021) Dutch Digitalisation Strategy 2.0. Government of the Netherlands. https://cutt.ly/GNtoiTq
Government of the United Kingdom (2022) UK Digital Strategy. Department for Digital, Culture, Media & Sport. The Government of the United Kingdom. https://www.gov.uk/government/publications/uks-digital-strategy/uk-digital-strategy
Gross JMS (2018). Document Analysis. In Frey, BB (ed) The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. SAGE, p 545-548. https://doi.org/10.4135/9781506326139.n209
Hall TE, Owens J (2011). The digital divide and e-government services. In ICEGOV ‘11: Proceedings of the 5th International Conference on Theory and Practice of Electronic Governance. ACM https://doi.org/10.1145/2072069.2072076
Hardill, I; O’Sullivan, R. E-government: Accessing public services online: Implications for citizenship. Local Econ; 2018; 33,
Heponiemi T, Kaihlanen AM, Kouvonen A, Leemann L, Taipale S, Gluschkoff K (2022). The role of age and digital competence on the use of online health and social care services: A cross-sectional population-based survey. Digit. Health https://doi.org/10.1177/20552076221074485
Inkinen T, Merisalo M, Makkonen T (2018). Variations in the adoption and willingness to use e-services in three differentiated urban areas. Eur. Plan. Stud. 26 (5). https://doi.org/10.1080/09654313.2018.1448756
Junta of Andalusia (2022) Plan de Capacitación Digital de Andalucía 2022-2025. https://cutt.ly/nwHkPHDR
Kayesa NK, Shung-King M (2021). The role of document analysis in health policy analysis studies in low and middle-income countries: Lessons for HPA researchers from a qualitative systematic review. Health Policy OPEN, 2. https://doi.org/10.1016/j.hpopen.2020.100024
Klich, A. Electronic Communication with Public Administration in the Time of COVID-19—Poland’s Experience. Int J Environ Res Public Health; 2021; 18,
Krippendorff, K (2018). Content Analysis: An Introduction to Its Methodology. Sage
Kumar, R; Sachan, A; Mukherjee, A. Qualitative approach to determine user experience of e-government services. Comput Hum Behav; 2017; 71, pp. 299-306. [DOI: https://dx.doi.org/10.1016/j.chb.2017.02.023]
Laenens W, Van den Broeck W, Mariën I (2018) Channel Choice Determinants of (Digital) Government Communication: A Case Study of Spatial Planning in Flanders. Media Commun 6(4). https://doi.org/10.17645/mac.v6i4.1652
Li Y, Shang H (2020). Service quality, perceived value, and citizens’ continuous-use intention regarding e-government: Empirical evidence from China. Inf. Manag 57(3). https://doi.org/10.1016/j.im.2019.103197
Lindgren, I; Madsen, CO; Hofmann, S; Melin, U. Close encounters of the digital kind: A research agenda for the digitalization of public services. Gov Inf Q; 2019; 36,
Lněnička, M; Máchová, R. A theoretical framework to evaluate ICT disparities and digital divides: Challenges and implications for e-government development. Rev Econ Perspect; 2022; 22,
Marien, I; Prodnik, J. Digital inclusion and user (dis)empowerment: a critical perspective. Info; 2014; 6,
Medina Molina, C; Rey Moreno, M; Rufín Moreno, R. Nivel de educación y adopción de e-servicios: una aplicación al caso del e-gobierno. Sociol Tecnocienc; 2012; 2, pp. 22-43.
Mensah, IK. Predictors of Electronic Government Services Adoption: The African Students’ Perspective in China. Int J Public Adm; 2019; 42,
Mesa D (2023). Digital divide, e-government and trust in public service: The key role of education. Front. Sociol 8. https://doi.org/10.3389/fsoc.2023.1140416
Ministry of Economic Affairs and Digital Transformation (2022) Digital Spain 2026. https://espanadigital.gob.es/index.php/en
Ministry of Investment, Regional Development, and Informatisation of the Slovak Republic (2022) The National Digital Skills Strategy of the Slovak Republic and the Action Plan 2023-2026. https://cutt.ly/wwDxkRRD
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, PRISMA-P Group (2015). Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement. Syst. Rev 4(1). https://doi.org/10.1186/2046-4053-4-1
Morte-Nadal T, Esteban-Navarro MA (2022). Digital Competences for Improving Digital Inclusion in E-Government Services: A Mixed-Methods Systematic Review Protocol. Int. J. Qual. Methods 21. https://doi.org/10.1177/16094069211070935
National Interoperability Framework Observatory (2022). Digital Public Administration Factsheets – European Union
OECD. Organisation for Economic Co-operation and Development (2020). Digital Government Index: 2019 results. OECD Public Governance Policy Papers, No. 03. OECD Publishing. https://doi.org/10.1787/4de9f5bb-en
Page MJ, McKenzie JE, Bossuyt PM et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst. Rev 10 (89). https://doi.org/10.1186/s13643-021-01626-4
Pejic, M; Zoroja, J; Bosilj, V. Review of corporate digital divide research: A decadal analysis (2003–2012). Int J Inf Syst Proj Manag; 2013; 1,
Pethig F, Kroenung J, Noeltner M (2021) A stigma power perspective on digital government service avoidance. Gov Inf Q 38(2). https://doi.org/10.1016/j.giq.2020.101545
Pérez-Morote R, Pontones-Rosa C, Núñez-Chicharro M (2020). The effects of e- government evaluation, trust and the digital divide in the levels of e-government use in European countries. Technol. Forecast. Soc. Change 154. https://doi.org/10.1016/j.techfore.2020.119973
Ragnedda M (2016). The Third Digital Divide: A Weberian Approach to Digital Inequalities. Routledge. https://doi.org/10.4324/9781315606002
Ragnedda M, Ruiu ML, Addeo F (2022). The self-reinforcing effect of digital and social exclusion: The inequality loop. Telemat Inform 72. https://doi.org/10.1016/j.tele.2022.101852
Rana, NP; Dwivedi, YK; Lal, B; Williams, MD; Clement, M. Citizens’ adoption of an electronic government system: towards a unified view. Inf Syst Front; 2017; 19, pp. 549-568. [DOI: https://dx.doi.org/10.1007/s10796-015-9613-y]
Region of Murcia (2022) Agenda Digital Región de Murcia 2022-2027. http://agendadigital.carm.es/
Regional Government of Castile and Leon (2023) Sociedad de la Información en Castilla y León. https://cienciaytecnologia.jcyl.es/web/es/sociedad-informacion/sociedad-informacion-castilla-leon.html
Republic of Bulgaria (2020) Digital Transformation of Bulgaria for the period 2020-2030. Ministry of Transport and Communications. https://cutt.ly/NwHkD6YE
Republic of Slovenia (2023) Digital Slovenia 2030 Strategy. https://nio.gov.si/nio/asset/strategija+digitalna+slovenija+2030?lang=en
Riggins FJ, Dewan S (2005). The Digital Divide: Current and Future Research Directions. J. Assoc. Inf. Syst 6 (2). https://doi.org/10.17705/1jais.00074
Rodríguez Hevía LF, Navío-Marco J, Ruiz-Gómez LM (2020). Citizens’ Involvement in E-Government in the European Union: the rising importance of the digital skills. Sustainability, 12(17). https://doi.org/10.3390/su12176807
Salemink, K; Strijker, D; Bosworth, G. Rural development in the digital age: A systematic literature review on unequal ICT availability, adoption, and use in rural areas. J Rural Stud; 2017; 54, pp. 360-371. [DOI: https://dx.doi.org/10.1016/j.jrurstud.2015.09.001]
Scheerder, A; van Deursen, AJAM; van Dijk, J. Determinants of Internet skills, uses and outcomes. A systematic review of the second- and third-level digital divide. Telemat Inf; 2017; 34,
Serrano Cinca, C; Muñoz-Soro, JF; Brusca, I. A Multivariate Study of Internet Use and the Digital Divide. Soc Sci Q; 2018; 99,
Siren, A; Knudsen, SG. Older Adults and Emerging Digital Service Delivery: A Mixed Methods Study on Information and Communications Technology Use, Skills, and Attitudes. J Aging Soc Policy; 2017; 29,
Soong, KK; Ahmed, EM; Tan, KS. Factors influencing Malaysian small and medium enterprises adoption of electronic government procurement. J Public Procure; 2020; 20,
Sundberg, L. Electronic government: Towards e-democracy or democracy at risk?. Saf Sci; 2019; 118, pp. 222-232. [DOI: https://dx.doi.org/10.1016/j.ssci.2019.04.030]
Taejun L, Byung-Kwan L, Seulki LG (2020). The effects of information literacy on trust in government websites: Evidence from an online experiment. J. Inf. Manag 52. https://doi.org/10.1016/j.ijinfomgt.2020.102098
Taewoon N, Sigit D (2011). Who Uses E-government? Examining the Digital Divide in E-government Use. ICEGOV ‘11 https://doi.org/10.1145/2072069.2072075
Taipale, S. The use of e-government services and the Internet: The role of socio- demographic, economic and geographical predictors. Telecommun Policy; 2013; 37, pp. 4-5. [DOI: https://dx.doi.org/10.1016/j.telpol.2012.05.005]
United Nations (2020) Digital Government in the Decade of Action for Sustainable Development. With addendum on COVID-19 Response. United Nations
United Nations (2022) E-Government Survey 2022. The future of Digital Government. United Nations
Van Deursen, AJAM; Helsper, EJ. Collateral benefits of Internet use: Explaining the diverse outcomes of engaging with the Internet. N. Media Soc; 2018; 20,
Van Deursen AJAM, Helsper EJ (2015). The third-level digital divide: who benefits most from being online? In: Robinson L, Cotton SR, Schulz J, et al. (eds) Communication and Information Technologies Annual. Emerald Group Publishing Limited, pp 29–52
Van Deursen, AJAM; van Dijk, J. Civil servants’ internet skills: Are they ready for e-government?. Lect Notes Comput Sci; 2010; 6228, pp. 132-143. [DOI: https://dx.doi.org/10.1007/978-3-642-14799-9_12]
Viñarás Abad M, Abad-Alcalá L, Llorente-Barroso C, Sánchez-Valle M,Pretel-Jiménez M (2017) Administración electrónica y e-inclusión de las personas mayores. Rev Lat Comun Soc (72):197–219. https://doi.org/10.4185/RLCS-2017-1161
Williams, MD; Rana, NP; Dwivedi, YK. The unified theory of acceptance and use of technology (UTAUT): a literature review. J Enterp Inf Manag; 2015; 28,
Wirtz, BW; Daiser, P. A meta-analysis of empirical e-government research and its future research implications. Int Rev Adm Sci; 2018; 84,
Zhao, F; Collier, A; Deng, H. A multidimensional and integrative approach to study global digital divide and e-government development. Inf Technol People; 2014; 27,
Zheng, Y; Schachter, HL. Explaining Citizens’ E-Participation Use: the Role of Perceived Advantages. Public Organ Rev; 2017; 17, pp. 409-428. [DOI: https://dx.doi.org/10.1007/s11115-016-0346-2]
Copyright Palgrave Macmillan Dec 2025