A Survey of Middleware Adoption in Nonprofit Sectors: A Sustainable Development Perspective

Abstract

Nonprofit Organizations (NPOs) are adopting technology to improve their quality of services, scale up, or reduce operation costs. However, due to the heterogeneity of systems they use, NPOs face system-integration challenges when collaborating with other organizations. Middleware is an intermediary software that assists dissimilar systems in working together. This paper explores middleware applications, opportunities, and challenges within the sector. It extensively reviewed the current state of research on middleware usage in the nonprofit sector for all papers published in Scopus and Web of Science (WoS) until 2023. Out of 127 papers returned, only 31 remained after removing duplicates, invalid entries, and out-of-scope publications. Then, we synthesized insights from a thorough survey of these selected papers. In light of the survey results, we observed that NPOs primarily use middleware in a few of the Sustainable Development Goals (SDGs), namely, health (SDG 3), NPO operations (SDG 8 and 9), NPO collaborations (SDG 17), development of sustainable cities (SDG 11), security and disaster management (SDG 16), and education (SDG 4). We also identified several challenges related to using middleware in the nonprofit sector, which include privacy, security, system development and performance, data processing and transfer, and volunteer attrition.

1. Introduction

Nonprofit Organizations (NPOs) and Non-Governmental Organizations (NGOs) constitute the nonprofit sector, which encompasses a variety of entities, including service providers, membership organizations, foundations, and community groups [1]: NPOs do not generate profit for their owners or shareholders [2]; instead, they primarily focus on producing social and cultural forms of capital for the common good of society [3]. They are crucial in providing services in arts and culture, health and social services, and education [1]. Additionally, nonprofit infrastructure organizations support the sector by strengthening capacities, mobilizing resources, and building alliances [4].

An NGO (also known as a Civil Society Organization (CSO)), as defined by the United Nations, is “a not-for-profit, voluntary citizen’s group that is organized on a local, national or international level to address issues in support of the public good” [5,6]. NGOs are task-oriented organizations composed of individuals with a common goal. They can either be operational or advocacy NGOs. The operational NGOs focus on development projects, while the advocacy NGOs promote a cause [7]. They carry out a range of humanitarian tasks and monitor the execution of policies and programs, bring citizens’ problems to the government’s attention, and promote the participation of their stakeholders at the local level.

NGOs are separate from the government but may depend on government funding [7]. They play a crucial role in society by delivering resources, serving social or political purposes, and addressing global risks [8]. They are considered key players in social welfare and engage in political influence, negotiating agendas, forming coalitions, and mobilizing constituencies for policy change [9]. The growth of government-organized NGOs (GONGOs) has also impacted state and society relations globally [10].

Although both NPOs and NGOs operate without the primary goal of making a profit, there are some differences between the two [7]: NPOs are often more locally focused and may have a broader range of functions like capacity strengthening, resource mobilization, and alliance building within the nonprofit sector, while NGOs focus on addressing social or environmental issues on a global scale [4]. Also, NPOs rely on a combination of funding sources, including donations, grants, and earned income [11]. Although NGOs operate independently from government control, they receive funding from governments or international donors [7]. We shall use NPOs for all nonprofits (including NGOs and GONGOs) since all aim to serve the public good.

In recent years, NPOs have increasingly adopted newer technologies to transform their operations and pursue their missions. However, the continuous adoption of newer software and hardware technologies gives rise to system heterogeneity, hindering interactions between systems and seamless user experience. Thereby slowing down operations or degrading their quality. Heterogeneity in NPO systems arises due to the following reasons:

1. NPOs have diverse needs and work practices, making it difficult for standardized infrastructure to work optimally in diverse organizational settings [12]

2. Contemporary NPOs use business-like practices to fulfill a wide range of societal roles [13], suggesting that they adopt heterogeneous systems to perform their unique functions.

3. The theory of demand heterogeneity argues that the nonprofit sector develops in response to the diverse societal needs that are unfulfilled by the government [14]. Thus, as NPOs strive to meet unique demands and enhance service delivery, they often implement a variety of technological solutions.

4. Recent developments in the nonprofit sector have created new partnerships, divisions of labor, and responsibilities between the different stakeholders [15]. Thus, modern NPOs operate in complex environments that require different systems working together.

Thus, NPOs use heterogeneous systems to meet their diverse needs and adapt to their complex environments. Middleware is among the technologies that have become important to the functioning of NPOs. The use of middleware in the nonprofit sector is a way to address the growing complexity of technological infrastructures, enabling them to make the most of their technological investments. A middleware facilitates smooth communication and integration by linking hardware and applications within the NPO’s departments, for example, linking the human resources department’s Customer Relations Management (CRM) System with the accounting department’s fundraising system to allow the management to make informed decisions. We call this method an intra-NPO integration. It also plays a vital role in inter-NPO integration, where two or more NPOs collaborate to achieve a cause. For example, NPOs can use a cloud-based data-integration platform to share information about refugee populations, needs assessments, and resource allocation during humanitarian response missions. Also, NPOs use middleware to collaborate with organizations outside the nonprofit sectors. This type of integration is extra-NPO collaboration. An example of extra-NPO integration is UNICEF Primero and Thailand’s Ministry of Public Health (MoPH) using OpenFn to share patient health information [16]. Section 4.1.1 discusses middleware integration and how it fosters inter-, intra-, and extra-NPO collaborations. In this paper, the authors refer to all types of integration or interoperability whenever it is unspecified.

Figure 1 shows a block diagram depicting the operation of a typical middleware. A middleware functions as an intermediary software layer between the operating system and the applications, facilitating the development of distributed applications [17]. It also aims to hide heterogeneity in the system and makes application development a much easier process [18]. Section 3 describes middleware in details.

1.1. Motivation

The value of the NPOs market size is at USD 289.25 billion and growing. Reports show that the NPO market had grown at a Compound Annual Growth Rate (CAGR) of 3.6% between 2015 and 2023, and the forecast predicts that it will reach USD 394.42 billion by 2030, growing at a CAGR of 4.53% from 2024 to 2030 [19]. The research cites three main factors causing the growth [19]: (1) An increase in disposable income in developing countries, leading to an increase in donations. (2) A growing trend among corporations to engage in Corporate Social Responsibility (CSR) activities, which often involve partnerships with nonprofit organizations, and (3) The rise in internet usage allows NPOs to reach a broader audience, engage with donors more effectively, and implement digital fundraising strategies.

Consequently, the nonprofit sector experienced a significant change in managing data. NPOs had to transition from relying solely on spreadsheets for donor and program management to using numerous software systems and applications. Each of these new tools serves a specific purpose, but they often operate independently from one another [20]. Researchers found that NPOs typically use between 10 to 20 different software solutions, leading to data silos that impede efficiency and growth [20]. A Nonprofit Technology Network (NTEN) survey found that 72% of NPOs use at least three unintegrated systems [20]. This fragmentation results in challenges such as increased time spent on manual data entry, decision-making bottlenecks, and impersonal donor communications. Middleware can integrate these applications breaking down these data silos and harness the full potential of donor information and encourage collaborative program activities by providing streamlined workflow.

However, middleware is little known in the nonprofit technology circle. A significant portion of nonprofit organizations appear to have limited knowledge about middleware. In a poll, 43% of respondents chose the option “What is middleware?” indicating a lack of understanding among a broad segment of nonprofits [21]. The concept of middleware, while not new, has not been effectively adapted to suit the nonprofit sector until years after its introduction in other sectors. This delay might contribute to the confusion and limited understanding in nonprofit circles [21]. Furthermore, there is little research on middleware for NPOs in the research community, as shown by Figure 2a,b. They show the annual number of scientific publications and citations of papers in middleware for NPOs, respectively.

Nevertheless, NPOs are now adopting technology in their day-to-day operations because of changes in the operation methods and the availability of robust and cheap information technologies [22]. Since NPOs rely on collaborations with each other, the government, and the for-profit sector, there are scenarios where they must use middleware to integrate their technologies with their partners’ systems to improve performance and reduce operation costs. This paper aims to stimulate interest and motivate efforts in middleware applications for the nonprofit sector. It provides valuable insights into middleware utilization in the nonprofit sectors, its applications, benefits, challenges, and opportunities. Thus, this paper aims to accomplish two goals: (1) introduce the nonprofit sector to middleware and how they can apply it in their operations, and (2) Present researchers with applications, challenges, open issues, and research opportunities for developing and utilizing middleware in the nonprofit sector.

1.2. Contribution

We survey the literature to present the state-of-the-art applications of middleware in the nonprofit sector. Identifying and analyzing these applications enables researchers and NPOs to uncover ways to improve workflows, strengthen data management, and promote better communication between various applications and partners. Other contributions of this paper are as follows:

1. We developed a taxonomy and discussed the different applications of middleware in the nonprofit sector.

2. We examined the middleware technologies used in the nonprofit sector as presented in the current body of literature.

3. We listed the benefits of middleware in the nonprofit sector from the perspective of the surveyed papers.

4. We discussed the challenges NPOs face in adopting middleware solutions, which clarifies the obstacles unique to the nonprofit sector, helping them navigate potential barriers in the future.

5. We also listed some research opportunities, suggesting ways to leverage middleware in such sectors to gain maximum benefit and open new horizons.

The remaining sections of this survey are as follows: Section 2 explains the methodology for this survey and discusses the research questions selected for this work. Section 3 introduces the reader to middleware, its architecture, and contemporary middleware for the nonprofit sector available in the market. Section 4 discusses middleware applications in the nonprofit sector. It also analyzes the relationship between the applications and the UN’s Sustainable Development Goals (SDGs [23]). The section also presents a taxonomy of middleware applications in the nonprofit sector according to the contemporary literature. Then Section 5 presents the types of middleware and technologies used. Section 6 discusses the benefits of using middleware in the nonprofit sectors. Section 7 summarizes the challenges and weaknesses encountered in this research. Finally, we discussed the opportunities and future work in Section 8 and concluded our work in Section 9.

2. Methodology

This survey used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA [24]) technique. It is a guideline for transparent and complete reporting of systematic reviews and meta-analyses [25,26]. It also provides recommendations for authors, peer reviewers, editors, and readers to ensure the validity and trustworthiness of the results [26]. Figure 3 shows the PRISMA flowchart for study identification and filtration processes. The figure breaks the process into Identification, Screening, Eligibility, and Inclusion.

In the Identification stage, PRISMA locates those publications that may be useful in the research. The flowchart’s first stage shows that we searched Scopus [27] and Web of Science (WoS) database [28], and Google search engine. We used Google to search for information about contemporary middleware technologies in the nonprofit sector and their applications. Section 3 used the search results to introduce middleware, its architecture, types, and the latest applications in the nonprofit sector. Figure 3 shows that we obtained 14 publications and website articles from the Google search.

The Scopus and WoS database search provided scientific publications for the remaining sections of this research because this paper aims to show what researchers are doing in this area. We carried out the Scopus-WoS search in two stages: In the first stage, we used “(TITLE-ABS-KEY (middleware) AND TITLE-ABS-KEY (nonprofit OR nonprofit) OR TITLE-ABS-KEY (philanthropy) OR TITLE-ABS-KEY (non-governmental) OR TITLE-ABS-KEY (humanitarian) OR TITLE-ABS-KEY (“third sector”) OR TITLE-ABS-KEY (“civic sector”))” as a search term in the Scopus database and its equivalent in the WoS database. The search term instructs the databases to return papers containing the word “Middleware” in addition to one or more keywords relating to NPOs (i.e., nonprofit, philanthropy, non-governmental, humanitarian, third sector, or civic sector) in their title, abstract, or keywords section. Narrowing the search to the nonprofit sector is necessary to ensure that we only include the papers that directly address the applications, benefits, and unique challenges of applying middleware in nonprofit settings. Figure 3’s Identification stage shows that the search returned 32 publications from both databases after removing nine duplicates.

In the second stage, we fine-tuned the search term to “(TITLE-ABS-KEY (middleware) OR TITLE-ABS-KEY (apache AND integration) OR TITLE-ABS-KEY (“privacy broker system”) OR TITLE-ABS-KEY (“ArcGIS”) OR TITLE-ABS-KEY (broker AND message) OR TITLE-ABS-KEY (“MQTT”)) AND (TITLE-ABS-KEY (nonprofit) OR TITLE-ABS-KEY (nonprofit) OR TITLE-ABS-KEY (“NPO”) OR TITLE-ABS-KEY (philanthropy) OR TITLE-ABS-KEY (non-governmental) OR TITLE-ABS-KEY (humanitarian) OR TITLE-ABS-KEY (“third sector”) OR TITLE-ABS-KEY (“NGO”))”. The new search term added keywords closely related to middleware, which we derived from the selected papers of the initial Scopus-WoS search. We experimented with several, retaining only those that contributed to an increase in the number of relevant publications in the results. These modifications increased the search result to 72 papers, becoming 55 publications after removing publications already in the first search. We used only the Scopus database in this stage because it returned more hits in the first stage.

The Identification stage generated a total of 101 publications after removing 26 duplicates. The next stage is the Screening for removing invalid entries and publications that did not meet the inclusion criteria or have met the exclusion criteria. Table 1 shows the inclusion criteria that decide which papers are eligible for the survey. A publication must meet all inclusion criteria to enter the survey paper pool. The attribute column shows the properties we used for the inclusion, while the description column explains the inclusion condition for that attribute. We were flexible with the conditions (especially Recency) to have enough papers to extract intelligible information. Table 2 shows the exclusion criteria. A publication is removed from the survey pool if it falls within one or more of the exclusion criteria. The first column is the serial number for brevity’s sake, the second column is the attributes we used for our exclusion criteria, the third column explains the condition for ineligibility, and the fourth column shows the number of excluded papers.

In the end, Figure 3 shows that only 31 papers remained after removing unqualified ones. Five entries were either invalid or the full text was unavailable. We also excluded nine publications whose titles did not match the scope of our research. Then, we excluded 33 others whose titles suggest that they are within the scope of our work, but a deeper investigation of their abstracts indicates that they are not a match. Table 2 shows the number of excluded papers and the respective exclusion criteria that warranted it. Although we started the screening process with 101 publications, we ended with only 31 papers. So, why did the search process start with numerous papers? We found that the unusually high false positives were due to searching all fields for the keywords in the WoS database. As a result, the database returned papers with our keywords in some authors’ affiliations. On the Scopus database side, the additional keywords in the second database search have many synonyms in the nonprofit sector. For example, a Broker in the nonprofit sector is a professional who acts as a middleman, connecting nonprofits with potential donors, funders, or resources. While in middleware, a broker is a software component that facilitates communication between different applications or services. Also, some software like ArcGIS v10.1has many applications in the nonprofit sector, but few of them use or mention the usage of its middleware (like Fiware Summit v3.4.18 (for ArcGIS Pro v3.3) and ArcSDE v10.2.2). Nevertheless, these few pioneering papers gave us an insight into research directions in Humanitarian Action and Development Engineering (HADE) and the nonprofit sector. They also show us the challenges researchers are facing in this area.

We downloaded and studied the articles and market reports [29,30,31,32], and read the website articles [21,33,34,35,36] from the Google search results. Section 3 presents them as a mini-survey of middleware, bringing readers up to speed about introductory information on middleware. Then, we downloaded the 18 articles [37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54] on middleware in the nonprofit sector from the database search and use them for the rest of the paper. We also added the website article on OpenFn [16]. It is an open-source middleware for NPOs. It deserves an appearance because it is an ideal example of the middleware for this research.

To comprehensively review the publications, we utilized an extraction table based on the set of research questions in Table 3. The table presents the research questions for this research. The first column shows the identifiers for each research question. The second column contains the research questions, and the third explains their meaning. This paper aims to inform researchers that some NPO operations require middleware technologies. Thus, we chose these questions to cover various aspects of middleware applications in NPOs. RQ1 investigates all applications of middleware in the nonprofit sector, whether in their internal operations or the services they provide. RQ2 investigates the types of middleware NPOs use, which allows researchers to study the diversity of middleware usage. The question also checks the type of technologies used together with the middleware to show the heterogeneity of the subsystems. RQ3 analyzes the payoff for deploying middleware in the various applications. RQ4 investigates the challenges of deploying and managing middleware-based systems in NPOs since their core principles (e.g., humanity, independence, impartiality, neutrality, openness) differ from their for-profit counterparts. Then, RQ5 helped us explore the areas of advancement, development, or integration of middleware in the nonprofit sector where researchers can contribute. Note that although we answer the research questions (RQ1–RQ5) using only the 18 papers [37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54], we go beyond them in explaining concepts, definitions, and elaborations to justify our answers.

3. Middleware Overview

3.1. Definition and Evolution

Bernstein defines middleware as a service that operates between applications and the OS’s API, as well as other processor architecture-defined low-level services and processing elements [29]. Microsoft defined it as software that sits between an operating system and applications, enabling communication and data management for distributed applications and connecting multiple applications to share data or databases [21]. Bishop and Karne gave a more comprehensive definition [30]: Middleware is the software that assists the interaction or communication of an application with other applications, networks, hardware, or OS, assists programmers by hiding distributed system’s complexity while providing tools for improving quality of service (QoS), security, message passing, directory services, file services, etc. that can be user transparent.

The term “middleware” appeared in the late 1960s to describe software that mediates between a client application and back-end resources like databases, mainframe applications, or specialized hardware from which the client might request data [33]. The rise of Personal Computers (PCs) in the 1980s significantly contributed to the emergence of widespread distributed systems, which led to the growth of middleware usage in facilitating connections between modern applications and older legacy systems [33]. The advent of distributed systems came with several challenges to the development of enterprise-wide information systems [29]:

1. Integration difficulties due to a variety of hardware systems and operating systems.

2. Inconsistent user experience due to distinct user interface for different servers.

3. Limited access to shared resources due to local-area network isolation.

4. Applications inaccessibility due to different local server configurations.

5. Limited wide-area server support for personal computers.

6. Complex login process due to numerous access requirements by multiple servers.

7. PCs may need additional programming to retrieve a remote application’s data.

To address these challenges, vendors offer distributed system services—middleware services or middleware—having standard programming interfaces and protocols. The standard programming interfaces facilitate the porting of applications across various server types, and the standard protocols enable programs using the same protocol to interoperate, such that a program on one system can access another system’s program and data [29].

Modern middleware forms the technological foundation for cloud-native architectures, particularly for organizations operating in multi-cloud and containerized environments. It enables cost-effective development and scaling of applications, which is essential in today’s dynamic digital landscape [34]. In the realm of the Internet of Things (IoT)—a vast network of interconnected devices (e.g., smartphones, tablets, sensors, smart watches, etc.) gathering real-time data—middleware plays a crucial role in managing a network of devices that collect real-time data, transmitting it to cloud or edge-computing services [31]. Addressing security, energy consumption, reliability, and communication is vital when developing IoT applications. Middleware—whether corporate, open-source, or device-specific—provides an abstract layer that facilitates these essential functions, streamlining the complexities involved in IoT application development [31].

3.2. Middleware Architecture and Properties

Figure 4 illustrates the top-level architecture of middleware, depicting several applications from different platforms connecting to the middleware via Application Programming Interfaces (APIs). The middleware interfaces with the platforms’ operating systems and hardware, facilitating communication between applications and various platforms through its components and services. This setup enables application-to-application, application-to-platform, and platform-to-platform communications. Table 4 presents components typical to most middleware, according to Amazon AWS [35]. The table’s first column lists the component names, and the second column briefly describes them.

Middleware, while challenging to define precisely, possesses distinct characteristics that set it apart from applications or platform-specific services. These key properties include [29]:

1. A middleware is inherently a distributed system. It allows remote access to services and applications.

2. An ideal middleware must be versatile. It should meet the needs of a variety of applications across many industries.

3. A middleware abstracts complex network and distributed system functionalities, providing high-level services that simplify application development.

4. Middleware services must provide platform-independent APIs, allowing applications to run on multiple platforms, which enhances their portability.

5. A middleware service should be API-transparent, enabling developers to use the API to access the middleware without needing to change the API itself.

6. A middleware should support standard or at least published protocols, ensuring interoperability across different implementations. It is crucial for developing multiple implementations that can work together.

3.3. Taxonomy

Figure 5 shows a taxonomy of middleware we compiled from [21,30,36]. Karne and Bishop [30] classify middleware into two main categories: integration and application. The integration category classifies middleware based on the heterogeneous systems integration method. The taxonomy further divides the middleware according to its architecture or location in the system. The location category shows that middleware can be Online-based or On-premises. The Online-based middleware integrates systems over the Internet or a Wide Area Network (WAN). The On-premises middleware integrates systems in-situ, in the same local area network (LAN), building, or PC.

Researchers designed middleware with a specific application in mind. These middleware fall into the application category. Karne and Bishop [30] listed data access, desktop, realtime, and specialty middleware. The data-access middleware integrates applications with local or remote databases, data source files, or data warehouses. Desktop middleware enhances data presentation, manages transport services like file transfer and printing, and provides backup and operational support. Realtime middleware supports time-sensitive heterogeneous systems. The authors grouped all middleware with a specific use case under the Specialty middleware category.

3.4. Nonprofit Middleware Examples

Middleware aims to integrate heterogeneous systems. A heterogeneous system is a computing environment with diverse devices and platforms that vary significantly in hardware, software, capabilities, and resources. Thus, there is an increase in the demand for middleware due to the current rise of IoT systems and cloud computing solutions. Technavio reported that the Global Middleware as a service Market size will increase by USD 45.44 billion, at a Compound Annual Growth Rate (CAGR) of 17.73% between 2023 and 2028 [32]. The adoption of cloud computing solutions by NPOs to cater to their myriad activities is driving the adoption of middleware solutions. These technologies offer enhanced agility, scalability, and interoperability, facilitating streamlined operations and increased social impact. The shift towards cloud-based middleware solutions empowers NPOs to better their services affordably.

Table 5 shows an example of some middleware used in the nonprofit sector available in the market. The columns show the market name of the middleware, the name of the company that developed the middleware, whether it is open source, a brief description of the middleware, and examples of some NPOs currently using the middleware, respectively. We linked the middleware names to NPO customer pages or the middleware’s website where the earlier is unavailable. The remaining sections of this paper discuss papers on the development, technologies, and application of middleware in the nonprofit sector.

4. Applications of Middleware in the Nonprofit Sector

Figure 6 shows the VOSviewer map’s network visualization of the selected papers. VOSviewer is a Java-based open-source application for creating and visualizing bibliometric and scientific maps [55,56,57]. The word VOS stands for visualization of similarities [56]. In Co-occurrence mode, the software uses the distance-based graph to show the relationship between the keywords in the given papers, like Figure 6. It is an undirected graph with the vertices representing the keywords. The edges represent the pair of vertices (i.e., keywords) that co-occur in at least $\delta$ papers, where $\delta$ is a threshold value. We selected $\delta =2$ in this research because we have few papers. It means a keyword must appear in at least two papers to appear on the map. Also, the node size for a given keyword is proportional to the number of publications where the keyword is found (i.e., the number of occurrences). The distance between two connected vertices and the thickness of their edge is directly proportional to the number of papers the keyword pair appears together (i.e., the number of co-occurrences). Also, VOSviewer groups the keywords into clusters with the same color. A cluster consists of a set of keywords closer to one another co-occurrence-wise. Thus, a keyword is more likely to co-occur with any member of its set than non-members. More details on VoSviewer’s clustering technique are available in the literature [58].

Figure 6 shows the network map of the keywords from the selected papers with $\delta =2$. VOSviewer has a thesaurus feature that allows users to merge similar keywords that refer to the same concept into a single term. It also allows users to correct typographical errors, unify spelling variations, and reconcile abbreviations with the full terms or vice versa [56]. A thesaurus file consists of two columns; the first column contains the keyword, and the second column contains the word replacing it [56]. Table A1 in Appendix A shows our thesaurus setting for Figure 6. The first column shows the label, acronym, or group word VOSveiwer uses to replace the publications keyword, while the second column shows the keywords from the publications. Thus, Figure 6 displays the labels in the first column for any words in the second column. We highlight the keyword “npo” and all the words linked to it for brevity. An overview of the figure shows “middleware” as the largest node on the map and the closest to npo, which is intuitive since the selected papers are about middleware and the nonprofit sector. Also, the figure shows three clusters. In the top-left cluster, the words information and management are dominant. In the bottom-left cluster, humanitarian is more pronounced, emphasizing the importance of IoT and related technologies in Humanitarian Assistance and Disaster Response (HADR). The right cluster has middleware as the most prominent, followed by services and npo, showing their close relation. Further reading of the selected papers supports the figure by showing that the main application of middleware in the nonprofit sector is to cut costs in information management, data integration, and humanitarian services.

We noticed some SDG-related keywords (i.e., disaster, interoperability, integration, education, humanitarian, and health) in Figure 6. So, we decided to group the publication keywords according to the SDGs as shown in Table A2 in Appendix A to produce the map in Figure 7. The figure shows the appearance of seven SDGs out of the 17 on the map. Note that three SDGs, climate action, clean water and sanitation, and life below water, did not appear on the map because they were in only one paper. Table 6 shows all the SDGs in the selected papers’ keywords section. The ones on the map are all connected to the npo keyword, which shows a strong relationship between NPOs, middleware, and the SDGs. However, the map shows that some SDGs occur more than others. The highest occurrence goes to SDG 9 (Industry, Innovation, and Infrastructure). It has 16 occurrences because most publications use middleware to improve information technology to increase NPOs’ performance. It is followed by SDG 16 (Peace, Justice, and Strong Institutions) because we linked it to civil defense, security, HADR, and disaster management. We included disaster management in the SDG 16 category because they are closely related to law enforcement institutions [59]. Then follows SDG 17 (Partnerships for the Goals) because some NPOs use the middleware to overcome system heterogeneity with their partners to ensure interoperability. SDG 4 (Quality Education) and SDG 11 (Sustainable Cities and Communities) have only three occurrences, while SDG 3 (Good Health and Well-being) and SDG 8 (Decent Work and Economic Growth) have two. Thus, the map shows that any middleware application in NPO operations requires progress in industry, innovation, and infrastructure (SDG 9). It also shows more middleware research on disaster management than other applications.

4.1. Taxonomy

Many NPOs identified that using middleware in their system will improve their performance and solve many communication and management problems due to its ability to bind different applications and utilize heterogeneous resources effectively. Figure 8 shows that we can categorize middleware applications in NPO according to their function or application domain. For brevity, we color-coded the figure such that the children nodes on the same level have the same color. For example, Education, Environment, Healthcare, Security, Volunteering, and Humanitarian Activities are on the same level under the Domain branch. Under the Education node are the Educational Access and Equity (EAE) and the Research nodes, while under the EAE is the Technology Enhanced Learning (TEL), and under the Research are Modeling and Simulation and Virtual Laboratories nodes.

4.1.1. Functions of Middleware in NPO

Classification by function refers to the purpose of the middleware in the system. Figure 8 shows that the NPOs use middleware for system or information management. The system-management category refers to how middleware manages the system’s hardware or software.

1. Integration: Integration middleware bridges disparate cloud-based or on-premises systems, subsystems, or applications, enabling them to connect, share data seamlessly, and work together as a unified whole [36,60]. The integration can occur at different levels, including hardware, software, communication systems, business applications, administrative processes, manufacturing, distribution, and communication to ensure seamless collaboration and cooperation [60,61]. Thus, integration middleware helps NPOs collaborate. However, NPOs collaborate in three ways: between departments in an NPO, between two or more NPOs, and between NPOs with entities that are not in the nonprofit sector. We call them intra-NPO, inter-NPO, and extra-NPO collaborations, respectively. Subsequently, we have three types of middleware integration, as shown in Figure 8:

   • (a). Intra-NPO Integration: It refers to connecting different systems used by one or more departments within the organization. A single department may have many applications performing the same function, leading to data silo with data fragments across multiple platforms. NPOs use integration middleware to solve this problem [62]: International Justice Mission (IJM) is an NPO that rescues victims from violence and slavery. They used Boomi integration Platform as a Service (iPaaS) middleware to integrate their CRM (Salesforce) and donor payment systems (Stripe and Braintree), giving them complete control of donors’ information [62]. Also, AnglicareSA, an Australian NPO, uses the Boomi iPaaS to connect all their business applications, allowing decision-makers to gather vital information.

   • (b). Inter-NPO Integration: This type of integration connects systems from two or more NPOs, enabling NPO-to-NPO collaborations. In the aftermath of a natural disaster, multiple NPOs often collaborate on relief efforts. A cloud-based integration platform (middleware) can significantly enhance these collaborative initiatives by providing a centralized hub for coordination, information sharing, and resource management.

   • (c). Extra-NPO Integration: Extra-NPO integration facilitates system connectivity between nonprofit organizations and entities in the public or private sector, enabling cross-sectoral collaboration and maximizing the societal impact of NPOs. The diverse nature of NPO operations often necessitates partnerships with organizations that may employ different systems, leading to technological heterogeneity. Middleware plays a crucial role in addressing this challenge. For instance, Winer and Pavkov [37] demonstrate how middleware fosters collaboration between civilian government agencies and NPOs in creating an integrated human service data system. This approach bridges the technological gap and enhances the efficiency and effectiveness of cross-sector partnerships, ultimately improving service delivery and outcomes. Other researchers [50,52,53] proposed a middleware framework for Military-NPO collaboration in Humanitarian Assistance and Disaster Relief (HADR).

Centralization: In this paper, system centralization refers to consolidating control, decision-making, or resources within a single central point or entity in a distributed system. Middleware can centralize a distributed system by providing a transparent and comprehensive view of the distributed resources with which it can interact [63]. It achieves this by managing the coordination of applications and providing mechanisms for their integration [63]. Middleware acts as a layer of abstraction, masking the complexity of the underlying distributed system and providing a high-level programming model [64]. It allows processes to communicate data between them without worrying about network configuration and management [65]. By centralizing the management and coordination of distributed resources, middleware simplifies the development and operation of distributed systems [63]. Oceans Network Canada (ONC) is a University of Victoria-based nonprofit oceans observatory [66]. They used Service-Oriented Architecture Middleware (SOAM, see Section 5) to collect data from heterogeneous sensors and make it available to the public [44].

Decentralization: Managing distributed systems are the most widely used application of middleware [67]. It plays a crucial role in the decentralization of a system architecture by providing a layer of abstraction and facilitating effective management of distributed systems [63,64,67]. It allows for the adaptation of distributed computing systems to different workloads and volumes of data, regardless of the system’s size [67]. By providing mechanisms for coordination and communication between applications and components, middleware enables the seamless integration of distributed resources and improves the efficiency and fault tolerance of the system [63,68]. Additionally, middleware helps address the complexity and heterogeneity of contemporary distributed systems, making them more dynamic, responsive to change, and suitable for emerging areas such as smart cities and emergency response systems [64]. An example of middleware for distributed nonprofit applications is the peer-to-peer (P2P) middleware used by NPOs for humanitarian missions and content delivery in areas with limited or no network connectivity [38,45].

Sometimes, researchers aim to manage information curated by an NPO rather than its system’s hardware or software. We found that several NPOs use middleware to optimize data storage, enhance information dissemination, or both.

1. Information Storage: Middleware is crucial in improving information storage by providing facilities that adapt distributed computing systems for different workloads [67]. It helps manage large volumes of data efficiently in distributed systems of any size [67]. Middleware also works with data of any volume, making it adaptable to changing storage needs [67]. By implementing middleware, NPOs can improve fault tolerance and ensure effective management of their storage systems [67]. It improves overall system response time and performance by managing data and I/O requests [69]. NPOs in the health sector use middleware for information storage. They are not interested in disseminating the health information of their beneficiaries. Instead, they want to gather the information from all branches and other sources for storage and further analysis for logistics and fundraising purposes. Hence, they use middleware to collect information from heterogeneous subsystems and store it in a unified format. Suprihadi et al. [47] developed a Service Oriented Middleware (SOM, see Section 5) that manages service information from heterogeneous tuberculosis (TB) data sources to make a comprehensive TB database.

2. Information Dissemination: Middleware improves information sharing by optimizing computing and communication resources usage through load balancing, improving fault tolerance and scalability [67]. Alternatively, middleware that supports peer-to-peer content distribution systems can improve information transfer, enabling wireless content dissemination between users without relying on infrastructure support and facilitating the exchange of contents between the wireless ad-hoc domain and the wired Internet [70,71]. NPOs in humanitarian action sometimes find themselves in areas without state-of-the-art communications infrastructure. They often operate in conflict-stricken places with destroyed, damaged, or decommissioned communication infrastructure. Other NPOs operate in remote areas without network infrastructure, such as the ocean, Antarctica, or villages in underdeveloped countries. Thus, they use middleware to develop P2P distributed networks to share information within their organization, with other organizations, or with their beneficiaries [38,45].

3. Hybrid Information Management: This category, in this paper, refers to those applications where NPOs use middleware to store, retrieve, and disseminate information. NPOs source data from distributed systems and share it with their branches, other organizations, or the public. Middleware is essential for managing such systems because of its fault tolerance and adaptation to different workloads, regardless of the system size or data volume [67]. Middleware also uses QoS settings to support users from various networks, which helps NPOs or sensors deployed in remote areas. It ensures reliable messaging, supports different communication models, and provides strategies for data recovery and traffic management, which makes it crucial for facilitating communication and data dissemination in NPO activities such as in financial [72] and IoT [73] scenarios. For example, ONC developed an SOAM-based Data-Management and -Archiving System (DMAS) [44]. The system gathers data from its vast network of IOT-based ocean observatories for storage in its database-management system (DBMS) and sharing with the public.

4.1.2. Application Domains of Middleware in NPO

The right-hand side of Figure 8 shows the classification of middleware applications by the NPOs’ domain of operation. We define application domains, in this context, as the specific fields or areas in which the organizations operate. They encompass a variety of sectors and industries [1]. However, the figure only shows those domains in the literature where the nonprofit sectors currently use middleware. They are as follows:

1. Education: Due to its importance, quality education is the fourth SDG after no poverty, no hunger, and good health SDGs. Education refers to the deliberate efforts to enhance people’s knowledge, skills, goals, and values [74]. NPOs contribute to education through various avenues, including but not limited to EAE, teacher training and support, curriculum development and educational programs, advocacy and policy work, technology and innovation in education, and research. However, the literature shows that researchers only used middleware in the EAE and the research area.

   • (a). Educational Access and Equity (EAE): In the EAE area, researchers argue that technologies can enhance learning. The term Technology-Enhanced Learning (TEL) refers to the usage of information and communication technologies in teaching and learning [75]. It has become a global phenomenon to use new communication technologies to eliminate barriers like location, communication, and cost of learning [76]. This study [41] proposed the SINUS system, which aims to provide an environment that allows its users to develop TEL applications. This environment has three layers: a storage layer made of heterogeneous repositories to store the data, a tool layer containing tools for processing different types of resources that support active learning, and the middleware layer that coordinates the tool and the storage layer.

   • (b). Research: In the Research area, scientists used middleware to enhance their modeling and simulation computer systems and the development of virtual laboratories. NPOs interested in water-related research use various simulation and modeling techniques to simulate the dynamics of water availability. However, sophisticated modeling requires gathering land shape, soil qualities, climate, and hydrography data. Dierauer et al. [49] proposed a Future Gateway System using Apache Airavata gateway middleware that integrates and organizes the data input in the simulation system. The gateway improves the evaluation of future projections and validation against historical data.

Environment: An environment is the surrounding conditions in which an organism or community lives. It encompasses the physical, chemical, and biological factors that influence the life and activities of living beings [77]. Hence, the NPOs that focus on the environment do research, monitor, or advocate policies that protect the waters, land, or cities.

• (a). Waters: Water is vital for all lives, and the accessible amount varies across different geographic locations and times [78,79]. One application of using middleware in the nonprofit sector is its usage in water science. Scientists have developed various modeling and simulation techniques to forecast the dynamics of water availability. However, sophisticated modeling requires compiling a variety of data such as the land shape, soil qualities, climate, and hydrography data, which researchers use middleware to help improve their simulator’s performance as we mentioned earlier [49]. According to the literature, scientists mainly use middleware to overcome sensor heterogeneity during data acquisition in the environment. Oceans 2.0 [44] and now Oceans 3.0 [80] are DMAS that utilize the SOAM to collect data from heterogeneous sensors, store them, and make them accessible to governmental and non-governmental users.

• (b). Land and Cities: In a similar system, ONC deployed sensors on the land and underwater for an earthquake early warning system for southwestern British Columbia [81]. The system quickly estimates the location and magnitude of earthquakes near the Cascadia Subduction Zone near Vancouver Island [81]. The authors developed a middleware that processes the data from the land-based stations. They reported that the middleware helps the system provide more reliable magnitude estimates in the early stages of an earthquake. Geographic Information Systems (GIS) have become invaluable tools for NPOs, allowing them to visualize data spatially, analyze patterns and trends, and make informed decisions. Fiware-ArcGIS [82] is a middleware that integrates data between the FIWARE platform and ArcGIS software. This integration allows seamless data exchange between IoT devices and geographic information systems, enabling more comprehensive and insightful spatial data analysis. Middleware also finds application in recording landslide information [54]. Landslides in Poland are significant hazards, but a lack of systematic data collection makes risk estimations difficult. In 2008, the Polish Geological Institute-National (PGI) Research Institute created SOPO, a national landslide inventory using GIS technology for improved terrain management and hazard assessment for administrative bodies, environmental protection inspectorates, and NPOs. It is a distributed web-based inventory system that uses the Oracle 10g database for data storage and ArcSDE [83] middleware to intermediate between ArcGIS Server and the database [54]. Experts verify and store the landslide forms reporting the incidence in the system’s database, which is accessible to the public. Thus, the system is not real-time-based. However, the developers are planning towards near-real-time monitoring in the future [54].

Healthcare: The aim of SDG 3 is to guarantee healthy lives and well-being for all, regardless of age. It encompasses various targets on improving global health outcomes, reducing morbidity and mortality rates, and achieving universal health coverage [84]. Good healthcare focuses on delivering effective care that leads to positive patient outcomes and satisfaction [85]. Different studies were conducted on using information technologies in the health field, especially for tuberculosis [86]. One application of middleware is in the development of Patient Information-Management Systems (PIMSs). Research shows a lack of PIMS for tuberculosis (TB) patients [87]. Therefore, further study is required to develop technologies that can enhance its treatment compliance [87]. Suprihadi et al. [47] applied SOM to manage TB information. TB NPOs are stakeholders involved in managing TB information. They provide accompanying data and information for TB patients. The middleware enhances information management from the heterogeneous integrated TB service data sources. As a result, complete and detailed data and information on TB can be acquired and processed according to the needs of each connected stakeholder, which aids in initiatives to increase patient medication adherence.

Humanitarian Activities: According to the Professionals in Humanitarian Assistance and Protection (PHAP), Humanitarianism involves addressing the needs of people affected by natural disasters, famines, epidemics, and conflicts [88]. Some of the areas where middleware finds application in humanitarian activities are:

• (a). Humanitarian Logistics: Humanitarian logistics refers to managing and coordinating logistics and transportation in humanitarian operations [89]. It is vital in effective humanitarian aid, ensuring speedy and effective relief efforts [90]. Samano et al. [40] focus on developing identification systems for humanitarian relief applications. They used low-cost Radio Frequency Identification (RFID) platforms with added sensing capabilities. The platform utilizes open-source middleware components and enhances a legacy RFID reader platform with two additional modules for sensor data collection and wireless communication. Although the system shows near-real-time tracking performance, it relies on network infrastructure, which is unavailable in many humanitarian efforts, especially in conflicts or natural disasters. Taylor and Arthanari [51] developed a Zigbee mesh Wireless Sensor Network (WSN) for Supply Chain Visibility (SCV). WSN is a network of resource-limited devices for remote environment sensing. The system solves the infrastructure issues since WSN does not require network infrastructure. Although Zigbee mesh is self-healing when some nodes are down, energy-efficient, and reliable, it has low bandwidth, limiting the transmission of logistic data [51].

• (b). Disaster Management: The United Nations Office for Disaster Risk Reduction (UNDRR) defines disaster management as the application of policies and strategies for disaster risk reduction aimed at preventing new disaster risks, reducing existing risks, and managing residual risks [91]. Thus contributing to strengthening resilience and reducing disaster losses [91]. Coping with catastrophic events requires the cooperation of different entities to bring together resources [92]. In the disaster-management field, multiple protection groups—the government, medical assistants, humanitarian aid workers, and firefighters—must communicate and respond effectively to satisfy the needs of the victims. Researchers [48] proposed an architecture that optimizes the interoperability platform between distributed information systems that eases the information exchange in emergencies. The architecture includes a middleware layer that coordinates the publishing or accessing of information for each system integrated into the platform. Another example of a disaster-management study is Aceso middleware [50]. It is a proof-of-concept smart city middleware that supports complex federated HADR operations involving civilian and military organizations. The middleware utilizes location and context-aware services to offer a range of functionalities (like resource discovery and secure information exchange) that require quick execution to anticipate unexpected and critical events. Pradhan [52] developed a similar system using MQTT free and open source middleware while [53] used Apache Kafka and Hyperledger Fabric blockchain to ensure security.

• (c). Incident management: There is a distinction between disaster and incident management. Disaster management encompasses the entire lifecycle of a disaster, including prevention and recovery [93]. But incident management specifically focuses on the decision-making and response phases during an incident or disaster [94]. Incident management is an area that has gained particular attention because failing to respond promptly will lead to loss of life and damages to private and public properties [95]. Moreover, it is receiving more attention because of the growing concerns regarding climate change and the increasing frequency of extreme weather events [96]. The study conducted by [42] developed a prototype middleware as a resource-sharing framework for incident-management systems. The system intends to support emergency responders with tools that help them to gather crucial incident-related data from disparate organizations and institutions, evaluate the data, share mission-critical information with other emergency responders efficiently, and coordinate their efforts. It also asses the decision-making based on the incident situation and stores incident-related data for analysis.

Security: For this research, security includes measures to safeguard against physical or digital crimes, protecting buildings, personnel, information, data, networks, and systems. Middleware assists security officers in the physical security of the community. It also finds applications in cybersecurity to secure systems from cybercriminals. The following are the applications of middleware in NPO security:

• (a). Physical Security: WSNs are effective during emergencies such as natural disasters or when securing a crime scene because they require less power and smaller sensors that are easy to maintain. In this research [43], a framework is offered to monitor crime scenes and areas affected by natural disasters using a WSN connected to the cloud. The framework utilizes a middleware to serve as the interface between the sensors and the cloud-based services. The sensors will send their collected data to the cloud, where the middleware processes, analyzes, normalizes, and sends it to the Rescue and Crime Information System (RCIS). The RCIS detects disasters and criminal activities and shows the severity and other necessary details. In [39], the authors focused on chaotic environments with damaged infrastructure requiring rapid intervention due to humanitarian or natural catastrophes and civil or political crises necessitating specialized civil or military missions. They proposed a ubiquitous mobile learning framework designed for training and preparing personnel for specialized civil or military missions. This framework uses a hybrid networking middleware that is platform-independent. It is built based on the Network Centric Operations (NCO) concept to enable flexibility and quick adjustment to shifting circumstances and surroundings. NCO is a control concept that uses networks for seamless collaboration and information sharing across different systems and organizations [97]. It also uses location immersion techniques to replicate the impact on the actual world.

• (b). Cybersecurity: Hosting providers play a vital role in preventing web compromise. However, their success lies in their customers’ following security measures as well [98]. The following study [38] contributes to helping NPOs and Small and Medium-sized Enterprises (SMEs) when their web servers go down due to flash crowd effects. Flash crowd effects, in the context of web service, refer to situations where too many users access a web service simultaneously, which causes problems such as system downtime, user abandonment, and inefficient resource utilization [99]. Alas, NPOs and SMEs cannot afford to host their websites on a proprietary Content Delivery Network (CDN) where they do not have a high load daily. The authors proposed a scalable Web Hosting environment by implementing a CDN on top of a structured P2P middleware system (Distributed K-nary Search) of collaborative web servers. It can distribute the load among the servers to improve the availability, scalability, and performance.

Volunteering: Volunteering is a form of prosocial behavior where individuals commit their time and effort to help others through organizations [100]. Volunteering impacts NPOs positively by providing valuable resources and support. Middleware finds applications in Volunteer Computing (VC) and Volunteer Thinking (VT). According to [101], VC is a type of distributed computing that allows participants to donate their unused computing resources to help run computationally intensive projects. While VT involves volunteers contributing their cognitive skills and knowledge to solving scientific problems [46]. Yadav [46] demonstrated how a job distribution middleware can integrate the Virtual Atom Smasher (VAS [102]) game and CitizenGrid to develop a web-based platform that integrates VC and VT. The system allows students to learn from the game using volunteers’ computing resources. The system is dynamic as it lets the VC providers select the VAS team to which they want to donate their computing power through the CitizenGrid platform.

5. Middleware Types and Technologies Used in the Nonprofit Sector

We surveyed the types of middleware used in the literature. Table 7 show the middleware for each study reviewed. The table has five columns; the first column cites the paper that presented the middleware, the second column is its application domain, the third column is the middleware type or its name, the fourth column presents the actual developer of the middleware, and the last column shows other software that works together with the middleware in the paper.

Several papers used an existing open-source middleware in their research [40,45,46,49,51,52,53,54]. It helps the NPOs reduce system development costs. However, developers may need to modify the middleware to fit the NPO’s requirements. Ramiro and Atilio [40] modified an existing middleware as part of the ASPIRE project [103] to reduce implementation costs and promote RFID technology adoption. They modified the Low-Level Reader Protocol (LLRP [104]) and the Application Level Event (ALE [105]) to handle and transport the sensor data coming from the reader.

Also, only two of the studies proposed middleware-based systems general applications [37,38]. Winer and Pavkov [37] listed middleware and technologies that can serve as middleware (see row 1, column 3 of Table 7) to enable data sharing between government agencies and NPOs, while Jernberg et al. [38] proposed DKS Organized Hosting (DOH), a middleware for content delivery in a P2P network.

Another observation is that many studies use distributed architecture styles [106]—service-based architecture (SA) and service-oriented architecture (SOA)—for their middleware design. SOM is a middleware developed based on SA, while SOAM is a middleware based on SOA. SA is an architecture that breaks the system’s function into separate units of well-defined independent domains, while SOA breaks the system into a set of business services connected to a set of enterprise services [106]. SOA offers high interoperability and abstraction, but it is more expensive to develop, deploy, and maintain than SA. From case studies and funding sources in the literature, we deduce that large NPOs use SOAM [41,42,44], while medium and small NPOs use SOM [43,47] to cut costs.

To further reduce development costs, some researchers customize existing middleware into SOM. Suprihadi et al. [47] converted Message-Oriented Middleware (MOM) in [107] into SOM because they pointed out that MOM is not suitable for TB information due to its dependency on a fixed predefined metadata structure, which limits its ability to meet the diverse needs of TB information. They successfully make the conversion by adding an analysis builder layer, which allows users to add data-analysis methods they deem vital for patients.

Table 7

Types of middleware and technologies used in each study.

Ref	Domain	Middleware	Source	Ecosystem
[37]	General	Authors proposed XML, Java, Powerbuilder, Information Builder’s EDA, or WebFocus middleware	Not Applicable	Email and messaging tools, Local area networks (LAN), Wide area networks (WAN), Virtual private networks (VPN), Routers and Firewall servers, Enterprise web servers, DSUCSU devices, RDBMS or OODBMS
[38]	General	Distributed K-nary Search (DKS)	Authors	P2P Network, Distributed Hash Tables (DHT), CDN
[39]	Security	Java Ad-Hoc Network Environment (JANE) middleware	Authors	JANE Simulator, Link Layer Network Emulator, XML
[40]	Humanitarian Activities	ASPIRE middleware	Modified [103]	RFID tags or Transponders, Reader or Interrogator, Middleware, Back-end processing servers, Sensor modules, Wireless communication modules
[41]	Education (EAE)	SOAM	Authors	Service-Oriented Architecture (SOA), Extended Search Engine, Semantic Annotation Editor, Ontology Editor, Learning task editor, Web service, Repositories, XML
[42]	Humanitarian Activities	SOAM	Authors	Web Service, Integration Services Sensor, Geographical interface system (GIS), Sensor and Visualization Services, Reasoning and inference engine
[43]	Security	SOM	Authors	Wireless Sensor Network (WSN), Cloud Computing, JAVA, Google App Engine and Google SQL, Mobile Ad-Hoc Network (MANET)
[44]	Environment	SOAM	Authors	Web service, Sensor Observation Services, SensorML markup language
[45]	Education (EAE)	Bittorrent Sync (BtSync)	[108]	Graasp, P2P Network
[46]	Volunteering	LiveQ middleware	[109]	VAS game and CitizenGrid [110]
[47]	Health	SOM	Authors	Analysis Builder, REST web service
[48]	Humanitarian Activities	Authors developed a middleware layer	Authors	MySQL, OpenVPN, WebGIS
[49]	Environment and Education (Research)	Apache AiravataTM gateway middleware	[111]	Soil and Water Assessment Tool (SWAT), MySQL, Wagtail content-management system, R-Shiny data visualization, Keycloak identity-management system
[50]	Humanitarian Activities	Aceso middleware	Authors	Multi-domain Asynchronous, Gateway-Of-Things (MARGOT), Sieve, Process, and Forward (SPF), Attribute-based access control (ABAC), Communication layer, Controller
[51]	Humanitarian Activities	Mosquitto Message Broker	Eclipse Mosquitto [112]	ZigBee, Mosquitto Message Broker, MySQL, Python, PageKite, Nginx HTTP Server, Sahana Eden Enterprise Resource Planning (ERP) System, Raspberry Pi
[52]	Humanitarian Activities	Mosquitto Message Broker	Eclipse Mosquitto [112]	MQTT, VerneMQ, HiveMQ, Vaadin framework, OpenStreetMaps, JSON
[53]	Humanitarian Activities	Apache Kafka	ASF	Apache Kafka, Hyperledger Fabric blockchain, IoT devices, Amazon AWS
[54]	Humanitarian Activities	ArcSDE	ESRI	Oracle 10g, Oracle Portal, ArcSDE, ArcGIS Server, Apache Tomcat, Java, API for JavaScript

Also, we studied the types of technologies they used in their implementations. Each study used different technologies to match their goals and objectives. We categorized them as hardware tools, repositories, knowledge representation, analysis tools, web services, programming languages, networking, and security. Table 8 shows technologies classifications. The table has two columns: The first column categorizes the technologies based on their type or function, while the second column lists specific technologies or tools corresponding to each type mentioned in the first column. We listed the technologies as named in the papers to ensure accuracy. For example, in the Repositories category, some of the publications mentioned OODBMS or RDBMS without any further specifications [37], while others specify the RDBMS as either MySQL [48,49] or GoogleSQL [43].

The table shows the hardware tools category features IoT devices such as sensors and actuators. Environmental NPOs use these technologies in monitoring land, waters, or cities. The actuators help in the remote control of systems. Most of the middleware applications for NPOs in the literature are web-based. For this reason, we have the repository, middleware service, and web services consisting of web application-related tools. The networking tools can fall into WSN, P2P, CDN, and traditional networking categories. The WSN is for monitoring remote environments, P2P networks are for communication in remote areas, the CDN ensures efficient content or knowledge delivery, and the traditional networking tools show that some papers use middleware to improve the performance or collaboration over their existing networking infrastructure.

The middleware service, analysis tools, and knowledge representation categories consist of technologies that support data-driven decisions, data analysis, and data visualization. The programming languages consist of web programming languages like PHP and JSP. The category also contains language that supports heterogeneous systems like JAVA. The literature also uses markup languages to transfer data in a structured format to ensure interoperability.

6. Middleware Benefits in the Nonprofit Sector

From Section 4, we could infer that some benefits of using middleware in the nonprofit sector revolve around improving organizational effectiveness, efficiency, collaboration, and impact, ultimately supporting nonprofits in achieving their mission of serving their communities and advancing social causes. This section discusses the benefits mentioned in the literature. These benefits empower NPOs to make informed decisions, advocate for supportive policies, and maximize the impact of technology in advancing their missions and serving their communities. Other benefits of middleware in the nonprofit sector, according to the selected papers, are as follows:

1. It enhances security and provides robust data-management tools crucial for nonprofits, which often handle sensitive information and require reliable systems to protect their data [30].

2. It also delivers software updates to improve overall system security [45].

3. It fosters interoperability between system components and provides an easy-to-use interface [41,42].

4. It enables accessing data from heterogeneous repositories and large sensor networks [36,38,44].

5. It helps NPOs share up-to-date knowledge with different audiences, and provide offline access [45].

6. It supports rapid prototyping. It allows developers to add new components easily, increasing system scalability and allowing nonprofits to implement and evaluate changes swiftly, promoting a culture of innovation and entrepreneurship within the nonprofit sector [31,42,48].

7. It utilizes replicated nodes to support availability and reliability due to the load distribution and provide additional data backup [38,45].

8. It offers free and open-source alternatives that are affordable and comply with standards [16,40,45,46,49,51,52,53,54].

9. It supports using a GIS-based interface to visualize the location of resource availability for effective decision-making [42,82].

10. It also aids in developing a scalable and distributed archival system for tracking vulnerable locations, such as potential landslide areas [54].

11. It increases the computational power of the system through cloud computing. Thus, the system could use smaller sensors, making it easier to maintain during natural calamities [43].

7. Middleware Challenges and Open Issues in the Nonprofit Sector

Understanding the challenges and weaknesses of utilizing middleware in the nonprofit sector is paramount. Recognizing existing obstacles serves as a precursor to addressing potential gaps requiring improvement. In this research, we meticulously reviewed papers to identify weaknesses and challenges associated with using middleware in the nonprofit sector. We documented the solutions found in the literature for some of these issues. For challenges without identified solutions, we highlighted them as open issues for future researchers to address. The challenges are as follows:

1. Financial Challenges: There are two main expenditures from the nonprofit sector perspective: program and support expenses. Program expenses refer to the funds allocated towards the core programs and activities that fulfill the organization’s mission directly. However, support expenses encompass all other expenditures necessary to support the organization’s operations and activities. Support expenses are overhead costs [113]. The overhead cost ratio is the ratio of overhead cost to the budget. It is a widely used benchmark for measuring NPO efficiency [113]. Donors generally favor a lower overhead cost ratio to ensure more money directly reaches beneficiaries. Thus, NPOs are always trying to reduce it by cutting support expenses. For example, many NPOs do not have high-capacity internet resources, sophisticated IT staff, and IT plans despite knowing their importance [37]. There is also training cost because the system may require specific training, lack of which may pose usability challenges for technically untrained users [39]. To address implementation cost issues, some researchers proposed using free and open-source middleware [40]. However, this solution does not solve maintenance cost issues.

2. Privacy and Legal Challenges: NPOs operate in areas touched by privacy laws. They often collect personal information from donors, volunteers, clients or beneficiaries, and other stakeholders. Also, the reviewed papers have shown us that NPOs use middleware for information integration, storage, or dissemination in healthcare provision, humanitarian activities, or security. According to the United Nations Conference on Trade and Development (UNCTAD), 137 out of 194 countries had enacted legislation on data and privacy protection [114]. Although this is progress towards security and consumer protection, it poses challenges to NPOs that want to deploy middleware to improve their performance. As a starter, most NPOs often lack the resources and expertise to assess themselves against legal regulations despite collecting and managing personal data [115]. Likewise, the privacy laws in some countries are stringent, making NPOs avoid trying new technologies like middleware out of fear of litigation. However, some countries have some consideration for NPOs in their regulations like the General Data Protection Regulation (GDPR) [116]. Also, the disparity in privacy laws across the countries makes it difficult for an international NPO coalition or multinational NPOs to deploy integration middleware systems. Vizuete et al. [48] proposed a middleware-based system that operates without interfering with the privacy protocols used by agencies involved. Yet, this requires developers versed in the privacy laws of the countries involved, which may increase the system development cost. Another solution is to anonymize the beneficiaries [117]. Another challenge due to legal barriers is data law restrictions for HADR. Mrozek et al. [54] cited unresolved legal issues as barriers to availing raw landslide data to the public, hindering the development of a near real-time landslide inventory system.

3. Security Challenges: Cybersecurity is a primary challenge in any IT solution [37]. Security in middleware for NPO applications is vital for safeguarding sensitive data, maintaining trust and compliance, preventing financial fraud, mitigating operational disruptions, and supporting donor confidence. One area where cybersecurity is necessary for the nonprofit sector is humanitarian missions in conflict areas. There may be vulnerabilities associated with the mobile terminal reliability and availability when employing a mobile terminal for wireless communication during humanitarian activities, particularly in remote or resource-constrained areas [40]. One or more belligerents may seek an advantage by debilitating the humanitarian effort. Some middleware-based systems use existing security protocols [51], such as HTTP or TLS [48], which saves the NPOs time and money. However, in IoT-based systems, some sensor nodes may be so resource-constrained that existing protocols like HTTP or TLS are unsuitable. Some hardware, like RFID platforms, present security threats [40], which the middleware must address to ensure the safety of the whole system. Sometimes, the presence of security infrastructure or policy is the challenge, not its absence. In the context of civil-military collaboration [50], security policies of participants in HADR operations can hinder information sharing, requiring the consideration of separate networks for security reasons. Jakub and Zieliński [53] solve this problem by proposing secure message streams in a federated HADR environment using Apache Kafka middleware for data acquisition, Hyperledger Fabric blockchain for device fingerprint identity storage and hardware-software IoT gateway for managing communication between microservices and the blockchain, ensuring that only authenticated devices can exchange data. The system discards any message with an invalid or without a device fingerprint. Hardware vandalism is another security threat dimension that attacks the system’s availability. It is a threat that can reduce the NPOs’ quality of services, especially in conflict areas where there is no law and order.

4. Collaboration Challenges: Collaboration is necessary for NPOs to accomplish their mission. As such, middleware is an essential technology for NPOs, as it bridges between different systems and applications. Nonprofits often collaborate with other organizations and share data. It could be NPO-NPO (i.e., inter-NPO collaboration). It could also be extra-NPO collaboration, such as government-NPO, private-NPO, or private-government-NPO collaboration. It could also be international or domestic collaboration. Usually, NPOs collaborate to achieve a common cause. However, using middleware to integrate their systems may face legal challenges when the NPOs are from different countries or regions. Also, they may face political challenges due to distrust between the concerned countries. Several challenges hinder the middleware implementation in the government-NPO collaboration: A case study found that most government and NPO agencies are willing to share information with other agencies. But there are no plans in place for the integration of IT systems [37]. The study also noted a legal barrier to establishing agreements for sharing information by the collaborators [37]. In [50], the authors proposed a Smart city-based middleware that enables civil-military collaboration in HADR. This type of collaboration has two challenges. The military may be apprehensive about connecting their system to the middleware in fear of security breaches due to the technical inexperience of the NPOs or the fact that the NPOs may be easier to infiltrate. While the NPOs may not want to collaborate with the military for fear of distrust from the beneficiaries, especially in humanitarian operations. Almadani et al. [118] cited several challenges to government-NPO collaboration in Smart Cities. Winer and Pavkov [37] sum middleware collaboration challenges as more human-related than technology challenges.

5. Data processing and transfer challenges: A challenge of using middleware in the nonprofit sectors related to integrating data from heterogeneous sources is that it requires time-consuming steps of pre-processing the input and post-processing the output data for validation [49]. Another problem related to data transfer is poor internet connections, which makes uploading the data challenging [45]. Slow internet can also significantly impede the use of middleware in the nonprofit sector, causing delays and inefficiencies in data processing and communication due to increased latency and reduced bandwidth, making it difficult for middleware to transmit and process data in real time. Consequently, delayed exchange of information could hinder collaboration and decision-making in the nonprofit sector. Additionally, cloud-based middleware solutions may face limitations in scalability and responsiveness under such conditions. This issue is significant because most NPOs do not use high-capacity Internet resources [37]. Managing conflicting versions of the same file is another issue. It is common for multiple users to update the same file simultaneously, resulting in many copies of the file [45]. Some researchers avoided Internet connection by using a P2P middleware system to improve system performance [45]. However, their system makes it difficult for regular users to understand that data was only added to a local node and not synced to the cloud yet [45]. Finally, some systems [44] use scalar data, which hinders the representation of complex relationships, causing loss of valuable contextual information and reducing expressiveness.

6. System performance challenges: Another challenge is related to the system framework and performance. Many of the proposed works were a prototype or frameworks, so they did not focus on the completeness of the developed system nor the graphical interfaces, which do not conform with standards [41]. Designing user interfaces for systems that require collaboration among users with diverse backgrounds poses several challenges. These include accommodating varying levels of technical proficiency and preferences, managing complexity while maintaining usability, ensuring clear communication and understanding across different user groups, balancing familiarity with introducing new features, addressing accessibility needs, designing intuitive training and onboarding processes, and implementing efficient collaboration features. Also, ensuring sensor nodes’ reliability and robustness under extreme conditions like natural disasters while maintaining constant data transmission and monitoring is another challenge [43]. In addition, humanitarian workers often operate in areas with inadequate or no internet connection. Although researchers proposed installing the P2P application [38,45], deploying such systems is not trivial. For example, some operating systems like iOS do not support installing apps not from the App Store. In this case, the user requires an internet connection. Another solution is to use the user browser without installing a standalone application. Despite being updated, the latest version of a popular browser still lacks some features, and field browsers tend to be outdated [45]. Van de Walle et al. [119] listed nine design factors that developers should focus on to develop reliable and efficient HADR systems. These factors are: (1) The system should offer training and simulation to keep users ready, (2) It should focus on important information only, (3) The system should help users learn from the crises, (4) It should also be able to detect events that are exceptions to the norms in each crisis, (5) The system should capture all stakeholders and resources according to crises scope and nature, (6) The system must offer role transferability, (7) It must also ensure information validity and timeliness, (8) It should also ensure free information exchange among stakeholder, and (9) It should ensure coordination among stakeholders.

7. Volunteering challenges: Nonprofits may struggle to find volunteers with the requisite technical skills or allocate resources for training and support. The NPO must align volunteer-management practices with the system’s requirements, especially if collaborating with other agencies. Also, developers must design intuitive user interfaces [51] to lower training time and costs. They must also safeguard sensitive data to ensure volunteers cannot tamper with them, which will incentivize collaborators to join the system. Also, NPOs must effectively communicate and engage with volunteers to accommodate their requirements during system development, and afterward, IT support should be available for them to avoid volunteer attrition. Regarding challenges related to the usage of middleware in volunteer computing, there are difficulties in attracting a diverse pool of volunteers in computing projects and the potential complexity of the simulation software to run on volunteers’ resources [46]. Also, integration requirements like specific resources, expertise, and real-time interaction limitations pose challenges to the availability of volunteers for volunteer computing [46].

8. Opportunities for Middleware in Nonprofits

Based on the reviewed papers [37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54], using middleware in nonprofit systems leverage some opportunities that could be improved or investigated more in the future, this could help researchers when developing a middleware system to ensure covering them.

1. Privacy: UN reported that 71% of countries in the world have legislation on data and privacy protection [114]. Although it is an achievement for security, it poses a challenge to NPOs that want to collaborate, especially at an international level. Therefore, researchers must review the privacy laws and develop frameworks or guidelines to help NPOs and developers navigate complex data privacy laws more effectively. This research could involve assessing the impact of different legal frameworks on NPOs and identifying best practices for ensuring compliance while still leveraging middleware technologies. Vizuete et al. [48] proposed a middleware that integrates multiple systems without affecting their privacy policies. This solution will encourage cross-border collaboration between NPOs if the system is scalable and transparent.

2. Security: Security is necessary for middleware systems because middleware serves as a conduit for data from multiple systems. It makes them a target for cybercriminals. We found that NPO middleware-based systems are especially susceptible to attack as belligerents could attack them to either obtain information or prevent them from providing services to their enemies. Therefore, researchers should study security challenges facing middleware systems deployed in humanitarian missions, particularly in conflict areas. This research could focus on identifying vulnerabilities, threat models, and risk mitigation strategies specific to these environments, considering factors such as hostile actors, limited infrastructure, and unpredictable operating conditions. Drawing inspiration from work in [48], scientists can develop a middleware system that can securely connect multiple systems without affecting their security performance. It will allow civil-military collaboration since it will allow the military to maintain the security maturity level of their system.

3. Middleware-based Serious Games: A serious game is designed with a primary purpose other than pure entertainment, typically to educate, train, inform, or persuade users [120]. These games often simulate real-world scenarios and can be used for learning, skill-building, behavior change, or raising awareness about specific issues. Höhfeld [39] developed a simulator that trains humanitarian mission personnel in various mission types. One of the suggested future works was to conduct the system experiments on a larger diverse group with dynamic scenarios to verify the system results [39]. In addition, middleware can help develop serious games for training all stakeholders on collaboration in humanitarian missions and disaster and incident management.

4. System Design: Researchers could explore user-centered design approaches to develop middleware systems that are privacy-compliant, user-friendly, and accessible for NPOs with limited technical expertise [46]. Such systems will reduce maintenance costs and avoid volunteer attrition due to system usability challenges. The literature also recommended flexible security, disaster, and incident-management middleware-based systems that offer a variety of cloud-based services and support real-time communication as well as different interaction possibilities of [43,48]. Additionally, NPOs need middleware that enhances sensor reading reliability through innovative combinations of hardware and software layers to raise the level of reading reliability and utilize internet-based technologies that are low-cost and accessible by local web browsing [40]. PIMS can efficiently generate prediction and automation services by developing a SOM that accommodates deep learning as an analytical module to manage large amounts of data [47]. Jernberg et al. [38] recommended researchers look into enhancing the caching method to improve system functionality, improving the system design to support web-based applications, and resolving problems with transaction handling, state management, failover, application deployment and replication, and state storage.

5. Interoperability, Connectivity and Compatibility: There is a need to improve interoperability standards for incident-management systems across different domains, including transportation, cyber systems, and healthcare. The standards should include security, privacy, access control privileges, data models, and sharing protocols [42]. To address poor internet issues, NPOs may need to explore alternative connectivity options or optimize their middleware systems to operate efficiently in low-bandwidth environments. Researchers can also look into developing middleware that functions over a delay-tolerant network. To enable NPOs in areas with poor Internet connectivity to collaborate with their colleagues in high Internet bandwidth areas. Another issue is compatibility. Some middleware versions are not backward compatible, making collaboration difficult, especially in a federated system [52].

9. Conclusions

NPOs use new technologies to empower their communities by improving their operations and services. They need a mechanism that binds together heterogeneous systems and facilitates communications and information sharing. Middleware can help achieve these functionalities by mediating between different systems and resources and enabling smooth integration and interoperability. Few studies research the application of middleware by NPOs. Also, to the best of our knowledge, there has not been a literature review in this area. This paper presents a comprehensive survey of the role of middleware in the nonprofit sector. This survey followed a systematic technique to collect the reviewed studies. We surveyed 31 papers to explore middleware applications in the nonprofit sector, their types, strengths, challenges, and opportunities.

We found that middleware is applicable in different domains. Mainly, it focuses on humanitarian activities such as humanitarian missions, humanitarian relief, disaster, incident, and crime management. Also, it finds applications in healthcare, water science, volunteering, and education. We also discovered that many researchers developed their middleware while a few used existing ones. Then, we listed technologies used in system implementations and classified them into different types. We found more web and IoT-related technologies in the lists, meaning current middleware-based applications are web-based or IoT-based systems. Our study shows that web-based technologies help communication, while IoT-based technologies help in environmental sensing. As the middleware is currently popular, we investigated its strengths and benefits (i.e., integration, centralization, flexibility, and complexity abstraction). We also identified its challenges from the papers (e.g., financial, privacy, and security). In addition, we summarized the opportunities that are potential future work in privacy, security, and compatibility.

Author Contributions

Conceptualization, B.A. and F.A.; methodology, F.A.; validation, E.A.-N., F.A. and B.A.; formal analysis, S.A. and R.A.; investigation, S.A. and R.A.; resources, S.A., R.A. and F.A.; data curation, S.A. and R.A.; writing—original draft preparation, S.A. and R.A.; writing—review and editing, F.A. and E.A.-N.; visualization, S.A., R.A. and F.A.; supervision, B.A.; project administration, B.A.; funding acquisition, B.A. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Acknowledgments

The authors would like to acknowledge all support provided by Alfozan Academy, the Center of Excellence in Development of Nonprofit Organizations (CEDNPO), the Deanship of Research Oversight and Coordination (DROC), and King Fahd University of Petroleum & Minerals (KFUPM).

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

The following abbreviations are used in this manuscript:

ABAC	Attribute-Based Access Control
ALE	Application Level Event
CDN	Content Delivery Network
CSO	Civil Society Organization
DBMS	Database-Management System
DKS	Distributed K-nary Search
DMAS	Database-Management and -Archiving System
EAE	Education Access and Equity
GDPR	General Data Protection Regulation
GIS	Geographical Interface System
GONGO	Government-Organized Non-Governmental Organizations
HADE	Humanitarian Action and Development Engineering
HADR	Humanitarian Assistance and Disaster Relief
LLRP	Low Level Reader Protocol
MANET	Mobile Ad-Hoc Networks
MOM	Message Oriented Middleware
NCO	Network Centric Operations
NGOs	Non-Governmental Organizations
NPOs	Nonprofit Organizations
OS	Operating System
P2P	Peer-to-Peer
PHAP	Professionals in Humanitarian Asistance and Management
PIMS	Patient Information-Management System
PRISMA	Preferred Reporting Items for Systematic Reviews and Meta-Analyses
RCIS	Rescue and Crime Information System
RFID	Radio Frequency Identification
SDGs	Sustainable Development Goals
SMEs	Small and Medium-sized Enterprises
SOAM	Service Oriented Architecture Middleware
SOM	Service Oriented Middleware
TB	Tuberculosis
TEL	Technology Enhanced Learning
UN	United Nations
UNCTAD	United Nations Conference on Trade and Development
UNDRR	United Nations Office for Disaster Risk Reduction
VAS	Virtual Atom Smasher
VC	Volunteer Computing
VOS	Visualization of Similarities
VPN	Virtual Private Networks
VT	Volunteer Thinking
WSDL	Web Services Description Language
WSN	Wireless Sensor Networks

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Table 1

Inclusion criteria.

Sn.	Attribute	Description
1	Relevance	Papers must directly address middleware technologies, frameworks, or solutions and their application within the nonprofit sector.
2	Focus	Papers should specifically examine or discuss the use of middleware in NPOs, highlighting the unique challenges, opportunities, and impacts within this sector.
3	Type	We included all publication types, peer-reviewed journal articles, conference papers, scholarly books, and technical reports. We also included middleware company articles on their architecture and usage in the nonprofit sector.
4	Recency	We include all papers regardless of the time frame since few are available.
5	Diversity	We accept papers from all geographical regions and perspectives to capture a comprehensive overview of the global application of middleware in NPOs.

Table 2

Exclusion criteria.

Sn.	Attribute	Description	# Excluded
1	Focus	Exclude all papers on middleware applications in for-profit enterprises or under commercial settings unless they provide transferable insights to the nonprofit sector, and papers on nonprofit systems but without middleware usage.	62
2	Language	Exclude all papers published in languages other than English.	3
3	Sources	Exclude all non-peer-reviewed sources such as websites *, blogs, and opinion pieces to maintain the academic rigor of the survey paper.	0
4	Duplicates	Exclude duplicate publications or papers that substantially overlap in their content to maintain diversity and originality.	26
5	Type	Exclude non-technical papers without technical depth or do not contribute substantial insights into the practical application of middleware in NPOs.	0
6	Completeness	Exclude publications whose content is either incomplete, inaccessible, or invalid database entry	5

* We allow website articles on middleware architecture and its NPO applications. We use them in the Middleware Overview sections.

Table 3

Research questions.

#RQ	Research Question	Remarks
RQ1	What are middleware applications in the nonprofit sectors?	To identify the domains where middleware applications are being used in the nonprofit sectors.
RQ2	What types of middleware and technologies are used in the nonprofit sectors?	To highlight the most commonly used middleware types and the technologies that support them.
RQ3	What are the benefits of using middleware in the nonprofit sectors?	To identify the advantages that middleware technology offers specifically to NPOs.
RQ4	What are the challenges of using middleware in the nonprofit sector?	To highlight the identified challenges of applying middleware in nonprofit systems.
RQ5	What are the opportunities for using middleware in the nonprofit sectors?	To identify the opportunities for middleware applications in nonprofit sectors which will help in indicating topics for future work.

Table 4

Common components of middleware software.

Component	Description
Management Console	Offers developers an overview of middleware activities, rules, and configurations.
Client Interface	The outer layer of middleware enabling communication with applications, databases, and microservices.
Internal Interface	Serves as the binding element for various components, allowing them to work together cohesively.
Platform Interface	Ensures compatibility of the middleware with different operating systems and platforms.
Contract Manager	Defines data exchange rules and alerts applications when they violate them.
Session Manager	Establishes secure communication channels and maintains records of data activity.
Database Manager	Integrates with various database types as needed by the middleware.
Runtime Monitor	Continuously monitor data movements and report any unusual activities to developers.

Table 5

Middleware available in the market.

Middleware	Company	Open Source	Description	NPOs
Apache Camel, https://camel.apache.org, accessed on 25 August 2024;Apache Kafka, https://kafka.apache.org, accessed on 2 September 2024	Apache Software Foundation (ASF), Wilmington, DE, USA	Yes	Integration framework that allows different systems to communicate using various protocols and data formats.	-
Boomi, https://bit.ly/boomiNPO, accessed on 25 August 2024	Dell Technologies Round Rock, TX, USA	No	Cloud-based integration platform for connecting cloud and on-premises systems.	Australian Red Cross, The Fred Hollows Foundation
Doxis, https://www.sergroup.com, accessed on 5 August 2024	SER Group, Bonn, Nordrhein-Westfalen, Germany	No	Intelligent content services platform that integrates document management, collaboration, and process automation	-
Fiware-ArcGIS, https://github.com/esri-es/Fiware-ArcGIS, accessed on 5 August 2024	ESRI, Redlands, CA, USA	No	This tool allows you to add sensor information received from a Context Broker into an ArcGIS Online or ArcGIS Server account.	Flying Labs, The Humanitarian OpenStreetMap Team (HOT), YouthMappers
Jitterbit, https://www.jitterbit.com/industry/nonprofits/, accessed on 25 August 2024	Jitterbit Inc., Alameda, CA, USA	No	It is a low-code integration platform that connects different applications, data sources, and APIs.	UNICEF UK, The Fire Fighters Charity
Mojo Middleware, https://www.mojomiddleware.com/, accessed on 8 August 2024	501Works, Alexandria, VA, USA	No	Mojo Middleware integrates Agency-Management System/Customer Relationship Management (AMS/CRM) with other software to communicate and share data.	-
Mulesoft, https://blogs.mulesoft.com/search/?q=nonprofit, accessed on 12 August 2024	Salesforce, San Francisco, CA, USA	No	Enterprise service bus that integrates multiple applications and services.	America SCORES, Boy Scouts of America
OpenFn, https://www.openfn.org/customers, accessed on 27 August 2024	OpenFn, New York, NY, USA	Yes	OpenFn is a middleware for NPOs that bridges different systems and applications, facilitating data integration and communication.	UNICEF, Mercy Corps, International Rescue Committee, Lwala Community Alliance, Wildlife Conservation Society
RabbitMQ, https://www.rabbitmq.com, accessed on 24 August 2024	VMware, Palo Alto, CA, USA	Yes	Message broker that facilitates communication between different systems and applications.	-
Talend Open Studio, https://www.talend.com/customers/?industries=Non+profit, accessed on 25 August 2024	Talend, San Mateo, CA, USA	Yes	Data-integration platform that connects various data sources and applications.	The Royal British Legion, Save the Children Germany
Workato, https://www.workato.com, accessed on 11 August 2024	Workato Inc., Mountain View, CA, USA	No	Intelligent automation platform that integrates apps and automates workflows across an organization.	Feeding America, American Society for the Prevention of Cruelty to Animals (ASPCA)
WSO2 Enterprise Integrator, https://wso2.com/customers/, accessed on 2 September 2024	WSO2, Santa Clara, CA, USA	Yes	Open-source-integration platform for connecting and managing APIs, applications, and systems.	TechSoup
Zapier, https://zapier.com/customer-stories, accessed on 27 August 2024	Zapier Inc., San Francisco, CA, USA	No	Connects different web applications to automate workflows.	Lasagna Love, One More Day, SisterLove

Table 6

SDGs with middleware applications in the nonprofit sector.

Title	# SDG	Label	# Papers	SDGs’ Description
Good Health and Well-being	3	health	2	Ensure healthy lives and promote well-being for all, at all ages.
Quality Education	4	education	3	Ensure inclusive and equitable quality education and promote lifelong learning.
‡ Clean Water andSanitation	6	cws	1	Ensure availability and sustainable management of water and sanitation for all.
Decent Work and Economic Growth	8	dweg	2	Promote sustained, inclusive, and sustainable economic growth, full and productive employment, and decent work for all.
Industry, Innovation, and Infrastructure	9	industry	13	Build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation.
Sustainable Cities and Communities	11	cities	3	Make cities and human settlements inclusive, safe, resilient, and sustainable.
‡ Climate Action	13	climate	1	Take urgent action to combat climate change and its impacts.
‡ Life Below Water	14	lbw	1	Conserve and sustainably use the oceans, seas, and marine resources for sustainable development.
Peace, Justice, and Strong Institutions	16	pjsi	7	Promote peaceful and inclusive societies for sustainable development, provide access to justice for all, and build effective, accountable, and inclusive institutions at all levels.
Partnerships for the Goals	17	partnership	4	Strengthen the means of implementation and revitalize the Global Partnership for Sustainable Development.

^‡ SDGs in the dataset but not on the map because they did not meet the threshold value ($\delta =2$).

Table 8

Technology type classifications.

Technologies Type	Technologies Used
Hardware Tools	Sensors, Scalar sensors, Actuators, Transponders, RFID tags, RFID Reader, Raspberry Pi, Single Board Computers (SBC)
Repositories	Distributed Hash Tables (DHT), Oracle 10g, MySQL, Google SQL, Object-Oriented Database-Management System (OODBMS)
Middleware Service	Apache Tomcat, ArcSDE, Mosquitto Message Broker, Integration Services Sensor, Sensor and Visualization Services, Sensor Observation Services
Web Services	Oracle Portal, PageKite, Nginx HTTP Server, Sahana Eden Enterprise Resource Planning (ERP) System, Extended Search Engine, Geographical interface system (GIS), Wagtail content-management system, R-Shiny web application framework, Graasp Online System, Enterprise web servers, Email and Messaging, Cloud Computing, Google App Engine, ArcGIS Server
Networking	Content Delivery Networks (CDN), Local Area Networks (LAN), Wide Area Networks (WAN), Virtual Private Networks (VPN), Peer-to-Peer Networks (P2P), Wireless Sensor Networks (WSN), Personal Area Network (PAN), Mobile Ad-Hoc Networks (MANET), OpenVPN, Routers
Security	Keycloak identity-management system, Attribute-based access control (ABAC), Firewall servers
Analysis Tools	Analysis Builder, Reasoning and inference engine, SWAT software version 2018/Rev 670
Knowledge Representation	Semantic Annotation Editor, Ontology Editor
Programming Languages	Python, Java, Java Server Pages (JSP), JavaScript, Extensible Markup Language (XML), Web Services Description Language (WSDL), SensorML markup language, C#, Hypertext Preprocessor (PHP)

Appendix A. Keyword Grouping for VOSviewer

Table A1

Grouping keywords by synonyms.

Label/Group Word	Keywords
cc	cloud, cloud computing, hybrid cloud
cost	cost reduction, implementation cost, low costs, rfid low cost reader platforms
cs	crowdsourcing
data	data and information, data integration, data visualization, data warehouse
disaster	disaster, disasters, disaster prevention
distr sys	distributed communications, distributed system
education	e-learning, science and engineering, technology enhanced learning, virtual laboratory
health	health facilities, health monitoring, hospital data processing, human body temperature, tuberculosis
humanitarian	humanitarian assistances, humanitarian assistance and disaster recovery (hadr), humanitarian missions, humanitarian relief
information	exchange of information, information analysis, information system, information systems, information use, rcic, rcis
integration	integrated infrastructure, integration architecture, system integration
interoperability	interoperability, interoperability testing
iot	iot, internet of things (iot), internet-of-things, internet of things
management	disaster management, emergency management, information management, knowledge management, land use and managements, risk-management
middleware	message oriented middleware, middle-ware, middleware components, middleware frameworks, middleware layer, open source middleware, service oriented middleware, service-oriented middleware
ngo	ngo, nongovernmental organizations, nonprofit sector
protocols	internet protocols, protocol, reader protocol
sensors	scalar sensors, sensor informations, sensor web enablement
services	context-aware services, information services, location based services (lbs), location-based services, sensor observation service, sensor observation services, service computing, service oriented architecture (soa), service-oriented architectures, web services
wsn	sensor networks, wireless sensor network (wsns), wireless sensor networks

Table A2

Grouping keywords by SDGs.

Group	Keywords
cities (sdg 11)	land use, land use and managements, location based services (lbs), location-based services, rural area, rural areas
climate (sdg 13)	climate change, futurewater science gateway, river basins climate interaction.
cws (sdg 6)	soil and water assessment tool
dweg (sdg 8)	cost reduction, implementation cost, low costs
education (sdg 4)	e-learning, science and engineering, technology enhanced learning, virtual laboratory
health (sdg 3)	health facilities, health monitoring, hospital data processing, human body
industry (sdg 9)	cloud, cloud computing, data warehouse, electronic product codes, enterprise server, fog computing, hybrid cloud, information analysis, information management, information services, information services management, information system, information systems, information use, integrated infrastructure, integration architecture, internet connection, internet of things (iot), internet protocols, internet-of-things, intuitive user interface, iot, internet of things, online games, radio frequency identification (rfid), real-time distributed computing, scalar sensor data, scalar sensors, semantic technologies, sensor informations, sensor networks, sensor observation service, sensor observation services, sensor web enablement, sensors, service computing, service oriented architecture (soa), service-oriented architectures, system integration, technologies, user interfaces, virtual private network (vpn), web services, websites, wireless connection, wireless sensor network (wsns), wireless sensor networks, wireless technologies, wireless telecommunication systems, workflow visualization, wsn, added value sensing, added values, application level, architecture, computer architecture, content delivery, context-aware services, data and information, data integration, data visualization, design, distributed communications, distributed system, framework, human computation, life cycle, local knowledge, parallel computing, peer to peer, proprietary systems, protocol, reader protocol, rfid low cost reader platforms, clientcentric, mobile terminal, poor connectivity
lbw (sdg 14)	ocean networks canada, oceanography, open geospatial consortium
middleware	middle-ware, open source middleware, service oriented middleware, service-oriented middleware, message oriented middleware, middleware components, middleware frameworks, middleware layer
ngo	ngo, nongovernmental organizations, nonprofit sector
partnership (sdg 17)	exchange of information, interoperability, interoperability testing, knowledge delivery, knowledge management, resource sharing
pjsi (sdg 16)	civil defense, disaster, disaster management, disaster prevention, disasters, emergency conditions, emergency management, humanitarian assistances, humanitarian assistance and disaster recovery (hadr), humanitarian missions, humanitarian relief, incident response ontology, rcic, rcis, recovery, risk management, security, unified incident command and decision supporte (uicds (tm)), privacy
volunteering	community grid, crowdsourcing, volunteer computing (vc), volunteer thinking (vt), citizen cyberscience (ccs)

References

1. Anheier, H.K.; Toepler, S. Nonprofit Organizations: Theory, Management, Policy; 3rd ed. Routledge: London, UK, 2022; pp. 1-605. [DOI: https://dx.doi.org/10.4324/9780429299681]

2. Ivanenko, Y.V. Economic definition of nonprofit organization. Actual Probl. Econ.; 2015; 167, pp. 8-19.

3. Coule, T.M.; Bain, C. Organizing Logics, Nonprofit Management and Change: Rethinking Power, Persuasion and Authority; Routledge: New York, NY, USA, 2021; pp. 1-175. [DOI: https://dx.doi.org/10.4324/9780429352676]

4. Prentice, C.R.; Brudney, J.L. Are you being served? Toward a typology of nonprofit infrastructure organizations and a framework for their assessment. J. Public Nonprofit Aff.; 2018; 4, pp. 41-58. [DOI: https://dx.doi.org/10.20899/jpna.4.1.41-58]

5. UN. The UN and Civil Society | United Nations; United Nations (UN): New York, NY, USA, 2021; Available online: https://www.un.org/en/get-involved/un-and-civil-society (accessed on 31 June 2024).

6. UN. About Us|United Nations; United Nations (UN): New York, NY, USA, 2022; Available online: https://www.un.org/en/civil-society/page/about-us (accessed on 27 November 2023).

7. Helmold, M.; Samara, W. PM in NGO and NPO. Manag. Prof.; 2019; Part F568, pp. 149-154. [DOI: https://dx.doi.org/10.1007/978-3-030-20534-8_11]

8. Archambault, E. Non-Governmental Organisations and Foundations; Edward Elgar Publishing: Cheltenham, UK, 2023; pp. 155-162. [DOI: https://dx.doi.org/10.4337/9781803920924.00033]

9. Matiatou, M. The Story Retold Visiting the Role and Mission of Non-Governmental Organizations in the Decision Making and Policy Formulation Process; IGI Global: Hershey, PA, USA, 2019; pp. 785-808. [DOI: https://dx.doi.org/10.4018/978-1-7998-1207-4.ch045]

10. Hasmath, R.; Hildebrandt, T.; Hsu, J.Y.J. Conceptualizing government-organized Non-Governmental Organizations. J. Civ. Soc.; 2019; 15, pp. 267-284. [DOI: https://dx.doi.org/10.1080/17448689.2019.1632549]

11. Abramson, A.J. Assessing the State of the U.S. Nonprofit Sector: What Indicators Should We Use?. Nonprofit Volunt. Sect. Q.; 2023; 52, pp. 544-559. [DOI: https://dx.doi.org/10.1177/08997640221091534]

12. Saeed, S.; Rohde, M. Technology Design for e-Governance in Nonprofit Organizations; IGI Global: Hershey, PA, USA, 2013; pp. 49-58. [DOI: https://dx.doi.org/10.4018/978-1-4666-3640-8.ch004]

13. Suykens, B.; Maier, F.; Meyer, M.; Verschuere, B. Business-Like and Still Serving Society? Investigating the Relationship Between NPOs Being Business-Like and Their Societal Roles. Nonprofit Volunt. Sect. Q.; 2023; 52, pp. 682-703. [DOI: https://dx.doi.org/10.1177/08997640221106979]

14. Paarlberg, L.E.; Zuhlke, S. Revisiting the Theory of Government Failure in the Face of Heterogeneous Demands. Perspect. Public Manag. Gov.; 2019; 2, pp. 103-124. [DOI: https://dx.doi.org/10.1093/ppmgov/gvz002]

15. Henriksen, L.S.; Smith, S.R.; Zimmer, A. Welfare Mix and Hybridity. Flexible Adjustments to Changed Environments. Introduction to the Special Issue. Voluntas; 2015; 26, pp. 1591-1600. [DOI: https://dx.doi.org/10.1007/s11266-015-9622-y]

16. OpenFn. UNICEF Thailand Primero Interoperability Phases 1 & 2; OpenFn: London, UK, 2023; Available online: https://openfn.github.io/primero-thailand/ (accessed on 27 August 2024).

17. Shorey, R.; Ananda, A.; Chan, M.; Ooi, W. Part III: Middleware, Application, and New Paradigm. Mobile, Wireless, and Sensor Networks: Technology, Applications, and Future Directions; IEEE Press: Piscataway, NJ, USA, Wiley: Hoboken, NJ, USA, 2006.

18. Sumathi, L.; Valarmathi, M. A survey on middleware for smart home application. Int. J. Latest Trends Eng. Technol.; 2017; 8, pp. 078-085.

19. Maximize Market Research. Nonprofit Organizations Market—Global Industry Analysis and Forecast 2024–2030; OpenFn: Pune, India, 2024; Available online: https://www.maximizemarketresearch.com/market-report/nonprofit-organizations-market/134550/ (accessed on 7 September 2024).

20. Jackson, H. Integration Solutions for Nonprofits: A Guide to Choosing the Perfect Fit for Your Organization—Omatic; Omatic: Mt. Pleasant, SC, USA, 2024; Available online: https://omaticsoftware.com/blog/integration-solutions-for-nonprofits-a-guide-to-choosing-the-perfect-fit-for-your-organization/ (accessed on 7 September 2024).

21. Manewith, S. Middleware, Ipaas, and Isaas for Nonprofits: Cutting through the Confusion—Omatic; Omatic: Mt. Pleasant, SC, USA, 2023; Available online: https://omaticsoftware.com/ebook/middleware-ipaas-and-isaas-for-nonprofits-cutting-through-the-confusion/ (accessed on 25 August 2024).

22. Zhang, W.; Gutierrez, O.; Mathieson, K. Information systems research in the nonprofit context: Challenges and opportunities. Commun. Assoc. Inf. Syst.; 2010; 27, 1. [DOI: https://dx.doi.org/10.17705/1CAIS.02701]

23. UN. The 17 Goals|Sustainable Development; United Nations (UN): New York, NY, USA, 2024; Available online: https://sdgs.un.org/goals (accessed on 31 June 2024).

24. Selçuk, A.A. A guide for systematic reviews: PRISMA. Turk. Arch. Otorhinolaryngol.; 2019; 57, 57. [DOI: https://dx.doi.org/10.5152/tao.2019.4058] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31049257]

25. Moher, D.; Altman, D.G.; Tetzlaff, J. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). Guidelines for Reporting Health Research: A User’s Manual; Wiley: Hoboken, NJ, USA, 2014; pp. 250-261.

26. Page, M.J.; Moher, D.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E. et al. PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews. BMJ; 2021; 372, n160. [DOI: https://dx.doi.org/10.1136/bmj.n160] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33781993]

27. Ballew, B.S. Elsevier’s Scopus® database. J. Electron. Resour. Med. Libr.; 2009; 6, pp. 245-252. [DOI: https://dx.doi.org/10.1080/15424060903167252]

28. Mongeon, P.; Paul-Hus, A. The journal coverage of Web of Science and Scopus: A comparative analysis. Scientometrics; 2016; 106, pp. 213-228. [DOI: https://dx.doi.org/10.1007/s11192-015-1765-5]

29. Bernstein, P.A. Middleware: A model for distributed system services. Commun. ACM; 1996; 39, pp. 86-98. [DOI: https://dx.doi.org/10.1145/230798.230809]

30. Bishop, T.A.; Karne, R.K. A Survey of Middleware. Proceedings of the 18th International Conference on Computers and Their Applications 2003, CATA 2003; Honolulu, HI, USA, 26–28 March 2003; pp. 254-258.

31. Gazis, A.; Katsiri, E. Middleware 101: What to know now and for the future. Queue; 2022; 20, pp. 10-23. [DOI: https://dx.doi.org/10.1145/3526211]

32. Technavio. Middleware as a Service (MWAAS) Market Analysis North America, Europe, APAC, South America, Middle East and Africa—US, China, Japan, UK, Germany—Size and Forecast 2024–2028; Technavio: Toronto, ON, Canada, 2024; Available online: https://www.technavio.com/report/middleware-as-a-service-mwaas-market-industry-analysis (accessed on 25 August 2024).

33. Susnjara, S.; Smalley, I. What is Middleware?; IBM: Armonk, NY, USA, 2024; Available online: https://www.ibm.com/topics/middleware (accessed on 12 August 2024).

34. Red Hat. What is Middleware?; Red Hat: Raleigh, NC, USA, 2022; Available online: https://www.redhat.com/en/topics/middleware/what-is-middleware (accessed on 12 August 2024).

35. Amazon AWS. What is Middleware? — Middleware Explained — AWS; Amazon AWS: Seattle, WA, USA, 2024; Available online: https://aws.amazon.com/what-is/middleware/ (accessed on 12 August 2024).

36. Danielkievych, A. Integration Middleware Explained: From Definition to Industry Applications. Integration Middleware 101: Types, Benefits, & Use Cases; Forbytes: Lviv, Ukraine, 2024; Available online: https://forbytes.com/blog/integration-middleware/ (accessed on 12 August 2024).

37. Winer, C.R.; Pavkov, T.W. Human Services Information Technology: A Shared System. Inf. Sci. Int. J. Emerg. Transdiscipl.; 2003; 6, pp. 135-141. [DOI: https://dx.doi.org/10.28945/521]

38. Jernberg, J.; Vlassov, V.; Ghodsi, A.; Haridi, S. Doh: A content delivery peer-to-peer network. Proceedings of the Euro-Par 2006 Parallel Processing: 12th International Euro-Par Conference; Dresden, Germany, 28 August–1 September 2006; Proceedings 12 Springer: Berlin/Heidelberg, Germany, 2006; pp. 1026-1039.

39. Hohfeld, A. A ubiquitous m-learning framework for Network Centric Operations. Proceedings of the Fifth Annual IEEE International Conference on Pervasive Computing and Communications Workshops (PerComW’07); White Plains, NY, USA, 19–23 March 2007; pp. 171-176.

40. Sámano-Robles, R.; Gameiro, A. Design of a low cost RFID platform with added value sensing capabilities for humanitarian relief applications. Proceedings of the 1st International Conference on Wireless Technologies for Humanitarian Relief; Amritapuri Kollam, Kerala, India, 18–21 December 2011; pp. 291-297.

41. Agre, G. SINUS–A Semantic Technology Enhanced Environment for Learning in Humanities. Cybern. Inf. Technol.; 2012; 12, pp. 5-24. [DOI: https://dx.doi.org/10.2478/cait-2012-0027]

42. Shafiq, B.; Chun, S.; Atluri, V.; Vaidya, J.; Nabi, G. Resource sharing using UICDS™ framework for incident management. Transform. Gov. People Process Policy; 2012; 6, pp. 41-61. [DOI: https://dx.doi.org/10.1108/17506161211214813]

43. Chandrakant, N.; Bijil, A.P.; Shenoy, P.D.; Venugopal, K.R.; Patnaik, L.M. Middleware service oriented rescue and crime information on cloud (RCIC) using heterogeneous nodes in WSNs. Proceedings of the TENCON 2012 IEEE Region 10 Conference; Cebu, Philippines, 19–22 November 2012; pp. 1-5. [DOI: https://dx.doi.org/10.1109/TENCON.2012.6412271]

44. Pirenne, B.; Guillemot, E. OGC Sensor Web Enablement compliance for Ocean Networks Canada scalar data. Proceedings of the 2013 OCEANS—San Diego; San Diego, CA, USA, 23–27 September 2013; pp. 1-3.

45. Vozniuk, A.; Holzer, A.; Mazuze, J.; Gillet, D. GraaspBox: Enabling Mobile Knowledge Delivery into Underconnected Environments. Proceedings of the Ninth International Conference on Information and Communication Technologies and Development; Lahore, Pakistan, 16–19 November 2017; Association for Computing Machinery: New York, NY, USA, 2017; ICTD ’17 [DOI: https://dx.doi.org/10.1145/3136560.3136572]

46. Yadav, P.; Charalampidis, I.; Cohen, J.; Darlington, J.; Grey, F. A collaborative citizen science platform for real-time volunteer computing and games. IEEE Trans. Comput. Soc. Syst.; 2018; 5, pp. 9-19. [DOI: https://dx.doi.org/10.1109/TCSS.2017.2771479]

47. Suprihadi, S.; Wijono, S.; Hartomo, K.D. Service Oriented Middleware for Tuberculosis’s Information Services Management. Proceedings of the 2020 International Seminar on Application for Technology of Information and Communication (iSemantic); Semarang, Indonesia, 19–20 September 2020; pp. 425-430. [DOI: https://dx.doi.org/10.1109/iSemantic50169.2020.9234198]

48. Zambrano Vizuete, M.; Pérez, F.; Zambrano, A.; Maya, E.; Dominguez, M. A New Approach to Interoperability in Disaster Management. Proceedings of the Advances in Emerging Trends and Technologies; Botto-Tobar, M.; León-Acurio, J.; Díaz Cadena, A.; Montiel Díaz, P. Springer: Cham, Switzerland, 2020; pp. 375-388.

49. Dierauer, J.R.; Zhu, C.; Gong, L.; Walsh, A.; Pamidighantam, S.; Wang, J.; Christie, M.; Abeysinghe, E. FutureWater Indiana: A Science Gateway for Spatio-Temporal Modeling of Water in Wabash Basin with a Focus on Climate Change. Proceedings of the Practice and Experience in Advanced Research Computing; Portland, OR, USA, 26–30 July 2020; Association for Computing Machinery: New York, NY, USA, 2020; PEARC ’20 pp. 252-261. [DOI: https://dx.doi.org/10.1145/3311790.3396651]

50. Campioni, L.; Poltronieri, F.; Stefanelli, C.; Suri, N.; Tortonesi, M.; Wrona, K. Enabling civil–military collaboration for disaster relief operations in smart city environments. Future Gener. Comput. Syst.; 2023; 139, pp. 181-195. [DOI: https://dx.doi.org/10.1016/j.future.2022.09.020]

51. Taylor, C.C.S.; Arthanari, T.S. ZigBee Architecture for Disaster Relief Supply Chain Visibility and Supply Chain Coordination. Proceedings of the Americas Conference on Information Systems; New Orleans, LA, USA, 16–18 August 2018.

52. Pradhan, M. Federation Based on MQTT for Urban Humanitarian Assistance and Disaster Recovery Operations. IEEE Commun. Mag.; 2021; 59, pp. 43-49. [DOI: https://dx.doi.org/10.1109/MCOM.001.2000937]

53. Sychowiec, J.; Zieliński, Z. An Experimental Framework for Secure and Reliable Data Streams Distribution in Federated IoT Environments. Proceedings of the 2023 18th Conference on Computer Science and Intelligence Systems (FedCSIS); Warsaw, Poland, 17–20 September 2023; pp. 769-780. [DOI: https://dx.doi.org/10.15439/2023F3882]

54. Mrozek, T.; Kułak, M.; Grabowski, D.; Wójcik, A. Landslide Counteracting System (SOPO): Inventory Database of Landslides in Poland. Proceedings of the Landslide Science for a Safer Geoenvironment; Sassa, K.; Canuti, P.; Yin, Y. Springer: Cham, Switzerland, 2014; pp. 815-820. [DOI: https://dx.doi.org/10.1007/978-3-319-05050-8_126]

55. Van Eck, N.; Waltman, L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics; 2010; 84, pp. 523-538. [DOI: https://dx.doi.org/10.1007/s11192-009-0146-3]

56. Van Eck, N.J.; Waltman, L. VOSviewer manual Version 1.6.20; Universiteit Leiden: Leiden, The Netherlands, 2023; Available online: https://www.vosviewer.com/documentation/Manual_VOSviewer_1.6.20.pdf (accessed on 31 June 2024).

57. Bukar, U.A.; Sayeed, M.S.; Razak, S.F.A.; Yogarayan, S.; Amodu, O.A.; Mahmood, R.A.R. A method for analyzing text using VOSviewer. MethodsX; 2023; 11, 102339. [DOI: https://dx.doi.org/10.1016/j.mex.2023.102339]

58. Waltman, L.; Van Eck, N.J.; Noyons, E.C. A unified approach to mapping and clustering of bibliometric networks. J. Informetr.; 2010; 4, pp. 629-635. [DOI: https://dx.doi.org/10.1016/j.joi.2010.07.002]

59. I, B.; Shaik, N. Training Programs for Police in Disaster Risk Reduction. International Handbook of Disaster Research; Springer Nature Singapore: Singapore, 2022; pp. 1-10. [DOI: https://dx.doi.org/10.1007/978-981-16-8800-3_164-1]

60. Peci, M.; Vazan, P.; Nemlaha, E. The levels of systems integration. Appl. Mech. Mater.; 2013; 373–375, pp. 1949-1953. [DOI: https://dx.doi.org/10.4028/www.scientific.net/AMM.373-375.1949]

61. Esterle, L.; Brown, J.N. The Competence Awareness Window: Knowing what I can and cannot do. Proceedings of the Proceedings—2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion, ACSOS-C 2020; Washington, DC, USA, 17–21 August 2020; pp. 62-63. [DOI: https://dx.doi.org/10.1109/ACSOS-C51401.2020.00031]

62. Boomi. Discover How Boomi Customers Accelerate Business Outcomes; Boomi: Conshohocken, PA, USA, 2024; Available online: https://boomi.com/customers/find-customer/#nonprofit (accessed on 25 August 2024).

63. Hourdin, V.; Ferry, N.; Tigli, J.Y.; Lavirotte, S.; Rey, G. Middleware in Ubiquitous Computing. Computer Science and Ambient Intelligence; Wiley: Hoboken, NJ, USA, 2013; pp. 71-88. [DOI: https://dx.doi.org/10.1002/9781118580974.ch5]

64. Blair, G. Complex distributed systems: The need for fresh perspectives. Proceedings of the Proceedings—International Conference on Distributed Computing Systems; Vienna, Austria, 2–6 July 2018; pp. 1410-1421. [DOI: https://dx.doi.org/10.1109/ICDCS.2018.00142]

65. Simó-Ten, J.E.; Munera, E.; Poza-Lujan, J.L.; Posadas-Yague, J.L.; Blanes, F. CKMultipeer: Connecting devices without caring about the network. Adv. Intell. Syst. Comput.; 2018; 620, pp. 189-196. [DOI: https://dx.doi.org/10.1007/978-3-319-62410-5_23]

66. ONC. Observatories | Ocean Networks Canada; Ocean Networks Canada (ONC): Victoria, BC, Canada, 2024; Available online: https://www.oceannetworks.ca/observatories/ (accessed on 31 June 2024).

67. Gankevich, I.; Gaiduchok, V.; Korkhov, V.; Degtyarev, A.; Bogdanov, A. Middleware for big data processing: Test results. Phys. Part. Nucl. Lett.; 2017; 14, pp. 1001-1007. [DOI: https://dx.doi.org/10.1134/S1547477117070068]

68. Dubey, P.; Srivastava, D.; Singh, K.; Singh, V. Middleware Architecture for Microservices based Distributed System. Proceedings of the 13th International Conference on Cloud Computing, Data Science and Engineering, Confluence 2023; Noida, Indian, 19–20 January 2023; pp. 200-207. [DOI: https://dx.doi.org/10.1109/Confluence56041.2023.10048814]

69. Prakash, V.S.; Zhao, X.; Wen, Y.; Shi, W. Back to the future: Using magnetic tapes in cloud based storage infrastructures. Lect. Notes Comput. Sci. (Incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinform.); 2013; 8275 LNCS, pp. 328-347. [DOI: https://dx.doi.org/10.1007/978-3-642-45065-5_17]

70. Helgason, O.; Kouyoumdjieva, S.T.; Pajević’, L.; Yavuz, E.A.; Karlsson, G. A middleware for opportunistic content distribution. Comput. Netw.; 2016; 107, pp. 178-193. [DOI: https://dx.doi.org/10.1016/j.comnet.2016.05.026]

71. Singh, A.; Haahr, M. A Survey of p2p Middlewares; Technical Report School of Computer Science & Statistics, Trinity College Dublin: Dublin, Ireland, 2005.

72. Chen, Y.; Mao, T.; Yu, B. A reliable messaging middleware for financial institutions. Proceedings of the ACM International Conference Proceeding Series; Tokyo, Japan, 24–26 November 2017; pp. 108-112. [DOI: https://dx.doi.org/10.1145/3162957.3163050]

73. Siegemund, G.; Turau, V. A self-stabilizing publish/subscribe Middleware for IoT applications. ACM Trans. Cyber-Phys. Syst.; 2018; 2, pp. 1-26. [DOI: https://dx.doi.org/10.1145/3185509]

74. Thomas, R. Education: Cultural and Religious Concepts. International Encyclopedia of the Social & Behavioral Sciences; Smelser, N.J.; Baltes, P.B. Pergamon: Oxford, UK, 2001; pp. 4197-4200. [DOI: https://dx.doi.org/10.1016/B0-08-043076-7/02333-0]

75. Kirkwood, A.; Price, L. Technology-enhanced Learning and Teaching in Higher Education: What is ‘enhanced’ and how do we know? A Critical Literature Review. Learn. Media Technol.; 2013; 39, pp. 6-36. [DOI: https://dx.doi.org/10.1080/17439884.2013.770404]

76. Gulati, S. Technology-Enhanced Learning in Developing Nations: A review. Int. Rev. Res. Open Distrib. Learn.; 2008; 9, [DOI: https://dx.doi.org/10.19173/irrodl.v9i1.477]

77. Buongiorno, F.; Chiaramonte, X. Environment. Handbook of the Anthropocene: Humans between Heritage and Future; Springer International Publishing: Cham, Switzerland, 2023; pp. 49-54. [DOI: https://dx.doi.org/10.1007/978-3-031-25910-4_8]

78. Flint, R.W. The sustainable development of water resources. Water Resour. Update; 2004; 127, pp. 48-59.

79. Arnold, J.G.; Srinivasan, R.; Muttiah, R.S.; Williams, J.R. LARGE AREA HYDROLOGIC MODELING AND ASSESSMENT PART I: MODEL DEVELOPMENT1. JAWRA J. Am. Water Resour. Assoc.; 1998; 34, pp. 73-89. [DOI: https://dx.doi.org/10.1111/j.1752-1688.1998.tb05961.x]

80. Kerschtien, S.C.; Aquino, T. Oceans 3.0 Knowledge Base; Ocean Networks Canada (ONC): Victoria, BC, Canada, 2023; Available online: https://wiki.oceannetworks.ca/display/O2KB (accessed on 31 June 2024).

81. Moran, K. Earthquake Early Warning System; Technical Report Ocean Networks Canada: Victoria, BC, Canada, 2019.

82. Ortega, R.J. Esri-es/Fiware-ArcGIS: Middleware that Allows Fiware Context Broker Notifications to be Stored in Arcgis Online or Arcgis Server; Esri España: Tetuán, Madrid, Spain, 2023; Available online: https://github.com/esri-es/Fiware-ArcGIS (accessed on 31 June 2024).

83. West, R. Environmental Systems Research Institute Redlands, C. Understanding ArcSDE: GIS by ESRI; ArcGis 8.1 Environmental Systems Research Institute: Redlands, CA, USA, 2001.

84. Leal Filho, W.; Azeiteiro, U.; Alves, F.; Pace, P.; Mifsud, M.; Brandli, L.; Caeiro, S.S.; Disterheft, A. Reinvigorating the Sustainable Development Research Agenda: The Role of the Sustainable Development Goals (SDG). Int. J. Sustain. Dev. World Ecol.; 2018; 25, pp. 131-142. [DOI: https://dx.doi.org/10.1080/13504509.2017.1342103]

85. Mohammad Mosadeghrad, A. Healthcare service quality: Towards a broad definition. Int. J. Health Care Qual. Assur.; 2013; 26, pp. 203-219. [DOI: https://dx.doi.org/10.1108/09526861311311409]

86. Rohmah, R.; Handaga, B.; Nurokhim, M.; Soesanti, I. A statistical approach on pulmonary tuberculosis detection system based on X-ray image. Telkomnika (Telecommun. Comput. Electron. Control); 2019; 17, pp. 1474-1482. [DOI: https://dx.doi.org/10.12928/telkomnika.v17i3.10546]

87. Ngwatu, K.; Ntwali, P.; Oxlade, O.; Mappin-Kasirer, B.; Linh, N.; Jaramillo, E.; Falzon, D.; Schwartzman, K. The impact of digital health technologies on tuberculosis treatment: A systematic review. Eur. Respir. J.; 2018; 51, 1701596. [DOI: https://dx.doi.org/10.1183/13993003.01596-2017]

88. PHAP. Humanitarianism | phap.org; Professionals in Humanitarian Assistance and Protection (PHAP): Brussels, Belgium, 2023; Available online: https://phap.org/PHAP/PHAP/Themes/Humanitarianism.aspx (accessed on 18 November 2023).

89. Kovács, G.; Vega, D. Humanitarian Logistics. International Encyclopedia of Transportation; Vickerman, R. Elsevier: Oxford, UK, 2021; pp. 190-194. [DOI: https://dx.doi.org/10.1016/B978-0-08-102671-7.10242-8]

90. Abdelbary, I.; Elshawa, R.H. The role of logistics operations in humanitarian aids: Lessons learned from the Syria crisis. Cases on International Business Logistics in the Middle East; IGI Global: Hershey, PA, USA, 2023; pp. 205-229. [DOI: https://dx.doi.org/10.4018/978-1-6684-4686-7.ch011]

91. UNDRR. Terminology on Disaster Risk Reduction. Technical Report; United Nations Office for Disaster Risk Reduction (UNDRR): Geneva, Switzerland, 2009; Available online: https://www.undrr.org/publication/2009-unisdr-terminology-disaster-risk-reduction (accessed on 31 November 2023).

92. Noran, O.; Bernus, P. Effective disaster management: An interoperability perspective. Proceedings of the On the Move to Meaningful Internet Systems: OTM 2011 Workshops: Confederated International Workshops and Posters: EI2N+ NSF ICE, ICSP+ INBAST, ISDE, ORM, OTMA, SWWS+ MONET+ SeDeS, and VADER 2011; Hersonissos, Crete, Greece, 17–21 October 2011; Proceedings Springer: Berlin/Heidelberg, Germany, 2011; pp. 112-121.

93. Saha, H.N.; Auddy, S.; Pal, S.; Kumar, S.; Pandey, S.; Singh, R.; Singh, A.K.; Banerjee, S.; Ghosh, D.; Saha, S. Disaster management using Internet of Things. Proceedings of the 2017 8th Industrial Automation and Electromechanical Engineering Conference, IEMECON 2017; Bangkok, Thailand, 16–18 August 2017; pp. 81-85. [DOI: https://dx.doi.org/10.1109/IEMECON.2017.8079566]

94. Abrishami Shirazi, N.; Omidvar, B.; Nohegar, A. 1—Introduction to decision-making and disaster management systems. Crises in Oil, Gas and Petrochemical Industries; Elsevier: Amsterdam, The Netherlands, 2023; pp. 3-24. [DOI: https://dx.doi.org/10.1016/B978-0-323-95154-8.00008-6]

95. Allen, D.K.; Karanasios, S.; Norman, A. Information sharing and interoperability: The case of major incident management. Eur. J. Inf. Syst.; 2014; 23, pp. 418-432. [DOI: https://dx.doi.org/10.1057/ejis.2013.8]

96. Karanasios, S. New & Emergent ICTs and Climate Change in Developing Countries; Center for Development Informatics, Institute for 1324 Development Policy and Managment, SED University of Manchester: Manchester, UK, 2011.

97. Trahtengerts, E.; Pashchenko, A. Principles of implementation and estimation of influence factors in network centric systems. Proceedings of the 11th IEEE International Conference on Application of Information and Communication Technologies, AICT 2017—Proceedings; Moscow, Russia, 20–22 September 2017; [DOI: https://dx.doi.org/10.1109/ICAICT.2017.8687300]

98. Tajalizadehkhoob, S.; Van Goethem, T.; Korczyński, M.; Noroozian, A.; Böhme, R.; Moore, T.; Joosen, W.; Van Eeten, M. Herding vulnerable cats: A statistical approach to disentangle joint responsibility for web security in shared hosting. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security; Dallas, TX, USA, 30 October–3 November 2017; pp. 553-567.

99. Tada, H.; Murata, M.; Aida, M. Web service flash crowd mitigation using feedback to users. Proceedings of the Proceedings—International Conference on Advanced Information Networking and Applications, AINA; Taipei, Taiwan, 27–29 March 2017; pp. 213-219. [DOI: https://dx.doi.org/10.1109/AINA.2017.121]

100. Wilson, J. Religion and Volunteering. The Economics of Religion; World Scientific: Singapore, 2023; pp. 187-244. [DOI: https://dx.doi.org/10.1142/9789811273148_0008]

101. Durrani, M.N.; Shamsi, J.A. Volunteer computing: Requirements, challenges, and solutions. J. Netw. Comput. Appl.; 2014; 39, pp. 369-380. [DOI: https://dx.doi.org/10.1016/j.jnca.2013.07.006]

102. Charalampidis, I. Virtual Atom Smasher. Presentation, S’Cool LAB; CERN: Meyrin, Switzerland, 2015; Available online: https://indico.cern.ch/event/373094/contributions/882384/attachments/742836/1019002/SCoolLab-Presentation-2.pdf (accessed on 31 November 2023).

103. CORDIS. Advanced Sensors and Lightweight Programmable Middleware for Innovative RFID Enterprise Applications: Aspire Project: Fact Sheet: FP7: CORDIS: European Commission; CORDIS: Cordis, Hong Kong, 2008.

104. GS1. Low Level Reader Protocol, 2010; GS1: London, UK, 2009; Available online: https://www.gs1.org/standards/epc-rfid/llrp/1-1-0 (accessed on 15 February 2024).

105. GS1. Application Level Events (ALE) Standard; GS1: London, UK, 2009; Available online: https://www.gs1.org/standards/rfid/ale (accessed on 15 February 2024).

106. Richards, M.; Ford, N. Fundamentals of Software Architecture: An Engineering Approach; O’Reilly Media: Sebastopol, CA, USA, 2020.

107. Wisswani, N.W.; Wijaya, I.W.K. Message Oriented Middleware for Library’s Metadata Exchange. TELKOMNIKA (Telecommun. Comput. Electron. Control); 2018; 16, pp. 2756-2762. [DOI: https://dx.doi.org/10.12928/telkomnika.v16i6.9475]

108. Farina, J.; Scanlon, M.; Kechadi, M.T. Bittorrent sync: First impressions and digital forensic implications. Digit. Investig.; 2014; 11, pp. S77-S86. [DOI: https://dx.doi.org/10.1016/j.diin.2014.03.010]

109. Charalampidis, I. LiveQ: An Interactive Volunteering Computing Batch System Source-Code; GitHub: San Francisco, CA, USA, 2015; Available online: https://github.com/wavesoft/LiveQ (accessed on 13 February 2024).

110. Yadav, P.; Cohen, J.; Darlington, J. CitizenGrid: An Online Middleware for Crowdsourcing Scientific Research. arXiv; 2017; arXiv: 1707.09489

111. Apache Airavata. Apache Airavata; Apache Airavata: Forest Hill, MD, USA, 2022; Available online: https://airavata.apache.org/ (accessed on 25 August 2024).

112. Eclipse. Eclipse Mosquitto; Eclipse Foundation AISBL: Brussels, Belgium, 2018; Available online: https://mosquitto.org/ (accessed on 25 August 2024).

113. Burkart, C.; Wakolbinger, T.; Toyasaki, F. Funds allocation in NPOs: The role of administrative cost ratios. Cent. Eur. J. Oper. Res.; 2018; 26, pp. 307-330. [DOI: https://dx.doi.org/10.1007/s10100-017-0512-9]

114. UNCTAD. Data Protection and Privacy Legislation Worldwide | UNCTAD; UN Trade and Development (UNCTAD): Geneva, Switzerland, 2023; Available online: https://unctad.org/page/data-protection-and-privacy-legislation-worldwide (accessed on 3 March 2024).

115. Henriksen-Bulmer, J.; Faily, S.; Jeary, S. Implementing GDPR in the Charity Sector: A Case Study. Privacy and Identity Management. Fairness, Accountability, and Transparency in the Age of Big Data: 13th IFIP WG 9.2, 9.6/11.7, 11.6/SIG 9.2.2 International Summer School, Vienna, Austria, August 20–24, 2018, Revised Selected Papers; Springer International Publishing: Cham, Switzerland, 2019; pp. 173-188. [DOI: https://dx.doi.org/10.1007/978-3-030-16744-8_12]

116. Voigt, P.; Von dem Bussche, A. The eu general data protection regulation (gdpr). A Practical Guide; 1st ed. Springer International Publishing: Cham, Switzerland, 2017; Volume 10, pp. 10-5555.

117. Douma, M.; Gamito, E.J. A culturally appropriate, Web-based technology for anonymous data collection for public health research in culturally diverse populations. J. Cases Inf. Technol.; 2007; 9, pp. 15-26. [DOI: https://dx.doi.org/10.4018/jcit.2007070102]

118. Almadani, B.; Aliyu, F.; Aliyu, A. Integrated Operation Centers in Smart Cities: A Humanitarian Engineering Perspective. Sustainability; 2023; 15, 11101. [DOI: https://dx.doi.org/10.3390/su151411101]

119. Van de Walle, B.; Van Den Eede, G.; Muhren, W. Humanitarian information management and systems. Proceedings of the Mobile Response: Second International Workshop on Mobile Information Technology for Emergency Response, MobileResponse 2008; Bonn, Germany, 29–30 May 2008; Revised Selected Papers 2 Springer: Berlin/Heidelberg, Germany, 2009; pp. 12-21.

120. Cerqueira, J.M.; Cleto, B.; Moura, J.M.; Sylla, C.; Ferreira, L. Potentiating Learning Through Augmented Reality and Serious Games. Springer Handbook of Augmented Reality; Springer International Publishing: Cham, Switzerland, 2023; pp. 369-390. [DOI: https://dx.doi.org/10.1007/978-3-030-67822-7_15]