1. Introduction
Citizen Observatories (COs) have emerged in the last decade as a vehicle to facilitate and promote the adoption of citizen science [1]. A broad interpretation of citizen science is envisaged, for example open innovation in healthcare [2]. Moreover, COs are not necessarily coupled to just the one scientific domain. Rather than solely focusing on traditional science-focused initiatives in collaboration with the professional scientific community, the CO seeks to foster community engagement. Such engagement is manifested in diverse ways—contributing to policy definition, fostering a sense of ownership in the stewardship of the environment, and crucially, promoting more meaningful involvement in the democratic process. Another essential objective is enabling further progress toward the UN Sustainable Development Goals (SDGs) [3]. Successful COs are prototyped, developed, and deployed in diverse application domains, for example, in water quality [4], weather [5], and disaster risk reduction [6]. Data from COs may be conflated with that from other sources, for example, satellites [7], to demonstrate added value.
COs are now at a critical stage in their development cycle. In the next few years, the CO concept will become mainstream, evolve, or be replaced by another model of participatory engagement. Indeed, it remains to be seen if research in COs has now passed its peak. There are reasons to be optimistic—citizen science is a cross-cutting initiative in the current Horizon Europe research programme; thus, more deployments can be realistically expected. However, COs are not immune to best-practice principles in research and innovation and must therefore adapt, complement, and align with best practices. One of the most significant policy initiatives in the European research landscape in the last decade is Responsible Research and Innovation (RRI) [8].
RRI has emerged as a concept, policy, and code of practice in the last decade. The concept is embedded in the European Research Area (ERA) [9] and has diffused beyond Europe. RRI seeks to align the public good with research and development [10] and currently constitutes an integral component of the EU Horizon Europe programme. Likewise, COs must be cognisant of the obligations conferred by RRI. Given the crucial stage in the evolution of COs and the importance of COs going forward, it is opportune to ask to what degree research in COs has reflected and manifested RRI principles. This paper seeks to answer this question through the instrument of a literature analysis.
This paper is structured as follows: Section 2 presents a background to the evolution of COs and RRI, after which the review question is fully articulated. Section 3 presents the methodology for answering the research question. Results are presented in Section 4 and discussed in Section 5. Future work is considered in Section 6, after which the paper is concluded.
2. Background
Citizen science has a long and distinguished history. Weather watching and bird counts are well-documented citizen science activities. It is often forgotten that the era of the professional scientist is a relatively recent phenomenon. In recent decades, there has been an upsurge in interest in citizen science. Developments in ICT and especially smartphones have proved transformational, offering novel platforms for public engagement. Many benefits accrue from having a public awareness of science, not least an ability to confront many of the global challenges of this time [11]. In Europe, a framework to promote the uptake of citizen science has emerged—namely, the Citizen Observatory (CO).
In South America, the term citizen observatory has been used since the 1990s. For example, the Citizen’s Observatory of Education (Observatorio Ciudadano de la Educación), is a publication informing public opinion on education policies [12]. However, the European Union has adopted and popularised the concept in its various research programmes since approximately 2010. Calls for prototypes, demonstrators, and implementations began at the end of the FP7 programme and continued throughout Horizon 2020, especially in the Science with and for Society (SWAFS) initiative. Currently, COs are included in Horizon Europe but more as a cross-cutting theme. For a detailed description of those projects funded under the various EU schemes, see: [13,14,15].
Citizen science and COs are tightly coupled concepts but not synonyms; subtle but vital differences exist in their goals. Citizen science is often regarded as a paradigm of scientific enquiry. However, such a singular view is quite restrictive, and a plurality and diversity of interpretations have been observed [16]. COs often encompass scientific research but prioritise policy and governance to create synergies with external actors to address community issues [17]. It is often implicitly assumed that such matters pertain to environmental monitoring, but this need not be the case. COs can also incorporate different models of participatory science, including Volunteered Geographic Information (VGI) and community-based monitoring.
No uniform definition of a CO exists; however, that of Rathnayake et al. [18] (p. 326) serves as a working definition: “Citizen observatories can be defined as socio-technical constellations designed to make people who are nonprofessional scientists empowered entities”. An alternative interpretation is offered by Liu et al. [19] (p. 1), who consider the essence of a CO as a “process that involves environmental monitoring, information gathering, data management and analysis, assessment and reporting systems”. Gonda et al. [20] (p. 9) prioritise collaboration and collaboration, envisaging a CO as a “collaborative and participatory effort by an interdisciplinary team that collects data from organisations, networks, and citizens with the objective of filling information gaps”, in this case, information concerning COVID-19 in Nicaragua. To date, contributions by COs include increased citizen engagement, environmental monitoring, improved dialogue between citizens and public authorities, and contributed valuable scientific knowledge, among others [14].
Successful COs are enabled by the commitment and engagement of all relevant stakeholders, from citizens to policymakers. However, many obstacles exist. Where there is a need for repetitive measurements, for example, complementary in situ data for remote sensing, the inherent strength of the community could be a weakness [1]. For land cover classification, citizen scientists’ diverse interpretations of semantics and affordances demanded consideration [21]. While the laudable objective of improving citizens’ scientific literacy is often highlighted, increasing the professional scientists’ knowledge of citizens as experts is of utmost importance [22]. A critical need to define pathways to enable data-informed decision making, for example, in transformations to sustainability that encompass socio-cultural contexts and practices, has been identified [23]. Fundamentally, the credibility of citizen science as a research paradigm is sometimes questioned. Elliott and Rosenberg [24] argue that, philosophically, these reservations are irrelevant in a general sense in that the quality of citizen science cannot be decoupled from the specific research context.
Professional scientists regularly encounter scenarios that can provide severe obstacles for the citizen science community. For example, ethical dilemmas concerning information management may pose significant difficulties in light of ownership, privacy, and security concerns. For COs, such issues are omnipresent, and thus, responsibilities and consequences—both intended and unintended—need careful consideration [25]. A synergy exists between open science and citizen science. However, a tension exists between openness and the need for data protection, which could hinder their mutual progress [26]—citizen science in the health domain is a case in point [27]. Thus, while the potential of citizen science and COs to confront grand societal challenges is acknowledged, there is a fundamental need for high-quality data practices as well as aligning data with decision making [28]. Data challenges may be best considered within the context of Responsible Research and Innovation.
2.1. Responsible Research and Innovation
Responsible Research and Innovation (RRI) has become central to EU policy in the last decade [29,30]. It was a cross-cutting theme in the H2020 programme. Concerns were expressed that RRI would not feature in the current Horizon Europe programme [31]; however, this is not the case. In the EU context, RRI has its roots in the early framework Programmes, including FP7, for example, the RRI Toolkit (
Broadly speaking, RRI seeks to align European research and innovation with the needs of society. An operational definition of RRI within an EU context is offered by Meijer and van de Klippe [32] (p. 171): “A process where all societal actors (researchers, citizens, policy makers, business representatives, third sector organisation representatives, etc.) work together during the whole R&I process in order to better align R&I outcomes with the values, needs and expectations of European society”.
However, a variety of definitions of RRI may be found in the literature; see, e.g., Schomberg [33]. Moreover, it must be emphasised that RRI is not just an EU initiative. However, its contextualisation to other regions, for example, the Global South, remains a work in progress [34]. Moreover, its dimensions are open to debate; Szymanski et al. [35] argue that RRI should learn from multispecies studies and expand its definition of stakeholders to include animals and plants.
Despite significant progress, problems remain, especially with the operationalisation of RRI. Interest in RRI originally grew from an observation that innovation was perceived as being disconnected from the societies it served. However, innovation is only beneficial if it meets societal needs economically, environmentally, and sustainably [36]. Indeed, synergies exist between RRI and the quadruple helix model of innovation [37]. Nevertheless, intrinsic difficulties exist with the concept as no agreed definition exists. This may be attributed to RRI being an evolving construct; see, e.g., Rip [38]. Alternatively, if RRI focuses on producing science for the public good, critical questions arise as to its epistemic nature or what the public good entails [10]. For researchers, such a lack of clarity is problematic. RRI within the context of the EU initiatives is essentially a prerequisite for funding. Nonetheless, significant flexibility in interpreting and implementing RRI exists for the actors on the ground. Such flexibility is understandable at this stage of the evolution of RRI; however, a lack of agreed metrics hinders the mainstreaming and monitoring of RRI, and an increased need for additional methodological information concerning RRI practices has been identified [39].
Traditionally, RRI was synonymous with six keys: public engagement, gender equality, science literacy and science education, open access, ethics, and governance. However, the issue of governance was perceived as problematic and was excluded [38]. Hence, for many researchers, RRI is a cross-cutting theme consisting of five keys. However, how the keys were implemented and measured was particular to individual projects.
2.2. Study Rationale
Currently, there is a concerted effort to mainstream citizen science in EU research. Thus, RRI must be considered in both citizen-driven initiatives as per the CO model and in collaborations with professional scientists and other societal actors. At present, it is unclear how RRI has permeated CO initiatives and been manifested therein. Understanding this gap will lead to a deeper understanding of what remedial actions might be taken by CO initiators to improve the contribution of citizen science to societal priorities, including policy development. Thus, the following research question was formulated: How has the influence of RRI been manifested in Citizen Observatory initiatives, as documented in the research literature? To answer the research question, the advice of Munn et al. [40] is adopted. Here, the authors recommend scoping reviews where the research question is imprecise and seeks to identify knowledge gaps, clarify concepts, and investigate research practices. Such reviews may be regarded as an independent study, consolidating knowledge and giving an overview of this body of knowledge such that progress may be assessed and avenues for further research identified [41].
In this paper, we identify emerging evidence for the adoption of RRI with COs. This research synergises with the systematic review undertaken by Rathnayake et al. [18]. Here, the authors undertake a detailed mapping of CO initiatives in environmental monitoring, drawing interesting findings concerning civic participation, data management, and societal implications. Though these themes naturally overlap with this study, RRI themes were not explicitly considered.
3. Methodology
This review is reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-SCR) [42,43]. The study was conducted in late 2021.
3.1. Eligibility Criteria
The key criterion for consideration for inclusion in this review was the inclusion of the term “citizen observatory” or equivalent in the title, abstract, or keywords. All papers were either from a journal or conference proceedings, or a chapter in an edited book. As such, all articles were peer-reviewed, thus ensuring quality. Exclusion criteria included the paper being an extended or poster abstract. Documents that were not available for downloading or written in English were likewise excluded.
3.2. Search Strategy
Three exemplar electronic databases were harnessed: the well-known Scopus and Web of Science databases [44] and EBSCO. Scopus is amongst the foremost bibliographic databases of peer-reviewed literature, including science and technology, arts and humanities, and social sciences. The Web of Science is an interdisciplinary database encompassing several individual bibliographic databases, for example, the Science Citation Index (SCI). EBSCO provides access to an extensive range of citation databases, e-books, and e-journals, including business and economics journals. There is an overlap in records between all three databases. Google Scholar was assessed; as experienced by Lasda Bergman [45], many duplicates were returned, so it was not used.
The search terms were (“citizen observatory” OR “citizens observatory” OR “citizen observatories” OR “citizens observatories”). These terms were applied to the title, abstract, and keywords. All records were exported in BibTex and imported into Citavi 6. Duplicates were removed as part of this process. Papers were then retrieved electronically as PDFs through the university library using the EZProxy web proxy server.
3.3. Selection of Sources of Evidence
An initial screening exercise excluded papers that were irrelevant to the study, for example, extended and poster abstracts, editorials, and other short entries such as those in magazines. A second more thorough screening involved a complete reading of the title, abstract, and keywords. Here, theoretical papers, for example, describing software frameworks, were identified and excluded. Articles were excluded if they did not include a practical citizen engagement dimension. In cases where such engagement was perceived as minimal, the paper was included for a thorough, detailed analysis.
3.4. Synthesis of Results
All studies were initially clustered according to whether they addressed each of the five keys synonymous with RRI, namely public engagement, gender equality, science literacy and science education, open access, and ethics. In the cases of gender and ethics, it was a binary decision. The situation was more complicated for the other keys in that synonyms for the terms education or literacy might be used. Thus, only when each paper was read was a final decision made concerning its RRI focus, noting that cluster membership is not mutually exclusive.
3.5. Study Selection
After duplicates were removed, a total of 129 papers remained for screening. After the initial screening of citations, 43 papers were removed. A more detailed screening resulted in a further 38 papers being removed. Finally, 86 studies were considered suitable for a complete detailed analysis. After a full analysis of the papers from the perspective of the RRI keys, a total of 48 articles were considered for full inclusion in this review. See Figure 1 for a PRISMA 2020 flow diagram.
3.6. Limitations
The focus of this study is unambiguously on citizen observatories. It does not include singular studies in citizen science, participatory research, crowdsourcing, VGI, or equivalent. The three databases are exemplary, but no claim is made that the corpus underpinning this review is complete and exhaustive. The grey literature did not inform this analysis. It is possible that more recent material such as project deliverables or preprints was published in open archives such as Zenodo or arXiv. Thus, there is a risk of academic bias in this study. Finally, all papers are in English, so some international studies may be overlooked.
4. Results
Each of the five RRI keys is now considered individually, detailing how the CO literature details and reports on each.
4.1. Open Access
Adoption of open access practices has surged in recent years. Research that is publicly funded is often under an obligation to be made freely available for all. Many in the research community see open access as a channel for disseminating project results. In this way, impact is increased and measured using a variety of metrics. Over 48% of the publications are available in an open access format, confirming a trend to open access and that this process remains ongoing.
Open data seek to make datasets open to all and are often assumed as best practice within citizen science. The need for open global datasets remains urgent; it is estimated that only 50% of the SDG indicators are recorded, and that the available datasets are frequently out of date [46]. While the potential for citizen science to contribute remains significant, the risk of stereotyping citizen scientists as mere data collectors may limit the potential of COs [3]. Data quality remains an omnipresent issue with data collected by COs. To counteract negative perceptions, Liu et al. [19] advise focusing on accuracy and uncertainty, privacy and security, and qualitative and quantitative indicators. However, despite potentially larger errors and uncertainties, Tian et al. [47] consider COs a valuable but alternative source of information in incidences of urban pluvial flooding. Gharesifard and Wehn [48] investigated motivations for sharing weather data. Trust was identified as a critical indicator of willingness to share data. Motivations reported include reducing risk, benefiting wider society through knowledge creation, and the perceived moral obligations. Interestingly, previous experience was identified as an indicator of trust.
Having collected data, a key question facing COs concerns the practicalities of making the data available. Weather enthusiasts offer a model that could be aspired to. There is a long tradition of weather watching that even predates the well-known Audubon Christmas Bird Count. Moreover, the technology is well-advanced, and amateur weather enthusiasts have a wide variety of online platforms where they can upload their observations. Thus, while there is an initial and longer-term monetary investment, the cost associated with data sharing is minimal [5]. Moreover, the data can then be automatically consumed by various popular weather networks, thus becoming an integral source of weather information [49]. However, for other citizen science initiatives, the problem is more complex. Few projects have official links with public or policy institutions outside of the well-known citizen science initiatives. The Citclops CO developed a public interface through which data could be downloaded [50]. The COBWEB CO provided data to a Local Environmental Record Centre (LERC), part of the UK National Biodiversity Network.
Aligning with the spirit of open access, some CO’s have harnessed open hardware and software platforms. Thus, any CO or citizen science initiative could replicate the methodology, at least in principle. KdUINO is a DIY kit based on the open-source Arduino platform that enables continuous monitoring of light attenuation throughout the water column [51]. Here, data can be visualised in near real time on an interactive map. SmartFluo follows a similar model [52]; this device can be manufactured using 3D printing techniques and is thus attractive as a frugal technology and open labware platform [53]. From a software perspective, there has been an understandable emphasis on open-source software. To maximise data access, an adherence to open standards is often manifested. The CitiSense platform is archetypical, adopting international standards for geospatial data, e.g., the OGC Web Feature Service (WFS), open web interface technologies, e.g., Representational State Transfer (REST), and standard encoding formats, such as JSON [54].
4.2. Public Engagement
As citizen participation is a key aspect of the CO model, it is the dimension of RRI that is most visible in CO research. Such participation is identical to that fostered in classic citizen science. However, the objective of citizen engagement in COs is more directed toward obtaining evidence for informing policy and enabling informed decision making that is more cognisant of the realities in the community. Enabling processes are bottom-up, participative, collaborative, and oriented towards problem articulation and solution generation. Such processes are not necessarily ICT-mediated; instead, public meetings, workshops, and social media also contribute [55]. Motivations for participation vary and include, for example, personal interest and altruism. Nevertheless, the most successful projects are those where citizens directly benefit [56].
Documented flood monitoring and water governance are typical of the approaches that COs has adopted in general. In the case of flooding, the WeSenseIT CO explored participation as experienced by citizens and authorities, harnessing face-to-face interviews and focus groups across three different countries [57]. Here, physical meetings were identified by citizens as occurrences with the highest impact on decision making. Likewise, the divergent roles ascribed to and assumed by citizens influence the dynamics of participation, the perception of local authorities, and the contribution to water governance [58]. In the case of flooding, the need for public participation has been acknowledged in policy, namely the European Flood Directive 2007/60/EC. Fundamental to involvement in a CO is its usability; Degrossi et al. [59] report on the user-centred design of a CO for floods in Brazil.
Pargman et al. [4] consider the potential of the CO as a model for the management and stewardship of water. A co-design approach solicited multistakeholder input via semi-structured interviews and workshops. Similar to the flood management scenario outlined above, a recurring theme concerning the disparity in expectations between citizens and policymakers emerged. The need for a critical mass of participation for stakeholder engagement was observed. Nevertheless, a critical observation concerned the difficulties in bringing the diverse stakeholders together when seeking to provide them with a space for deliberation. This intrinsic need for comprehensive, active multistakeholder, multidisciplinary dialogues was also observed by Liu et al. [19].
Community Champions were pioneered by the GROW CO. Serving as ambassadors on the ground, they liaised with local communities, offering support and training. Moreover, ambassadors acted as intermediaries between participants on the ground and the CO development team [60]. The establishment of communities in thirteen EU countries testifies to this model of facilitating community engagement. The Osservatorio del Mare a Molfetta is unique in that it was founded by volunteers interested in coastal and marine biodiversity [61]. It is an excellent example of a bottom-up CO initiative, but one that actively collaborates with the professional science community. Participation is open to motivated individuals who use their own devices to participate in an underwater visual census. The LandSense CO sought to stimulate civic engagement in Vienna, enabling the public to provide their perceptions of urban space, thus influencing the planning process [62].
4.3. Science Education
RRI seeks to transform the ethos of how research is conducted throughout the EU, including in higher education institutions. Thus, it significantly influences the training and preparation of the next generation of researchers. However, science education has a significant role outside of formal education. COs are essentially bottom-up or community-driven initiatives. Thus, a key role is to equip citizen scientists and other actors to engage in responsible research and innovation. In this way, a more scientifically literate community emerges and is thus far better prepared to contribute to and engage in policy definition.
Citizen science offers many education opportunities, and COs have engaged in various learning and training activities. However, informal learning dominates, despite the prevalence of academic institutions in CO research. One exception occurred in Vienna via the LandSense CO [62]. Here, students undertaking a course unit on landscape architecture evaluated a mobile application (app) as part of their course work. The GROW CO developed Massive Open Online Courses (MOOCs) that facilitated enrolment in various free courses, thus empowering citizens to become involved in climate action [3]. A further initiative of the GROW CO was to provide contributors with extensive training in protocols for data collection. In this way, the CO sought to confront and mitigate the omnipresent data quality problem in citizen science [63].
COs are seen predominantly as contributing to both informal and lifelong learning, seeking to foster a sense of autonomy and responsibility for citizens [54]. Active participation and collaboration foster knowledge generation and cultivate values of solidarity and shared responsibility essential to sustainable development; research-based learning enabled by COs offers an apt approach here [64]. Moreover, such activities may alleviate eco-anxiety, a growing problem amongst young adults, especially in Western countries [65]. When the CitClops CO evaluated the kdUINO platform, school children contributed to usability testing.
A common approach to validating the operation of a CO was to undertake several studies with potential end communities of users. In many cases, an app constituted the user-facing dimension of the CO, and a usability or equivalent study was undertaken. Training was provided, and potential end users were requested to complete some tasks. The methodologies followed by the COBWEB CO and in a case study in Brazil [66] are archetypical. In such cases, however, learning can be minimal in that potential participants act as little more than data collectors. Extreme citizen science [67] envisages citizen scientists contributing to a full range of activities, from problem specification to methodology definition to interpretation and dissemination of results. The GROW CO adopted such an approach as part of its Living Soils mission, focusing on hypothesis-driven rather than observation-driven research [54]. In the Osservatorio del Mare a Molfetta CO, in the Italian Southern Adriatic Sea, citizen scientists contributed to all aspects of the scientific process and collaborated with professional scientists outside of academic institutions.
4.4. Gender
Gender within the context of RRI seeks to ensure gender equality in all research and innovation activities. Gender is often perceived as synonymous with gender balance, but it is only one characteristic. A second dimension is more holistic and far-reaching, requiring that all activities are considered through the lens of potential gender differences. Research activities should first consider and then incorporate gender differences into their methodologies. Sex and gender analysis offers one approach for assessing the gender dimension in RRI [68].
Gender is not considered either broadly or deeply in the CO literature. Gender was sometimes recorded as part of surveys or evaluations. Gharesifard and Wehn [69] considered gender when exploring the drivers and barriers when sharing data in amateur weather networks. Likewise, gender was recorded when validating an app for collecting water and land cover data [70], and in a usability study on flood risk [6]. However, a holistic treatment of gender was not manifested.
4.5. Ethics
Ethics permeate all aspects of research and innovation, representing an essential dimension of research integrity. Thus, RRI demands that all potential ethical implications be considered. In addition to issues such as avoiding falsification and plagiarism, matters pertaining to governance, informed consent, and privacy need further consideration. In many instances, initiatives must conform with international and national guidelines and be cognitive of best practices in the domain under investigation. Moreover, ethics is omnipresent. Ethics approval is often seen as a paper exercise at the start of an initiative. However, contexts invariably change as efforts progress in research and innovation, thus demanding a reassessment of ethical implications as projects mature.
A lack of reference to ethics characterises the published literature on COs. In the medical domain, a statement concerning ethical approval is a prerequisite for review and publication. No equivalent practices were discernible in the CO literature.
5. Discussion
Penetration of RRI principles in COs remains underdeveloped. Overall, the findings concur with those of Novitzky et al. [71], who identified a lack of consistent integration of RRI across the EU H2020 programme. Further effort is needed by the citizen science community, society, and government to fulfil the objectives and potential of RRI. Many of the publications examined as part of this scoping review were funded under various EU schemes. Thus, these explicitly include an RRI element as a prerequisite. Moreover, it is unreasonable to expect every project publication to cover all RRI keys. Nonetheless, it was evident from this review that many RRI themes were hinted at; however, opportunities to develop themes were not always availed of. Ethics are a case in point. There can be little doubt that ethical approval was needed and given for many of these studies. However, this was not mentioned, even in passing. Thus, for future CO research, strengthening the RRI dimension is strongly encouraged and should be reported where appropriate. Otherwise, as highlighted by Lowry and Stepenuck [72], issues of bias, inclusion, exploitation, and practicability may result in citizen science losing its place in the scientific process. Some guidelines are suggested for each of the five keys when undertaking CO activities.
5.1. Open Access
A solid commitment to open access is manifested, encompassing publication, open-source software and hardware, open standards, and open data. Open access to scientific publications is reasonably well-advanced. Open data remain problematic as they encompass diverse issues such as security, GDPR, privacy, and the definition of meta-data. More importantly, they demand a complete understanding of licensing. It is fundamental that citizen scientists understand the need to explicitly define the conditions under which others may use the data. It must be observed that where data are collected as part of validation and testing, such data may not be suitable for long-term archiving.
P1: All COs should have a documented policy for open access management, including, if necessary, a training plan for citizen scientists.
5.2. Public Participation
COs offer several opportunities for public engagement. Such engagement has encompassed classic data collection, usability, and requirements definition. Thus, participation in CO broadly replicates that of citizen science. However, both RRI and the CO concept demand more meaningful engagement, including contributing to local democracy through policy definition. Such engagement is lacking. Moreover, the vision of how such engagement and participation might be achieved, and its overall objectives, are rarely developed.
P2: COs should define the role of the public and how pathways to local democratic input and policy derivation are actively supported.
5.3. Science Education
Education within the context of a CO is informal and may be classified under lifelong learning. How it is provided is flexible; the GROW CO used MOOCs to good effect, but such a model is not always practical. In some cases, learning is trivial, for example, how to use an app, though in this case, some domain-specific training is almost invariably needed. When conceptualising a CO, a holistic interpretation should be considered when constructing a plan for participants that is cognisant of their role and training needs while specifying how the educational dimensions of the CO should be accomplished. Though RRI highlights science education, a broader education interpretation is more appropriate for a CO.
P3: COs should define the educational dimension of their mission and plan for its implementation.
5.4. Gender
Equality and inclusion are intrinsic to COs, especially when considering their overall mission of increasing democracy and driving evidence-driven policymaking. To date, gender is treated in general terms, for example, ensuring that a population in a survey may be categorised as usual for subsequent statistical analysis. The deeper motivations for gender analysis are not meaningfully considered. A simple example might be the design of the WWW interface and ensuring that an inclusive approach is adopted from the design stage through to testing. Nevertheless, gender is much more profound; gender sensitivity must permeate all aspects of the CO, including its rationale. If this rationale is to gather data to support a particular policy intervention, then the proposed methodology needs a detailed analysis of how it facilitates or impedes gender mainstreaming.
P4: COs should define how gender mainstreaming will be accomplished.
5.5. Ethics
As people are central to the CO concept, ethics are intrinsic to its activities. Of all the RRI keys, ethics is the most weakly developed. As a starting point, all reports on CO activities should, where appropriate, confirm that ethical approval was given. Where obtaining ethical approval was challenging, some explanation should be given of the relevant issues and how they were addressed. Such information can be of great practical use to the CO community. Moreover, it should not be forgotten that as COs evolve, the ethical landscape changes. Thus, the CO is obligated to continuously monitor all activities and review the ethical implications as these arise. Indeed, any CO seeking to operate over an extended period should develop its own guidelines for ethics.
P5: COs should seek ethical approval and establish guidelines as appropriate.
5.6. Governance
Governance is the sixth key of RRI and seeks to integrate the other five keys. It was not considered in detail for the reasons explained earlier. While governance was considered within a broader context, for example, Wehn et al. [58] considered water governance, the RRI dimension did not manifestly permeate the CO discourse. How to effectively operationalise governance within CO remains an open question.
P6: COs should apply a holistic approach to RRI by defining and implementing an appropriate governance model.
6. Future Work
Several potential avenues exist for extending this work. One of the most promising is to replicate the study in the broader citizen science domain and see to what degree RRI has been conceptualised, implemented, and reported. As many COs have been involved in a European context, they are invariably influenced by the EU definition of RRI. However, other interpretations of RRI exist, for example, MoRRI. Moreover, it may be argued that enacting RRI in COs requires additional indicators. Thus, additional information from the CO and broad citizen science community concerning the practical issues of operationalising RRI and developing a roadmap for its full enactment would be beneficial.
RRI is applicable across all research and innovation activities. However, how apt this model is for COs and citizen science in general is still questionable. For example, is focusing on science education to the detriment of other education categories beneficial and sustainable? A more inclusive interpretation of education may be more appropriate for COs. A more fundamental question arises about the nature of innovation and what it means in a CO context. For example, is open innovation a dimension of open science, or is it an optional characteristic of a CO?
As COs evolve, some fundamental questions concerning their very nature arise, perhaps challenging some of the current definitions in use and demanding a more holistic definition. Rather than being viewed as a socio-technical constellation, perhaps a knowledge-based organisation or an epistemic community is a more appropriate model [73], where issues such as agility and both knowledge and quality management are prioritised [74]. Alternatively, the CO may be regarded as a high-sociability organisation that adopts an interpretivist approach to knowledge management [75].
7. Conclusions
The present landscape of RRI adoption in COs is fragmented. Thus, this paper proposes some guidelines to remedy this situation when reporting CO advancements. Looking forward, both COs and RRI will continue to evolve. As efforts to mainstream citizen science continue, a corresponding need for cognisance of RRI by COs community increases. Moreover, difficulties will be exacerbated as communities themselves increasingly drive CO initiatives. Thus, there is an urgent need to identify strategies to promote and ultimately mainstream RRI in COs. More transdisciplinary research is needed to operationalise RRI within COs effectively.
Conceptualisation, M.O. and E.M.; methodology, M.O.; writing—review and editing, M.O. and E.M. All authors have read and agreed to the published version of the manuscript.
The encouragement and support of the ISEED consortium is gratefully acknowledged.
The authors declare no conflict of interest.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1. Grainger, A. Citizen Observatories and the New Earth Observation Science. Remote Sens.; 2017; 9, 153. [DOI: https://dx.doi.org/10.3390/rs9020153]
2. Ciasullo, M.V.; Carli, M.; Lim, W.M.; Palumbo, R. An open innovation approach to co-produce scientific knowledge: An examination of citizen science in the healthcare ecosystem. Eur. J. Innov. Manag.; 2021; 25, pp. 365-392. [DOI: https://dx.doi.org/10.1108/EJIM-02-2021-0109]
3. Ajates, R.; Hager, G.; Georgiadis, P.; Coulson, S.; Woods, M.; Hemment, D. Local Action with Global Impact: The Case of the GROW Observatory and the Sustainable Development Goals. Sustainability; 2020; 12, 10518. [DOI: https://dx.doi.org/10.3390/su122410518]
4. Pargman, T.C.; Joshi, S.; Wehn, U. Experimenting with Novel Forms of Computing: The case of the Swedish Citizen Observatory for Water Quality Conservation. Proceedings of the 5th Workshop on Computing within Limits (LIMITS’ 19); Lappeenranta, Finland, 10–11 June 2019; Easterbrook, S.; Chen, J.; Bates, O. ACM: New York, NY, USA, 2019; pp. 1-10.
5. Gharesifard, M.; Wehn, U.; van der Zaag, P. Towards benchmarking citizen observatories: Features and functioning of online amateur weather networks. J. Environ. Manag.; 2017; 193, pp. 381-393. [DOI: https://dx.doi.org/10.1016/j.jenvman.2017.02.003]
6. O’Grady, M.J.; Evans, B.; Eigbogba, S.; Muldoon, C.; Campbell, A.G.; Brewer, P.A.; O’Hare, G.M.P. Supporting participative pre-flood risk reduction in a UNESCO biosphere. J. Flood Risk Manag.; 2019; 12, e12520. [DOI: https://dx.doi.org/10.1111/jfr3.12520]
7. Kibirige, D.; Dobos, E. Soil Moisture Estimation Using Citizen Observatory Data, Microwave Satellite Imagery, and Environmental Covariates. Water; 2020; 12, 2160. [DOI: https://dx.doi.org/10.3390/w12082160]
8. Owen, R.; von Schomberg, R.; Macnaghten, P. An unfinished journey? Reflections on a decade of responsible research and innovation. J. Responsible Innov.; 2021; 8, pp. 217-233. [DOI: https://dx.doi.org/10.1080/23299460.2021.1948789]
9. Mejlgaard, N.; Bloch, C.; Madsen, E.B. Responsible research and innovation in Europe: A cross-country comparative analysis. Sci. Public Policy; 2019; 46, pp. 198-209. [DOI: https://dx.doi.org/10.1093/scipol/scy048]
10. Delgado, A.; Åm, H. Experiments in interdisciplinarity: Responsible research and innovation and the public good. PLoS Biol.; 2018; 16, e2003921. [DOI: https://dx.doi.org/10.1371/journal.pbio.2003921]
11. Lönngren, J.; van Poeck, K. Wicked problems: A mapping review of the literature. Int. J. Sustain. Dev. World Ecol.; 2021; 28, pp. 481-502. [DOI: https://dx.doi.org/10.1080/13504509.2020.1859415]
12. Reimers, F. The Social Context of Educational Evaluation in Latin America. International Handbook of Educational Evaluation; Kellaghan, T.; Stufflebeam, D.L.; Wingate, L.A. Kluwer Academic: Dordrecht, The Netherlands, London, UK, 2003; pp. 441-463. ISBN 9781402008498
13. WeObserve Consortium. Roadmap for the Uptake of the Citizen Observatories’ Knowledge Base: Report Submitted to the European Commission. Available online: https://zenodo.org/record/4646774#.YkNOjTUo9PY (accessed on 12 June 2022).
14. Hager, G.; Gold, M.; Wehn, U.; Ajates, R.; See, L.; Woods, M.; Tsiakos, C.; Maso, J.; Fraisl, D.; Moorthy, I. et al. Onto new horizons: Insights from the WeObserve project to strengthen the awareness, acceptability and sustainability of Citizen Observatories in Europe. J. Sci. Commun.; 2021; 20, A01. [DOI: https://dx.doi.org/10.22323/2.20060201]
15. Montargil, F.; Santos, V. Citizen Observatories: Concept, Opportunities and Communication with Citizens in the First EU Experiences. Beyond Bureaucracy. Public Administration and Information Technology; Paulin, A.A.; Anthopoulos, L.G.; Reddick, C.G. Springer: Cham, Switzerland, 2017; pp. 167-184. ISBN 9783319541426
16. Haklay, M.; Fraisl, D.; Greshake Tzovaras, B.; Hecker, S.; Gold, M.; Hager, G.; Ceccaroni, L.; Kieslinger, B.; Wehn, U.; Woods, S. et al. Contours of citizen science: A vignette study. R. Soc. Open Sci.; 2021; 8, 202108. [DOI: https://dx.doi.org/10.1098/rsos.202108] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34457323]
17. Spasiano, A.; Grimaldi, S.; Braccini, A.M.; Nardi, F. Towards a Transdisciplinary Theoretical Framework of Citizen Science: Results from a Meta-Review Analysis. Sustainability; 2021; 13, 7904. [DOI: https://dx.doi.org/10.3390/su13147904]
18. Rathnayake, C.; Joshi, S.; Cerratto-Pargman, T. Mapping the current landscape of citizen-driven environmental monitoring: A systematic literature review. Sustain. Sci. Pract. Policy; 2020; 16, pp. 36-334. [DOI: https://dx.doi.org/10.1080/15487733.2020.1829845]
19. Liu, H.-Y.; Kobernus, M.; Broday, D.; Bartonova, A. A conceptual approach to a citizens’ observatory—Supporting community-based environmental governance. Environ. Health; 2014; 13, 107. [DOI: https://dx.doi.org/10.1186/1476-069X-13-107]
20. Gonda, N.; Prado Córdova, J.P.; Huybrechs, F.; van Hecken, G.T. Exploring the Love Triangle of Authoritarianism, Populism, and COVID-19 Through Political Ecology: Time for a Break-Up?. Front. Hum. Dyn.; 2022; 4, 653990. [DOI: https://dx.doi.org/10.3389/fhumd.2022.653990]
21. Comber, A.; Mooney, P.; Purves, R.S.; Rocchini, D.; Walz, A. Crowdsourcing: It Matters Who the Crowd Are. The Impacts of between Group Variations in Recording Land Cover. PLoS ONE; 2016; 11, e0158329. [DOI: https://dx.doi.org/10.1371/journal.pone.0158329]
22. Bedessem, B.; Ruphy, S. Citizen Science and Scientific Objectivity: Mapping Out Epistemic Risks and Benefits. Perspect. Sci.; 2020; 28, pp. 630-654. [DOI: https://dx.doi.org/10.1162/posc_a_00353]
23. Porto de Albuquerque, J.; Anderson, L.; Calvillo, N.; Coaffee, J.; Cunha, M.A.; Degrossi, L.C.; Dolif, G.; Horita, F.; Klonner, C.; Lima-Silva, F. et al. The role of data in transformations to sustainability: A critical research agenda. Curr. Opin. Environ. Sustain.; 2021; 49, pp. 153-163. [DOI: https://dx.doi.org/10.1016/j.cosust.2021.06.009]
24. Elliott, K.C.; Rosenberg, J. Philosophical Foundations for Citizen Science. Citiz. Sci. Theory Pract.; 2019; 4, 9. [DOI: https://dx.doi.org/10.5334/cstp.155]
25. Lynn, S.J.; Kaplan, N.; Newman, S.; Scarpino, R.; Newman, G. Designing a Platform for Ethical Citizen Science: A Case Study of CitSci.org. Citiz. Sci. Theory Pract.; 2019; 4, 14. [DOI: https://dx.doi.org/10.5334/cstp.227]
26. Berti Suman, A.; Pierce, R. Challenges for Citizen Science and the EU Open Science Agenda under the GDPR. Eur. Data Prot. Law Rev.; 2018; 4, pp. 284-295. [DOI: https://dx.doi.org/10.21552/edpl/2018/3/7]
27. Majumder, M.A.; McGuire, A.L. Data Sharing in the Context of Health-Related Citizen Science. J. Law Med. Ethics; 2020; 48, pp. 167-177. [DOI: https://dx.doi.org/10.1177/1073110520917044] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32342743]
28. De Sherbinin, A.; Bowser, A.; Chuang, T.-R.; Cooper, C.; Danielsen, F.; Edmunds, R.; Elias, P.; Faustman, E.; Hultquist, C.; Mondardini, R. et al. The Critical Importance of Citizen Science Data. Front. Clim.; 2021; 3, 650760. [DOI: https://dx.doi.org/10.3389/fclim.2021.650760]
29. Spaini, C.; Peter, V.; Bloch, C.; Mejlgaard, N.; Costa, R.; Meijer, I.; Woolley, R.; Maier, F. Monitoring the Evolution and Benefits of Responsible Research and Innovation: The Evolution of Responsible Research and Innovation—The Indicators Report; European Commission—Directorate-General for Research and Innovation: Bruxelles, Belgium, 2018; ISBN 9789279928505
30. European Commission—Directorate General for Research and Innovation. Responsible Research and Innovation: Europe’s Ability to Respond to Societal Challenges; EU Publications Office: Luxembourg, 2014; ISBN 9789279382390
31. Gerber, A.; Forsberg, E.-M.; Shelley-Egan, C.; Arias, R.; Daimer, S.; Dalton, G.; Cristóbal, A.B.; Dreyer, M.; Griessler, E.; Lindner, R. et al. Joint declaration on mainstreaming RRI across Horizon Europe. J. Responsible Innov.; 2020; 7, pp. 708-711. [DOI: https://dx.doi.org/10.1080/23299460.2020.1764837]
32. Meijer, I.; van de Klippe, W. Monitoring Responsible Research and Innovation in the European research area: The MoRRI project. Assessment of Responsible Innovation; Routledge: Oxfordshire, UK, 2020; pp. 171-195.
33. Von Schomberg, R. A Vision of Responsible Research and Innovation. Responsible Innovation; Owen, R.; Bessant, J.; Heintz, M. John Wiley & Sons, Ltd.: Chichester, UK, 2013; pp. 51-74. ISBN 9781118551424
34. Wakunuma, K.; de Castro, F.; Jiya, T.; Inigo, E.A.; Blok, V.; Bryce, V. Reconceptualising responsible research and innovation from a Global South perspective. J. Responsible Innov.; 2021; 8, pp. 267-291. [DOI: https://dx.doi.org/10.1080/23299460.2021.1944736]
35. Szymanski, E.A.; Smith, R.D.J.; Calvert, J. Responsible research and innovation meets multispecies studies: Why RRI needs to be a more-than-human exercise. J. Responsible Innov.; 2021; 8, pp. 261-266. [DOI: https://dx.doi.org/10.1080/23299460.2021.1906040]
36. Burget, M.; Bardone, E.; Pedaste, M. Definitions and Conceptual Dimensions of Responsible Research and Innovation: A Literature Review. Sci. Eng. Ethics; 2017; 23, pp. 1-19. [DOI: https://dx.doi.org/10.1007/s11948-016-9782-1]
37. Paredes-Frigolett, H. Modeling the effect of responsible research and innovation in quadruple helix innovation systems. Technol. Forecast. Soc. Chang.; 2016; 110, pp. 126-133. [DOI: https://dx.doi.org/10.1016/j.techfore.2015.11.001]
38. Rip, A. The clothes of the emperor. An essay on RRI in and around Brussels. J. Responsible Innov.; 2016; 3, pp. 290-304. [DOI: https://dx.doi.org/10.1080/23299460.2016.1255701]
39. Schuijff, M.; Dijkstra, A.M. Practices of Responsible Research and Innovation: A Review. Sci. Eng. Ethics; 2020; 26, pp. 533-574. [DOI: https://dx.doi.org/10.1007/s11948-019-00167-3] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31845176]
40. Munn, Z.; Peters, M.D.J.; Stern, C.; Tufanaru, C.; McArthur, A.; Aromataris, E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med. Res. Methodol.; 2018; 18, 143. [DOI: https://dx.doi.org/10.1186/s12874-018-0611-x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30453902]
41. Lim, W.M.; Kumar, S.; Ali, F. Advancing knowledge through literature reviews: ‘what’, ‘why’, and ‘how to contribute’. Serv. Ind. J.; 2022; 42, pp. 481-513. [DOI: https://dx.doi.org/10.1080/02642069.2022.2047941]
42. Sarkis-Onofre, R.; Catalá-López, F.; Aromataris, E.; Lockwood, C. How to properly use the PRISMA Statement. Syst. Rev.; 2021; 10, 117. [DOI: https://dx.doi.org/10.1186/s13643-021-01671-z] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33875004]
43. Tricco, A.C.; Lillie, E.; Zarin, W.; O’Brien, K.K.; Colquhoun, H.; Levac, D.; Moher, D.; Peters, M.D.J.; Horsley, T.; Weeks, L. et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann. Intern. Med.; 2018; 169, pp. 467-473. [DOI: https://dx.doi.org/10.7326/M18-0850]
44. Pranckutė, R. Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World. Publications; 2021; 9, 12. [DOI: https://dx.doi.org/10.3390/publications9010012]
45. Lasda Bergman, E.M. Finding Citations to Social Work Literature: The Relative Benefits of Using Web of Science, Scopus, or Google Scholar. J. Acad. Librariansh.; 2012; 38, pp. 370-379. [DOI: https://dx.doi.org/10.1016/j.acalib.2012.08.002]
46. Allen, C.; Smith, M.; Rabiee, M.; Dahmm, H. A review of scientific advancements in datasets derived from big data for monitoring the Sustainable Development Goals. Sustain. Sci.; 2021; 16, pp. 1701-1716. [DOI: https://dx.doi.org/10.1007/s11625-021-00982-3]
47. Tian, X.; ten Veldhuis, M.-C.; Schleiss, M.; Bouwens, C.; van de Giesen, N. Critical rainfall thresholds for urban pluvial flooding inferred from citizen observations. Sci. Total Environ.; 2019; 689, pp. 258-268. [DOI: https://dx.doi.org/10.1016/j.scitotenv.2019.06.355]
48. Gharesifard, M.; Wehn, U. To share or not to share: Drivers and barriers for sharing data via online amateur weather networks. J. Hydrol.; 2016; 535, pp. 181-190. [DOI: https://dx.doi.org/10.1016/j.jhydrol.2016.01.036]
49. Nipen, T.N.; Seierstad, I.A.; Lussana, C.; Kristiansen, J.; Hov, Ø. Adopting Citizen Observations in Operational Weather Prediction. Bull. Am. Meteorol. Soc.; 2020; 101, pp. E43-E57. [DOI: https://dx.doi.org/10.1175/BAMS-D-18-0237.1]
50. Ceccaroni, L.; Piera, J.; Wernand, M.R.; Zielinski, O.; Busch, J.A.; van der Woerd, H.J.; Bardaji, R.; Friedrichs, A.; Novoa, S.; Thijsse, P. et al. Citclops: A next-generation sensor system for the monitoring of natural waters and a citizens’ observatory for the assessment of ecosystems’ status. PLoS ONE; 2020; 15, e0230084. [DOI: https://dx.doi.org/10.1371/journal.pone.0230084] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32214341]
51. Bardaji, R.; Sánchez, A.-M.; Simon, C.; Wernand, M.R.; Piera, J. Estimating the Underwater Diffuse Attenuation Coefficient with a Low-Cost Instrument: The KdUINO DIY Buoy. Sensors; 2016; 16, 373. [DOI: https://dx.doi.org/10.3390/s16030373]
52. Friedrichs, A.; Anke Busch, J.; van der Woerd, H.J.; Zielinski, O. SmartFluo: A Method and Affordable Adapter to Measure Chlorophyll a Fluorescence with Smartphones. Sensors; 2017; 17, 678. [DOI: https://dx.doi.org/10.3390/s17040678] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28346338]
53. Drack, M.; Hartmann, F.; Bauer, S.; Kaltenbrunner, M. The importance of open and frugal labware. Nat. Electron.; 2018; 1, pp. 484-486. [DOI: https://dx.doi.org/10.1038/s41928-018-0133-x]
54. Kobernus, M.; Berre, A.J.; Gonzalez, M.; Liu, H.-Y.; Fredriksen, M.; Rombouts, R.; Bartanova, A. A practical approach to an integrated citizens’ observatory: The CITI-SENSE framework. Proceedings of the Workshop ‘Environmental Information Systems and Services—Infrastructures and Platforms 2013′ (ENVIP 2013), Neusiedl am See, Austria, 10 October 2013; Norwegian Institute for Air Research: Kjeller, Norway, 2015.
55. Castell, N.; Kobernus, M.; Liu, H.-Y.; Schneider, P.; Lahoz, W.; Berre, A.J.; Noll, J. Mobile technologies and services for environmental monitoring: The Citi-Sense-MOB approach. Urban Clim.; 2015; 14, pp. 370-382. [DOI: https://dx.doi.org/10.1016/j.uclim.2014.08.002]
56. Fritz, S.; Fonte, C.C.; See, L. The Role of Citizen Science in Earth Observation. Remote Sens.; 2017; 9, 357. [DOI: https://dx.doi.org/10.3390/rs9040357]
57. Wehn, U.; Evers, J. The social innovation potential of ICT-enabled citizen observatories to increase eParticipation in local flood risk management. Technol. Soc.; 2015; 42, pp. 187-198. [DOI: https://dx.doi.org/10.1016/j.techsoc.2015.05.002]
58. Wehn, U.; Rusca, M.; Evers, J.; Lanfranchi, V. Participation in flood risk management and the potential of citizen observatories: A governance analysis. Environ. Sci. Policy; 2015; 48, pp. 224-236. [DOI: https://dx.doi.org/10.1016/j.envsci.2014.12.017]
59. Degrossi, L.C.; Abe, B.B.; Albuquerque, J.P.; de Mattos Fortes, R.P. Enhancing usability of a citizen observatory based on user-centered design. Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion; Thessaloniki, Greece, 20–22 June 2018; ACM International Conference Proceeding Series: New York, NY, USA, 2018; [DOI: https://dx.doi.org/10.1145/3218585.3218674]
60. Kovács, K.Z.; Hemment, D.; Woods, M.; van der VELDEN, N.K.; Xaver, A.; Gi Esen, R.H.; Burton, V.J.; Garrett, N.L.; Zappa, L.; Long, D. et al. Citizen observatory based soil moisture monitoring—The GROW example. Hung. Geogr. Bull.; 2019; 68, pp. 119-139. [DOI: https://dx.doi.org/10.15201/hungeobull.68.2.2]
61. De Virgilio, M.; Cifarelli, S.; de Gennaro, P.; Garofoli, G.; Degryse, B. A first attempt of citizen science in the genetic monitoring of a Posidonia oceanica meadow in the Italian Southern Adriatic Sea. J. Nat. Conserv.; 2020; 5, [DOI: https://dx.doi.org/10.1016/j.jnc.2020.125826]
62. Wannemacher, K.; Birli, B.; Sturn, T.; Stiles, R.; Moorthy, I.; See, L.; Fritz, S. Using citizen science to help monitor urban landscape changes and drive improvements. J. Geogr. Inf. Sci.-GI_Forum; 2018; 1, pp. 336-343. [DOI: https://dx.doi.org/10.1553/giscience2018_01_s336]
63. Downs, R.R.; Ramapriyan, H.K.; Peng, G.; Wei, Y. Perspectives on Citizen Science Data Quality. Front. Clim.; 2021; 3, 615032. [DOI: https://dx.doi.org/10.3389/fclim.2021.615032]
64. Mominó, J.M.; Piera, J.; Jurado, E. Citizen observatories as advanced learning environments. Analysing the Role of Citizen Science in Modern Research; IGI Global: Pennsylvania, PA, USA, 2017; pp. 192-212. [DOI: https://dx.doi.org/10.4018/978-1-5225-0962-2.ch009]
65. Coffey, Y.; Bhullar, N.; Durkin, J.; Islam, M.S.; Usher, K. Understanding Eco-anxiety: A Systematic Scoping Review of Current Literature and Identified Knowledge Gaps. J. Clim. Chang. Health; 2021; 3, 100047. [DOI: https://dx.doi.org/10.1016/j.joclim.2021.100047]
66. Degrossi, L.C.; de Albuquerque, J.P.; Fava, M.C.; Mendiondo, E.M. Flood citizen observatory: A crowdsourcing-based approach for flood risk management in Brazil. Proceedings of the International Conference on Software Engineering and Knowledge Engineering, SEKE; Vancouver, BC, Canada, 1–3 July 2014.
67. Chiaravalloti, R.M.; Skarlatidou, A.; Hoyte, S.; Badia, M.M.; Haklay, M.; Lewis, J. Extreme citizen science: Lessons learned from initiatives around the globe. Conserv. Sci. Pract.; 2022; 4, e577. [DOI: https://dx.doi.org/10.1111/csp2.577]
68. Tannenbaum, C.; Ellis, R.P.; Eyssel, F.; Zou, J.; Schiebinger, L. Sex and gender analysis improves science and engineering. Nature; 2019; 575, pp. 137-146. [DOI: https://dx.doi.org/10.1038/s41586-019-1657-6]
69. Gharesifard, M.; Wehn, U. Participation in citizen science: Drivers and barriers for sharing personally-collected weather data via web-platforms. Proceedings of the International Conferences on ICT, Society and Human Beings 2015, ICT 2015, Web Based Communities and Social Media 2015, WBC 2015 and Connected Smart Cities 2015, CSC 2015—Part of the Multi Conference on Computer Science and Information Systems 2015; Las Palmas de Gran Canaria, Spain, 21–23 July 2015.
70. Assumpcao, T.H.; Jonoski, A.; Theona, I.; Tsiakos, C.; Krommyda, M.; Tamascelli, S.; Kallioras, A.; Mierla, M.; Georgiou, H.V.; Miska, M. et al. Citizens’ Campaigns for Environmental Water Monitoring: Lessons from Field Experiments. IEEE Access; 2019; 7, pp. 134601-134620. [DOI: https://dx.doi.org/10.1109/ACCESS.2019.2939471]
71. Novitzky, P.; Bernstein, M.J.; Blok, V.; Braun, R.; Chan, T.T.; Lamers, W.; Loeber, A.; Meijer, I.; Lindner, R.; Griessler, E. Improve alignment of research policy and societal values. Science; 2020; 369, pp. 39-41. [DOI: https://dx.doi.org/10.1126/science.abb3415]
72. Lowry, C.S.; Stepenuck, K.F. Is Citizen Science Dead?. Environ. Sci. Technol.; 2021; 55, pp. 4194-4196. [DOI: https://dx.doi.org/10.1021/acs.est.0c07873]
73. Håkanson, L. The firm as an epistemic community: The knowledge-based view revisited. Ind. Corp. Chang.; 2010; 19, pp. 1801-1828. [DOI: https://dx.doi.org/10.1093/icc/dtq052]
74. Ong, E.C.; Tan, C.L. Soft TQM, agility, and knowledge management deliver organisational performance: A study of Malaysian manufacturing organisations in the electrical and electronics sector. Glob. Bus. Organ. Excell.; 2022; 41, pp. 28-47. [DOI: https://dx.doi.org/10.1002/joe.22155]
75. Kimble, C. Successful knowledge management in high-sociability organisations. GBOE; 2020; 39, pp. 38-44. [DOI: https://dx.doi.org/10.1002/joe.21993]
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Mangina, Eleni