Citizen Science for Nature Conservation in Hungary A Three-Dimensional Approach

Abstract

Nature conservation–related citizen science (NCCS) has grown rapidly worldwide in previous years. In Hungary, a few citizen science (CS) projects have been operating for years and some have only recently launched. Our aim herein is to assess the performance of eight Hungarian NCCS projects in three dimensions: a) science, b) nature conservation, and c) participants’ development. An evaluation framework was developed for the assessment. Our results show that the Common Bird Monitoring Program performed the best overall. This is also the oldest NCCS project in the country. When comparing the performance per dimension, the majority of the projects tended to have good performances in the science dimension. Most of the projects ensure data quality using different strategies. However, the need for open data and processing the project results for generating scientific publications still needs to be tackled by some NCCS initiatives. Regarding the nature conservation dimension, data generated have been mostly used in monitoring species/ecosystems, whereas data is less commonly used for conservation policy-making. It was identified that the participants’ development dimension has not received sufficient attention, and neither learning outcomes nor behavioral and attitude change has been evaluated by any projects. Our results of evaluating Hungarian NCCS initiatives in a complex way may offer insights for project managers and coordinators to identify which dimension are performing well and which areas need improvement. Also, our framework serves as a model that can be applied to current and future NCCS initiatives.

Introduction

Citizen Science (CS) is an emerging area of research and practice where volunteers collaborate with professional scientists by taking part in the research process while enhancing their scientific literacy and generating new knowledge (Turrini et al. 2018; Bela et al. 2016; Bonney 2021). CS projects are becoming an important component of biodiversity and nature conservation research in multiple ways (Peter et al. 2021; Suškevičs et al. 2021). International nature conservation–related citizen science (NCCS) projects such as iNaturalist (Mesaglio and Callaghan 2021) and eBird (Sullivan et al. 2017) are extending the evidence-based needed for policy-making by enriching global biodiversity databases (Chandler et al. 2017; Bonney 2021). One such example is the Global Biodiversity Information Facility (GBIF), which compiles all observations about species occurrences worldwide; at least 50% of their inputs come from NCCS initiatives (Chandler et al. 2017; Miller 2022). The long temporal scope, broad taxonomic coverage, and wide geographic range of the NCCS projects’ datasets are assisting in filling the large-scale data gaps that scientists have identified in the literature (Danielsen et al. 2014; Pocock et al. 2015). Volunteers who engage in NCCS projects generally participate in tracking, monitoring, or documenting biological data (Theobald et al. 2015). The new knowledge generated commonly relates to long-term datasets on phenology, species occurrence, distribution, or abundance. The new data also helps to inform policy and decision-makers as well as to achieve biodiversity conservation goals (Theobald et al. 2015; Chandler et al. 2017; McKinley et al. 2017). Participation in NCCS projects has shown corollary effects on the development of volunteers, which are mainly linked to an increase in scientific and conservation knowledge, raising nature conservation awareness, promoting environmental education, and influencing attitude and behavioral change (Jordan et al. 2016; Bela et al. 2016; Bonney 2021; Santori et al. 2021).

In Europe, CS has predominantly contributed to the life sciences (Hecker, Garbe, and Bonn 2018). However, while NCCSs are well known in Western Europe, they are only now gaining recognition in Central and Eastern Europe (Vohland et al. 2021). In 2018, the European Commission released an inventory of 503 NCCS projects from the European Union that provided policy support, contributed to science, and encouraged meaningful citizen engagement (Bio Innovation Service 2018). Central and Eastern European countries were underrepresented in this document. CS research in Hungary is still scarce (Vohland et al. 2021) despite the fact that several NCCS projects are in operation. The Common Bird Monitoring Program of the Hungarian Ornithological and Nature Conservation Society, also known as BirdLife Hungary (MME), has been active since the late 1990s (Szép and Gibbons 2000), and the WildWatcher (Vadonleső) has been operating for more than a decade (Vadonleső Group et al. 2018, 2019). Since the 2020s, CS initiatives have been supported by the National Research, Development and Innovation Office’s policies, growing expert networks, and funding schemes (Czeglédi 2022).

Evaluation of CS projects is highlighted in one of the principles (principle 9) of The European Citizen Science Association (ECSA) as a core requirement (Robinson et al. 2018). Previous studies have adopted frameworks to evaluate CS projects along specific dimensions including their scientific impacts (D’Andrea et al. 2022), learning-education outcomes (Phillips et al. 2018), the role of CS on conservation (Blaney et al. 2016; Yang et al. 2019), and the quality of participation (Shirk et al. 2012). Several dimensions (social, scientific, and socio-ecological/-economic) are integrated into the open framework of Kieslinger et al. (2018), which can be used for a variety of CS projects from different objectives and fields.

Recommendations for CS evaluation in the literature encourage CS frameworks to be developed for specific fields (Davis et al. 2022).

Most of the evaluation frameworks that specifically focus on NCCS projects usually cover only one dimension such as data quality (Kosmala et al. 2016), learning outcomes (Phillips et al. 2018), ecological literacy (Jordan et al. 2016), pro-environmental behavior (Pocock et al. 2023), or the role of CS in conservation (Yang et al. 2019). Only a few frameworks focus on NCSS projects from a multidimensional perspective. Turbé et al. (2019) assessed European NCCS projects that already have policy relevance by considering citizen scientist, science, socio-economic, and policy dimensions. Price-Jones et al. (2022) evaluated 103 NCCS projects specifically targeting invasive species in Europe. They assessed the projects considering different variables such as data quality, data management, and participants’ engagement.

However, frameworks that would enable the comparison of NCCS projects in more dimensions are still needed (Bonney 2021; Somerwill and Wehn 2022).

Therefore, based on the CS literature, we propose an evaluation framework for NCCS projects in three dimensions: science, nature conservation, and participants’ development. We then apply our framework to assess the performance of eight Hungarian NCCS projects.

Research and Design Methods

Conceptual framework

The evaluation framework shown in Figure 1 was developed to assess NCCS projects, incorporating relevant CS literature that covers different evaluation frameworks and recommendations for the advancement of CS projects (see references in Table 1). We address three dimensions that seem very relevant to NCCS projects: (science, nature conservation, and participants’ development). The science dimension was chosen because there is a discussion in the CS literature about the scientific value of CS outputs (Balázs et al. 2021). While our evaluation framework targets NCCS projects, the primary goal of a nature conservation–related CS project is to contribute to the conservation of natural resources, ecosystems, or species (McKinley et al. 2017; Suškevičs et al. 2021). That is why the nature conservation dimension was added. CS projects are based on the work of volunteers, and their participation might have added value to their personal development, which is starting to be noticed in some studies (Turrini et al. 2018; Phillips et al. 2018; Santori et al. 2021). This is the reason for the choice of the participants’ development dimension. Table 1, which shows the criteria used for assessing NCCS projects, is based on the evaluation framework shown in Figure 1.

[Table Omitted: See PDF]

Figure 1 The proposed evaluation framework (dimensions and components).

We compiled a comprehensive list of criteria that were suggested in the literature for each dimension. No criteria were disregarded, yet several were merged. The questions for each criterion of assessment were designed by the authors, ensuring that they are clear, concise, and effective in gathering information that can be answered through yes/no responses.

Selection of nature conservation–related citizen science projects for the assessment

Eight Hungarian NCCS projects were chosen for analysis in a three-step process from November 2021 to January 2022: 1) We conducted a desk study to explore past and ongoing NCCS projects, using information from official web pages of CS associations and European Union (EU) CS platform repositories (e.g., the EU-funded online platform eu-citizen.science co-developed and run by ECSA). Also, we explored the websites of universities and scientific institutes from Hungary; some of these sites guided us to the NCCS project websites. We built an initial list of fifteen potential projects. 2) To include NCCS projects in our final list, we applied the following selection criteria: a) NCCS projects started in Hungary, b) the aim of the project was linked to nature conservation or biodiversity, and c) they had flora or fauna target species. 3) We sent out online invitations to fifteen projects. We received positive responses from nine projects who accepted the invitation to the interviews. We excluded the project “Looking for Cowslips” because it originated in Estonia. We interviewed coordinators from the selected eight NCCS projects, who all became co-authors of this paper.

Data gathering

Interviews

From April to July 2022, six online and two in-person semi-structured interviews (Newing et al. 2011) were conducted with NCCS project coordinators. We aimed to get an in-depth understanding of experts’ experiences running NCCS projects in Hungary. The questions were designed based on our conceptual framework. The interviews were structured in five blocks, 1) project description, 2) science dimension, 3) nature conservation dimension, 4) participants’ development, and 5) outcomes and challenges (see the interview questions in Supplemental File 1). The in-person interviews were recorded using a digital voice recorder (Olympus WS-832), while online interviews were made via the Skype platform with recording. During the interviews, notes were taken, and by utilizing MAXQDA software (VERBI 2021), a literal transcription of every interview was compiled. The interviews lasted an average of 80 minutes; the shortest interview was 60 minutes, and the longest was 120 minutes, which was done in a two-day session because of the time availability of the interviewee.

Data analysis

Following our conceptual framework, the transcripts of the interviews were analyzed with qualitative content analysis, and based on that, the eight Hungarian NCCS projects were evaluated. First, a priori codes were created for the main characteristics of the projects (aim, institution affiliation, target species, periodicity, geographical scope, the task of the participants, number of participants, and number of observations until April 2022. Codes were developed based on the three dimensions and their components (Figure 1), then they were applied to the transcripts of the interviews using the MAXQDA software (VERBI 2021).

A summary table was then created by going through the coded text and based on the main characteristics of the projects (see Supplemental File 2: Table S1). By answering the questions in Table 1 related to the performance criteria, scores were assigned (see Supplemental File 3). After this process, we summed up the scores for each criterion within and across the studied NCCS projects, and we compared the eight projects using the percentage of maximum scores. We did not carry out a more exhaustive quantitative analysis. The scoring was reviewed and accepted by interviewees who are all co-authors of this study. To explain our findings, we employed qualitative data from the content analysis of the interviews. We base our analysis on the interviews showing the state of the NCCS projects in 2022. Yet, there have been changes and further developments in some projects since the interviews were completed, which are reflected in the Discussion.

Results

Overall performance of NCCS projects

The total achieved points of the eight projects are presented in Supplemental File 3: Table S2. The maximum score that could be reached was 22 points. Based on this overview, the Common Bird Monitoring Program scored the best by reaching a total of 17 points, whereas the Arthropods NCCS project had the lowest total performance score of 11 points (Figure 2). The next section contains further information about performance according to each dimension.

Figure 2 Dimension performance of each nature conservation–related citizen science (NCCS) project in 2022. * BeaverMap and MyPond started in 2021.

Performance of the nature conservation–related citizen science projects in each dimension

Figure 2 shows the percentages of scores of NCCS projects achieved per dimension. In the science dimension, all the projects reached 60% or more of the possible maximum scores. In this first dimension, the Common Bird Monitoring Program achieved 90% of the possible maximum scores followed by the Mosquito Monitor and MyPond (82% each). The Common Bird Monitoring Program had the best performance in the nature conservation dimension, along with the Amphibian and Reptile Mapping and WildWatcher, scoring 100% each. The two projects, Arthropods and Butterfly-Net, followed, each demonstrating a 67% performance in the nature conservation dimension. BeaverMap and MyPond received lower scores in nature conservation because they were fairly new projects (began in 2021) at the time of the interview (began in 2022); the same was true for Mosquito Monitor (began in 2019). However, their coordinators intend to extend the use of the project data for nature conservation in the future. The Butterfly-Net effectively represented the participants’ development dimension, achieving 88% of the possible maximum score, while the rest of the seven projects reached less than 50%.

Science dimension and its components

The methodology component includes the research question, methodology for data collection, and technology use. The Common Bird Monitoring Program, Mosquito Monitor, BeaverMap, and MyPond had a scientific question developed prior to project commencement. The methodology for data collection varies among projects. The Common Bird Monitoring Program uses a randomized sampling design, a methodology adopted from the well-recognized Breeding Bird Survey from the UK (Szép and Gibbons 2000). When recording an observation, not all the projects request the mandatory upload of supporting information (e.g., WildWatcher does not request uploaded pictures, whereas in the Amphibian and Reptile Mapping project, uploading pictures is optional, but supporting GPS coordinates and filling out a short questionnaire are mandatory). Every project uses technology to support data collection, including smartphones with specialized applications, GIS map visualization, and websites—with the exception of Butterfly-Net. The data quality component includes data validation and institutional affiliation. The Butterfly-Net and the Common Bird Monitoring Program validated the protocol process for data collection by testing it before the official start of the project. Experts are in charge of data validation in every project, but how they are involved differs. While experts in charge of data validation are employed by the majority of the projects, their participation in the Arthropods NCCS project is entirely voluntary. In the Arthropods project, after applying for the volunteer expert position, they are granted special access to the project platform to review the records. The Arthropods project manager highlighted the lack of taxonomists in the country willing to sign up for the task. Data validation forms that involved requesting more information from participants and site visits were rare. The majority of the NCCS initiatives were affiliated with nongovernmental (NGOs), conservation, or scientific organizations in 2022, except for the Arthropods project, which was launched through private efforts and its coordinator had no intention to pursue an affiliation at that time.

The last component of the science dimension is data management practices, which includes long-term datasets, open data, and publications. All the projects maintain methods of suitable data management in terms of storage and releasing data. Only Arthropods offers open access raw data, but all have a willingness to share datasets on request. Having a platform (e.g., an online website, an app) resulted in the primary strategy to release data through summaries of validated data (e.g., distribution of maps and graphs). For example, on the WildWatcher and Mosquito Monitor websites, observation points can be updated, visualized, and filtered per year or per species using maps. Arthropods as well as Amphibian and Reptile Mapping offer the same, with the extra opportunity to view the recorded points with the added photo provided by the participants. BeaverMap shows the observations of the volunteers on a map that is periodically updated. Project coordinators emphasized that all the data are managed carefully, especially the geolocation of endangered or strictly protected species (Amphibian and Reptile Mapping, BeaverMap, WildWatcher, and Arthropods), as well as personal information of the participants such as email addresses (MyPond, Amphibian and Reptile Mapping, BeaverMap). In the case of BeaverMap, coordinators emphasized that they will not share information about beaver dams until the protection of valuable beaver-made wetlands is guaranteed by regulations.

Writing scientific publications was a common strategy to share data by older projects that have released scientific papers under their project affiliations (e.g., the Common Bird Monitoring Program (Szép and Gibbons 2000; Szép et al. 2012; Szép et al. 2021), the WildWatcher (Vadonleső Group et al. 2018, 2019; Vadonleső Group et al. 2019)). All the other projects were planning to prepare scientific publications, except for the Butterfly-Net due to their short-term running period. Another data management practice was the use of data by other researchers not involved in the programs. Arthropods’ data have been used most widely in scientific papers of other scholars (e.g., Károlyi and Rédei 2017; Merkl, Károlyi, and Korányi 2017; Vétek et al. 2018; Kóbor et al. 2021).

Nature conservation dimension and its components

The environmental monitoring component includes data used in species/ecosystem monitoring. All analyzed projects generated data that was helpful for environmental monitoring, with all of them targeting species, and MyPond specifically focusing on ecosystems. The Arthropods project’s open data led to the identification of new species occurrences in the country e.g., A. heegeri (Károlyi and Rédei 2017), Cybocephalus nipponicus (Merkl, Károlyi, and Korányi 2017) and invasive species such as Acanalonia conica (Kóbor et al. 2021) or Halyomorpha halys (Vétek et al. 2018).

The environmental management component includes data used in species/ecosystem management. In the Mosquito Monitor project, participants catch specimens of invasive mosquitoes, helping in the eradication of these species that are not only causing conservation-related problems but also spread viruses and pathogens that pose a threat to human health. In WildWatcher, the Hedgehog (Erinaceus roumanicus) is the most frequently observed species. Reports from participants about this species’ occurrence outside their distribution range (where food is limited) helped to initiate action for habitat management. Data from Amphibian and Reptile Mapping served to protect the habitat of the fire salamander (Salamandra salamandra) in a district of Budapest. During the operation of Butterfly-Net, observations by students helped to discover a new population of butterflies, which was followed by action for habitat protection by the Őrség National Park Directorate.

The conservation policy component contains the data used in conservation policy; thus older projects perform well in this component. For example, WildWatcher data have been included in the Hungarian Nature Conservation Information System (TIR), and the national report of Article 17 of the EU Habitat Directive. Butterfly-Net data was included in the biodiversity platform called OpenBioMaps (Bán et al. 2022), which is available for decision-makers to support biodiversity and management. Amphibian and Reptile Mapping supplies the European Atlas of Amphibians and Reptiles with data atlases (SEH) (NA2RE) (Sillero et al. 2014), and the project provides regular reporting according to Article 17 of the EU Habitat Directive. The Common Bird Monitoring Program, with the largest ornithological dataset in Hungary, has been providing data for common EU bird monitoring and the annual bird atlas for Hungary for several years (Szép et al. 2021). All these contributions increase the possibility of impacting local, national, regional, or EU policy-making.

Participants’ development and its components

The training component includes the forms of training provided. The Butterfly-Net was a unique initiative that combined personal training (lectures on butterfly biology, and species identification) with other forms of knowledge transfer (e.g., species identification books, phone applications, environmental education events). The other projects utilize non-personal training strategies through their websites, where participants can find descriptive information about instructions for field observation, data collection protocols, and species. The Common Bird Monitoring Program offers extra materials such as a bird-sound guide, maps, and field diaries. MyPond trained people through videos and sampling kits (informative guides with conservation and scientific content, sampling tools for water chemistry and eDNA, data sheets, and user-friendly manuals). Mosquito Monitor also uses videos to show how to trap mosquito specimens.

The additional learning-support component focuses on educational events and communication for knowledge gain. Some projects organize events (e.g., The Mammal of the Year [WildWatcher] or The Amphibian and Reptile of the Year [Amphibian and Reptile Mapping]) that serve to promote environmental education and nature conservation awareness. Social media is used to promote these events. The Common Bird Monitoring Program organizes recognition events where observers gather to receive awards. Participants have established a community of common interests in which they can learn from each other’s experiences. In many projects, experts are freely available for the participants if they have questions or concerns. Volunteers, mainly farmers, commonly contact experts of BeaverMap to find common solutions for beaver-related conflicts. Participants of Amphibian and Reptile Mapping and WildWatcher use social media (Facebook) to communicate with experts and with each other. By exchanging photos and seeking advice on species identification, participants engage in discussions that help to reinforce their knowledge about their observations and ensure species recognition before data recording on the website.

The last component is the measurement strategies for participants’ personal development that involve the measurement of knowledge and skills gained/reinforced, measurement of attitude/behavioral change, and strategies for reinforcing and increasing knowledge through feedback.

Although all eight projects have generated new knowledge that improves the understanding of species in the territory of Hungary, none of them took action to measure the knowledge/skills gained by participants after their contribution. None of the projects measured the attitudes or behavioral changes of participants either. In the specific case of MyPond, coordinators emphasized that even though no behavior/attitude–change survey was used, participants’ willingness to move on to the second phase of MyPond (kitbox distributed for measurements) already demonstrates a good attitude towards involvement. Six projects (the Common Bird Monitoring Program, Amphibian and Reptile Mapping, Butterfly-Net, Arthropods, Mosquito Monitor, and MyPond) provide some form of feedback that helps in reinforcing or increasing the knowledge of the participants (e.g., feedback via email, sending email via post or app). Although project coordinators mentioned that giving feedback encourages participants to continue contributing, no formal measurements have been made to know the impact of it on NCCS projects or volunteers.

Discussion

The evaluation framework

Literature on CS calls for new evaluation frameworks that are beyond the traditional impacts of CS, focus on specific fields, have standardized approaches, and are easy to use (Bonney et al. 2014; Schaefer et al. 2021). We designed our framework for NCCS projects, which includes three dimensions with components and a set of criteria. A scoring system has been developed based on yes and no questions related to each criterion. Other multidimensional frameworks are either not specifically designed for NCCS projects and therefore the nature conservation dimension is missing (e.g., Kieslinger et al. 2018), or in the case of NCCS evaluation frameworks, they focus on partly different dimensions and components (e.g., Turbé et al. 2019; Price-Jones et al. 2022). Kieslinger et al. (2018) proposed a set of supporting questions for the evaluation, but the questions were partly open-ended and did not lead to a scoring system. Turbé et al. (2019) conducted a survey among NCCS projects with policy relevance with closed and open-ended questions. Their evaluation was based on statistical analysis; they did not aim to develop a standardized scoring system either.

Science dimension

It has been demonstrated that NCCS efforts can serve as an essential data source for research when data quality standards are maintained and ensured (Wiggins et al. 2013). For example, eBird, an international CS project with a rigorous data collection and validation protocol, has published at least 90 peer-reviewed scientific studies (Bonney et al. 2014; Kelling et al. 2015). In our study, the Common Bird Monitoring Program in Hungary, which scored the highest in the science dimension, adopted the data collection protocol of the Breeding Bird Survey from the UK. It has consistently maintained high scientific standards for methodology, data quality, and validation, creating one of Hungary’s longest-running (23 years), highest-quality ornithological datasets.

A handbook for CS projects released by the United States Environmental Protection Agency (EPA 2019) emphasizes that data produced by projects in collaboration with professional organizations increases the chance to meet data quality standards and extend scientific outputs. In our study, we found that all projects, with the exception of Arthropods, were linked to a scientific or a conservation institution at the time of the interviews. Now Arthropods is also connected to an NGO (Közösen a Természetért Alapítvány – Together for Nature Foundation), showing that affiliation is becoming important for stable operation.

Turbé et al. (2019) found that projects with open access data typically generate more scientific outputs than those without open data. In our study, only Arthropods had an open access and downloadable data policy, which has led to a long-term dataset of more than 400,000 records. Although all other analyzed projects are prioritizing the scientific dimension, and some newer projects have started to publish their results (e.g., Mosquito Monitor [Garamszegi, Kurucz, and Soltész 2023] and MyPond [Márton et al. 2023; Hamer et al. 2024]), we recognize that there is still a lot of opportunity for developments in terms of data accessibility and data-sharing standards.

Von Gönner et al. (2023) also assessed CS projects in Germany, Austria, and Switzerland from a multidimensional perspective including the science dimension. In line with our findings, they discovered that although data quality assurance is widely applied in most of the projects, not all have open raw data, and scientific publications still need to be released in half of the evaluated projects.

Nature conservation dimension

Citizen research projects all over the world assist natural resource management decisions in international biodiversity monitoring and conservation policy (Chandler et al. 2017; McKinley et al. 2017; Turbé et al. 2019) at all levels—individual, community, national, and international. However, in their study, Runnel et al. (2016) stress that in Central and Eastern European countries, integrating CS in biodiversity monitoring and management is still needed. Based on our findings, all of the analyzed NCCS projects already support monitoring of species or ecosystems (MyPond). Additionally, observations in some evaluated projects are also used for habitat protection (Amphibian and Reptile Mapping, BeaverMap), detecting potential human-wildlife conflicts, (BeaverMap), and invasive species control (Mosquito Monitor).

Some scholars emphasize that data from large-scale and long-term CS projects are commonly used for population management decisions and even for assisting international environmental and conservation policies (Chandler et al. 2017; McKinley et al. 2017). Our findings also show that NCCS projects with long experience (Amphibian and Reptile Mapping, WildWatcher, and Common Bird Monitoring Program) had the highest score in the nature conservation dimension, including contributions to environmental management and policy. At the time of the interviews, most NCCS projects did not use their data in policy-making.

Similar to our findings, Sullivan et al. (2017) found in their evaluation that habitat management and protection, rather than law and policy, were the primary uses of eBird data.

Since the interviews, some additional projects used the NCCS project for conservation goals, indicating rapid development of the field in Hungary. For example, the BeaverMap that was initiated in 2021 already shows an impact on species/ecosystem management. They contributed to the first Hungarian model project that maintained a beaver-made wetland and solved related human-wildlife conflicts. Additionally, CS data allowed for the exploration of high-conservation-value beaver-made wetlands, and in the future, these data will improve the distribution map of the Hungarian beaver population (Juhász and Biró 2024).

Participants’ development dimension

According to Kieslinger et al. (2018), it is an indicator of a successful CS project when the benefits of the project equally serve the goals of the project and the participants. In our study, Butterfly-Net demonstrated the best performance in this dimension even though it was the project with the shortest duration. This project offered in-person training and constant connection with experts. Some authors, including Evans et al. (2005) and Phillips et al. (2018), have highlighted these characteristics as essential for influencing participants’ learning outcomes. Starr et al. (2014) state that personal training is not always cost-efficient. In the case of Butterfly-Net, personal training was possible because of the small number of participants (24). The other projects had much greater numbers of participants (e.g., Amphibian and Reptile Mapping: 23,000, Common Bird Monitoring Program: 10,712), and all continue to increase. Bonney et al. (2009) found that participants acquire confidence in their data collection skills as they work with supporting materials. We found that all projects offered some form of additional learning support through their website (e.g., online guides for supporting data collection, printed material, an app for species recognition).

Somerwill and Wehn (2022) showed in their review that even though a wide range of different validated approaches exist to measure changes in attitude toward the environment, in behavior, and in participant knowledge, only a few projects apply them. We have similar findings because, in the projects evaluated in our study, no assessment has been undertaken on project impact on the knowledge, acquired skills, attitude, or behavior of participants.

Leonard et al. (2023) showed in their study that CS projects in 25 countries evaluated the knowledge and behavioral change of participants. The findings indicated that volunteers exhibited positive attitudes and achieved considerable knowledge acquisition. A key recommendation for the evaluation was to consider demographics. Moving forward, the established NCCS projects in Hungary should enhance their data collection protocols to gather more information about participants, allowing for a deeper exploration of the participants’ development dimension.

Conclusion

In this paper, we developed a conceptual framework to assess the performance of NCCS projects in three dimensions: science, nature conservation, and participants’ development, and applied it to evaluate eight Hungarian NCCS projects. Overall, the investigated initiatives earned high scores in the science dimension, which strengthened the quality of data generated. Hungarian CS initiatives are becoming an effective tool for supporting the monitoring of biodiversity and developing conservation strategies. Since all of them have the vision to expand their impact in the nature conservation dimension, we recommend improving the strategies of data sharing with multiple stakeholders, fulfilling open data requirements, and promoting cooperation between NCCS projects and institutions with experience in informing and implementing conservation policies. All NCCS initiatives could further progress the participants’ development dimension, in terms of evaluation of knowledge gain and changes in attitude and behavior. Our results assist Hungarian NCCS managers and coordinators to understand the strengths of their projects and identify project dimensions that can be improved.

The newly developed evaluation framework can be applied to assess the performance of NCCS initiatives in three dimensions (science, nature conservation, and participants’ development). The main strength of the framework is its simplicity: yes-no questions need to be answered for each indicator. In most cases, with the help of the project coordinators, the questions can be easily answered. The framework’s simplicity and free availability enable its application by diverse end users, including project managers (for self-assessment and planning purposes) and external evaluators who are interested in the evaluation of NCCS projects. Although the evaluation framework was designed based on the Hungarian context, it can be applied to evaluate other NCCS projects as well.

However, the evaluation needs to be repeated from time to time because even since the interviews were conducted (2022), the projects have been further developed. The framework can be improved (e.g., combining composite indices for each dimension, changing the binary scoring system to a 0–3, 0–5 scale, focusing on only achieved results without plans, and altering some criteria, for example, institutional affiliation and extending the knowledge-attitude-behavior part). The framework can be further developed in a participatory way involving coordinators of NCCS projects and experts from other fields (e.g., conservation psychology).

Supplementary Files

The supplementary files for this article can be found as follows:

Supplemental File 1

Interview questions. DOI: https://doi.org/10.5334/cstp.762.s1

Supplemental File 2

Table S1. The main characteristics of the nature conservation–related citizen science projects in Hungary. DOI: https://doi.org/10.5334/cstp.762.s2

Supplemental File 3

Table S2. Results of the performance assessment of nature conservation–related citizen science projects (state in 2022). DOI: https://doi.org/10.5334/cstp.762.s3

Ethics and Consent

During the interviews we followed the fundamental ethical principles of the Code of Research Ethics of the Hungarian Academy of Sciences (HAS 2010) and the European Code of Conduct of Scientific Integrity (ALLEA 2017). A signed informed consent including information about data use, confidentiality, anonymity, voluntary participation, permission for recording, and causing no harm to participants was sought from the participants before the interview.

Acknowledgements

We acknowledge the time and help of the participants of all the NCSS projects.

We thank the funding of the eight CS projects involved in this study.

We are thankful to Bálint Balázs for the suggestions during the conceptualization of the study and Jocelyn Weyala Burudi and Yvonne Garcia for the English proofreading.

Funding Information

Johanna Soria was financed by Tempus Public Foundation, Hungary. Stipendium Hungaricum Scholarship (ID number: 2021_422428).

Erika Juhász received support through the National Laboratory for Health Security (RRF-2.3.1-21-2022-00006), HUN-REN Centre for Ecological Research, Budapest, Hungary.

Competing Interests

The authors have no competing interests to declare.

Author contributions

Conceptualization and methodology: SJ, TKE, interviews, analysis: SJ, data provision and validation: VO, BM, JE, SZ, BB, MZ, ST, HB, SI, KB, original draft -writing: SJ, review and editing: TKE, VO, BM, JE, SZ, BB, MZ, ST, HB, SI, KB, CA, BG, supervision: TKE.

References

1. ALLEA. (2017) The European Code of Conduct for Research Integrity. Available at https://www.etag.ee/wp-content/uploads/2017/04/ALLEA-European-Code-of-Conduct-for-Research-Integrity-2017-really-final.pdf. (Last accessed 27 June 2024)

2. Balázs, B., Mooney, P., Nováková, E., Bastin, L. and Jokar, A.J. (2021) Data quality in citizen science, in: Vohland, K, Land-Zandstra, A, Ceccaroni, L, Lemmens, R, Perelló, J, Ponti, M, Samson, R and Wagenknecht, K. (Eds.) The Science of Citizen Science. Cham, Switzerland: Springer International Publishing, pp. 139–157. DOI: https://doi.org/10.1007/978-3-030-58278-4_8

3. Bán, M., Boné, G.M., Bérces, S., Barta, Z., Kovács, I., Ecsedi, K. and Sipos, K. (2022) OpenBioMaps – self-hosted data management platform and distributed service for biodiversity related data. Earth Science Informatics, 15, pp. 2007–2016. DOI: https://doi.org/10.1007/s12145-022-00818-3

4. Bela, G., Peltola, T., Young, J.C., Balázs, B., Arpin, I., Pataki, G., Hauck, J., et al. (2016) Learning and the transformative potential of citizen science: Lessons from the Study of Nature. Conservation Biology, 30(5), pp. 990–999. DOI: https://doi.org/10.1111/cobi.12762

5. Bio Innovation Service. (2018) Citizen science for environmental policy: development of an EU-wide inventory and analysis of selected practices. Final report for the European Commission, DG Environment under the contract 070203/2017/768879/ETU/ENV.A.3, in collaboration with Fundacion Ibercivis and The Natural History Museum. Available at at https://data.europa.eu/doi/10.2779/961304 last accessed 29 October 2023)

6. Blaney, R.J.P., Philippe, A.C.V., Pocock, M.J.O. and Jones, G.D. (2016) Citizen Science and Environmental Monitoring: Towards a Methodology for Evaluating Opportunities, Costs and Benefits Final Report, UK Environmental Observation Framework, National Environment Research Council, Swindon, Wiltshire, UK.

7. Bonney, R. (2021) Expanding the Impact of Citizen Science. BioScience, 71: 448–451. DOI: https://doi.org/10.1093/biosci/biab041

8. Bonney, R., Cooper, C.B., Dickinson, J., Kelling, S., Phillips, T., Rosenberg, K.V. and Shirk, J. (2009) Citizen science: a developing tool for expanding science knowledge and scientific literacy BioScience, 59(11): 977–984, DOI: https://doi.org/10.1525/bio.2009.59.11.9

9. Bonney, R., Shirk, J.L., Phillips, T.B., Wiggins, A., Ballard, H.L., Miller-Rushing, A.J. and Parrish, J.K. (2014) Next steps for citizen science. Science, 343(6178): 1436–1437, DOI: https://doi.org/10.1126/science.1251554

10. Brossard, D., Lewenstein, B. and Bonney, R. (2005) Scientific knowledge and attitude change: The impact of a citizen science project. International Journal of Science Education, 27, pp. 1099–1121. DOI: https://doi.org/10.1080/09500690500069483

11. Ceccaroni, L. and Piera, J. (Eds). (2017) Analyzing the Role of Citizen Science in Modern Research: Advances in Knowledge Acquisition, Transfer, and Management. IGI Global. DOI: https://doi.org/10.4018/978-1-5225-0962-2

12. Chandler, M., See, L., Copas, K., Bonde, A.M., López, B.C., Danielsen, F., Legind, J.K., et al. (2017) Contribution of citizen science towards international biodiversity monitoring. Biological conservation, 213, pp. 280–294. DOI: https://doi.org/10.1016/j.biocon.2016.09.004

13. Cox, J., Oh, E.Y., Simmons, B., Lintott, C., Masters, K., Greenhill, A., Graham, G., et al. (2015) Defining and Measuring Success in Online Citizen Science: A Case Study of Zooniverse Projects. Computing in Science & Engineering, 17, pp. 28–41. DOI: https://doi.org/10.1109/MCSE.2015.65

14. Czeglédi, A. (2022) Citizen science in the making: mapping participatory science projects in Hungary, in: Proceedings of Engaging Citizen Science Conference 2022 — PoS (CitSci2022). Engaging Citizen Science Conference 2022, Sissa Medialab, Aarhus University, Denmark. p37. DOI: https://doi.org/10.22323/1.418.0037

15. D’Andrea, L., Kalpazidou S.E., Bužan, E., Vidal Merino, M., Dall, E., Colonnello, C., Graversen, E.K., et al. (2022) Evaluating citizen science initiatives through a citizen science-based approach. Journal for Research and Technology Policy Evaluation, 54, pp. 43–50. DOI: https://doi.org/10.22163/fteval.2022.570

16. Danielsen, F., Pirhofer-Walzl, K., Adrian, T.P., Kapijimpanga, D.R., Burgess, N.D., Jensen, P.M., Bonney, R., et al. (2014) Linking Public Participation in Scientific Research to the Indicators and Needs of International Environmental Agreements: Monitoring environmental agreements. Conservation Letters, 7: 12–24. DOI: https://doi.org/10.1111/conl.12024

17. Davis, C., Del Bianco, V., Peterman, K., Grover, A., Phillips, T. and Becker-Klein, R. (2022) Diverse and Important Ways Evaluation can Support and Advance. Citizen Science: Theory and Practice, 7, 30. DOI: https://doi.org/10.5334/cstp.482

18. EPA. (2019) Quality Assurance Handbook and Guidance Documents for Citizen Science Projects (EPA QA Handbook) Environmental Protection Agency, USA. Available at https://www.epa.gov/participatory-science/quality-assurance-handbook-and-toolkit-participatory-science-projects (Last accessed 05 February 2024)

19. Evans, C., Abrams, E., Reitsma, R., Roux, K., Salmonsen, L. and Marra, P.P. (2005) The Neighborhood Nestwatch Program: Participant Outcomes of a Citizen-Science Ecological Research Project. Conservation Biology, 19, pp. 589–594. DOI: https://doi.org/10.1111/j.1523-1739.2005.00s01.x

20. Freitag, A., Meyer, R., and Whiteman, L. (2016) Strategies Employed by Citizen Science Programs to Increase the Credibility of Their Data. Citizen Science: Theory and Practice, 2, pp. 1–11. DOI: https://doi.org/10.5334/cstp.6

21. Garamszegi, L., Kurucz, K. and Soltész, Z. (2023) Validating a Surveillance Program of Invasive Mosquitoes Based on Citizen Science in Hungary. Journal of Applied Ecology, 60(7):1481–94. DOI: https://doi.org/10.1111/1365-2664.14417

22. Hamer, A.J., Barta, B., Márton, Z., Vad, C.F., Szabó, B., Tornero, I. and Horváth, Z. (2024). Patterns and correlates in the distribution, design and management of garden ponds along an urban–rural gradient. Urban Ecosystems. DOI: https://doi.org/10.1007/s11252-024-01559-5

23. HAS Hungarian Academy of Sciences. (2010) Science Ethics Code. Hungarian Academy of Sciences. Available at https://mta.hu/data/dokumentumok/english/background/Science_Ethics_Code_English.pdf (Last accessed 26 June 2024)

24. Hecker, S., Garbe, L. and Bonn, A. (2018) The European Citizen Science Landscape – a Snapshot. Citizen Science, Innovation in Open Science, Society and Policy. In: Hecker, S, Haklay, M, Bowser, A, Makuch, Z, Bonn, A and Vogel, J (Eds.) UCL Press. London, UK. DOI: https://doi.org/10.2307/j.ctv550cf2

25. Jordan, R., Gray, S., Sorensen, A., Newman, G., Mellor, D., Newman, G., Hmelo-Silver, C., et al. (2016) Studying citizen science through adaptive management and learning feedbacks as mechanisms for improving conservation. Conservation Biology, 30: 487–495. DOI: https://doi.org/10.1111/cobi.12659

26. Jordan, R.C., Gray S.A., Howe, D.V., Brooks, W.R. and Ehrenfeld, J.G. (2011) Knowledge Gain and Behavioral Change in Citizen-Science Programs: Citizen-Scientist Knowledge Gain. Conservation Biology, 25, pp. 1148–1154. DOI: https://doi.org/10.1111/j.1523-1739.2011.01745.x

27. Juhász, E., and Biró, M. (2024) BeaverMap, a Citizen Science Program in the Service of Discovering the Beaver-Made Landscape Alteration. In: Abstract Book of Citizen Science Conference and Workshop SEEN HUNGARY 2024. 13–13, Available at https://ecolres.hun-ren.hu/wp-content/uploads/2024/01/SEEN_2024_abstractbook.pdf (Last accessed 15 June 2024)

28. Károlyi, B. and Rédei, D. (2017) Acrosternum heegeri Fieber, 1861 (Hemiptera: Heteroptera: Pentatomidae), another Mediterranean bug expanding to the north. Zootaxa, 4347. DOI: https://doi.org/10.11646/zootaxa.4347.2.13

29. Kelling, S., Fink, D., La Sorte, F.A., Johnston, A., Bruns, N.E. and Hochachka, W.M. (2015) Taking a ‘Big Data’ approach to data quality in a citizen science project. A Journal of Environment and Society, 44: 601–611. DOI: https://doi.org/10.1007/s13280-015-0710-4.

30. Kieslinger, B., Schäfer, T., Heigl, F., Dörler, D., Richter, A. and Bonn, A. (2018) Evaluating citizen science-Towards an open framework. In: Hecker, S, Haklay, M, Bowser, A, Makuch, Z, Vogel, J and Bonn, A (Eds.), Citizen Science: Innovation in Open Science, Society and Policy. London: UCL Press, pp. 81–95. DOI: https://doi.org/10.14324/111.9781787352339

31. Kóbor, P., Kondorosy, E., Nagy, C. and Orosz, A. (2021) Acanalonia conica (Say, 1830): A new alien planthopper species established in Hungary (Auchenorrhyncha: Fulgoroidea: Acanaloniidae). Acta Phytopathologica et Entomologica Hungarica, 56, pp. 201–207. DOI: https://doi.org/10.1556/038.2021.00132

32. Kosmala, M., Wiggins, A., Swanson, A. and Simmons, B. (2016) Assessing data quality in citizen science. Frontiers in Ecology and the Environment. 14, pp. 551–560. DOI: https://doi.org/10.1002/fee.1436

33. Leonard, A., Wheeler, S. and McCulloch, M. (2023) Does Citizen Science Bring “Power to the People”? Evaluating a Remote Mapping Project to Identify Best Practices for Positive Impact on Volunteers. Citizen Science: Theory and Practice, 8(1): 4, pp. 1–15. DOI: https://doi.org/10.5334/cstp.534

34. Márton, Z., Barta, B., Vad, C., Szabó, B., Hamer, J., Kardos, V., Laskai, C., et al. (2023). Effects of Urbanisation, Habitat Characteristics, and Management on Garden Pond Biodiversity: A Citizen Science Approach. preprint. Ecology. DOI: https://doi.org/10.1101/2023.12.08.570789

35. Maynard, L., Monroe, M.C., Jacobson, S.K. and Savage, A. (2020) Maximizing biodiversity conservation through behavior change strategies. Conservation Science and Practice, 2(6), pp. 1–11. DOI: https://doi.org/10.1111/csp2.193

36. McKinley, D.C., Miller-Rushing, A.J., Ballard, H.L., Bonney, R., Brown, H., Cook-Patton, S.C., Evans, D.M., et al. (2017) Citizen science can improve conservation science, natural resource management, and environmental protection. Biological Conservation, 208, pp. 15–28. DOI: https://doi.org/10.1016/j.biocon.2016.05.015

37. Merkl, O., Károlyi, B. and Korányi, D. (2017) First record of Cybocephalus nipponicus in Hungary (Coleoptera: Cybocephalidae). Folia Entomologica Hungarica. 78, pp. 71–76. DOI: https://doi.org/10.17112/FoliaEntHung.2017.78.71

38. Mesaglio, T. and Callaghan, C.T. (2021) An overview of the history, current contributions and future outlook of iNaturalist in Australia. Wildlife Research. 48, pp. 289–303. DOI: https://doi.org/10.1071/WR20154

39. Miller, J. (2022) GBIF’s (Global Biodiversity Information Facility) New Strategic Framework to Address Biodiversity Data Needs. BISS Biodiversity Information Science and Standards, 6: e91093. DOI: https://doi.org/10.3897/biss.6.91093

40. Murray-Rust, P. (2008) Open Data in Science. Nature Precedings, pp. 52–64. DOI: https://doi.org/10.1038/npre.2008.1526.1

41. Newing, H., Eagle., C., Puri, R. and Watson, C.W., (2011) Conducting Research in Conservation. London. Routledge. DOI: https://doi.org/10.4324/9780203846452

42. Peter, M., Diekötter, T., Höffler, T. and Kremer, K. (2021) Biodiversity citizen science: Outcomes for the participating citizens. People and Nature, 3, pp. 294–311. DOI: https://doi.org/10.1002/pan3.10193.

43. Phillips, T., Porticella, N., Constas, M. and Bonney, R. (2018) A Framework for Articulating and Measuring Individual Learning Outcomes from Participation in Citizen Science. Citizen Science. Citizen Science: Theory and Practice, 3, pp. 1–19, DOI: https://doi.org/10.5334/cstp.126

44. Pocock, M.J.O., Hamlin, I., Christelow, J., Passmore, H. and Richardson, M. (2023) The Benefits of Citizen Science and Nature-noticing Activities for Well-being, Nature Connectedness and Pro-nature Conservation Behaviours. People and Nature. DOI: https://doi.org/10.1002/pan3.10432.

45. Pocock, M.J.O., Roy, H.E., Preston, C.D. and Roy, D.B. (2015) The Biological Records Centre: a pioneer of citizen science. Biological Journal of the Linnean Society, 115, pp. 475–493. DOI: https://doi.org/10.1111/bij.12548

46. Price-Jones, V., Brown, P.M.J., Adriaens, T., Tricarico, E., Farrow, R.A., Cardoso, A.C., Gervasini, E., et al. (2022) Eyes on the aliens: citizen science contributes to research, policy and management of biological invasions in Europe. NeoBiota 78, pp. 1–24. DOI: https://doi.org/10.3897/neobiota.78.81476

47. Robinson, L.D., Cawthray, J.L., West, S.E., Bonn, A. and Ansine, J. (2018) Ten principles of citizen science. In Hecker, S, Haklay, M, Bowser, A, Makuch, Z, Vogel, J and Bonn, A (Eds.), Citizen Science: Innovation in Open Science, Society and Policy, pp. 27–40. UCL Press. DOI: https://doi.org/10.2307/j.ctv550cf2.9

48. Runnel, V., Wetzel, F., Groom, Q., Koch, W., Nils-Valland, I., Panteri, E. and Kõljalg, U. (2016) Summary Report and Strategy Recommendations for EU Citizen Science Gateway for Biodiversity Data. Research Ideas and Outcomes, 2:e11563. DOI: https://doi.org/10.3897/rio.2.e11563

49. Santori, C., Keith, R.J., Whittington, C.M., Thompson, M.B., Van-Dyke, J.U. and Spencer, R. (2021) Changes in participant behavior and attitudes are associated with knowledge and skills gained by using a turtle conservation citizen science app. People and Nature, 3, pp. 66–76. DOI: https://doi.org/10.1002/pan3.10184

50. Schaefer, T., Kieslinger, B., Brandt, M., and Van Den Bogaert, V. (2021) Evaluation in citizen science: the art of tracking a moving target. In Vohland, K, et al (Eds.), The Science of Citizen Science, pp. 495–514. Springer International Publishing. DOI: https://doi.org/10.1007/978-3-030-58278-4_25

51. Serrano-Sanz, F., Holocher-Ertl, T., Kieslinger, B., Sanz García, F. and Silva, C.G. (2014) White Paper on citizen science for Europe. European Commission. Available at www.zsi.at/object/project/2340/attach/White_Paper-Final-Print.pdf (Last accessed 15 March 2023)

52. Shirk, J., Ballard, H.L., Wilderman, C.C., Phillips, T., Wiggins, A., Jordan, R., McCallie, E., et al. (2012) Public Participation in Scientific Research: A Framework for Deliberate Design. E&S, 17 (2). DOI: https://doi.org/10.5751/ES-04705-170229.

53. Sillero, N, Campos, J, Bonardi, A, Corti, C, Creemers, R, Crochet, PA, Crnobrnja-Isailović, J., et al. (2014) Updated distribution and biogeography of amphibians and reptiles of Europe. Amphibia-Reptilia, 35, pp. 1–31. DOI: https://doi.org/10.1126/science.1103538

54. Somerwill, L. and When, U. (2022) How to Measure the Impact of Citizen Science on Environmental Attitudes, Behaviour and Knowledge? A Review of State-of-the-Art Approaches. Environmental Sciences Europe, 34(1):18. DOI: https://doi.org/10.1186/s12302-022-00596-1

55. Starr, J., Schweik, C.M., Bush, N., Fletcher, L., Finn, J., Fish, J. and Bargeron, C.T. (2014) Lights, Camera…Citizen Science: Assessing the Effectiveness of Smartphone-Based Video Training in Invasive Plant Identification. PLoS ONE 9, e111433. DOI: https://doi.org/10.1371/journal.pone.0111433

56. Stevenson, R., Merrill, C. and Burn, P. (2021) Useful biodiversity data were obtained by novice observers using iNaturalist during college orientation retreats. Citizen Science: Theory and Practice. 6(1): 27, pp. 1–12. DOI: https://doi.org/10.5334/cstp.407

57. Sullivan, B.L., Phillips, T., Dayer, A.A., Wood, C.L., Farnsworth, A., Iliff, M.J., Davies, I., et al. (2017) Using open access observational data for conservation action: A case study for birds. Biological Conservation 208, pp. 5–14. DOI: https://doi.org/10.1016/j.biocon.2016.04.031

58. Suškevičs, M., Raadom, T., Vanem, B., Kana, S., Roasto, R., Runnel, V. and Külvik, M. (2021) Challenges and opportunities of engaging biodiversity-related citizen science data in environmental decision-making: Practitioners’ perceptions and a database analysis from Estonia. Journal for Nature Conservation, 64: 126068. DOI: https://doi.org/10.1016/j.jnc.2021.126068

59. Szép, T. and Gibbons, D. (2000) Monitoring of common breeding birds in Hungary using a randomized sampling design. The Ring, 22, pp. 45–55.

60. Szép, T., Csörgő, T., Halmos, G., Lovászi, P., Nagy, K. and Schmidt, A. (Eds.). (2021) Magyarország madáratlasza – Bird Atlas of Hungary. Ministry of Agriculture, Magyar Madártani és Természetvédelmi Egyesület-Hungarian Ornithological and Nature Conservation Society, pp. 799.

61. Szép, T., Nagy, K., Nagy, Z. and Halmos, G. (2012) Population trends of common breeding and wintering birds in Hungary, decline of long distance migrant and farmland birds during 1999–2012. Ornis Hungarica, 20: 13–63. DOI: https://doi.org/10.2478/orhu-2013-0007.

62. Tang, J., Zhou, X., (Chris) Zhao, Y. and Wang, T. (2021) How the type and valence of feedback information influence volunteers’ knowledge contribution in citizen science projects. Information Processing & Management, 58: 102633. DOI: https://doi.org/10.1016/j.ipm.2021.102633

63. Theobald, E.J., Ettinger, A.K., Burgess, H.K., DeBey, L.B., Schmidt, N.R., Froehlich, H.E., Wagner, C., et al. (2015) Global change and local solutions: Tapping the unrealized potential of citizen science for biodiversity research. Biological Conservation, 181, pp. 236–244. DOI: https://doi.org/10.1016/j.biocon.2014.10.021.

64. Turbé, A., Barba, J., Pelacho, M., Mugdal, S., Robinson, L.D., Serrano-Sanz, F., Sanz, F., et al. (2019) Understanding the Citizen Science Landscape for European Environmental Policy: An Assessment and Recommendations. Citizen Science: Theory and Practice, 4(1), pp. 34. DOI: https://doi.org/10.5334/cstp.239

65. Turrini, T., Dörler, D., Richter, A., Heigl, F., and Bonn, A. (2018) The threefold potential of environmental citizen science – Generating knowledge, creating learning opportunities and enabling civic participation. Biological Conservation, 225, pp. 176–186. DOI: https://doi.org/10.1016/j.biocon.2018.03.024.

66. VADONLESŐ GROUP. (2019) Experiences of the first decade of the WildWatcher Programme. Results of the National Biodiversity Monitoring System II, pp. 181–198.

67. VADONLESŐ GROUP, Berndt, M, Lukács, A, Oláh, BK and Farkas, A. (2019) Urban hedgehogs (Erinaceus roumanicus) in Budapest: Live or let die. Hungarian Agricultural Research, 28(3), pp. 20–26.

68. VADONLESŐ GROUP, Geng, B., Bakó, B., Bata, K., Bokor, V., Koczka, K., Váczi, O., et al. (2018) WildWatcher Programme: Volunteer Based, Biodiversity Data-Collector System. Results of the First Ten Years. Hungarian Agricultural Research, 27(4), pp. 11–18.

69. VERBI, S. (2021) MAXQDA 2022 (computer software).

70. Vétek, G., Károlyi, B., Mészáros, Á., Horváth, D. and Korányi, D. (2018) The invasive brown marmorated stink bug (Halyomorpha halys) is now widespread in Hungary. Entomologia, 38, pp. 3–14. DOI: https://doi.org/10.1127/entomologia/2018/0631.

71. Vohland, K., Göbel, C., Balázs, B., Butkevičienė, E., Daskolia, M., Duží, B., Hecker, S., et al. (2021) Citizen Science in Europe. in The Science of Citizen Science. In: Vohland, K, Land-Zandstra, Ceccaroni, L, Lemmens, R, Perelló, J, Ponti, M, Samson, R and Wagenknecht, K (Eds.) Springer International Publishing, pp. 35–53. DOI: https://doi.org/10.1007/978-3-030-58278-4_3

72. Von Gönner, J., Herrmann, T.M., Bruckermann, T., Eichinger, M., Hecker, S., Klan, F., Lorke, J., et al. (2023). Citizen science’s transformative impact on science, citizen empowerment and socio-political processes. Socio-Ecological Practice Research 5, pp. 11–33. DOI: https://doi.org/10.1007/s42532-022-00136-4

73. Wiggins, A., Bonney, R., Graham, E., Henderson, S., Kelling, S., LeBuhn, G., Litauer, R., et al. (2013) Data management guide for public participation in scientific research. DataOne Working Group.

74. Wiggins, A., Newman, G., Stevenson, R.D. and Crowston, K. (2011) Mechanisms for Data Quality and Validation in Citizen Science, in: 2011 IEEE Seventh International Conference on E-Science Workshops. Presented at the 2011 IEEE Seventh International Conference on e-Science Workshops (eScienceW), IEEE, Stockholm, Sweden, pp. 14–19. DOI: https://doi.org/10.1109/eScienceW.2011.27.

75. Yang, D., Wan, H., Huang, T.K and Liu, J. (2019) The Role of Citizen Science in Conservation under the Telecoupling Framework. Sustainability, 11, pp. 2–17. DOI: https://doi.org/10.3390/su11041108