1. Introduction
Since the signing of the European Landscape Convention (ELC) of the Council of Europe on 20 October 2000, the landscape now has a regulatory document with an international scope [1] that provides it with legal certainty in those countries adhering to the treaty [2,3,4]. The ELC’s articles provide, among other aspects, common rules to ensure that all the countries involved have the same point of departure regarding the treatment of the landscape [5]. A series of measures are proposed to enable countries to design and develop the necessary measures for managing and organising it [6,7] and educating people accordingly [8]. In addition, the ELC seeks to foster a network of shared experiences across countries with a view to boosting those teaching and management techniques that are providing the best results [3,4]. What is more, the ELC ensures that each country has the freedom required to adapt its decision-making process according to its own idiosyncrasies and characteristics [3], thereby catering for the proposal of measures suited to social and territorial requirements [3,7]. The ELC is designed to cover all matters involving the landscape [6], which means it does not focus solely on the management and organisation of landscape resources or on territorial features with an impact upon a given landscape [7], as it also considers that landscape teaching and education from early schooling onwards is a step that will help to raise awareness about it [8]. At the same time, it proposes training professionals to acquire the right skills for managing and teaching about the landscape [8,9]. The latter considerations will be the focus of our attention here, namely, the landscape’s importance in education and in teacher training, with the ELC being used as the international referent.
The landscape has traditionally featured in schooling as part of different subjects in which it has been studied depending on the overall perspective adopted in each specific case [5]: art, history, literature, geography, etc. This interdisciplinary nature [10] favours its inclusion in the teaching–learning process and provides teachers with an instrument that allows them to apply a mainstream approach to territorial issues [11,12]. The landscape has also been a major component of the subject of geography because of both its integrating nature [10] and its classroom use according to two main approaches: the landscape as curricular content and the landscape as a didactic instrument [13]. This means that the teaching of geography takes the landscape into account during the training process [14] for teachers responsible for early schooling in the form of primary education. At this stage, the landscape is part of the syllabus that pupils are required to learn [13], and although it appears in a range of subjects, as noted earlier, it plays a key role in the social sciences, and especially in geography [14]. Due to this twin purpose it serves as both content and didactic instrument. Primary education in Spain is divided into six years, with a total of five core subjects: Social Sciences, Natural Sciences, Spanish Language and Literature, Maths, and a Foreign Language [15]. This means that the Social Sciences, whose syllabus consists mainly of content involving geography and history, are taught during the six years of primary education, thereby highlighting the importance of suitably instructing teachers in landscape-related matters [14] including its interpretation and assessment. It is necessary to know how the landscape is perceived in order to value it as an integrated educational resource for teaching curricular content, values (conservation and respect for landscape diversity) [16,17], cultural and identity issues [18]. The lived landscape provides nuances and singularities that differentiate it from the immediate areas [18,19,20]. Therefore, the teacher must be able to understand how this specific landscape is interpreted by the students, because their behaviour in relation to the landscape (conservation, feelings, etc.) will be conditioned by the way they perceive it [16,19,20,21]. Moreover, depending on the teacher’s knowledge and the perception and sensitivities about the landscape, the way of teaching will be adapted [19]. Additionally, it is necessary to identify how the landscape is perceived, analysed, and interpreted [22] from the teacher’s perspective. In addition, landscape has a significant symbolic charge associated with identity and emotions [20]. The teaching–learning process of landscape has been evolving and adapting to different teaching methodologies and different didactic instruments [19] conditioned by the subjectivity of landscape [20,22]. The subjective character of landscape influences the teaching process [20]. According to Zanato [20], landscape has a variety of didactic characteristics that can be summarised in three functions: hermeneutic function; pragmatic function and social function. Zanato [20] emphasises also the possibilities offered by landscape from a pedagogical perspective. Landscape is a didactic resource that facilitates the development of a critical thinking [20]. When students compare landscapes, they can identify the cultural aspects underlying each landscape and learn habits of respect for different territorial identities [18]. In other words, landscape must be identified as something more than a didactic instrument and as something more than a curricular content; it must be treated as an element of identity and as an element for people’s comfort. The perception of the landscape by teachers and students as well as its symbolic charge will influence the way it is interpreted, taught, and learnt [20].
This informs the need to provide prospective teachers with sufficient knowledge and skills to conduct the teaching–learning process [23], whereby their future primary pupils can achieve the subject’s goals and the specific targets in this stage of schooling. Teacher training in Spain involves a Degree in Primary School Teaching, and this is the qualification upon which we shall be focusing our research, seeking to analyse the knowledge these prospective teachers have of the landscape. Furthermore, the teaching–learning processes involving the landscape need to adapt to new teaching methods and include, among other aspects, Information and Communication Technologies (ICTs) [24,25]. New instruments are appearing that support the teaching both of and with the landscape; for example, augmented reality (AR), virtual reality (VR), and 3D modelling. All these resources are now seen as useful instruments in the teaching process [24].
Faced with this new paradigm for teaching the landscape, there is a need to understand the students’ perception of it; to identify this perception as a didactic resource or instrument, as curricular content, and identify those methods or techniques they consider most suitable for their teaching and whether they look upon ICTs as a useful resource [23]. The survey technique is one way of addressing this problem of understanding the students’ perception [26]. There are different types of instruments and procedures that furnish us with information and enable us to conduct descriptive research based on survey studies that include interviews and questionnaires [26,27]. This research has used a bigger sample and compiled information from a higher number of participants [27] by using questionnaires as the data-gathering instruments. Education research often uses these kinds of tools for collecting information [26] and obtaining, amongst other things, the perception that students and future teachers have of some or other aspect of their profession [26]. It is commonplace to encounter studies in the field of education that use a questionnaire for data-gathering purposes [26], as it provides a general snapshot of some specific aspect that needs to be mastered or understood by the subjects in the sample because it is part of their training process or professional careers.
The aim here, therefore, is the design and validation of an ad hoc questionnaire using qualitative and quantitative analysis techniques [26] for creating a valid and reliable instrument for garnering information on the knowledge of students studying for their Degree in Primary School Teaching as regards the landscape (Cuestionario sobre Percepción del Paisaje—CPP) [Questionnaire on Landscape Perception]. This article has the following specific objectives regarding the CPP: analyse the validity of its content, study its construct validity, and confirm its reliability.
2. Materials and Methods 2.1. Participants: Panel of Experts and Sample
The instrument’s validation process (validity of content, validity of construct, and reliability) involved three kinds of participants. In the first case, the validity of content was performed by a panel of experts [28,29,30,31,32,33]. This provided us with more significant information on specific aspects of the content to be measured with the questionnaire and analyse the consistency between the study’s objectives and those of the instrument [29,31,34,35], as well as the pertinence of the items [36]. According to the suggestion made by Escobar-Pérez and Cuervo-Martínez [34] (pp. 30–31), the following steps were taken for implementing the experts’ judgements: define the panel of experts’ remit (i.e., validate the content of the measuring instrument; select the experts taking part; explain the measuring instrument’s purpose; explain the dimensions and indicators measured by each one of the questionnaire’s items; determine the weighting of the questionnaire’s dimensions; design the experts’ assessment scorecards; conclusions of the assessment. Our research involved five experts [31,37,38,39,40] in geography, in the teaching of landscape, and in working with questionnaires, three of whom were male and two were female. Some scholars recommend between two and ten experts, indicating that this number will depend on their level of expertise in the subject [31,41]. The participating experts were selected by means of causal or deliberate non-probability sampling [42,43], as according to García and Cabero, Merino-Barrero, Valero-Valenzuela, and Moreno-Murcia, and Singh [44,45,46] they had to meet a series of requirements related to their experience and backgrounds in a specific field of education and research. Our case involved university lecturers in three knowledge areas (Social Sciences, Natural Sciences, and the Humanities) with a proven research record in landscape-related topics and more than fifteen years’ service in teaching, following the guidelines provided by Galán [47], who recommends a minimum term of service of five years within the field of knowledge [47].
Secondly, continuing with the content validation process, deliberate non-probability sampling [48] was used to select two students from each year and from each campus, providing a total of sixteen students (eight females and eight males). The following selection criteria were applied: parity between sexes and enrolment as a student for the Decree in Primary School Teaching at Salamanca University. This trial run was used to identify those aspects that could be confusing or prompt the wrong interpretations in the drafting of the questionnaire’s items [43,48]. Besides analysing syntax, this application also enabled us to measure the questionnaire’s duration and check the wording of the instructions [26,48]. This meant that sixteen students with the same characteristics as those in the overall sample could assess the measuring instrument. None of them were included in the final sample, and they were pledged to secrecy so as not to interfere with the results. In this case, the students undertaking the abridged trial run, and as opposed to the experts, did not have a specific instrument for this purpose, but instead answered only the CPP and simply added comments on the instructions and those items they might find confusing.
Finally, the analysis of the construct’s validity and the questionnaire’s reliability involved a sample made up of undergraduates studying for their Degree in Primary School Teaching at Salamanca University. Simple random sampling was administered to all the students enrolled in the degree’s four-year course at the two campuses the university has in Ávila and Salamanca for taking part in the study. Out of the overall number of students enrolled, 432 subjects answered the questionnaire, accounting for approximately 60% of the total (N = 715), whereby our sample had more than the 200 subjects that some scholars consider to be the minimum sample size [49,50,51]. Of these, 172 students were enrolled at the Ávila campus (Escuela Universitaria de Educación y Turismo—University College for Education and Tourism) and 260 at the Faculty of Education in Salamanca. The sample was broken down into 65.74% females and 34.26% males, with an age range of 18–38, although 87.73% of the sample were 24 or under. In terms of the distribution by years, 28–24% (122) were first-years, 33–33% (144) were in second, 27–55% (119) in third, and the remaining 10.88% (47) in fourth.
2.2. Instruments The entire process involved the use of two measuring instruments: a questionnaire for measuring the expert judges’ rating of the items in the CPP and the main questionnaire considered in this research.
Firstly, the questionnaire compiled for gathering the expert judges’ ratings was based on the models proposed by Backhoff, Aguilar, and Larrazolo; García and Cabero; Martín-Romera and Molina, [36,43,44]. They informed the design of a questionnaire divided into two blocks (see Figure 1). The first block focused on identifying the expert and on gathering data related to their time of service in teaching and research. The second block consisted of a total of 70 items in which the judges had to use a score of 1 to 10 [48] to rate their consistency, relevance, and clarity [44]. Furthermore, each item had a space set aside for comments so that the experts could add any remarks on the item they considered necessary [43]. The aim was therefore to gather information that would allow us to conduct the statistical calculations for measuring the validity of the content and thus discard or modify those items that did not comply with minimum quality requirements.
The second instrument was the CPP, and it was designed to measure the students’ perception of the landscape as a concept, as a teaching tool and as educational content. It began with a brief outline of the study and of the researchers involved, as well as specifying how long it should take to fill in and providing basic instructions for doing so. This questionnaire was divided into two main blocks: the first one was used to gather sociodemographic data on the subjects taking part (see Figure 2); the second one, in turn, was used for actually exploring the students’ perception of the landscape.
The sociodemographic data were collated through twelve items, involving an open format (4) and a closed one (8) [48] in which the subjects provided information on the year they were studying, campus, sex, age, town/city, pre-university studies in geography, and their parents’ occupation and studies, amongst others.
The second part of the questionnaire initially involved 70 items with a five-point Likert score for the purpose of addressing the matter of continuity [51,52]: fully disagree (1); disagree (2); neither agree nor disagree (3); agree (4); fully agree (5) [53]. In this case, and according to McMillan and Schumacher [48], the decision was made to include an intermediate category (3) so that the subjects were not forced to select an option that they did not agree with. Based on a literature review and according to the theoretical underpinnings, the 70 variables were grouped into four dimensions, although they were randomly presented in the questionnaire [48]: Theoretical knowledge of the landscape and its features; Landscape and a sense of identity; Measures for managing and protecting the landscape, and Pedagogical expertise (teaching–learning the landscape). The variables were introduced through statements [53] to which the subjects indicated their degree of agreement or disagreement [53]. The dimension dealing with Theoretical knowledge of the landscape comprised 22 variables, all designed to identify the student’s potential understanding of the landscape as a theoretical concept and its component features [54]. This was based on a series of variables centred around the definition of the term landscape (as provided by the ELC), for subsequently identifying whether the subjects had any understanding of the features (units) that make up the landscape and their influence on its development or dynamism. The ELC’s definition was used because there are numerous scholars that consider it to be both all-inclusive and covering a broad spectrum that groups together all the landscape’s features, whether anthropic or natural [6,55,56,57,58], and which includes educational considerations by linking landscape and society through perception [5,59]. What is more, there was also a variable that focused exclusively on the ELC, thereby seeking to discover whether the students were familiar with this convention. The second dimension, Landscape and a sense of identity, involved variables for verifying whether the subjects considered there might be a sense of identity linked to the landscape [9,54,60,61,62,63,64], and regarding their experiences in it [65]. Accordingly, this group consisted of eight variables, all of which focused on aspects related to the subjective values the landscape may evoke, such as positive or negative feelings, moods, and territorial identity or belonging. The dimension called Measures for managing and protecting the landscape had ten variables for discovering whether the students had any notion of the measures adopted accordingly (of a natural or cultural nature), as well as their opinion on those aspects of the landscape that might be more significant when designing these management and protection measures [60,66,67]. The most numerous dimension was the one involving Pedagogical expertise (teaching–learning the landscape), which comprised 30 items overall. The aim here was to identify the students’ perception of the landscape in the teaching–learning process from the viewpoint of the landscape as a didactic instrument and as part of the syllabus, as well as single out those teaching methods considered more appropriate [13,62,67,68,69,70,71]. This group included teaching methods and techniques, curricular knowledge on the landscape as content in primary education, the use of the landscape as an instrument for teaching certain aspects of geography [72] (e.g., climate, plant life, relief, and land usages) and the adoption of specific approaches for teaching the landscape (e.g., fieldwork [72], photos, and textbook). The teaching methods considered included items focusing on constructivism [73,74,75,76] and the flipped classroom [77].
2.3. Procedure
2.3.1. Design and Content Validation
This study adopted two approaches: a descriptive one [48,78] and a mainstream one [27] using the survey technique [48], involving an ad hoc questionnaire as a data-gathering instrument [27]. Based on the guidelines for designing a questionnaire proposed by Cohen, Manion, and Morrison (p. 472) [26] the process was orchestrated into four stages (see Figure 3) based on the need to design and validate a questionnaire on landscape perception amongst undergraduates studying for their degree in primary school teaching.
Stage 1 (see Figure 3) was devoted to the literature review and the selection of the measuring instrument. A thorough study of the literature [26] revealed theoretical aspects on the landscape and its teaching. This process brought us up to speed with the state-of-the-art and enabled us to study the different data-gathering mechanisms used [26]. At the same time, this thorough review enabled us, according to the guidelines provided by Martín-Romera and Molina [43] (p. 198), to define the theoretical variables, dimensions and indicators corresponding to the research goals proposed. Thus, now in stage 2 (see Figure 3), we began the operationalization process that led to the conversion of the theoretical aspects into items on the questionnaire [26,43]. This gave way to the questionnaire’s initial design [26,43], identifying those sociodemographic details needed to obtain and draft the items considering the theoretical contributions obtained during the literature review [43].
Stage 3 (see Figure 3) covered the validation of the questionnaire’s content and its final design. This involved calling upon experts to give their opinion [26,27,28,29,30,31,32,33] and a brief trial run involving a small group of students with similar characteristics to the subjects in the sample [43,48]. The experts submitted their opinions in September 2019 by completing a questionnaire in which they assessed the coherence, relevance, and clarity of the items, using a score from 1 to 10 to rate each one. The judges’ scores were analysed using Aiken’s V content validity coefficient [79]. This involved using the critical value V0 = 0.70 [80]. Aiken’s V coefficient was calculated using the formula adapted by Penfield and Giacobbi [81]:
v=x¯−lk
wherex¯= mean of the judges’ ratings; l = lowest possible rating (1); k = highest possible rating (10). Aiken’s V calculation reports the judges’ rating of each item.
The critical value was used to discard those items [82] whose score was below or outside the confidence interval set by the Wilson score method [83], whereby the lower and upper thresholds were calculated according to the following equations:
L=2nkV+z2−z4nkv(1−v)+z22(nk+z2)
U=2nkV+z2+z4nkv(1−v)+z22(nk+z2)
where L = the interval’s lower threshold; U = the interval’s upper threshold; z = value in standard normal distribution; V = Aiken’s V; n = number of judges. With these equations the confidence interval can be calculated. The equations have been adapted by Penfield and Giacobbi [81] in order to use Aiken’s V as a ratio.
The analysis of the experts’ ratings was followed by the rejection or modification of any problematic items. The final questionnaire then underwent a trial run with a small group of students. The trial was held in a classroom during class-time in order to replicate the same conditions as the overall sample would experience when administered the final questionnaire as a pilot test [84]. The trial students simply assessed the wording of the items and reported any that were difficult to understand [85]. The experts’ ratings and the brief trial run therefore informed the definitive version of the questionnaire.
2.3.2. Construct’s Validity
Stage 4 involved an analysis of the construct’s validity, its reliability, and the questionnaire’s definitive version. A pilot test was conducted in November 2019. With the prior permission of the lecturer responsible for the subject, a member of the research team administered the questionnaires to the students for their completion in the classroom and provided the instructions on how to do so. The confidentiality and anonymity of the data gathered were guaranteed at all times. The trial lasted 35 min once the instructions had been given and any queries resolved. Once the questionnaire had been returned, the students’ results were used to study the construct’s validity through an exploratory factor analysis (EFA) [51] with a view to detecting an underlying structure in the set of variables analysed [86]. An EFA first requires analysing the normality of the sample distribution by calculating the coefficients of skewness and kurtosis [51] taken as confidence interval in both cases (−1–1) [87]. The variables scoring outside this interval are discarded from the test. This was followed by a statistical analysis of Bartlett’s test of sphericity and the KMO (Kaiser-Meyer-Olkin) sampling test for verifying the data’s adequacy for the EFA [50,51,52,88]. A sample is considered to be adequate for the application of an EFA when the KMO value >0.70 and the value provided by Bartlett’s test of sphericity is p < 0.001 [50,51,52,88]. The next step involved an EFA with a principal components extraction method and a varimax rotation that enabled us to remove those variables whose saturation in the factor was lower than 0.40 [89]. The result of the EFA prompted the creation of a series of factors, considering solely those with an eigenvalue ≥1 [51,90,91] and taking into account other aspects such as the interpretability of the solution found and the initial theory [51] (p. 1161). In turn, the reliability analysis involved Cronbach’s alpha [92]. All the statistical calculations involved the IBM SPSS Statistics 26 package, except for those tests related to Aiken’s V coefficient and its confidence interval, which were calculated with Microsoft Excel.
3. Results 3.1. Content Validity and Reliability
Content validity was calculated according to two key processes: the panel of expert judges [31,41] and the trial run involving a small group of subjects (n = 16). The expert judges analysed and rated each one of the items in the measuring instrument, as well as the overall questionnaire. The five judges concurred on the comments made about the questionnaire’s first block referring to sociodemographic data, and did not request any changes to the variables in it. As regards the variables in the questionnaire’s second block, which dealt with the students’ knowledge of the landscape, the judges assessed the items by rating their consistency, relevance, and clarity from 1 to 10 points. They also added any comments they wished to make. The assessment of the variables was followed by the calculation of Aiken’s V in order to discard those items that scored below the critical value established (>0.70) [80] (see Table 1). In this case, items 3, 19, 21, 30, 35, 40, 44, 47, and 53 were removed because they did not comply with the critical value and recorded an Aiken’s V of less than 0.70, and would not therefore be included in the validation process involving 16 students or in the final questionnaire.
In qualitative terms, the judges provided a number of comments on certain variables. One such comment that was common to all the experts involved the item referring to the definition of landscape proposed by the ELC. This definition initially appeared in only one item. The experts recommended dividing it into two new variables, which meant that instead of having one item that read ‘The landscape is any part of the territory people perceive, whose character is the outcome of the action and interaction of natural and/or human factors’, there were two items that were worded as follows: The landscape is any part of the territory that people perceive. The landscape is the outcome of the action and interaction of natural and/or human factors. This simplified the variable, rendering it easier to understand. The experts also unanimously agreed that it would be more appropriate to modify the way of answering the item on the students’ familiarity with the ELC, replacing a five-point Likert-type scale with a Yes/No dichotomous variable.
In qualitative terms, the questionnaire’s technical structure and its content were also validated. In this case, a small sample of subjects (n = 16) with the same characteristics as the subjects that would later be administered the final questionnaire, studied its features, and completed it. This helped to quantify the estimated time needed to carry out the task [43] and uncover any issues involved in interpreting the variables’ meaning [84]. The average time required for completing the questionnaire wasx¯= 28.31 min (SD = 2.99), and no comments were made on the variables’ wording, whereby it was understood that there was no issue with the way the questionnaire was drafted. The subjects’ assessment indicated that the approximate time for satisfactorily undertaking the task was around 35 min.
The reliability analysis involved the calculation of overall internal consistency via the Cronbach’s alpha coefficient [48], with the result being 0.860. It may therefore be concluded that the questionnaire has a high degree of internal consistency [93,94] as it recorded a coefficient of close to 1, which is the value considered to indicate an ideal correlation [48,93].
3.2. Construct Validity
Following the ratings provided by the experts and their comments, the next step was to administer a pilot test to 432 undergraduates studying for a Degree in Primary School Teaching at Salamanca University. Firstly, following the indications provided by Ferrando and Anguiano-Carrasco; Lloret-Segura, Ferreres-Traver, Hernández-Baeza, Tomás-Marco; Morales, Urosa, and Blanco [50,51,88] the sample size was deemed suitable for analysing the instrument’s technical quality. A series of statistical calculations were made, with their results being reported in the following paragraphs.
The statistical treatment began with an analysis of kurtosis and skewness to verify whether each variable’s data approached a normal distribution. Considering the values forthcoming to be valid, in terms of both kurtosis and skewness, between −1 and 1 [50], those values scoring outside this interval were discarded to avoid any errors in the EFA. This led to the removal of eighteen variables that did not comply with a normal distribution. All the variables were analysed to identify the problem they had, finding that these variables alluded to highly specific theoretical issues on the landscape or to banal matters that did not add anything to the study, whereby their removal had no impact on the final data on landscape perception.
The verification of the normality of the distribution was followed by an analysis of the pertinence of the factorial test. This involved calculating the KMO mean and Bartlett’s test of sphericity, with results of 0.814 (KMO) and a significance of p < 0.001 (Bartlett) (see Table 2). The results were satisfactory in both cases for applying an EFA [50,88].
An EFA was conducted on the remaining variables that revealed six factors that explain 50.803% of the variance [95]. All those variables that did not reach a minimum coefficient of 0.40 in the assigned factor were removed from the questionnaire [89], thereby providing twenty variables distributed across the six factors (see Table 3 and Table 4).
Five factors had at least three variables with saturations of over 0.40, whereby we could assume that we had enough items to define them appropriately [88,91,96]. By contrast, one of the factors had only two variables, which meant it was not fully defined, even though both recorded saturations >.80. These variables were not removed from the questionnaire because the coefficients they saturated with exceeded 0.50 [88,91] and signalled a relationship between them. In turn, this could be creating subconstructs or factors that were part of other factors [97], which might be combining with each other. This case involved two variables used to put forward a definition of landscape with its theoretical grounding being the definition provided by the ELC. It was therefore deemed necessary to include new definitions of the term landscape in order to create a construct that was more robustly defined. This would require the use of a sample of subjects N > 500 and conducting a confirmatory factor analysis (CFA) to confirm that the variables centred on the definition of landscape were grouped into the same factor.
Factor number 1 consisted of four variables in all, recording an eigenvalue of 3.915 and explaining 19.573% of the variance. It involved items 51, 57, 58, and 59, all of which addressed theoretical aspects of the possibilities the landscape provides as a teaching instrument. In our case, this factor was referred to as: The landscape as a didactic instrument. In turn, the second factor defined was called Qualities of the landscape and teaching practice. This factor had an eigenvalue of 1.613 and explained 8.065% of the variance. It consisted of five items (items 29, 36, 48, 55, and 62), which addressed theoretical issues involving the landscape’s qualities and the part it plays as content in the teaching–learning process. Factor 3 had an eigenvalue of 1.377 and explained 6.886% of the variance, grouping three items (items 8, 38 and 52). It was referred to as Anthropic features in teaching the landscape and consisted of a series of variables designed to define the presence of cultural aspects in the dynamics of the landscape and in its teaching. Factor number 4 explained 5.855% of the variance, in total grouping three items (items 17, 18 and 31), with an eigen value of 1.171. This factor theoretically explained socialisation in the landscape teaching–learning process. It grouped variables that measured aspects centred on the social construct and the perception of the landscape, and was referred to as Social context and landscape. Factor 5 involved the definition of landscape, recording an eigenvalue of 1.090 and explaining 5.449% of the variance. It consisted of two items (items 1 and 2). In theoretical terms, it grouped those variables that involved the definition of the concept of landscape, and was referred to as Concept of landscape. As noted earlier, there was a need to define two new variables focusing on the definition of landscape that helped to provide a more robust definition of this factor’s characteristics [91]. The last of these factors had an eigenvalue of 1.001 and explained 5.003% of the variance. It was referred to as Interpreting and teaching the landscape. It focused on issues involving the positive or negative feelings the landscape arouses when it is contemplated and how this may have an impact on the teaching process. It comprised items 4, 41, and 61.
3.3. Final Questionnaire on Landscape Perception: Cuestionario Sobre Percepción del Paisaje-CPP
The various processes involved in validating the questionnaire on the perception of the landscape, of both a qualitative and quantitative nature, culminated in the design of the Cuestionario sobre Percepción del Paisaje—CPP (see Table 5). The use of EFA allowed reducing the number of variables, as well as analysing the internal and underlying structure grouping the variables [51,86,88]. This ultimately led to the design of a reliable questionnaire that can be used to interpret [51] the perception that students have of the landscape during their education. The final questionnaire consists of two blocks; a first one involves the compiling of sociodemographic data, and the second one focuses on the students’ perception of the landscape as theoretical content and a didactic instrument in the teaching–learning process. The first block is unchanged from the way it has been presented in the second section of this paper. By contrast, the second block has undergone substantial changes that have informed a questionnaire made up in total of 21 variables (20 involving a Likert-type scale and one requiring a dichotomous yes/no answer). This has led to a reduction in the number of variables, discarding those that made no significant contribution to the study and those that could distort the final results.
4. Discussion and Conclusions
Following the ELC’s postulates, the signatory countries are to put in place measures for managing and organising the landscape and educating in this subject [6,7,8]. As far as Spain is specifically concerned, schemes have been rolled out to include the population at large in these processes [5] and the landscape has a noticeably greater presence as curricular content in basic education [13]. This last aspect, education and the landscape’s presence in the teaching–learning process, is the one that has focused our attention here. According to Pedroli and Van Mansvelt; García; Fernández [8,13,14], the landscape currently serves a dual process in this process; on the one hand, it continues to be a didactic instrument in the teaching of geography, and on the other, it is another content in the general syllabus. This means there is a need to analyse the perception that prospective teachers have of the landscape, as they will be the ones to teach it. This article therefore addresses the design and validation of a questionnaire for identifying this perception. The results obtained after administering this questionnaire to a sample of the population of undergraduates studying for their Degree in Primary School Teaching at Salamanca University support this instrument’s reliability and validity. Yet at the same time, we should also proceed with caution, for according to Álvarez-García et al.; Arribas [84,98], questionnaires may contain biases when the subjects surveyed reply to the items in what they consider to be the most appropriate manner and according to their prior responses, without expressing their true opinion. Hence the need for subsequent studies to include qualitative analyses that assess the subjects’ answers and compare the results involving different cohorts [26,48].
Based on a proposal made by Cohen, Manion, and Morrison [26] for the design and construction of a questionnaire, this research has conducted a literature review on theoretical aspects of the landscape and its teaching. This has been used to define a series of variables and construct an initial questionnaire made up of 70 items involving a Likert-type assessment scale. This has been followed by a quantitative and qualitative content validity analysis conducted by a panel of expert judges [28,31,41] and a brief trial run involving a small group of students [42,48]. This initial validation process has led to the removal of the nine variables that the judges have not deemed suitable for the study according to the scores they have given them, and the average time for completing the questionnaire has been established. The instrument’s overall reliability analysed by means of Cronbach’s alpha has recorded a result of 0.860. This indicates a high overall internal consistency [48,93,94]. After confirming the sample’s suitability for an EFA involving an analysis of the normality of the distribution and calculation of the KMO statistic (0.814) and Bartlett’s test of sphericity (p < 0.001), a further eighteen items were removed from the questionnaire, as they did not record a normal distribution and could be problematic when conducting the EFA. Their analysis and individual assessment have revealed that they all involve theoretical matters that might be challenging for students studying subjects other than geography. We should not forget that the sample consists of students studying for their Degree in Primary School Teaching.
The EFA provided an initial view of the structure underpinning the variables [51,52], which in this case have been grouped into six factors in total. The literature review and the ELC as a reference were used to design an initial questionnaire that in theoretical terms was arranged into four dimensions (Theoretical understanding of the landscape; Landscape and a sense of identity; Measures for the management and protection of the landscape; and Pedagogical knowledge) and 70 variables. The EFA has indicated that there are six constructs (The landscape as a didactic instrument; Qualities of the landscape and teaching practice; Anthropic features in teaching the landscape; Social context and landscape; Concept of landscape; and Interpreting and teaching the landscape) and has helped us to reduce the number of variables to 21. According to Frías-Navarro and Pascual; Lloret-Segura et al.; Pérez and Medrano [51,52,97], the EFA is also a sound technique for reducing the number of variables and avoiding overly long questionnaires that may make the subjects in the sample lose interest. In addition, it provides the guidelines for validating a measuring instrument of proven reliability, although it requires a CFA calculated on the basis of a matrix of data gathered from subjects other than those involved in the pilot task and with a larger sample according to Kline [99] of N > 500. The CFA is used to confirm whether the same dimensions are obtained as in other cohorts of the same characteristics, as we now have a starting hypothesis in which we can set the number of factors to be extracted [100,101]. According to Batista-Foguet, Coenders, and Alonso; Fernández; Herrero; Urrutia et al. [102,103,104,105], this analysis will allow confirming the dimensions and underlying characteristics of the questionnaire’s variables and suitably concluding its validation process.
In short, this research has prompted the design and validation of a questionnaire for analysing the perception of the landscape amongst undergraduates studying for a Degree in Primary School Teaching at Salamanca University: Cuestionario sobre Percepción del Paisaje-CPP. In the future, this analysis must be confirmed with a CFA certifying the results obtained with the EFA.
![View Image - Figure 1. Model of scorecard for the judges' assessment of the items on the questionnaire. Source: compiled according to Backhoff, Aguilar, and Larrazolo; García and Cabero; Martín-Romera and Molina [36,43,44].](https://media.proquest.com/media/hms/THUM/2/AbWVI?_a=ChgyMDI1MDkxMDE0MjcxMzk4Nzo3NTA1NDASBzEzODMzMjMaCk9ORV9TRUFSQ0giDjIxNi43My4yMTYuMTAzKgcyMDMyNDA1MgoyNTAxMzA3NzI4OgtJbmxpbmVJbWFnZUIBMFIGT25saW5lWgNJTEliBFRIVU1qCjIwMjEvMDEvMDFyCjIwMjEvMTIvMzF6AIIBIFAtMTAwMDA4NC0yMDEzOTUtU1RBRkYtbnVsbC1udWxskgEGT25saW5lygFnTW96aWxsYS81LjAgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbzsgY29tcGF0aWJsZTsgQ2xhdWRlQm90LzEuMDsgK2NsYXVkZWJvdEBhbnRocm9waWMuY29tKdIBElNjaG9sYXJseSBKb3VybmFsc5oCB1ByZVBhaWSqAhtPUzpFTVMtRnVsbFRleHQtZ2V0RnVsbFRleHTKAg9BcnRpY2xlfEZlYXR1cmXiAgFO8gIA%2BgIBWYIDA1dlYooDHENJRDoyMDI1MDkxMDE0MjcxMzk4NzoyMzk2ODM%3D&_s=2SURZyLSSJfznqPR%2FQ5kEAdqFdQ%3D "View Image - Figure 1. Model of scorecard for the judges' assessment of the items on the questionnaire. Source: compiled according to Backhoff, Aguilar, and Larrazolo; García and Cabero; Martín-Romera and Molina [36,43,44].")
Enlarge this image.Figure 1. Model of scorecard for the judges' assessment of the items on the questionnaire. Source: compiled according to Backhoff, Aguilar, and Larrazolo; García and Cabero; Martín-Romera and Molina [36,43,44].
![View Image - Figure 3. Diagram of the methodological process for the design and validation of the Questionnaire on Landscape Perception (CPP) based on the proposal made by Cohen, Manion, and Morrison. Source: adapted from Cohen et al. [26] (p. 472).](https://media.proquest.com/media/hms/THUM/6/AbWVI?_a=ChgyMDI1MDkxMDE0MjcxMzk4Nzo3NTA1NDASBzEzODMzMjMaCk9ORV9TRUFSQ0giDjIxNi43My4yMTYuMTAzKgcyMDMyNDA1MgoyNTAxMzA3NzI4OgtJbmxpbmVJbWFnZUIBMFIGT25saW5lWgNJTEliBFRIVU1qCjIwMjEvMDEvMDFyCjIwMjEvMTIvMzF6AIIBIFAtMTAwMDA4NC0yMDEzOTUtU1RBRkYtbnVsbC1udWxskgEGT25saW5lygFnTW96aWxsYS81LjAgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbzsgY29tcGF0aWJsZTsgQ2xhdWRlQm90LzEuMDsgK2NsYXVkZWJvdEBhbnRocm9waWMuY29tKdIBElNjaG9sYXJseSBKb3VybmFsc5oCB1ByZVBhaWSqAhtPUzpFTVMtRnVsbFRleHQtZ2V0RnVsbFRleHTKAg9BcnRpY2xlfEZlYXR1cmXiAgFO8gIA%2BgIBWYIDA1dlYooDHENJRDoyMDI1MDkxMDE0MjcxMzk4NzoyMzk2ODM%3D&_s=65Tu1I0gZLxoe0ZWlEcneMg3MjI%3D "View Image - Figure 3. Diagram of the methodological process for the design and validation of the Questionnaire on Landscape Perception (CPP) based on the proposal made by Cohen, Manion, and Morrison. Source: adapted from Cohen et al. [26] (p. 472).")
Enlarge this image.Figure 3. Diagram of the methodological process for the design and validation of the Questionnaire on Landscape Perception (CPP) based on the proposal made by Cohen, Manion, and Morrison. Source: adapted from Cohen et al. [26] (p. 472).
| Items | x¯ | Aiken’s V | CI (95%) | Items | x¯ | Aiken’s V | CI (95%) |
|---|---|---|---|---|---|---|---|
| Item 1 | 8.4 | 0.74 | 0.59–0.85 | Item 36 | 8.8 | 0.78 | 0.63–0.88 |
| Item 2 | 8.4 | 0.74 | 0.59–0.85 | Item 37 | 9 | 0.8 | 0.65–0.89 |
| Item 3 | 5 | 0.40 | 0.25–0.55 | Item 38 | 9.4 | 0.84 | 0.70–0.92 |
| Item 4 | 9.2 | 0.82 | 0.67–0.91 | Item 39 | 9 | 0.8 | 0.65–0.89 |
| Item 5 | 9.2 | 0.82 | 0.67–0.91 | Item 40 | 5.6 | 0.46 | 0.32–0.61 |
| Item 6 | 9.8 | 0.88 | 0.74–0.95 | Item 41 | 9.8 | 0.88 | 0.74–0.95 |
| Item 7 | 9 | 0.8 | 0.65–0.89 | Item 42 | 9.4 | 0.84 | 0.70–0.92 |
| Item 8 | 8.8 | 0.78 | 0.63–0.88 | Item 43 | 9.2 | 0.82 | 0.67–0.91 |
| Item 9 | 9.4 | 0.84 | 0.70–0.92 | Item 44 | 6.2 | 0.52 | 0.37–0.67 |
| Item 10 | 9.2 | 0.82 | 0.67–0.91 | Item 45 | 9 | 0.8 | 0.65–0.89 |
| Item 11 | 9.8 | 0.88 | 0.74–0.75 | Item 46 | 9.4 | 0.84 | 0.70–0.92 |
| Item 12 | 8.8 | 0.78 | 0.63–0.88 | Item 47 | 6.4 | 0.54 | 0.39–0.68 |
| Item 13 | 9.6 | 0.86 | 0.72–0.94 | Item 48 | 9 | 0.8 | 0.65–0.89 |
| Item 14 | 10 | 0.9 | 0.77–0.96 | Item 49 | 9.8 | 0.88 | 0.74–0.95 |
| Item 15 | 10 | 0.9 | 0.77–0.96 | Item 50 | 9.4 | 0.84 | 0.70–0.92 |
| Item 16 | 9.8 | 0.88 | 0.74–0.95 | Item 51 | 9 | 0.8 | 0.65–0.89 |
| Item 17 | 10 | 0.9 | 0.77–0.96 | Item 52 | 8.6 | 0.76 | 0.61–0.87 |
| Item 18 | 9.2 | 0.82 | 0.67–0.91 | Item 53 | 3 | 0.2 | 0.10–0.35 |
| Item 19 | 4 | 0.3 | 0.18–0.45 | Item 54 | 8.6 | 0.76 | 0.61–0.87 |
| Item 20 | 8.8 | 0.78 | 0.63–0.88 | Item 55 | 9.4 | 0.84 | 0.70–0.92 |
| Item 21 | 5.8 | 0.48 | 0.33–0.63 | Item 56 | 9 | 0.8 | 0.65–0.89 |
| Item 22 | 9 | 0.8 | 0.65–0.89 | Item 57 | 9.4 | 0.84 | 0.70–0.92 |
| Item 23 | 8.2 | 0.72 | 0.57–0.84 | Item 58 | 9.2 | 0.82 | 0.67–0.91 |
| Item 24 | 9 | 0.8 | 0.65–0.89 | Item 59 | 9.4 | 0.84 | 0.70–0.92 |
| Item 25 | 9.6 | 0.86 | 0.72–0.94 | Item 60 | 9.4 | 0.84 | 0.70–0.92 |
| Item 26 | 8.2 | 0.72 | 0.57–0.84 | Item 61 | 8.8 | 0.78 | 0.63–0.88 |
| Item 27 | 9.4 | 0.84 | 0.70–0.92 | Item 62 | 8.6 | 0.76 | 0.61–0.87 |
| Item 28 | 9.6 | 0.86 | 0.72–0.94 | Item 63 | 10 | 0.9 | 0.77–0.96 |
| Item 29 | 9 | 0.8 | 0.65–0.89 | Item 64 | 9.2 | 0.82 | 0.67–0.91 |
| Item 30 | 4.8 | 0.38 | 0.25–0.53 | Item 65 | 9.4 | 0.84 | 0.70–0.92 |
| Item 31 | 9.8 | 0.88 | 0.74–0.95 | Item 66 | 9.2 | 0.82 | 0.67–0.91 |
| Item 32 | 8.8 | 0.78 | 0.63–0.88 | Item 67 | 9.4 | 0.84 | 0.70–0.92 |
| Item 33 | 9.8 | 0.88 | 0.74–0.98 | Item 68 | 9.8 | 0.88 | 0.74–0.95 |
| Item 34 | 9 | 0.8 | 0.65–0.89 | Item 69 | 9.4 | 0.84 | 0.70–0.92 |
| Item 35 | 5.8 | 0.48 | 0.33–0.63 | Item 70 | 9.4 | 0.84 | 0.70–0.92 |
| KMO and Bartlett Tests | ||
| Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy | 0.814 | |
| Bartlett’s test of sphericity | Approx. Chi-squared | 1719.359 |
| df | 190 | |
| Signif. | 0.000 | |
| Total Variance Explained | |||||||||
| Component | Initial Eigenvalues | Extraction Sums of Squared Loadings | Rotation Sums of Squared Loadings | ||||||
| Total | % of Variance | Cumulative % | Total | % of Variance | Cumulative % | Total | % of Variance | Cumulative% | |
| 1 | 3.915 | 19.573 | 19.573 | 3.915 | 19.573 | 19.573 | 2.450 | 12.248 | 12.248 |
| 2 | 1.613 | 8.065 | 27.638 | 1.613 | 8.065 | 27.638 | 1.996 | 9.980 | 22.228 |
| 3 | 1.377 | 6.886 | 34.524 | 1.377 | 6.886 | 34.524 | 1.595 | 7.977 | 30.205 |
| 4 | 1.171 | 5.855 | 40.379 | 1.171 | 5.855 | 40.379 | 1.436 | 7.179 | 37.383 |
| 5 | 1.090 | 5.449 | 45.828 | 1.090 | 5.449 | 45.828 | 1.389 | 6.946 | 44.330 |
| 6 | 1.001 | 5.003 | 50.830 | 1.001 | 5.003 | 50.830 | 1.300 | 6.500 | 50.830 |
| 7 | 0.962 | 4.810 | 55.640 | ||||||
| 8 | 0.916 | 4.578 | 60.218 | ||||||
| 9 | 0.906 | 4.530 | 64.748 | ||||||
| 10 | 0.816 | 4.082 | 68.830 | ||||||
| 11 | 0.771 | 3.856 | 72.686 | ||||||
| 12 | 0.737 | 3.684 | 76.370 | ||||||
| 13 | 0.720 | 3.598 | 79.968 | ||||||
| 14 | 0.682 | 3.409 | 83.377 | ||||||
| 15 | 0.645 | 3.226 | 86.603 | ||||||
| 16 | 0.613 | 3.067 | 89.670 | ||||||
| 17 | 0.570 | 2.850 | 92.520 | ||||||
| 18 | 0.542 | 2.711 | 95.230 | ||||||
| 19 | 0.497 | 2.485 | 97.715 | ||||||
| 20 | 0.457 | 2.285 | 100.000 | ||||||
| Extraction method: principal component analysis. | |||||||||
| Rotated Component Matrix a | ||||||
| Component | ||||||
| 1 | 2 | 3 | 4 | 5 | 6 | |
| Item57 | 0.767 | |||||
| Item58 | 0.630 | |||||
| Item59 | 0.522 | |||||
| Item51 | 0.480 | |||||
| Item36 | 0.665 | |||||
| Item55 | 0.587 | |||||
| Item48 | 0.517 | |||||
| Item62 | 0.490 | |||||
| Item29 | 0.442 | |||||
| Item38 | 0.713 | |||||
| Item52 | 0.693 | |||||
| Item8 | 0.403 | |||||
| Item17 | 0.846 | |||||
| Item31 | 0.450 | |||||
| Item18 | 0.439 | |||||
| Item1 | 0.804 | |||||
| Item2 | 0.802 | |||||
| Item4 | 0.716 | |||||
| Item41 | 0.463 | |||||
| Item61 | −0.447 | |||||
Extraction method: principal component analysis. Rotation method: Varimax with Kaiser normalisation. a. Rotation converged in 12 iterations.
| Number of the Item in the Questionnaire Prior to Validation | Number in Final Questionnaire | Variable | Likert (1 to 5) |
| Item 0 | Item 1 | Are you familiar with the European Landscape Convention? | YES/NO |
| Item 1 | Item 2 | The landscape is any part of the territory that people perceive | 1 to 5 |
| Item 2 | Item 3 | The landscape is everything that can be seen from a given place | 1 to 5 |
| Item 29 | Item 4 | The landscape needs to be taught through fieldwork or day trips | 1 to 5 |
| Item 36 | Item 5 | The landscape should be taught as a mainstream subject in primary education | 1 to 5 |
| Item 48 | Item 6 | The landscape is a dynamic environment | 1 to 5 |
| Item 62 | Item 7 | The landscape is part of the syllabus in primary education | 1 to 5 |
| Item 38 | Item 8 | The landscape should be taught through a textbook | 1 to 5 |
| Item 52 | Item 9 | The cultural landscape is made up of human features | 1 to 5 |
| Item 8 | Item 10 | The cultural landscape is more important than the natural landscape | 1 to 5 |
| Item 17 | Item 11 | The social context we live in conditions the way we study the landscape | 1 to 5 |
| Item 31 | Item 12 | Pupils should know how to identify anthropic features in order to interpret the landscape | 1 to 5 |
| Item 18 | Item 13 | A combination of photographs and fieldwork is useful for studying the landscape’s social features | 1 to 5 |
| Item 4 | Item 14 | Contemplating any kind of landscape generates positive feelings | 1 to 5 |
| Item 41 | Item 15 | Simulation techniques enable me to understand how the landscape and the feelings it conveys change | 1 to 5 |
| Item 61 | Item 16 | There is a need to further our understanding of the landscape because it is a feature that arouses positive feelings | 1 to 5 |
| Item 57 | Item 17 | The city helps me to understand the urban landscape | 1 to 5 |
| Item 58 | Item 18 | The landscape helps me to teach about rural life | 1 to 5 |
| Item 59 | Item 19 | The main reason for studying the landscape is to provide pupils with tools for understanding geography | 1 to 5 |
| Item 51 | Item 20 | Augmented reality is very useful for teaching about the landscape | 1 to 5 |
| Item 55 | Item 21 | The landscape is a didactic instrument | 1 to 5 |
Author Contributions
Conceptualization, R.F.Á.; methodology, R.F.Á.; software, R.F.Á. and J.F.; formal analysis, R.F.Á. and J.F.; investigation, R.F.Á. and J.F.; resources, R.F.Á.; data curation, J.F.; writing-original draft preparation, R.F.Á.; writing-review and editing, R.F.Á.; visualization, R.F.Á.; supervision, R.F.Á.; project administration, R.F.Á.; funding acquisition, R.F.Á. All authors have read and agreed to the published version of the manuscript.
Funding
This research was supported by the University of Salamanca under grant OMEU (Observatorio de la Marca España en Europa) and Ministerio de Ciencia, Innovación y Universidades under grant CSO2017-84623-R La Realidad Aumentada como herramienta para explicación del paisaje. Aplicaciones a la docencia y al turismo.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
1. Fernández, R. La aplicación de Landscape Character Assessment a los espacios de montaña media: El paisaje del macizo de Las Villuercas. Ciudad. Territ. Estud. Territ. 2015, 185, 499-518.
2. Nogué, J.; Puigbert, L.; Sala, P.; Bretcha, G. Landscape and Public Participation. The Experience of the Landscape Catalogues of Catalonia; Catalan Landscape Observatory and Regional Government of Catalunya: Barcelona, Spain, 2010.
3. Ortega, M. El Convenio Europeo del Paisaje: Claves para un compromiso. Ambienta 2007, 63, 18-26.
4. Zoido, F. El paisaje, patrimonio público y recurso para la mejora de la democracia. Boletín Inst. Andal. Patrim. Hist. 2004, 50, 66-73.
5. Fernández, R.; Plaza, J.I. Participación ciudadana y educación en materia de paisaje en el marco del Convenio Europeo del Paisaje en España. Cuad. Geogr. 2019, 58, 262-286.
6. Prieur, M.; Durousseau, S. Landscape and Public participation. In Landscape and Sustainable Development: Challenges of the European Landscape Convention; Prieur, M., Luginbühl, Y., Zoido, F., De Montmollin, B., Pedroli, B., Van Mansvelt, J.D., Durousseau, S., Eds.; Council of Europe: Strasbourg, France, 2006; pp. 165-208.
7. Zoido, F. Landscape and spatial planning policies. In Landscape and Sustainable Development: Challenges of the European Landscape Convention; Prieur, M., Luginbühl, Y., Zoido, F., De Montmollin, B., Pedroli, B., Van Mansvelt, J.D., Durousseau, S., Eds.; Council of Europe: Strasbourg, France, 2006; pp. 52-88.
8. Pedroli, B.; Van Mansvelt, J.D. Landscape and awareness-raising, training and education. In Landscape and Sustainable Development: Challenges of the European Landscape Convention; Prieur, M., Luginbühl, Y., Zoido, F., De Montmollin, B., Pedroli, B., Van Mansvelt, J.D., Durousseau, S., Eds.; Council of Europe: Strasbourg, France, 2006; pp. 119-142.
9. Casas, M.; Puig, J.; Erneta, L. El paisaje en el contexto curricular de la LOMCE: Una oportunidad educativa, ¿aprovechada o desaprovechada? Didact. Geogr. 2017, 18, 39-68.
10. Delgado, E. El paisaje en la formación de maestros, un recurso educativo de alto interés para la Educación Primaria. Tabanque Rev. Pedagog. 2015, 28, 117-138.
11. García, A. Un enfoque innovador en didáctica del paisaje: Escenario y secuencia geográfica. In Innovación en la Enseñanza de la Geografía ante los Desafíos Sociales y Territoriales; De Miguel, R., De Lázaro, M.L., Marrón, M.J., Eds.; Instituto Fernando el Católico: Zaragoza, Spain, 2013; pp. 257-277.
12. Tort, J. El paisaje como pedagogía del territorio. Didact. Geogr. 2004, 6, 133-153.
13. García, A. El paisaje: Un desafío curricular y didáctico. Rev. Didact. Especificas 2018, 4, 7-26.
14. Fernández, R. La enseñanza del paisaje desde una concepción constructivista: Propuesta didáctica. Dedica. Rev. Educ. Hum. 2019, 15, 135-159.
15. BOE (Boletín Oficial del Estado-Spain's Offocial State Gazette). Royal Decree 126/2014, of 28 de February, establishing the basic syllabus for Primary Education, 2014. Available online: https://www.boe.es/buscar/pdf/2014/BOE-A-2014-2222-consolidado.pdf (accessed on 1 July 2020).
16. Busquets, J. La educación en paisaje. Iber. Didact. Cienc. Soc. Geogr. Hist. 2010, 65, 7-17.
17. Nardi, A. El paisaje como instrumento de intermediación cultural en la escuela. Iber. Didactica Cienc. Soc. Geogr. Hist. 2010, 65, 25-37.
18. Domínguez, A.; López, R. Patrimonio, paisaje y educación: Formación inicial del profesorado y educación cívica del alumnado de primaria. Clio. Hist. Hist. Teach. 2014, 40, 1-26.
19. Gómez-Zotano, J.; Riesco-Chueca, P. Landscape learning and teaching: Innovations in the context of The European Landscape Convention. In Proceedings of the INTED2010 Conference, Valencia, Spain, 8-10 March 2010; pp. 1-13.
20. Zanato, O. Lo sguardo sul paesaggio da una prospettiva padagógico-ambientale. In Il Paesaggio Vicino a Noi. Educazione, Consapevolezza, Responsabilità; Castiglioni, B., Celi, M., Gamberoni, E., Eds.; Museo Civico di Storia Naturale e Archeologia: Montebelluna, Italy, 2007.
21. Santana, D.; Morales, A.J.; Souto, X.M. Las representaciones sociales del paisaje en los trabajos de campo con Educación Primaria. In Nuevas Perspectivas Conceptuales y Metodológocas para la Educación Geográfica; Martínez, R., Tonda, E.M., Eds.; Asociación Española de Geografía: Córdoba, Spain, 2014; pp. 167-182.
22. Castiglioni, B. Education on landscape: Theoretical and practical approaches in the frame of the European Landscape Convention. In Geographical View on Education for Sustainable Development; Reinfried, S., Schleicher, Y., Rempfler, A., Eds.; Geographiedidaktische Forschungen: Lucerne, Switzerland, 2007.
23. McMahon, M.; Pospisil, R. Laptops for a digital lifestyle: Millennial students and wireless mobile technologies. Available online: https://www.researchgate.net/publication/49280225_Laptops_for_a_Digital_Lifestyle_Millennial_Students_and_Wireless_Mobile_Technologies/stats (accessed on 8 September 2020).
24. Cabero, J.; Barroso, J. The educational possibilities of Augmented Reality. New Approaches Educ. Res. 2016, 5, 44-50.
25. Prensky, M. Digital Natives, Digital Immigrants. Horizon 2001, 9, 1-20.
26. Cohen, L.; Manion, L.; Morrison, K. Research Methods in Education; Routledge: New York, NY, USA, 2011.
27. Torrado, M. Estudios de Encuesta. In Metodología de la Investigación Educativa, 5th ed.; Bisquerra, R., Ed.; La Muralla: Madrid, Spain, 2016; pp. 223-249.
28. Ayuga, E.; González, C.; Ortiz, M.A.; Martínez, E. Diseño de un cuestionario para evaluar conocimientos básicos de estadística de estudiantes del último curso de ingeniería. Form. Univ. 2012, 5, 21-32.
29. Cabero, J.; Llorente, M.C. La aplicación del juicio de experto como técnica de evaluación de las tecnologías de la información (TIC). Eduweb. Rev. Tecnol. Inf. Comun. Educ. 2013, 7, 11-22.
30. Ding, C.; Hershberger, S. Assessing content validity and content equivalence using structural equation modelling. Struct. Equ. Modeling Multidiscip. J. 2002, 9, 283-297.
31. Escurra, L.M. Cuantificación de la validez de contenido por criterio de jueces. Rev. Psicol. 1988, 6, 103-111.
32. Garrido, M.; Romero, S.; Ortega, E.; Zagalaz, M. Diseño de un cuestionario para niños sobre los padres y madres en el deporte (CHOPMD). J. Sport Health Res. 2011, 3, 153-164.
33. Oloruntegbe, K.O.; Zamri, S.; Saat, R.M.; Alam, G.M. Development and validation of measuring instruments of contextualization of science among Malaysian and Nigerian serving and preservice chemistry teachers. Int. J. Phys. Sci. 2010, 5, 2075-2083.
34. Escobar-Pérez, J.; Cuervo-Martínez, A. Validez de contenido y juicio de expertos: Una aproximación a su utilización. Av. En Med. 2008, 6, 27-36.
35. Utkin, L.V. A method for processing the unreliable expert judgments about parameters of probability distributions. Eur. J. Oper. Res. 2005, 175, 385-398.
36. Backhoff, E.; Aguilar, J.; Larrazolo, N. Metodología para la validación de contenidos de exámenes normativos. Rev. Mex. Psicol. 2006, 23, 79-86.
37. Delgado-Rico, E.; Carretero-Dios, H.; Ruch, W. Content validity evidences in test development: An applied perspective. Intern. J. Clin. Health Psych. 2012, 31, 67-74.
38. Gable, R.K.; Wolf, J.K. Instrument Development in the Affective Domain: Measuring Attitudes and Values in Corporate and School Settings; Kluwer Academic: Boston, MA, USA, 1993.
39. Grant, J.S.; Davis, L.L. Selection and use of content experts for instrument development. Res. Nurs. Health 1997, 20, 269-274.
40. Lynn, M. Determination and quantification of content validity. Nurs. Res. 1986, 35, 382-385.
41. McGartland, D.; Berg-Weger, M.; Tebb, S.S.; Lee, E.S.; Rauch, S. Objectifying content validity: Conducting a content validity study in social work research. Soc. Work Res. J. 2003, 27, 94-104.
42. Chacón, S.; Pérez-Gil, J.A.; Holgado, F.P.; Lara, A. Evaluación de la calidad universitaria: Validez de contenido. Psicothema 2001, 13, 294-301.
43. Martín-Romera, A.; Molina, E. Valor del conocimiento pedagógico para la docencia en Educación Secundaria: Diseño y validación de un cuestionario. Estud. Pedagog. 2017, 43, 195-220.
44. García, E.; Cabero, J. Diseño y valoración de un cuestionario dirigido a describir la evaluación en procesos de educación a distancia. Edutec-E 2011, 35, 1-26.
45. Merino-Barrero, J.; Valero-Valenzuela, A.; Moreno-Murcia, J. Análisis psicométrico del cuestionario estilos de enseñanza en Educación Física (EEEF). Rev. Int. Med. Cienc. Act. Fis. Deporte 2017, 17, 225-241.
46. Singh, K. Quantitative Social Research Methods; Sage Publications: London, UK, 2007.
47. Galán, M. Desarrollo y validación de contenido de la nueva versión de un instrumento para clasificación de pacientes. Rev. Latinoam. Enferm. 2011, 19, 1-9.
48. McMillan, J.H.; Schumacher, S. Investigación Educativa. Una Introducción Conceptual, 5th ed.; Pearson Educación: Madrid, Spain, 2011.
49. De Winter, J.C.F.; Dodou, D.; Wieringa, P.A. Exploratory factor analysis with small sample sizes. Multivar. Behav. Res. 2009, 44, 147-181.
50. Ferrando, P.J.; Anguiano-Carrasco, C. El análisis factorial como técnica de investigación en Psicología. Pap. Psicólogo 2010, 31, 18-33.
51. Lloret-Segura, S.; Ferreres-Traver, A.; Hernández-Baeza, A.; Tomás-Marco, I. El análisis factorial exploratorio de los ítems: Una guía práctica, revisada y actualizada. An. Psicol. 2014, 30, 1151-1169.
52. Frías-Navarro, D.; Pascual, M. Prácticas del análisis factorial exploratorio (AFE) en la investigación sobre la conducta del consumidor y marketing. Suma Psicol. 2012, 19, 45-58.
53. Joshi, A.; Kale, S.; Chandel, S.; Pal, D.K. Likert Scale: Explored and Explained. Br. J. Appl. Sci. Technol. 2015, 7, 396-403.
54. Zube, E.H.; Sell, J.L.; Taylor, L.G. Landscape perception: Research, application and theory. Landsc. Urban Plan. 1982, 9, 1-33.
55. Mata, R. Métodos de estudio del paisaje e instrumentos para su gestión. Consideraciones a partir de experiencias de planificación territorial. In El Paisaje y la Gestión del Territorio. Criterios Paisajísticos en la Ordenación del Territorio y el Urbanismo; Mata, R., Torroja, A., Eds.; Diputación (Provincial Council) de Barcelona: Barcelona, Spain, 2006; pp. 199-239.
56. Oliva, J.; Iso, A. Diseños metodológicos para la planificación participativa del paisaje. Empiria. Rev. Metodol. Cienc. Soc. 2014, 27, 95-120.
57. Serrano, D. Paisajes y políticas públicas. Investig. Geográficas 2007, 42, 109-123.
58. Zoido, F. El paisaje, ideas para la actuación. In Estudios Sobre el Paisaje; Martínez de Pisón, E., Ed.; Fundación Duques de Soria: Madrid, Spain, 2000; pp. 293-311.
59. Calcagno, A. Landscape and education. In Landscape Dimensions. Reflections and Proposals for the Implementation of the European Landscape Convention; Council of Europe: Strasbourg, France, 2017; pp. 55-119.
60. Beauchamp, E.; Clements, T.; Milner-Gulland, E.J. Investigating Perceptions of Land Issues in a Threatened Landscape in Northern Cambodia. Sustainability 2019, 11, 5881.
61. Cornwall, A. Locating citizen participation. Inst. Dev. Stud. Bull. 2002, 33, 9-19.
62. Liceras, A. Didáctica del paisaje. Didáctica Cienc. Soc. Geogr. Hist. 2013, 74, 85-93.
63. Peng, J.; Yan, S.; Strijker, D.; Wu, Q.; Chen, W.; Ma, Z. The influence of place identity on perceptions of landscape change: Exploring evidence from rural land consolidation projects in Eastern China. Land Use Policy 2020, 99.
64. Rodríguez, E.J.; Granados, C.S.T.; Santo-Tomás, R. Landscape perception in Peri-Urban Areas: An expert-based methodological approach. Landsc. Online 2019, 75, 1-22.
65. Yli-Panula, E.; Persson, C.; Jeronen, E.; Eloranta, V.; Pakula, H.M. Landscape as experienced place and worth conserving in the drawings of Finnish and Swedish students. Educ. Sci. 2019, 9, 93.
66. Council of Europe. Explanatory Report to the European Landscape Convention; Council of Europe: Florence, Italy, 2000.
67. Iurk, M.C.; Biondi, D.; Dlugosz, F.L. Perception, landscape and environmental education: An investigation with students from the municipality of Irati, state of Paraná, Brazil. Floresta 2018, 48, 143-152.
68. Batllorí, R.; Serra, J.M. D'ensenyar geografía a través del paisatge a educar en paisatge. Doc. Anal. Geogr. 2017, 63, 617-630.
69. García, A. Perspectivas de futuro en el aprendizaje del paisaje. Didact. Geogr. 2019, 20, 55-77.
70. Liceras, A. Observar e Interpretar el Paisaje. Estrategias Didácticas; Grupo Editorial Universitario: Granada, Spain, 2003.
71. Nevrelová, M.; Ružicková, J. Educational Potential of Educational Trails in Terms of Their Using in the Pedagogical Process (Outdoor Learning). Eur. J. Contemp. Educ. 2019, 8, 550-561.
72. García, A. El itinerario geográfico como recurso didáctico para la valoración del paisaje. Didact. Geogr. 2004, 6, 79-95.
73. Coll, C. Constructivismo y educación escolar: Ni hablamos siempre de los mismo ni lo hacemos siempre desde la misma perspectiva epistemológica. Anu. Psicol. 1996, 69, 153-178.
74. Díaz, F.; Hernández, G. Estrategias Docentes Para un Aprendizaje Significativo. Una Interpretación Constructivista; McGraw Hill: Mexico City, Mexico, 2002.
75. García, A. Aplicación didáctica del aprendizaje basado en problemas al análisis geográfico. Rev. Didact. Especificas 2010, 2, 43-60.
76. Solé, I.; Coll, C. Los profesores y la concepción constructivista. In El Constructivismo en el Aula; Coll, C., Mauri, T., Miras, M., Solé, I., Zabala, A., Eds.; Graó: Barcelona, Spain, 1993; pp. 7-24.
77. Zamora-Polo, F.; Corrales-Serrano, M.; Sánchez-Martín, J.; Espejo-Antúnez, L. Nonscientific University Students Training in General Science Using an Active-Learning Merged Pedagogy: Gamification in a Flipped Classroom. Educ. Sci. 2019, 9, 297.
78. Mateo, J. La Investigación "Ex-Post-Facto"; Universitat Oberta de Catalunya: Barcelona, Spain, 1997.
79. Aiken, L.R. Three coefficients for analyzing the reliability and validity of ratings. Educ. Psychol. Meas. 1985, 45, 131-142.
80. Charter, R.A. A Breakdown of reliability coefficients by test type and reliability method, and the clinical implications of low reliability. J. Gen. Psychol. 2003, 130, 290-304.
81. Penfield, R.D.; Giacobbi, P.R. Applying a score confidence interval to Aiken's item content relevance index. Meas. Phys. Educ. Exerc. Sci. 2004, 8, 213-225.
82. Merino, C.; Livia, J. Intervalos de confianza asimétricos para el índice de validez de contenido: Un programa Visual Basic para la V de Aiken. An. Psicol. 2009, 21, 169-171.
83. Wilson, E.B. Probable inference, the law of succession and statistical inference. J. Am. Stat. Assoc. 1927, 22, 209-212.
84. Álvarez-García, O.; Sureda-Negre, J.; Comas-Forgas, R. Diseño y validación de un cuestionario para evaluar la alfabetización ambiental del profesorado de primaria en formación inicial. Profr. Rev. Curric. Form. Profr. 2018, 22, 265-284.
85. León-Larios, F.; Gómez-Baya, D. Diseño y validación de un cuestionario sobre conocimientos de sexualidad responsable en jóvenes. Rev. Esp. Salud Publica. 2020, 92, 1-15.
86. Méndez, C.; Rondón, M.A. Introducción al análisis factorial exploratorio. Rev. Colomb. Psiquiatr. 2012, 41, 197-207.
87. Forero, C.G.; Maydeu-Olivares, A.; Gallardo-Pujol, D. Factor and analysis with ordinal indicators: A monte Carlo study comparing DWLS and ULS estimation. Struct. Equ. Modeling 2009, 16, 625-641.
88. Costello, A.B.; Osborne, J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pract. Assess. Res. Eval. 2005, 10, 1-9.
89. Hair, J.F.; Anderson, R.E.; Tatham, R.; Black, W.C. Análisis Multivariante, 5th ed.; Prentice Hall: Madrid, Spain, 1999.
90. Kaiser, H.F. The varimax criterium for analytic rotation in factor analysis. Psychometrika 1958, 23, 187-200.
91. Landa, M.R.; Ramírez, M.Y. Diseño de un cuestionario de satisfacción de estudiantes para un curso de nivel profesional bajo modelo de aprendizaje invertido. Rev. Paginas Educ. 2018, 11, 153-175.
92. Hernández, R.; Fernández, C.; Baptista, P. Metodología de la Investigación; Mc-Graw Hill: Mexico City, Mexico, 2006.
93. Pegalajar, M.C. Diseño y validación de un cuestionario sobre percepciones de futuros docentes hacia las TIC para el Desarrollo de prácticas inclusivas. Pixel-Bit. Rev. Medios Educ. 2015, 47, 89-104.
94. Santos, M.A.; Jover, G.; Naval, C.; Álvarez, J.L.; Vázquez, V.; Sotelino, A. Diseño y validación de un cuestionario sobre práctica docente y actitud del profesorado universitario hacia la innovación (CUPAIN). Educ. Xxi 2017, 20, 39-71.
95. Merenda, P. A guide to the proper use of factor analysis in the conduct and reporting of research: Pitfalls to avoid. Meas. Eval. Couns. Eval. 1997, 30, 156-163.
96. Kim, J.; Mueller, C.W. Factor analysis, statistical methods and practical issues. In Factor Analysis and Related Techniques; Lewis-Beck, M., Ed.; Sage Publications: London, UK, 1994; pp. 75-155.
97. Pérez, E.R.; Medrano, L. Análisis Factorial Exploratorio: Bases conceptuales y metodológicas. Rev. Argent. Cienc. Comport. 2010, 2, 58-66.
98. Arribas, M. Diseño y validación de cuestionarios. Matronas Prof. 2004, 5, 23-29.
99. Kline, P. An Easy Guide to Factor Analysis; Sage: Newbury Park, Ventura Country, CA, USA, 1994.
100. Long, J.S. Confirmatory Factor Analysis: A preface to LISREL; Paper Series on Quantitative Applications in the Social Sciences: London, UK, 1983.
101. Stapleton, C.D. Basic concepts in Exploratory Factor Analysis (EFA) as a tool to evaluate score validity: A right-brained approach. In Proceedings of the Annual Meeting of the Southwest Educational Research Association, Austin, TX, USA, 24 January 1997.
102. Batista-Foguet, J.M.; Coenders, G.; Alonso, J. Análisis Factorial Confirmatorio. Su utilidad en la validación de cuestionarios relacionados con la salud. Med. Clin. 2004, 122, 21-27.
103. Fernández, A. Aplicación del análisis factorial confirmatorio a un modelo de medición del rendimiento académico en lectura. Rev. Cienc. Econ. 2015, 33, 39-66.
104. Herrero, J. El análisis factorial confirmatorio en el estudio de la estructura y estabilidad de los instrumentos de evaluación: Un ejemplo con el Cuestionario de Autoestima CA-14. Interv. Psicosoc. 2010, 19, 289-300.
105. Urrutia, M.; Barrios, S.; Gutiérrez, M.; Mayorga, M. Métodos óptimos para determinar validez de contenido. Educ. Med. Super. 2014, 28, 1-9.
Rubén Fernández Álvarez
1,* and
José Fernández
2
1Department of Geography, School of Education and Tourism, University of Salamanca, 05003 Ávila, Spain
2Department of Geography, University of Salamanca, 37007 Salamanca, Spain
*Author to whom correspondence should be addressed.
© 2021. This work is licensed under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.