Construction of Questionnaire-Scale USOTIC “Social Networks in Primary and Secondary School Students: Use and Digital Coexistence”

Abstract

In the present work, the construction process, namely the design, elaboration and validation of a questionnaire-scale is described to evaluate the attitudes and interactions in the way of living in the digital world of social networks in primary and secondary students. The current aim was to analyse the means, uses and risks, as well as the sources of training and advice those students have to interact with the diverse range of devices with an internet connection that provide access to social networks. The participating sample for validation was 1073 students from the Primary and Secondary Education stages of centres with organic dependency: Private, concerted and public. The obtained results express the satisfactory metric quality of the questionnaire, thus presenting a theoretically based and operationally defined instrument, considering four relevant dimensions: Special, environmental, perceptual and motivational.

1. Introduction

The way children and young people coexist in the world of technology and social networks has a great impact and echo in today’s society due to the attitudes that are reflected through them, causing conflicts that alter coexistence inside and outside the classroom, sometimes moving beyond the barriers of educational centres. These conflicts can arise due to inappropriate use of social networks, which makes them vulnerable [1]. This vulnerability can be generated in this group as a result of a lack of knowledge of the repercussions and the importance of incorrect use of social networks, and on too many occasions, due to a lack of digital training and critical thinking in their use [2].

The aim of this work was to develop an evaluation instrument using a questionnaire-scale to explore the digital coexistence of children and adolescents through social networks, as a means of obtaining information and analysing the risks for Primary and Secondary School pupils when interacting on social networks. It focuses on dimensions such as the use they make of them and their relationship with learning, digital coexistence, participation and attitudes. The information obtained may constitute relevant data to detect the key elements of the use of social networks in children and adolescents and guide them towards taking preventive measures, fostering the development of critical thinking and, in certain cases, correcting incorrect use.

In order to carry out the interest of this research, a questionnaire entitled “Study of Social Networks in primary and secondary education students” was employed. In this article, we present its reliability and validity.

2. Fundamentals

In today’s society, technology has become a necessity in our way of life, causing a cultural change and transforming our ways of communication, interaction and knowledge. This transformation triggers a break from the linear form of communication, giving birth to a new form of interaction and communication through ICTs, opening a window to the digital world, transferring young people to scenarios where they are the protagonists of their own actions: The way they adhere to the diversity of media content, which they share, disseminate and comment on with other users [3]. In this sense, the content offered to them through the use of ICTs facilitates interaction and socialisation, as they share hobbies, interests and tastes for the same activities with their peers (music, sports, travel, etc.). This way of interacting will be determined by their attitude, which can lead them to become vulnerable to new scenarios, which, through ICTs, turn them into actors defined by their attitudes, altering digital coexistence. In this line, there is a diversity of studies, such as Project SAVE [4], the ANDAVE school anti-violence project [5], the programme to combat bullying launched by the regional government [6], the Cyber-experts programme [7] and the Cybermentors programme for the safe and responsible use of ICTs [8]. Reports of the state observatory on coexistence were carried out in 2008, as were studies by [9], all with the same objective of preventing disruptive attitudes that make young people vulnerable and improving conflicts in digital coexistence.

In line with this research, together with professional experience in the field of education, the interest in analysing the way in which Primary and Secondary School pupils live together in the digital world was born, and to carry it out, the construction of a questionnaire (Appendix A) was implemented.

Importance of ICTs

Technologies offer a wide range of possibilities for interaction, communication and knowledge through a diversity of tools with easy access to an internet connection. In this sense, children and adolescents are increasingly acquiring devices with a variety of attractive digital applications that allow them to open up to the digital world in a quick and easy way, but without being aware of the risks and consequences to which they are exposed, making them vulnerable.

Vulnerability is determined as the dimension in which adolescents have found a way of life in Social Networks and mobile devices in which “they use them very naturally and playfully because they have been socialised in technological contexts through processes of self-learning and informal knowledge” [10]. Through them, they show their creativity, as it allows them to personalise their pages, and make and publish their own musical productions, photos and videos with the aim of making them visible and obtaining likes or opinions from others. This makes them more vulnerable as they expose their lives and there is a lack of privacy of which they are unaware, which can lead to future consequences and repercussions because adolescents’ knowledge of technologies is mainly instrumental [11].

In this sense, we consider it relevant that in order to understand the process of inclusion of children and adolescents in the digital world of social networks, it is necessary to be clear about the diversity of concepts related to digital life, starting from the definition of a social network, which is detailed below:

“A structure made up of people who are interconnected through the internet by different types of links: affective, work, sexual, friendship, etc. It constitutes a dynamic group in continuous evolution, where there are people who join, others who appear and others who disappear. They allow contact with unknown people in a simple way” [12]. They are formed between children and adolescents who have common interests, which allows them to form friendships, follow up on past relationships and get to know people intimately to a certain extent.

In this line, there are studies [13,14] in which the most attractive device for adolescents is the mobile phone, which is becoming more and more advanced and sophisticated, allowing them to do a multitude of things and giving them a degree of self-esteem, freedom and personal intimacy. These devices, from adolescence onwards, serve them for fun, leisure and communication, through the resources available on them, such as chatting, sending messages, sharing photos, events, etc. Thus, they cover their basic psychological needs: Making themselves visible, reaffirming their identity in front of a group, having fun or simply being connected to their friends.

This way of relating is characterised by the ease with which they overcome embarrassment, overcome shyness or fear of ridicule, express their own identity, create personal profiles and establish relationships, in which they express emotions, which encourages participation; although, in some cases, exclusion can occur, which leads to two important aspects: On the one hand, it facilitates the creation of links, and on the other hand, their destruction.

Along these lines, [15] highlight three positions in their studies that shed light on the motives and uses those adolescents give to ICTs:

• The effects produced by media content affect and condition adolescents’ behaviour since they act as models of that content.

• Adolescents integrate the use of content into their daily lives in an active way so that they can choose what is interesting and convenient for them.

• The use of devices becomes part of a social process in adolescents playing an important role in the construction of identity.

In this sense, the use of ICTs can be understood from the perspective that they are part of everyday life for adolescents, and consequently, they are linked to problems of conflicts and risks that adolescents face in a new digital scenario. Among the risks in which adolescents may find themselves, we highlight the following:

• Not having a critical capacity in the selection and publication of content, being irresponsible in terms of exposing their personal information and privacy to a world in which they do not know who is behind the screen [15].

• Abuse in the use of technologies sometimes causes them to become disordered, leading to online addiction [12].

• The protection of being on the other side of the internet gives rise to disruptive behaviour among adolescents towards others, making them vulnerable to falling victim to cyberbullying [16]. Different forms of cyberbullying can take place through electronic insults, harassment by sending offensive messages, impersonation, disclosure and coaxing of personal information and exclusion and ostracism [17].

3. Materials and Methods

The aim of this study was to construct and validate a questionnaire-scale to explore the digital coexistence of primary and secondary school pupils with social networks, analysing the means, uses and risks, as well as the sources of training and advice they obtain for interacting with them.

3.1. Data Collection Procedure

The questionnaires were completed on a scheduled basis at different educational centres that had previously been asked to participate on a voluntary basis. In this study, the questionnaires were completed in person in paper format at the different schools by appointment, day and time, established by the head teacher at the school. It was carried out in the classrooms of the educational centre, under the supervision of the person responsible for the research and authoring of this questionnaire [1] during the months of March, April and May of the 2018–2019 academic year. To this end, before the students began the questionnaire, they were informed of the aim and purpose of the study and the importance of their participation in it, which was voluntary and anonymous. In addition, they were made aware of the law on data protection and confidentiality for research purposes when completing the questionnaire.

3.2. Participants in the Pilot Study

In order to carry out the statistical studies required for the validation of the questionnaire under study, we relied on a pilot study involving 1073 Primary and Secondary Education students from the province of Madrid, 47% of whom were male and 52% female.

As for the type of centre according to its organic nature, of the total number of participants in the study, 17.1% belonged to private centres, 43.7% to public centres and 39.2% to subsidised centres. Regarding the distribution of the sample by stage, of the total number of respondents, 17.3% correspond to the 4th stage of Primary Education, 17.5% correspond to the 5th level of Primary Education, 18.3% correspond to the 6th stage of Primary Education, 13% correspond to the 1st stage of Secondary Education, 12.8% correspond to the 2nd stage of Secondary Education, 12.7% correspond to the 3rd stage of Secondary Education and 8.4% correspond to the 4th stage of Secondary Education. In relation to the distribution of the sample by age, 21.4% of the total participants were aged between 9 and 10, 34.7% were aged between 11 and 12, 24.9% were aged between 13 and 14, 3.3% were aged between 15 and 16 and 3.7% were aged between 17 and 18.

3.3. Instrument

To address the proposed general objective, the questionnaire is designed and used as a tool for data collection as determined by studies such as [18,19]. The choice of the instrument was in response to the quantitative methodology and nature of the data we are concerned with acquiring, which in our case is related to the motivation and perception of various attributes of the classroom space [20].

4. Justification for the Development of the Questionnaire

Regarding the identification of the dimensions in the development of this questionnaire, we started with a review of authors such as [21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38]. We were able to explore how the different contributions they provide can be grouped into the following specific argumentative dimensions in relation to digital coexistence:

• Culture of uses, which includes device access, internet connection, social networks and connection time. This is in line with authors [22,23,25,30,38], in which their studies analyse the use of smartphones, social networks and the culture of connectivity in children and adolescents.

• Need for and dependence on networks. Authors such as [32,33,34,35] establish how the need for children and adolescents to be connected makes them dependent on them, and these will be linked to the purpose of achieving prestige on the different social networking sites.

• Security and harassment on the networks. As a reference, we take legal aspects such as articles 7 and 84 of the Spanish Data Protection Agency [39], which establish the minimum requirements and responsibilities for minors’ access to the internet and social networks. In addition, we include the studies by [29,36], in which they demonstrate the need to train critical citizens in the face of the challenges of technologies for their safety and protection, and the dimension is elaborated. Given this need to train critical citizens, one of the key elements is attitude, as shown in studies by [21,24,26]; therefore, we find that the factor of attitude towards the usefulness and purpose of interaction through social networks is a factor that will determine the risks to which children and adolescents may be subjected in the digital world, such as cyberbullying, invasion of privacy and intimacy, etc., as well as the risks to which they may be exposed.

• Utility and purpose. It is worth highlighting in this aspect how networks can be used as a means to promote learning, as stated by authors such as [40,41,42], who highlight the potential of learning networks as a means to offer a greater wealth of access to different sources and resources that facilitate learning.

• Another aspect to take into account is the sources of training from the educational centre (teachers, counsellors and external agents); in this line, as a reference, we take the Horizon report [43] in relation to the time students spend connected to the internet where they interact with others and exchange a diversity of information. In this sense, the role of the different training agents is essential to promote critical thinking and the responsible use of social networks.

• A final dimension that can be seen in the different studies consulted is in the area of family support. In this sense, authors such as [28,31,37] focus on how to help pupils in the process of socialisation and development in order to develop in a society that requires competent digital citizenship.

5. Validation Process of the Tool

5.1. Content Validation

Following the literature review, and once an initial structure based on theoretical dimensions had been established, items were developed for each group. This initial battery of items consisted of 63 items, distributed across the five dimensions.

This initial proposal was submitted to the judgement of experts, who assessed it on the basis of three indicators: Quality in the wording of the question, coherence with the object of study and relevance and importance within the instrument in terms of the information it can provide.

Expert judgement is a strategy with many advantages; Ref. [44] summarises its importance as follows:

… the theoretical quality of the response we obtain from the person, the level of depth of the assessment offered, its ease of implementation, the fact that it does not require many technical or human requirements for its execution, the possibility of using different strategies to collect information is very useful for determining knowledge on difficult, complex, novel or little studied contents and topics, and the possibility of obtaining detailed information on the subject under study.

In this sense, and following [44], there are discrepancies regarding the number of experts that should participate in this validation process, following, in order of the dates of publication, Ref. [45], who suggest that the number should range between 15 and 20; Ref. [46], who places them between 15 and 35; Ref. [47], between 7 and 30 and Ref. [48], between 15 and 25.

Even so, there is another relevant issue, namely the consideration of “expert”. For this, we rely on the coefficient of expert competence [49]. In this reference, several self-assessed indicators are proposed by the proposed experts, and in our case, we summarise these indicators around three factors: Teaching experience in the field, level of publications and research experience in the field of knowledge.

In short, 15 experts were selected, who obtained scores above 0.85 in the coefficient of expert competence (CEC). Table 1 shows the overall data, although the overall scores can be found in Appendix B.

5.2. Expert Validation Data

Each expert gave a score between 0–1 to each item and for each factor (quality, coherence, relevance), obtaining a final score for each item as an average of the previous scores.

The initial version of the questionnaire had 67 items, with 4 of a socio-demographic nature and 63 of content on the dimensions. After judgement, the questionnaire was reduced to 44 items, whereby 4 socio-demographic items and 40 research items were maintained. The eliminated questions did not exceed an average of 0.80 in the overall total score. Appendix C shows the item-by-item scores for each expert, as well as the overall score for each one.

The configuration of the instrument after the expert assessment is presented in Table 2.

Finally, the recommendation made by the experts to establish seven levels of response on the Likert scale was taken into account in the final drafting of the instrument. This was taken on board in the final drafting of the instrument: 1 (NEVER), 2 (RARELY), 3 (SOMETIMES), 4(OFTEN), 5 (NORMALLY), 6 (USUALLY) and 7 (ALWAYS).

5.3. Construct Validation

Once the questionnaire had been updated with the experts’ contributions, the statistical study was carried out by means of confirmatory factor analysis. To this end, the pilot test was carried out with the aforementioned participants.

The factor analysis was carried out, in which the number of factors to be extracted was not fixed, nor were the minimum values to be excluded. Varimax rotation was applied, as well as the KMO test and Bartlett’s test of sphericity as shown in Table 3.

From these data we can assume that as the KMO value is 0.0832 (close to the ideal value of 1), it indicates a remarkable fit of the data [50].

Likewise, the level of Bartlett’s test (sig = 0.000) allows us to affirm that there is perfect significance, considering the adjustment of the variables by means of the ideal factor analysis.

In terms of principal components, six factors explain 57.765% of the total variance. From the sixth factor onwards, percentages of less than 2% of the variance are explained and should therefore be discarded (Table 4).

The model was then established using the structural equation procedure with the support of AMOS software version 24.

In the first analysis, the non-statistical values did not support the proposed factorial model, so we proceeded to rectify the model according to our own data. Seven items with low correlations were eliminated. The results obtained for this second model are presented in Table 5.

According to [51], the obtained values confirm this assessment.

After review, it was considered that the remaining items could explain the information of those eliminated.

The questionnaire was therefore structured as shown in Figure 1. We then list the appropriate statistics, which, by virtue of sample size and response format, best fit the study (Table 6).

6. Results

A reliability study was carried out on the items that make up the final structure (Table 7), whereby Cronbach’s alpha was calculated for the entire questionnaire, by category, item by item, as well as the value if these items were eliminated.

Analysed independently, each dimension yields the values shown in Table 8.

Taking into account the length of the questionnaire (33 items) and according to [52], the alpha values indicate that the instrument can be considered reliable overall. It should be noted that the data corresponding to dimensions 1 and 2 are below a value of 0.7, but even so, they can be accepted as reliable data.

7. Conclusions

In recent decades, interest in the presence of social networks and their involvement in learning processes has been increasing, and it is necessary to identify how this is carried out and what functions are given to these uses in students in compulsory education.

The obtained results express the satisfactory metric quality of the questionnaire, thus presenting a theoretically grounded and operationally defined instrument by considering four relevant dimensions: Spatial, environmental, perceptual and motivational. This approach can be considered relevant and differentiated in relation to other instruments, both in its theoretical approach and in its operationalization in items, thus making it consistent. The analysis of its technical characteristics shows adequate overall reliability in terms of internal consistency, which facilitates its applicability in different contexts.

On the other hand, it should be noted that the instrument presents an optimal content validity, guaranteed by the consistency and rigour of the theoretical work and the evaluations that the selection of experts from education and university research professionals conferred on the configuration of the instrument’s items.

To conclude, the proposed instrument represents a relevant contribution from a theoretical approach due to the approximation in the elaboration and dimensions that it addresses.

Author Contributions

Author Contributions: Conceptualization, M.C.L.B., C.S.R.; methodology, M.C.L.B., C.S.R., L.O.J.; formal analysis, M.C.L.B., L.O.J.; investigation, resources, M.C.L.B., C.S.R.; writing—original draft preparation, writing—review and editing, visualization, supervision, M.C.L.B., C.S.R. and L.O.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figure and Tables

Figure 1. USOTIC questionnaire structural model.

Table 1

Coefficient average of expert competence (CEC).

	Teaching	Publications	Investigation	CEC
Expert media	0.93	0.89	0.88	0.90

Table 2

Distribution of items in the questionnaire.

Factor	Identification Factor	Items
1	Culture of network use	19, 20, 21, 22, 23, 27 y 31
2	Network dependency	1, 3, 4, 5, 6, 7, 8, 10, 15 y 16
3	Safety and harassment on networks	24, 25, 26, 28, 29 y 30
4	Use of networks for learning	11, 12, 13, 14, 17 y 18
5	Teacher support	32, 33, 34, 35 y 36
6	Family support	2, 9, 37, 38, 39, 40

Source: Distribution data.

Table 3

KMO and Bartlett tests.

Kaiser-Meyer-Olkin Measure of Sampling Adequacy		0.832
Bartlett’s test of sphericity	Approx. Chi-cuadrado	10,731.409
	gl	780
	Sig.	0.000

Table 4

Factorial analysis.

Factor	Initial Eigenvalues
	% of Variance	Cumulated %
1	16.621	16.621
2	13.579	30.199
3	7.932	38.131
4	7.360	45.491
5	6.334	51.825
6	5.940	57.765

Source: Research data.

Table 5

Values of the proposed model.

p Value	RMSEA	CFI	TLI	NFI
0.07	0.049	0.964	0.949	0.921
>0.05	<0.50	>0.90	>0.90	>0.90

Source: Research data.

Table 6

Structure of the questionnaire.

Factor	Identification Factor	Items
1	Culture of network use	19, 20, 21 y 31
2	Network dependency	1, 4, 6, 7, 8, 10, 15 y 16
3	Safety and harassment on networks	25, 26, 28, 29 y 30
4	Use of networks for learning	11, 12, 13, 14, 17 y 18
5	Teacher support	32, 33, 34, 35 y 36
6	Family support	9, 37, 38, 39, 40

Source: Research data.

Table 7

Cronbach’s global alpha of the instrument.

Alpha of Cronbach	N of Elements
0.729	33

Table 8

Cronbach’s alpha for each dimension of the instrument.

Factor	Identification Factor	Alfa of Cronbach
1	Culture of network use	0.613
2	Network dependency	0.699
3	Safety and harassment on networks	0.701
4	Use of networks for learning	0.709
5	Teacher support	0.805
6	Family support	0.734

Source: Research data.

Appendix A. Questionnaire Used in the Present Study

Spatial Variables

I. DEMOGRAPHIC DATA

How old are you? (Mark with an X)

(  ) Less than 9 y.o.

(  ) 9–10 y.o.

(  ) 11–12 y.o.

(  ) 13–14 y.o.

(  ) 16–17 y.o.

(  ) 17–18 y.o.

(  ) More than 18 y.o.

What year and stage are you currently in? (Mark with an X)

(  ) 4°PS

(  ) 5°PS

(  ) 6°PS

(  ) 1°CSE

(  ) 2°CSE

(  ) 3°CSE

(  ) 4°CSE

What is your gender? (Mark with an X)

(  ) Male

(  ) Female

The school where you study is in what environment: (Mark with an X)

(  ) Rural

(  ) Urban

In which province is the school where you study.

The school where you study What type of school is it?

(  ) Private

(  ) Public

(  ) Privately-owned but state-funded

I.–DIMENSION: AGE OF DEVICE ACQUISITION, INTERNET CONNECTION, SOCIAL NETWORKS AND TIME SPENT ONLINE.

1. What was your first internet-enabled device? (Mark with an X).

(  ) Mobile telephone

(  ) Computer/laptop

(  ) Tablet

(  ) Other, Which?

2. At what age did you get your first internet-enabled device? (Mark with an X)

(  ) Less than 9 y.o.

(  ) 9 –10 y.o.

(  ) 11–12 y.o.

(  ) 13–14 y.o.

(  ) 16–17 y.o.

(  ) 17–18 y.o.

(  ) More than 18 y.o.

3. What is your favourite device for connecting to social networks? (Mark with an X)

(  ) Mobile telephone

(  ) Computer/laptop

(  ) Tablet

(  ) Other, Which?

4. At what age did you first sign up to a social network? (Mark with an X)

(  ) Less than 9 y.o.

(  ) 9–10 y.o.

(  ) 11–12 y.o.

(  ) 13–14 y.o.

(  ) 16–17 y.o.

(  ) 17–18 y.o.

(  ) More than 18 y.o.

5. How many days a week do you log on to social media? (Mark with an X)

(  ) Every day of the week

(  ) One day a week

(  ) Two days a week

(  ) Three days a week

(  ) Four days a week

(  ) Five days a week

(  ) Six days a week

(  ) Weekends only

SCALE OF ATTITUDES

Read slowly and answer each question with an X from 1 to 7, with 1 being the lowest value and 7 the highest value, bearing in mind that:

1	2	3	4	5	6	7
NEVER	RARELY	SOMETIMES	OFTEN	NORMALLY	USUALLY	ALWAYS

II. DIMENSION: NEED AND DEPENDENCY	1N	2R	3S	4O	5N	6U	7A
6. Do I turn my device off at night?
7. I feel nervous when I can’t connect you to social networks.
8. I respond immediately when I receive messages on social networks or SMS.
9. When you don’t have an internet connection, do you look for places to be?
10. I am not able to disconnect from social media for several days.
11. I look at my profile every day when I wake up
12. I see my friends’ profiles every day.
III. DIMENSION: SAFETY AND SECURITY	1N	2R	3S	4O	5N	6U	7A
13. Do you share your device’s password with your family?
14. Do you use social networks to establish new friendships?
IV. DIMENSION: UTILITY AND PURPOSE	1N	2R	3S	4O	5N	6U	7A
15. Do you use social media to do your homework?
16. Do social networks support your learning?
17. Frequency of your use of information content on social networks	1N	2R	3S	4O	5N	6U	7A
• To work with my friends in class.
• Search for information for school work.
• To talk to friends.
• Watch videos on youtube: games, music…
• To share files
• To upload and share photos
• To upload videos.
• To see the story of my friends.
• To see the story of famous people.
• Online games: Fortnite.
• Others: Which ones?
V. DIMENSION–DIGITAL COEXISTENCE	1N	2R	3S	4O	5N	6U	7A
18. Social networks promote coexistence with others.
19. I use social media in a way that respects others.
20. I think about the consequences before I post a comment or photo.
21. I post photos of my friends without their consent.
22. If I feel hurt by a comment on social media, I try to calm down and do not answer.
23. If I feel hurt by a comment on social media I impulsively reply with insults.
24. Do you think it is easier to suffer harassment through networks?
25. Have you ever been threatened through social media?
26. Have you ever felt helpless on social media?
27. Do you think that social networks are a medium in which it is easier for acts of harm to be done to you?
28. Have they posted rumours about you on social media?
29. Have you been excluded from participating in social media?
30. Have you been insulted on social media or through SMS messages?
31. Do you consider yourself capable of detecting situations of harassment that other people may be suffering?    (  ) YES (  ) NO
How?
VI. DIMENSION–RISK PREVENTION IN EDUCATION AND IN THE FAMILY ENVIRONMENT	1N	2R	3S	4O	5N	6U	7A
32. Do my teachers at school help me to learn about the usefulness of social media?
33. Do my teachers explain to me strategies for prevention in the use of social networks?
34. Do my teachers explain to me strategies for using social media safely?
35. Do my teachers inform me about the risks of using social networks?
36. Have you received talks at school about social networking by people other than your teachers?	1N	2R	3S	4O	5N	6U	7A
• Guidance counsellor
• Policeman/Policewoman
• Spanish military police
• Experts
37. I have help from my parents if I face a problem on social media.
38. My parents monitor the time I spend online.
39. My parents help me how to search for information on social networks.
40. My parents explain to me how to make good use of social networks.
COMMENTS AND CONTRIBUTIONS:______________________________________________________________

Thank you very much for your cooperation

Appendix B

Expert judges selected by expert competence coefficient: Judges rate themselves with scores between 0–1 on each factor.

Table A1

Expert competence coefficient.

Judge	Exp. Teaching	Publications	Investigation	Global
1	0.9	0.85	0.85	0.86
2	1	0.9	0.9	0.93
3	0.85	0.85	0.85	0.85
4	0.9	0.9	0.9	0.9
5	0.9	0.9	0.9	0.9
6	1	0.85	0.9	0.91
7	0.95	0.95	0.95	0.95
8	1	0.9	0.85	0.91
9	0.85	0.9	0.9	0.88
10	1	0.85	0.85	0.9
11	0.95	0.95	0.95	0.95
12	0.9	0.85	0.9	0.88
13	1	0.85	0.85	0.90
14	0.95	0.95	0.9	0.93
15	0.85	0.9	0.85	0.86
AVERAGE	0.93	0.89	0.88	0.90

Appendix C

Table A2

Mean scores obtained in the expert judgement for each item.

Items	Quality	Coherence	Relevance	Global	Decision
1	100	100	100	1	Maintain
2	100	100	100	1	Maintain
3	100	100	100	1	Maintain
4	100	100	100	1	Maintain
5	70	52	30	0.50	Discard
6	85	62	45	0.64	Discard
7	65	53	49	0.55	Discard
8	72	62	40	0.58	Discard
9	66	57	51	0.58	Discard
10	85	89	91	0.88	Maintain
11	95	91	93	0.93	Maintain
12	92	93	91	0.92	Maintain
13	91	89	90	0.90	Maintain
14	89	90	89	0.89	Maintain
15	89	88	91	0.89	Maintain
16	85	87	88	0.86	Maintain
17	89	90	93	0.90	Maintain
18	61	51	45	0.52	Discard
19	72	65	56	0.64	Discard
20	64	59	56	0.59	Discard
21	71	69	68	0.69	Discard
22	55	45	49	0.49	Discard
23	80	77	56	0.71	Discard
24	65	68	55	0.62	Discard
25	88	91	94	0.91	Maintain
26	91	94	93	0.92	Maintain
27	72	65	65	0.67	Discard
28	73	65	57	0.65	Discard
29	76	70	69	0.71	Discard
30	75	65	60	0.66	Discard
31	95	89	91	0.91	Maintain
32	95	91	93	0.93	Maintain
33	95	90	92	0.92	Maintain
34	90	91	90	0.90	Maintain
35	90	89	94	0.91	Maintain
36	85	88	91	0.88	Maintain
37	95	92	93	0.93	Maintain
38	70	65	59	0.64	Discard
39	70	65	57	0.64	Discard
40	75	65	65	0.68	Discard
41	75	67	60	0.67	Discard
42	70	58	58	0.62	Discard
43	70	60	65	0.65	Discard
44	95	99	95	0.96	Maintain
45	90	91	90	0.90	Maintain
46	90	89	92	0.90	Maintain
47	93	92	89	0.91	Maintain
48	93	90	93	0.91	Maintain
49	94	92	93	0.93	Maintain
50	93	90	89	0.90	Maintain
51	94	91	90	0.91	Maintain
52	94	89	90	0.91	Maintain
53	92	89	88	0.89	Maintain
54	89	90	89	0.89	Maintain
55	70	65	65	0.66	Discard
56	91	90	93	0.91	Maintain
57	89	90	91	0.90	Maintain
58	85	88	90	0.87	Maintain
59	88	90	91	0.89	Maintain
60	87	91	93	0.90	Maintain
61	91	90	93	0.91	Maintain
62	95	92	95	0.94	Maintain
63	94	93	95	0.94	Maintain
64	90	89	92	0.90	Maintain
65	94	89	95	0.92	Maintain
66	93	89	95	0.92	Maintain
67	95	91	93	0.93	Maintain

References

1. López Berlanga, M.C. La Convivencia Digital de los Estudiantes de Primaria y Secundaria en las Redes Sociales: Necesidades de Formación. Tesís de Doctorado (UNED) 2019; Available online: http://e-spacio.uned.es/fez/view/tesisuned:ED-Pg-Educac-Mclopez (accessed on 30 September 2021).

2. Romero, C.S.; González, E.Á. Actitudes nocivas y riesgos para los menores a través de los dispositivos móviles. REXE Rev. Estud. Exp. Educ.; 2018; 2, pp. 147-161.

3. Koerner, S.; Ernst, D.; Jenkinks, H.; Chisholm, A. Pathways to Measuring Consumer Behavior in an Age of Media Convergence. Proceedings of the ESOMAR Conference; Cannes, France, 22–23 June 2002.

4. Del Rey, R.; Ruiz, R.O. La formación del profesorado como respuesta a la violencia escolar: La propuesta del modelo de Sevilla Antiviolencia Escolar (SAVE). Rev. Interuniv. Form. Profr.; 2001; 41, pp. 59-71.

5. Ortega Ruiz, R.; Rey Alamillo, R.D. El Proyecto Anti-Violencia Escolar: ANDAVE; B/Ilustre Colegio Oficial de Doctores y Licenciados en Filosofía y Letras y en Ciencias: Madrid, Spain, 2003.

6. Pereira, G.V.; Macadar, M.A.; Luciano, E.M.; Testa, M.G. Delivering public value through open government data initiatives in a Smart City context. Inf. Syst. Front.; 2016; 19, pp. 213-229. [DOI: https://dx.doi.org/10.1007/s10796-016-9673-7]

7. Cirberexpertos Programme. CAM. 2016; Available online: https://www.educa2.madrid.org/web/educamadrid/principal/files/6753e847-3c12-4997-9058-5162593b8bba/DIPTICO%20CIBEREXPERT%40.pdf?t=1474961909700 (accessed on 30 September 2021).

8. Pereira, A.E. Cibermanagers/Cibermentores aprendizaje entre iguales al servicio de la ciudadanía digital. Rev. Asoc. Proy. Hombre; 2017; 93, pp. 29-30.

9. Del Rey, R.; Elipe, P.; Ortega-Ruiz, R. Bullying and cyberbullying: Overlapping and predictive value of the co-occurrence. Psicothema; 2012; 24, pp. 608-613.

10. Rodríguez, R.M. Desarrollo local y Análisis de Redes Sociales: El valor de las relaciones com factor del desarrollo socioeconómico. REDES; 2010; 18, pp. 278-304. [DOI: https://dx.doi.org/10.5565/rev/redes.396]

11. Sabana, C. Oportunidad Irremediable. Vanguardia; 2012; 34.

12. Echeburúa, E.; Requesens, A. Adicción a las Redes Sociales y Nuevas Tecnologías en Niños y Adolescentes; Morata: Madrid, Spain, 2012.

13. Amorós, P.; Buxarrais, M.R.; Casas, F. La Influència de les Tecnologies de la Informació i Comunicació en la Vida dels Nois i Noies de 12 a 16 Anys; Institut d’Infància i Món Urbà (CIIMU), Observatori de la Infància i la Família: Madrid, Spain, 2002.

14. Naval, C.; Sádaba-Chalezquer, C.; Bringué, X. Impacto de las Tecnologías de la Información y de la Comunicación (TIC) en las Relaciones Sociales de los Jóvenes Navarros. 2003; Available online: https://hdl.handle.net/10171/18444 (accessed on 30 September 2021).

15. Morduchowicz, R.; Marcon, A.; Sylvestre, V.; Ballestrini, F. Los Adolescentes y Las Redes Sociales; FCE: Buenos Aires, Argentina, 2012; Available online: https://www.sap.org.ar/docs/congresos_2015/37%20CONARPE/morduchowicz.adolescentesyredessociales.pdf (accessed on 30 September 2021).

16. Ortega, R.; Rey, R.D.; Casas, J.A. Redes sociales y cyberbullying: El proyecto ConRed. Convives; 2013; 3, pp. 34-44.

17. Kowalski, R.; Kowalski, R.; Limber, S.; Agatston, P. Cyber Bullying: El Acoso Escolar en la Era Digital; Academia: San Francisco, CA, USA, 2009; Volume 24, 207. [DOI: https://dx.doi.org/10.1111/j.1467-9604.2009.01431_5.x]

18. Del Rincón, D.; Arnal, J.; Latorre, A.; Sans, A. Técnicas de Investigación en Ciencias Sociales; Dykinson: Madrid, Spain, 1995.

19. Buendía, L.; Colas, P.; Hernandez, F. La investigación observacional. Métodos de Investigación en Psicopedagogía; McGraw-Hill: Madrid, Spain, 1997; pp. 157-206.

20. Miralles Martínez, P.; Molina Puche, S.; Ortuño Molina, J. La Investigación en Didáctica de las Ciencias Sociales. Educ. Siglo; 2011; XXI, pp. 149-174. Available online: https://revistas.um.es/educatio/article/view/119921 (accessed on 30 September 2021).

21. Tejada, E.; Castaño, C.; Romero, A. Los hábitos de uso en las redes sociales de los preadolescentes, R.I.E.D. Rev. Iberoam. Educ. Distancia; 2019; 22, pp. 119-133.

22. Van Dijck, J. La Cultura de la Conectividad: Una Historia Crítica de las Redes Sociales; Siglo XXI Editores: Madrid, Spain, 2019.

23. Escoda, A.P. Uso de smartphones y redes sociales en alumnos/as de educación primaria. Prism. Soc. Rev. Investig. Soc.; 2018; 20, pp. 76-91.

24. Livingstone, S.; Mascheroni, G.; Staksrud, E. European research on children’s internet use: Assessing the past and anticipating the future. New Media Soc.; 2018; 20, pp. 1103-1122. [DOI: https://dx.doi.org/10.1177/1461444816685930]

25. Mitchelstein, E.; Boczkowski, P.J. Juventud, estatus y conexiones. Explicación del consumo incidental de noticias en redes sociales. Rev. Mex. Opin. Pública; 2018; 24, pp. 131-145. [DOI: https://dx.doi.org/10.22201/fcpys.24484911e.2018.24.61647]

26. Giménez, A.M.; Luengo, J.A.; Bartrina, M.J. ¿Qué hacen los menores en internet? Usos de las TIC, estrategias de supervisión parental y exposición a riesgos. Electron. J. Res. Educ. Psychol.; 2017; 15, pp. 533-552. [DOI: https://dx.doi.org/10.25115/ejrep.43.16123]

27. Del Barco, B.L.; Castaño, E.F.; Bullón, F.F.; Carroza, T.G. Cyberbullying en una muestra de estudiantes de Educación Secundaria: Variables moduladoras y redes sociales. Electron. J. Res. Educ. Psychol.; 2017; 10, pp. 771-788. [DOI: https://dx.doi.org/10.25115/ejrep.v10i27.1527]

28. Cantón-Mayo, I.; Cañón-Rodríguez, R.; Grande-de-Pedro, M. La comunicación como subdimensiones de la competencia digital en futuros maestros de primaria. Pixel-Bit. Rev. Medios Educ.; 2016; 50, pp. 33-47. [DOI: https://dx.doi.org/10.12795/pixelbit.2017.i50.02]

29. García, L.R.; Benedito, J.R.M. Perspectiva de los jóvenes sobre seguridad y privacidad en las redes sociales. Icono14; 2016; 14, pp. 24-49. [DOI: https://dx.doi.org/10.7195/ri14.v14i1.885]

30. Gutiérrez, N.D.C. Hábitos del Usuario Mexicano y la Infoxicación en Internet. 2016; Available online: http://hdl.handle.net/20.500.11777/2460 (accessed on 30 September 2021).

31. Roldán, E.L.; Ayala, M.C.; Pérez Pineda, D.P.; Romero, N.Y. Redes sociales de apoyo a la crianza de los menores en etapa escolar primaria. Rev. Cient. Gen. José María Córdova; 2016; 14, pp. 73-95. [DOI: https://dx.doi.org/10.21830/19006586.43]

32. Ruiz-Palmero, J.; Sánchez-Rodríguez, J.; Trujillo-Torres, J.M. Utilización de Internet y dependencia a teléfonos móviles en adolescentes. Rev. Latinoam. Cienc. Soc. Niñez Juv.; 2016; 14, pp. 1357-1369.

33. Araujo, E.D. Indicadores de adicción a las redes sociales en universitarios de Lima. Rev. Digit. Investig. Docencia Univ.; 2016; 10, pp. 48-58. [DOI: https://dx.doi.org/10.19083/ridu.10.494]

34. Bejarano, G.M.; Guerrero, J.P.; Zúñiga, D.Q. Adicción a facebook y habilidades sociales en estudiantes de una universidad privada de Lima. Rev. Cient. Cienc. Salud; 2015; 7, pp. 7-15. [DOI: https://dx.doi.org/10.17162/rccs.v7i1.985]

35. Mercedes, G. Migración de los jóvenes españoles en redes sociales, de Tuenti a Facebook y de Facebook a Instagram: La segunda migración. ICONO 14 Rev. Comun. Tecnol. Emerg.; 2015; 13, pp. 48-72. [DOI: https://dx.doi.org/10.7195/ri14.v13i2.821]

36. Montalvo, J.F.; Vélez, A.P.; Irazabal, I. Hábitos de uso y conductas de riesgo en Internet en la preadolescencia. Comun. Rev. Cient. Iberoam. Comun. Educ.; 2015; 44, pp. 113-120. [DOI: https://dx.doi.org/10.3916/c44-2015-12]

37. Moreira, M.A.; Machado, J.F.B.; Santos, M. Educar a la generación de los Millennials como ciudadanos cultos del ciberespacio: Apuntes para la alfabetización digital. Rev. Estud. Juv.; 2015; 109, pp. 13-32.

38. Sánchez-Rodríguez, J. Uso problemático de las redes sociales en estudiantes universitarios/Problematic use of social networks in university students. Rev. Complut. Educ.; 2015; 26, pp. 159-174. [DOI: https://dx.doi.org/10.5209/rev_RCED.2015.v26.46360]

39. Organic Law 3/2018, of 5 December, on Personal Data Protection and Guarantee of Digital Rights. 2018; Available online: https://www.pbjuridico.com/the-organic-law-3-2018-of-december-5-on-data-protection-and-guarantee-of-digital-rights-enters-into-force-2/?lang=en (accessed on 30 September 2021).

40. Cabero, J.; Córdoba, M. Inclusión educativa: Inclusión digital. Rev. Educ. Inclusiva; 2009; 2, pp. 61-77. Available online: http://hdl.handle.net/10272/11296 (accessed on 30 September 2021).

41. Pérez, L. Usabilidad de los Entornos Personales de Aprendizaje Institucional. J. Educ. Teach. Train.; 2016; 7, pp. 77-94. Available online: http://hdl.handle.net/11201/149550 (accessed on 30 September 2021).

42. Castañeda, L.; Gutiérrez, I. Redes sociales y otros tejidos online para conectar personas. Aprendizajes con REDES Sociales: Tejidos EDUCATIVOS para los Nuevos Entornos; Castañeda, L. Editorial MAD: Sevilla, Spain, 2010; pp. 16-40.

43. Horizon, I. Higher Education Edition; The New Media Consortium: Austin, TX, USA, 2014.

44. Cabero, J.; y Llorente, M.C. La aplicación del juicio de experto como técnica de evaluación de las tecnologías de la información (TIC). Eduweb. Rev. Tecnol. Inf. Comun. Educ.; 2013; 7, pp. 11-22. Available online: http://servicio.bc.uc.edu.ve/educacion/eduweb/v7n2/art01.pdf (accessed on 30 September 2021).

45. Malla, F.; Zabala, I. La previsión del futuro en la empresa (III): El método Delphi. Estud. Empres.; 1978; 39, pp. 13-24.

46. Gordon, T.J. The Delphi Method. 1994; Available online: http://www.futurovenezuela.org/_curso/5-delphi.pdf (accessed on 30 September 2021).

47. Landeta, J. El Método Delphi: Una Técnica de Previsión del Futuro; Ariel: Barcelona, Spain, 2002.

48. García, L.; Fernández, S.J. Procedimiento de aplicación del trabajo creativo en grupo de expertos. Ing. Energ.; 2008; 29, pp. 46-50. Available online: http://www.redalyc.org/articulo.oa?id=329127758006 (accessed on 30 September 2021).

49. Barroso, J.M.; Cabero, J. La utilización del juicio de experto para la evaluación de TIC: El coeficiente de competencia experta. Rev. Pedagog.; 2013; 65, pp. 25-38. [DOI: https://dx.doi.org/10.13042/brp.2013.65202]

50. Bernal, J.J.; Martínez, M.D.; Sánchez, J.F. Modelización de los Factores más Importantes que Caracterizan un Sitio en la Red; ASEPUMA: Valencia, Spain, 2004; Available online: https://www.um.es/asepuma04/comunica/bernal_martinez_sanchez.pdf (accessed on 30 September 2021).

51. Schulenberg, S.E.; Melton, A.M.A. A Confirmatory Factor-Analytic Evaluation of the Purpose in Life Test: Preliminary Psychometric Support for a Replicable Two-Factor Model. J. Happiness Stud.; 2010; 11, pp. 95-111. [DOI: https://dx.doi.org/10.1007/s10902-008-9124-3]

52. Bisquerra, R.; López-Gonzalez, L. Validación y Análisis de una Escala Breve para Evaluar el Clima de Clase en Educación Secundaria. 2016; Available online: http://online.ucv.es/resolucion/files/L%C3%B3pez-L.-y-Bisquerra (accessed on 30 September 2021).