Rehability and vahdity in Quahtative Research Revisited and the Role of AI

Abstract

The evolving nature of research and the integration of new technologies, such as Artificial Intelligence (AI), introducé new challenges and threats, necessitating a re-examination of the key criteria of reliability and validity. In this paper, I revisit these concepts in the context of quahtative research, highlighting their evolving meanings and significance. I begin by defining reliability and validity and explaining their importance. Following this, I discuss strategies for establishing these criteria within quahtative studies and offer guidelines for maintaining rigor in the current research landscape. While emphasizing the need for human oversight, intervention, and interpretation, I underscore how to ensure trustworthiness in quahtative research outcomes amidst technological advancements.

Headnote

The evolving nature of research and the integration of new technologies, such as Artificial Intelligence (AI), introducé new challenges and threats, necessitating a re-examination of the key criteria of reliability and validity. In this paper, I revisit these concepts in the context of quahtative research, highlighting their evolving meanings and significance. I begin by defining reliability and validity and explaining their importance. Following this, I discuss strategies for establishing these criteria within quahtative studies and offer guidelines for maintaining rigor in the current research landscape. While emphasizing the need for human oversight, intervention, and interpretation, I underscore how to ensure trustworthiness in quahtative research outcomes amidst technological advancements.

Keywords: reliability, validity, technology, artificial intelligence, AI

Introduction

Building upon previous theoretical discussions on the topic of reliability and validity (Coleman, 2022; Golafshani, 2003; Rosé &Johnson, 2020), I discuss the use of reliability and validity in quahtative studies, as informed and shaped by specific dynamics and changes in the current research scène, such as the use of advanced technological means to conduct research and analyze quahtative data. First, the meanings and understandings of reliability and validity are explained, foliowed by a justification and clarification of the need to revisit these crucial research criteria. Second, the topic of establishing reliability and validity within the context of quahtative research is covered in more detail, emphasizing the importance of establishing rigor in research due to changes in the recent research landscape, mainly as a result of Artificial Intelligence (AI). Third, some crucial suggestions are provided for researchers to consider while seeking to establish the reliability and validity of their research output. As the author of this paper, I have been specializing in quahtative inquiry and quahtative analysis, as well as the implications of AI in research and social fields. My interest in AI arises firom its growing utility across various fields and in research, coupled with concerns about potential risks, including the diminishing human element, ethical and academie integrity changes. Consequently, I am exploring the applications, potentials, dynamics, limitations, and implications of AI in research and in this specific paper, in regards to reliability and validity.

Understandings of Reliability and Validity

Reliability, as a term, has evolved in its usage and meaning over time, with its roots deeply embedded in the concept of binding or connecting securely. In Latin, religdre was used to describe the act of binding or tying something securely (Etymonline, 2021). Over time, the focus shifted firom the physical act of binding to the metaphorical sense of depending on, or trusting in someone or something. This shift reflects the transition firom a literal binding to a more abstract concept of reliance, and "trust." Just as religdre implies a firm and dependable connection, reliability in research ensures that the findings are consistently tied to the methods used, allowing for stable and repeatable results, highlighting the importance of a secure and dependable research process to achieve accurate and trustworthy outcomes. In the context of research, reliability refers to the consistency and dependability of the data collection and analysis process. Crucial notions in the context of reliability are consistency, replicability, and repeatability of results or observations (Golafshani, 2003; Kirk &Miller, 1986). In quantitative research, this typically involves ensuring that measurements yield the same results under consistent conditions. In qualitative research, reliability takes on a different yet equally crucial meaning. It pertains to the trustworthiness of the procedures and the stability of the interpretations over time. Achieving reliability in qualitative research implies that the research process is transparent and can be foliowed by others to produce similar findings, even if exact replication is not possible due to the nature of qualitative data. That is, due to the unique, context-specific, and subjective nature of qualitative data.

Validity, as a term, is derived firom the Latin word validus, meaning strong and robust. The term's origin reflects the essential quality that research must possess strength in terms of the research process and findings, mirroring the strength implied by the term's origin, and not being weak, debatable, misleading, or of limited value. The term has been closely linked with a positivistic approach. Nonetheless, it has been embraced by researchers as a shared Standard for both quantitative and qualitative research (Adcock &Collier, 2001), with Maxwell (1992) providing insights into validity within the context of qualitative research. Whittemore et al. (2001) distinguished between primary (such as credibility and authenticity) and secondary (such as explicitness, thoroughness, and sensitivity) validity criteria in qualitative research. While referring to validity within the context of qualitative research, Rosé and Johnson (2020) highlighted the key notion of trustworthiness, with techniques such as sharing anonymized collected data with participants, using multiple sources, and providing rich descriptions.

Is There a Need to Revisit Reliability and Validity?

One may argue for the need to constantly revisit crucial research concepts and criteria that have been used for decades to evaluate the overall rigor and integrity of research, given the abundance of studies and reports that explore these issues in depth (Cypress, 2017; Kirk &Miller, 1986; Morse et al, 2002; Noble &Smith, 2015). Of course, this has not stopped other researchers from addressing these issues while taking into consideration shifts in research and trends (Cho &Trent, 2006). As a researcher and author of this paper, I have also argued the usefulness of a rather repetitive paper on the same topic. Nonetheless, the need to revisit the concepts of reliability and validity arises from several evolving factors within the current research landscape. Research methodologies continually evolve, introducing new techniques and approaches, while the rise of mixed-methods necessitates a re-examination of how reliability and validity are applied. The increasing emphasis on understanding the context and subjective experiences in qualitative research also calls for a re-evaluation of these concepts.

Additionally, and perhaps most importantly, there is a growing emphasis on methodological rigor and transparency in research, especially in an era when concerns over biases and ethical issues are heightened due to the influence of Artificial Intelligence (AI), the proliferation of fake news, and the pervasive spread of misinformation (Gichoya et al, 2023; Varsha, 2023). Transparency in data collection and analysis is essential to mitigate ethical concerns and ensure that research findings and conclusions have not been influenced by biases. As AI technologies, such as machine learning and natural language processing, become integral to data collection and analysis (Christou, 2024), traditional notions of reliability and validity must be adapted to address the new challenges introduced. Similarly, with AI's involvement, it is essential to ensure that AI tools correctly capture and reflect the context of the data they analyze. Thus, as AI becomes increasingly prevalent in qualitative research, revisiting reliability and validity is imperative to uphold research rigor.

Establishing Reliability and Validity in Qualitative Research

In their article, Noble and Smith (2015) provided alternative terminologies associated with the credibility of qualitative research. Specifically, they referred to validity as the precision with which the findings accurately reflect the data in quantitative research. Alternatively, they used the terms truth value within the context of qualitative research. This, as they noted, recognizes that multiple realities exist and that findings should accurately represent the participants' experiences and their realities (such as through the recording of interviews to allow repeated revisiting of the data to check emerging themes, and the use of rich and thick verbatim extracts). They also referred to reliability as the consistency of the analytical procedures in quantitative research. Alternatively, within the context of qualitative research, they used the term consistency, which relates to trustworthiness, where the researcher maintains a decision trail with transparent and clear decisions so that other researchers may arrive at similar or comparable findings. They also used neutrality or confirmability, which acknowledges the complexity of engagement with participants and differentiates the researcher's position and perspectives from participants' accounts. As such, they proposed certain strategies to address consistency and neutrality, which, as stated previously, correspond to reliability, such as being transparent and providing a clear description of the whole research process. Also, in the case of specific forms of qualitative analyses, such as thematic analysis, they recommended discussing the emerging themes with research team members who have relevant qualitative research expertise, to challenge assumptions and reach consensus.

Other researchers have made clear reference to the exact terms of reliability and/or validity while describing how these are established in qualitative research (Morse et al, 2002; Whittemore et al, 2001). For example, Rosé and Johnson (2020) provided various techniques to address validity issues in qualitative research, such as establishing prolonged engagement with participants, member checking (having participants see results in a form of a draft report and considering their feedback), and offering rich descriptions obtained through the research process.

As an academie and qualitative (or better, qualitative-oriented) researcher myself, I find it challenging to distinguish which specific tactics and techniques address validity or reliability within the context of qualitative research, or whether to use other alternative terms such as trustworthiness, instead of validity and reliability. Before I am being criticized for not examining the notions in depth, I would urge readers to consider the bewilderment resulting from the plethora of papers discussing these issues from differing prisms, researcher perspectives, and their terminology choices. Whether researchers choose to use the terms validity and reliability in their qualitative inquiry or replace them with alternative terms and notions, what remains crucial is maintaining rigor throughout the research process. This rigor refers to the strict application of scientific and research principles and standards across different phases of the research process, such as methodology, analysis, interpretation, and delivery of findings, while acknowledging the idiosyncrasies of qualitative inquiry, and the subjectivity and reflexivity it entails (Braun &Clarke, 2019; Davies &Dodd, 2002).

Rigor in Research in the Era of Artificial Intelligence

The simulation of human intelligence in machines that are programmed to think, learn, and problem-solve in ways that mimic human cognitive functions is referred to as AI. Whether machines can replace researchers is a large and debatable topic that falls beyond the scope of this paper. What is certain, though, is that AI's impact on qualitative research is particularly profound. AI algorithms are transforming the research landscape by automating the research process and the analysis of large volumes of qualitative data. Qualitative data analysis software tools have incorporated AI technology to enhance their capabilities. AI tools may explain complex concepts, theories, and methodologies, summarize information from research papers, highlight important trends in a specific field, identify patterns and sentiments within qualitative data, and aid the interpretation and writing process of articles. Key qualitative data analysis software (also referred to by the acronym QDA) have also incorporated AI technologies to assist researchers. For instance, NVivo claims that its autocoding with AI allows researchers to spend less time on the time-consuming process of coding, and more on analyzing results. As Lumivero (2025) claims, NVivo autocoding makes it easy for researchers to gain an understanding of their data. MAXQDA has also incorporated what it calls AI Assist, designed to support the analysis process. AI Coding automates coding by analyzing documents and providing coding recommendations for text segments that match the coding criteria set by the researcher. As MAXQDA (2025) states, analysts can benefit from the potential of dialogue by chatting and engaging with their data. Similarly, ATLAS.ti, powered by OpenAFs GPT model, introduced AI Coding Bèta, highlighting benefits such as reducing the overall data analysis time by up to 90%, uncovering insights and patterns that might have been missed, and shifting the focus to data interpretation (ATLAS.ti, 2025).

Despite the opportunities that AI technology offers within the research landscape, it also presents various challenges and even threats (Christou, 2023). Some key threats include privacy and ethical concerns, and bias (i.e., if the data used to train AI systems is biased, the analysis and output will also be biased). Also, the loss of nuance and meaning (i.e., human researchers can interpret context, tone, nuance, and emotions in ways that AI might miss), and lack of transparency, where researchers are unable to understand how an Al-assisted system has conducted a specific analysis or reached a particular conclusion. There is also the issue of lack of human control in the analysis process, and over-reliance on technology (where researchers become overly dependent on AI tools while neglecting their own analytical skills).

While acknowledging such perils, key QDA software providers offer reassuring comments on how AI is used in the analysis process. For instance, ATLAS.ti (2025) states on their official website: "Think of it (referring to AI Coding Bèta) as your personal research assistant... ensuring 100% transparency and human supervision." Even so, it may be argued that researchers and students might rum to specific AI tools, such as ChatGPT (a widely used Large Language Model that employs AI to understand and generate human-like text), for coding their data or even directly generating themes. Despite the efficiency of such tools, this raises critical questions about how the tooi was utilized by the researcher, how codes or themes were generated, and how conclusions were reached. In recent years, there have been increasing scholarly debates and various theoretical discussions in qualitative research academie circles and conferences regarding both the opportunities and the perils associated with Al-assisted analysis. For instance, a study by Morgan (2023) concluded that ChatGPT performed reasonably well in qualitative analysis and successfully reproduced concrete, descriptive themes, yet was less effective at identifying subtle and interpretive themes.

Given the rapid advancement and evolution of AI, it is critical to closely monitor its impact on qualitative inquiry and analysis. Issues related to research rigor remain paramount and must be continually considered and addressed by researchers, as society moves toward an era of Artificial General Intelligence (AGI) and AI Superintelligence, characterized by advanced AI with increasingly augmented capabilities and possible perils, including within the research context.

Discussion and Recommendations

Returning to the very origins and core principles of reliability, validity, and associated concepts (or should I simply use rigor) in research, I urge researchers to establish a solid (validus) and strongly binding (religare) case, where all their procedures are carefully considered, applied, and well-justifiedthroughoutthe research process. This includes selecting the appropriate and relevant methodological approach to address the research aim, maintaining a well-documented process for choosing and engaging participants in the study, and clearly outlining the exact process foliowed in the analysis (phase by phase) to develop specific themes (as in the case of thematic analysis, if applicable) and reach conclusions. In simple words, researchers should be rigorous, meaning thorough and accurate while following and documenting research procedures. By doing so, we build the trustworthiness of our study, establishing trust between us (as researchers, and the processes that we have foliowed), and the others (whoever may read, find useful, and rely on our findings). While taking into consideration the challenges that AI may pose in this process, I offer certain key suggestions in this regard.

First, we must consider whether there is an actual need to use AI technology for our research, and for what purpose. Researchers may consult language models for simple tasks, such as help with their writing process, or find a suitable theoretical framework. Even so, there are specific AI tools that may claim to perform specific forms of analysis, such as thematic analysis. Careful consideration should be given, since in many cases what some AI systems do is content analysis, relying heavily on numbers, and not thematic analysis. Usually, if themes are supported by frequencies or numbers of terms, then this is a strong indicator that a content analysis was most likely performed, rather than a thematic analysis. Additionally, the use of AI tools, or the use of AI features in qualitative data analysis software, does not necessarily imply a better and deeper form of analysis compared to one performed in a more traditional manual marmer by researchers (as in Braun &Clarke, 2006; Christou, 2022). At the same time, an Al-deification culture should not be cultivated in which editors and reviewers underestimate or reject papers that do not make use of technological assistance and/or AI technology. If, however, a researcher chooses to use Al-assisted tools, a fundamental prerequisite is to develop a solid understanding of not only the system, but also what qualitative research and analysis truly entail. Radiker (2024) stresses the need for researchers to comprehend research methods prior to their use and AI means for differing research purposes.

Second, it is important to become aware of possible ethical and bias issues involved, and to address these as much as possible. Conceivably, this issue primarily pertains to AI engineers, developers, and organizations. Even so, researchers should use any form of AI technology for research purposes in an ethical and responsible way, while addressing any possible biases that may have affected the research outcome. Ethical research emphasizes fairness, integrity, and objectivity, while bias can compromise these values. Returning to the basics, when using Al-assisted tools for confidential interviews or their analysis, we must ensure that we use pseudonyms (anonymized) and such information that does directly or indirectly reveal the informant's identity. Besides this, it is very important that researchers inform participants regarding how their personal data will be analyzed and protected. The use of any AI tools or means should also be exposed by researchers in their research papers. Furthermore, researchers are encouraged to use systems that clearly explain their analysis process and how AI is used in this process, to manually verify codes against their raw data, and if possible, to conduct an inter-coder reliability check where researchers review the AI's categorization.

Third, it is advisable that researchers use AI tools and systems that allow for human intervention (such as, in the analysis process). Some examples were provided in the previous section of this paper. Even so, other AI tools may not be transparent in the marmer in which they reach output and conclusions. This affects the trust between us and the AI tool/outcomes, as well as between our research output and the receiving end. This poses certain crucial questions: Is information and analysis output generated or assisted by an AI tooi or system valid, and can we rely on it? In this regard, I would refer readers directly to Christina Silver's ongoing and helpful work. She is often engaged in expert panel discussions, international qualitative research conferences (Silver, 2024; Silver &Paulus, 2025), and podcasts, sharing her extensive expertise in the relationship between qualitative methodology and technology, and providing useful and practical examples of how to use various forms of technology including Al-assisted tools, in qualitative analysis.

Fourth, we must document clearly and well-justify all the procedures that we have foliowed throughout the research process. As researchers, we must avoid secrecy and obfuscation (withholding information, or making information intentionally unclear or confusing), particularly if we have used any form of AI technology. Even so, through the constant advancement of AI technology and its increased integration in our daily lives and in various research tools and systems, researchers will struggle to establish boundaries for AI exposure and use. For instance, researchers may not be able to distinguish whether a program/tooi is actually using AI, the extent to which it uses such technology, and if researchers are able to eliminate (if deemed necessary, or desirable) its use. Programs and tools (including some key QDAs) may state that they make use of AI technologies, yet may fail to explain to researchers clearly how AI is used, and their rationale behind suggesting a specific code, or even theme. In such, or similar instances, researchers may not be able to understand how a conclusion was made, and hence not able to explain or interpret it.

Fifth, we must ensure that the human element is not marginalized throughout the research process. In recent years, there is much discussion about human-centered AI, with institutes, such as Stanford University HAI (Human-Centered Artificial Intelligence), studying, guiding, and developing human-centered AI technologies and applications. Despite this, there is the crucial issue of securing the human element in research and analysis. Major QDA players, as previously explained, do allow this human intervention in the analysis (such as coding) process. Though this topic may not be directly linked to issues such as reliability and validity, how can our research be genuinely trustworthy if there is an over-reliance on technological (AI) means to perform various research tasks and reach conclusions? Where is the exhibition of analytical skills, and overall "human touch," which are crucial elements in qualitative inquiry that involve investigating and interpreting complex, context-rich data, subtleties of human experience, and social phenomena?

As a concluding statement, the above questions are crucial to consider, and important guidelines to follow in addressing reliability and validity issues in the constantly changing and evolving research landscape, particularly impacted by AI technology. Obviously, as AI technology evolves, we, academies and researchers, will be called to closely monitor, evaluate it, and propose ways in which AI is used rigorously, effectively and ethically in qualitative inquiry and analysis.

References

References

Adcock, R., &Collier, D. (2001). Measurement validity: A shared Standard for qualitative and quantitative research. American Political Science Review, 95(3), 529-546. httüs://doi.ora/10.1017/S0003055401003100

ATLAS.ti. (2025). Introducins: AI Codins Bèta powered bv OpenAI. https://atlasti.com/ai-coding-powered-bv-openai

Braun, V., &Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psvcholosv. 3(2), 77-101. https://doi.org/10.1191/1478088706ap063oa

Braun, V., &Clarke, V. (2019). Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, H(^), 589-597. https://doi.org/10.1080/2159676X.2019.1628806

Cho, J., &Trent, A. (2006). Validity in qualitative research revisited. Qualitative Research, 6(3), 319-340. https://doi.org/10.1177/1468794106065006

Christou, P. A. (2022). How to use thematic analysis in qualitative research. Journal of Qualitative Research in Tourism, 3(2), 79-95. https://doi.org/10.4337/iqrt.2023.0006

Christou, P. A. (2023). How to use artificial intelligence (AI) as a resource, methodological and analysis tooi in qualitative research? The Qualitative Report, 28(1), 1968-1980. https://doi.org/10.46743/2160-3715/2023.6406

Christou, P. A. (2024). Thematic analysis through artificial intelligence (AI). The Qualitative Report, 29(2), 560-576. https://doi.org/10.46743/2160-3715/2024.7046

Coleman, P. (2022). Validity and reliability within qualitative research for the caring sciences. International Journal of Caring Sciences, 14(3), 2041-2045.

Cypress, B. S. (2017). Rigor or reliability and validity in qualitative research: Perspectives, strategies, reconceptualization, and recommendations. Dimensions of Critical Care Nursins. 36(4). 253-263. https://doi.org/10.1097/DCC.0000000000000253

Davies, D., &Dodd, J. (2002). Qualitative research and the question of rigor. Qualitative Health Research. 12(2). 279-289. https://doi.org/10.1177/104973230201200211

Etymonline. (2021). Relv. https://www.etvmonline.com/word/relv.

Gichoya, J. W., Thomas, K., Celi, L. A., Safdar, N., Banerjee, I., Banja, J. D., Seyyed-Kalantari, L., Trivedi, H., &Purkayastha, S. (2023). AI pitfalls and what not to do: Mitigating bias in AI. The British Journal of Radiology, 9(5(1150), https://doi.org/10.1259/bir.20230023

Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The Qualitative Report. 8(4). 597-607. https://doi.org/10.46743/2160-3715/2003.1870

Kirk, J., &Miller, M. L. (1986). Reliability and validity in qualitative research (Vol. 1). Sage.

Lumivero. (2025). Nvivo automated coding with AI https://lumivero.com/resources/product-tutorials/nvivo-automated-coding-with-ai/

MAXQDA. (2025). Discover what is possible with AI Assist. https://www.maxqda.com/products/ai-assist.

Maxwell, J. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 62(3), 27'9-301.

Morgan, D. L. (2023). Exploring the use of artificial intelligence for qualitative data analysis: The case of ChatGPT. International Journal of Qualitative Methods, 22. https://doi.org/10.1177/16094069231211248

Morse, J. M., Barrett, M., Mayan, M., Olson, K., &Spiers, J. (2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 13-22. https://doi.Org/10.17763/haer.62.3.8323320856251826

Noble, H., &Smith, J. (2015). Issues of validity and reliability in qualitative research.

Evidence-basedNursins, 18(2), 34-35. https://doi.org/10.1136/eb-2015-102054

Radiker, S. (2024). AI in research: Opportunities and challenges. Documentation of the Symposium about the opportunities and challenges of AI in (qualitative) research at the MOIC 2024. https://www.maxqda.com/bloepost/ai-in-research-opportunities-and-challenges

Rosé, I, &Johnson, C. W. (2020). Contextualizing reliability and validity in qualitative research: Toward more rigorous and trustworthy qualitative social science in leisure research. Journal of Leisure Research, 51(A), 432-451. https://doi.org/10.1080/00222216.2020.1722042

Silver, C. (2024). UsingMAXQDAto analyze open-ended survey responses: An example firom an evaluation project. In S. Radiker &M. C. Gizzi (Eds.), The practice of qualitative data analysis: Research examples using MAXQDA (Vol. 2, p. 87). MAXQDA Press.

Silver, C, &Paulus, T. (2025). The process of QDA in the era of Generative-AI. Panel Discussion at the WCQR, 9th World Conference on Qualitative Research in Krakow.

Varsha, P. S. (2023). How can we manage biases in artificial intelligence systems-A systematic literature review. International Journal of Information Management Data Insights, 3(1). 100165. https://doi.Org/10.1016/i.iiimei.2023.100165

Whittemore, R., Chase, S. K., &Mandle, C. L. (2001). Validity in qualitative research. Qualitative Health Research, -^(4), 522-537. https://doi.org/10.1177/104973201129119299

Author Note

Prokopis A. Christou is an Assistant Professor at Cyprus University of Technology, specializing in qualitative inquiry and the implications of AI in research and social fields. He has published books and numerous articles in prestigious academie journals, including conceptual papers, critical perspective articles, systematic reviews, and empirical studies, such as ethnographic and phenomenological research. His interest in AI arises firom its growing utility across various fields, coupled with concerns about potential risks, including the diminishing human element, reduced investment in critical and evaluative thinking, and ethical and academie integrity challenges. Consequently, he has explored the applications, potentials, dynamics, limitations, and implications of AI in research. Please direct correspondence to prokopis. christou@cut. ac. cy

Author Contribution: The author (Prokopis Christou) contributed in all aspects of this paper.

Acknowledgment: While acknowledging the role of AI in research (as noted in Christou, 2023b), I declare that I have consulted programs and tools that utilize AI technology, such as NVivo and MAXQDA, to gain a deeper understanding of AI's application in research, and I used a natural language processing tooi (ChatGPT) for minor editing purposes.