-
Notifications
You must be signed in to change notification settings - Fork 0
/
07_analysis.qmd
205 lines (125 loc) · 14.4 KB
/
07_analysis.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
---
title: "Analyzing Qualitative Data"
filters:
- naquiz
---
## The Role of Contextuality
Qualitative data can be explored and re-analyzed to uncover hidden meanings and deeply unfold historical, social, and cultural contexts and their entanglement with human subjects' attitudes, behaviors, and opinions. A crucial contextual issue regarding qualitative data concerns the ‘traces’ left by different perspectives on the material. Texts, whether from interviews, social media, or embedded in an artifact, are not just produced under certain material conditions rooted within socio-cultural contexts, but they are also produced to do something and become part of the context to be understood.
## A Few Words about Secondary Analysis
Archived qualitative data offers opportunities for reanalysis, reinterpretation, and comparison with existing and newly collected data sources. While many steps in secondary analysis parallel those of primary data analysis—such as data processing, analysis, and quality control—unique challenges arise, particularly in aligning the archived data with the specific objectives of the new study and assessing the data's value in that context.
The success of secondary analysis is also heavily dependent on the quality and comprehensiveness of the accompanying documentation, which must provide detailed context about the original data collection, including the methodology, sampling, and any potential biases. Without this, the data may be misinterpreted, leading to inaccurate conclusions or limiting its applicability to new research questions. In future hands-on activities of this course, we will reuse pre-existing data and assess the documentation provided. Later, we will also explore recommendations on what type of metadata and documentation should be preserved and archived along with the data.
## Understanding Qualitative Data Analysis Methods
### Coding & Themes
Picture this: a treasure trove of data brimming with stories, emotions, and insights waiting to be uncovered. That's the realm of qualitative data analysis — a journey through the complexities of human attitudes, behaviors, and perceptions to unearth the gems that lie within.
Qualitative analysis is about finding patterns, unfolding both explicit and implicit attitudes, behaviors, and beliefs, understanding meanings, and making sense of findings concerning the research questions at hand.
This process involves identifying and assigning "codes," which are specific labels applied to small pieces of data, such as text fragments, attributes of objects or photographs, or key topics in video recordings. These codes act as foundational building blocks that, when further developed and refined, can combine to form "themes." Themes are broader concepts that emerge from grouping related codes, offering a deeper interpretation of the data. It is an outcome of coding, categorization, and analytic reflection. In other words, codes provide detailed elements that contribute to the development of broader themes. Coding is the initial step of analyzing data by assigning labels to relevant sections while theming involves interpreting and grouping codes to identify recurring patterns and meanings.
{width="549"}
Let's imagine you’ve conducted a study on students’ perceptions of online learning. The responses you collected highlight various challenges participants face, such as:
- Challenges for active participation in discussions
- Difficulties staying focused and motivated
- Feeling of isolation and lack of social interactions with other students and instructors
- Internet connectivity and bandwidth issues
- Learning Management System (software-related) problems
*Assigning Codes*
To analyze these responses effectively, you can assign short codes to capture the key aspects of the student's experiences:
1. Engagement: Refers to the challenges students face in actively participating in discussions.
2. Focus & Motivation: Represents students' difficulties in maintaining concentration and motivation.
3. Isolation: Reflects feelings of loneliness or disconnection from peers and instructors in the online learning environment.
4. Technical Issues: Encompasses problems related to internet connectivity, bandwidth limitations, and software malfunctions.
*Developing Themes*
Once you have your codes, you can group them into broader themes that provide deeper insights into the students' experiences:
1. Barriers to Active Participation: This theme includes the code "Engagement" and "Isolation" and highlights the obstacles preventing students from joining discussions effectively.
2. Challenges in Learning Environment: This theme combines "Focus and Motivation" with "Technical Issues," reflecting how psychological and technical difficulties impact students' learning experiences.
::: {.callout-important collapse="true"}
## Can we start with themes?
Every research project begins with some foundational knowledge, whether drawn from existing literature or from the researchers’ assumptions developed while exploring their topic and creating data collection instruments. So, researchers often have general categories in mind. However, these initial categories are unlikely to be fully refined or precise at the outset. This is where multiple iterations and deeper engagement with the data become essential.
:::
### Approaches to Data Analysis
Researchers use various methods to extract codes and develop themes from qualitative data, such as *thematic analysis*, *content analysis*, *comparative analysis*, and *discourse analysis*, to unearth and understand the complexities of human experiences, attitudes, behaviors, and perceptions captured in qualitative data. The goal is to generate rich and nuanced understandings of phenomena rather than producing numerical summaries or generalizable findings. These methods have been used to inform theory development, policy, and practice across disciplines and research domains and applied to various data sources, including written documents, social media posts, news articles, advertisements, photographs, videos, and interviews.
- *Thematic analysis*: perhaps the most common method for qualitative data analysis, it allows researchers to spot overarching topics and main themes in the data that uncover recurring ideas, topics, or concepts.
- *Content analysis*: digs deeper into themes to see how often they appear. It's like zooming in to see the details. Content analysis involves systematically analyzing and interpreting the content of textual, visual, or audio materials to uncover patterns, trends, or meanings. It focuses on the specific elements within the content, such as words, phrases, images, or themes, rather than the broader themes or concepts.
- *Comparative analysis*: delves into the intricate web of causal relationships between events and outcomes across diverse cases. By scrutinizing the nuances and variations, it focuses on causal relationships between events and outcomes in different cases.
Discourse analysis helps us understand how language reflects different ideas and cultures. It focuses on spoken or written conversational language.
- *Sentiment analysis*: is a branch of discourse or content analysis particularly interested in determining whether the emotional tone of a message or discourse (speech or written) is positive, negative, or neutral or exploring a broader spectrum of sentiments. Depending on the corpus of interest, it can also be heavily computational and quantitative-oriented.
#### Thematic Analysis
Our practical exercises will focus on the most common approach; *thematic analysis*. So, here is a recommended workflow we suggest you follow for this method:
 adapted from [Braun, V., & Clarke, V. (2006)](https://doi.org/10.1191/1478088706qp063oa){target='_blank'}](images/thematic-analysis-workflow.png){fig-alt="An illustration of Braun and Clarke’s thematic analysis approach for developing themes from qualitative data (adapted from Braun & Clarke, 2006, p 87)." fig-align="left" width="500"}
Whether you're new to research or a pro, qualitative analysis is an active process of reflexivity in which your subjective experience and pre-knowledge about the phenomenon of interest inform your process of making sense of the data and finding patterns and relationships. It is all about close familiarization with the data, categorizing, discovering, reviewing, and iterating until you find meaningful insights to inform your research questions before you can draw conclusions and articulate your findings.
Data can be arranged and coded following two types of approaches: a more inductive (bottom-up) approach, where insights and themes emerge more organically and directly from the data without prior assumptions, or even a combination of both as researchers engage with and learn more details from the data. Or a deductive (top-down) approach, where researchers apply preexisting theoretical frameworks or concepts to the data.
{width="746"}
Adapted from: <https://delvetool.com/blog/deductiveinductive>
The top-down approach begins with a specific theoretical framework or research question. Data collection and analysis are guided by preconceived hypotheses, focusing on confirming or refuting these hypotheses. While it provides more direction to the research process, it may limit alternative perspectives or emergent themes not accounted for in the initial framework. In contrast, the bottom-up approach is exploratory and highly flexible as there are no predetermined categories or hypotheses; it has its basis in grounded theory, meaning that themes emerge directly from the data through open coding and analysis, allowing a wide range of topics and potential unexpected insights to be extracted from the data.
It’s important to remember that, although it’s helpful to understand the differences between these two approaches, they can actually work well together. Combining them allows you to take advantage of each method’s strengths, leading to a richer and more nuanced analysis that incorporates real-world data and theoretical insights.
Check our handout for more information on these two approaches:
This handout provides a compilation with some helpful tips:
<iframe width="50%" height="800" src="https://rcd.ucsb.edu/sites/default/files/2024-06/DLS-202406-QualCoding.pdf">
</iframe>
Source: UCSB Library Data Literacy Series ([perma.cc/4L6T-4ND5](https://perma.cc/4L6T-4ND5){target='_blank'}).
### 🧠 Knowledge Checking
Based on our discussion about bottom-up versus top-down approaches to coding, please answer the following questions:
::: question
***What is the primary focus of top-down coding?***
::: choices
::: {.choice .correct-choice}
Applying pre-existing categories or frameworks to the data.
:::
::: choice
Identifying themes that emerge directly from the data.
:::
::: choice
Allowing codes to develop organically without prior assumptions.
:::
::: choice
Analyzing data without an structured approach.
:::
:::
:::
::: question
***Which of the following best describes bottom-up coding?***
::: choices
::: choice
It begins with hypotheses that guide the analysis.
:::
::: choice
It utilizes existing literature to create codes.
:::
::: choice
It follows a structured approach to analyze the data.
:::
::: {.choice .correct-choice}
It starts with the raw data and develops codes from specific observations.
:::
:::
:::
::: question
***Which of the following scenarios is most suitable for using top-down coding?***
::: choices
::: choice
A study aiming to discover new patterns in data.
:::
::: {.choice .correct-choice}
A study testing a specific theory or framework.
:::
::: choice
Explore a new topic, not yet well-established in the literature.
:::
:::
:::
::: {.callout-note collapse="true"}
### A Note about Saturation
How many interviews are enough? Determining how many qualitative interviews are sufficient is a complex question that often yields nuanced answers.
Data saturation is frequently cited as a measure of quality in qualitative research and used to justify purposive small samples. It is primarily reliant on the point at which little new information is obtained, known as thematic saturation. In theory, thematic saturation occurs when interviews or observations start lacking variability to reveal only recurring themes, indicating that further data collection is unlikely to yield new insights. When researchers observe a plateau in these findings, it often signals that they may have reached an adequate sample size.
However, assessing saturation is not straightforward and is much left to interpretation, as it involves subjective judgments about rigor, precision, and confidence. Often, researchers may claim to have achieved saturation to meet certain criteria without adequately explaining what it entails in their specific context or how they reached that conclusion.
This lack of justification can undermine the transparency and credibility of qualitative studies. Data saturation is often insufficiently examined and can be problematic and contradictory when applied broadly to qualitative research. It might be more appropriate to reserve the concept of data saturation for grounded theory, where there is a clear framework for its application.
While there is no consensus on minimal sample sizes for qualitative research. There is a general recommendation of 6–30 interviews for most qualitative research projects.
:::
::: {.callout-tip collapse="true"}
### Time to Practice! 💪🏼
**Optional**
Now that we've gained some knowledge about coding, let's explore how we can put these concepts into practice. In pairs, access the [worksheet](https://docs.google.com/document/d/1Kxjx_Wp0PQ29e3Xhs1GZXTW2K3Da4LxnVd8zq1N2afY/edit){target='_blank'} and follow the instructions.
:::
We will get more into this in later episodes, where we will have a chance to perform coding assisted by a QDA open and free software.
------------------------------------------------------------------------
**Recommended/Cited Sources:**
Guest, G., Namey, E., & Chen, M. (2020). A simple method to assess and report thematic saturation in qualitative research. PloS one, 15(5), e0232076. <https://doi.org/10.1371/journal.pone.0232076>
van Rijnsoever F. J. (2017). (I Can't Get No) Saturation: A simulation and guidelines for sample sizes in qualitative research. PloS one, 12(7), e0181689. <https://doi.org/10.1371/journal.pone.0181689>