Collecting data isn’t the beginning of a researcher’s journey. The real work starts before carrying out an interview. We believe data collection and tagging should go hand-in-hand to set your research up for success. With a little advance preparation, you can save time and reach insights faster using Marvin.
First, we break down the importance of tagging to qualitative research. We then run through a real-life example of how to effectively tag within Marvin to conduct a quick and efficient analysis. The faster you analyze your data, the faster you gain valuable insights about your users.
Table of Contents
- A Qualitative Research Refresher
- What is Tagging?
- Start with Output
- Draft a Discussion Guide
- Tagging, Coding or Labeling
- Pre-Interview
- During Interview
- Post-Interview
- Use Graphs
- Build Narratives
- Create a Report
A Qualitative Research Refresher
Humans have been counting, performing calculations and using quantitative methodology for thousands of years. Its qualitative counterpart only took off with the advent of the social sciences in the mid 1900’s, but is no less scientific. Today, a large amount of policy making depends on qualitative research.
Quantitative data analysis tests hypotheses:
“Design A is rated more favorably among Management Consultants than Design B.”
Qualitative data analysis is used to understand our environment better, and provides a deeper understanding of what we observe. Asking the same participants:
“What do you think of the design and experience of a research tool?”
Using data, qualitative analysis uses scientific rigor to build theories. These theories must be repeatable — multiple people looking at the same data must come to similar conclusions about it. This adds credibility to the qualitative research study and the narratives we uncover from the data. Crucial to conducting effective qualitative research is tagging.
What is Tagging?
Tags are the building blocks of qualitative research.
So what is tagging? Tagging (also known as “coding” or “labeling,” which we will use interchangeably throughout the article) essentially breaks down large chunks of qualitative data into smaller observations.
Tags help us turn unstructured data into semi-structured data, taking large and complex datasets and turning them into more manageable pieces. By tagging data, we provide context, delving deeper into understanding and analyzing these bite-sized observations.
Tagging in research has several benefits. It makes data:
- Analyzable - easy to look for patterns; tease out and report these patterns to build out narratives to make decisions. The science we alluded to earlier lies in extracting these patterns and narratives.
- Searchable & Discoverable - even if you are not directly involved in the study, you can always look at the data at any point in the future. Tagging gives us the opportunity to quickly slice and dice data and tie together observations from various different places. This helps broaden our research perspective.
- Shareable - by making observations accessible to your peers, you elevate the voice of the user and create user empathy across your organization. This also allows people to challenge your assumptions and builds confidence in research results.
Let's sink our teeth into a real-life example of how to use tagging within Marvin to conduct quick and effective qualitative research.
During the testing of our product (we promise this won’t get too meta), we interviewed several Management Consultants about their expectations and challenges using research tools. This is how we approached it:
[DISCLAIMER: The following is only a suggested method for conducting quick user research in Marvin. To each their own.]
Step 0: Start with Output
Sounds odd, doesn’t it? All scientific inquiry begins with a question (or questions). You don’t necessarily have to have a well-formed hypothesis when you begin — it can be unclear. However, when you craft questions, you’ll have a ballpark idea of potential answers that may emerge from your research.
Going into our study, we knew certain challenges would be more common for consultants than other professionals. We had no idea of this output ahead of time.
We also knew that certain features would be more useful to consultants. Here, we knew the features we had built. For this output, we were able to anticipate some answers that we might receive — this helped us to prepare the data before beginning our study.
We suggest thinking deeply about the output you have in mind. The more thought you put into how your project will be structured, i.e. the questions you are going to ask and the output that you may get, the easier it becomes during analysis.
Step 1: Draft a Discussion Guide
Good output comes from good input. A discussion guide is a set of questions, tasks or topics that a researcher will address during the research process. These topics enable us to better understand participants' experience and help guide research sessions. A discussion guide isn’t simply a set of questions, it's also foundational to making the tagging or coding process easier along the way.
To create a discussion guide, navigate to the project folder of your choice and click “Discussion Guides” in the sidebar. Hit “/” to begin drafting questions in the document.
The best discussion guides move from easy, specific questions to difficult, higher level ones. Guides help facilitate conversations and act as a reference tool for the researcher. Their purpose is to keep the research focused on the output we mentioned earlier.
Q: If I already have a hypothesis in mind, how do I create questions around it?
A: Turning a hypothesis into questions is an artform, whereby you’re not trying to expose too much of that hypothesis to your participants. Validating or invalidating the hypothesis is your job, not the participants. So, ask questions that go into the context of the hypothesis, not the actual hypothesis itself.
Example
Our hypothesis: Live Notes is very powerful and will resonate with our users.
We showed participants a design with and without Live Notes and asked the following questions:
- What is your process like?
- Do you take notes during a conversation?
- Does this design make your workflow easier?
Step 2: Tagging, Coding or Labeling
Pre-Interview Prep
Tags, codes, or labels are a powerful and flexible tool that make qualitative analysis more useful.
One of our biggest time-savers? Tag your questions! Starting with questions gives you a launch off point on how to group and categorize your labels. From your discussion guide, you can start to build out labels in advance that you know will aid your analysis.
We’ve labeled our labels below (we lied about going meta):
- Use foundational labels based on overarching topics or subtopics. “Goal/Aim”, “Background” and “Process” are labels we knew would come in handy during our interviews.
- With structured and semi-structured questions, you can easily prepare labels in advance. We showed our participants several designs and asked them to provide feedback with responses on a 5-point Likert scale.
- If you have to map user personas, creating a simple label for each can help with analyzing personas individually. Here, we wanted to tag any mention of our partners.
- Establish and build a label taxonomy and hierarchy. To build on the user persona example, we built parent and child relationships so we can go into more detail and view data based on which partner the interviewee was talking about.
You’ll notice that the Likert tags above are purple, while others are in Marvin pink. The reason - templates. We suggest creating templates to make life easier and save precious time for you and other Marvin users.
There are two ways to create templates:
- Project>>Label - Add and modify labels to your project using the text box at the top of the page. Drag and drop labels to create a taxonomy and hierarchy. Hit the button “Save labels as template” when you’re done.
- Home>>Templates - Hit “New label template” and follow the on-screen instructions.
Templates create standardization across the company, and makes label taxonomy easy to import into a project. Import templates by using the pink “+ Add labels” button at the top right corner. If you have recurring interview questions / responses across interviews and projects, you can save time and effort by saving templates early.
“More is greater than less” - this rather obvious statement applies to labeling. At the beginning of your project, use as many labels as you can. Labels are a flexible and powerful tool that you should continue to modify and refine as you dig deeper into your analysis.
Use Live Notes during Interviews
It’s estimated that for every hour of interview recorded, a qualitative researcher spends 3-7 hours analyzing and coding the data afterwards. Marvin does the heavy lifting and speeds up this process by building a verbatim transcript of every interview, easily downloadable in a time-stamped text format. Live Notes is a collaborative note-taking functionality that encourages researchers and their peers to annotate a transcript in real time. We spoke earlier about adding context to your observations - Live Notes frees up your time to add this context to what interviewees are saying.
Add a note, a label, tag a question from the discussion guide, or simply add a bookmark at key moments during your interview.
You can create new questions and labels or lean on the existing library you’ve already created. Use the pane on the right to click on the one you want, wait for the response to be completed and hit “Return”. Alternatively, type in “/” for questions and “#” for labels to pull up your preloaded data, or create new ones on the fly. Hit “Return” to close the loop. If you’re new to the process, simply bookmark key moments during the interview - just press “Return”. Revisit these bookmarks to annotate and add notes later. Every note you make will be appropriately time stamped according to when you created it during the interview.
Tagging in real-time is hard work. Finding consistency in your labels and maintaining that within and across interviews can be confusing while they’re going on. A significant amount of tagging can be done in real time by tagging questions. Using the discussion guide (we told you it was worth it) is an easy way to tag questions at the beginning without having to revisit the data frequently. Tag questions from the discussion guide as you navigate your interview - the minimum Live Note taking we recommend is to tag questions and their responses. Tagging with Live Notes helps organize your data for analysis.
Post-Interview Analysis
Some might prefer to code using Marvin’s Annotation tool - where you interact with one interview at a time. You can highlight parts of the transcript and add labels, questions and notes on the right.
This method enables the researcher to conduct an in-depth review of individual recordings, but isn’t necessarily time-efficient. Personally, we use this view to look for new tags, ones that haven’t been created in advance.
So how do you dive deeper into coding the data? We recommend starting from the Analyze page:
At first glance, going into 377 notes seems daunting. Remember when we said “Tag your questions”? Here’s where the fruits of your labor emerge. Notice below how we grouped by questions at the top of the page:
Marvin categorized the 377 notes across our questions and now we have a clean, global view of the data immediately. This gives us a sandbox or playground where we can tinker with existing labels and create and refine our taxonomy going through the material one topic at a time. Tagging questions at the outset is helpful as it makes interviews easy to go through without having to go through entire recordings.
Go into each response by clicking on the pencil icon and add any labels directly to the interview. We added a new label “Recommendation” to the third participant’s response to the goal/aim question(above).
We then went back to the Labels page and nested the newly created “Recommendation” label under the foundational “Goal/Aim” label from earlier.
This interplay between Labels and Analyze sections is responsible for the bulk of your analysis. Looking at data through the lens of questions gives us perspective on how to create and group labels together and establish hierarchies and relationships between labels.
Going question-by-question is a systematic way to deepen our understanding of the data. As you go through the questions, look out for recurrences of old patterns and potential new patterns as well. Start with certain labels and as you go through the data, you will re-use and refine old ones. This constant back and forth establishes consistency in your approach to labeling data. We recommend this step after (at least) your second interview, as that provides you with a substantial enough dataset to begin analysis.
Step 3: Use Graphs
Once you begin building relationships and tagging your data, resume thinking about the output of your study. Graphs aren’t used exclusively in quantitative research - they’re a great way to view your qualitative data differently (bear in mind that while conducting qualitative research, no one realistically has the resources to reach statistical significance).
Numbers tell stories - viewing your data in a graphical representation can help crystallize your thinking and help you build out narratives.
You can click into the graph to go one level deeper in complexity. Double clicking on the gray “Challenge” slice of the pie chart allows us to drill down and explore further:
It’s apparent that “Scheduling Interviews” is a significant challenge faced by consultants.
A helpful way of using graphs is to group by question(top right):
We drill down to view the various tools and methodologies used by consultants:
After closer examination of Tools and Methodologies, we rewatch clips from different interviews and decide that participants use the terms “Analytics” and “Data Visualization” interchangeably. Graphs can help refine labeling or coding that has been previously carried out. The labeling process never stops - we visit the Labels page to merge these two labels:
People can quickly look at graphs and arrive at conclusions about answer distributions. It’s clear from the graph above that the most used tools and methodologies are qualitative interviews and data visualization.
Overlapping labels can help build overarching narratives for your project. In our case, we were wondering which of the designs that we showed participants would resonate the most with them and make their lives easier. We're interested in a distribution of results, charting which design features performed best.
We graph by labels (top left) and select “Strongly Agree” from the Likert scale responses. This is because we only want to see the responses that were overwhelmingly positive. Additionally, we check all “Product Feature” tags, as we want to see which performed best. The “Relation between labels” operator AND is selected to signify that both conditions must be met.
After clicking through, we end up with the following distribution:
Among Likert scale responses of 5 (Strongly agree), the Transcription tool seems to be most popular in terms of making consultants’ lives easier.
You can flip this by choosing a single product feature (e.g. Live Notes), choosing all the responses from Likert scale labels, with the relational operator AND, to see what people thought of this feature in particular.
There are infinite ways to slice the data, but keeping your output in mind and viewing results through a distribution lens helps build out narratives. Reader beware - the less data you go with, the more shaky your conclusions. You should aim at getting at least a minimum amount of data before drawing any conclusions or telling any stories with your data.
Step 4: Build a Narrative
Once you’ve begun exploring the data via questions, labels, the Analyze page and graphs, you’ll start to see potential narratives within your data. You’re beginning to dive into the meat of qualitative analysis — building out these arguments or narratives and validating them with data points.
If you see unifying themes across clips, you can tie them together into a highlight reel to make your point hit home. Highlight reels are a common way for researchers to provide reasons behind why they reach a certain conclusion.
Narrative: Live Notes is a generally positive feature that people like to see.
We want to view the positive feedback provided by participants regarding this feature. In the Analyze page, we filter by label for “Agree” & “Strongly Agree” on the Likert scale, and “Live Notes” within the Product Features. We use the Relation AND, and get the following output:
Selecting all the clips we want to combine, we hit “Export” (top right) and select “Playlist”.
You can re-order the videos and add a title and description before publishing (excuse any spelling or grammatical errors — this is for demonstration purposes only!). Once you hit “Create new playlist”, you are redirected to this page:
Step 5: Create a Report
The word ‘report’ conjures up a rather boring image. We’re here to change all that. In Marvin, you can assimilate all your data and build an interactive report within the app.
On the Analyze page, hit the pink “Insights” button at the top right hand side and click “+ New insight”:
After naming the insight and providing a brief description, you can now begin to compile your findings into your report:
Playlists or highlight reels are fundamental to your recommendations — embed playlists throughout the report to lend credibility to your claim.
- Add graphs to your report to give it some numerical nous. Simply take a screenshot of a graph and paste it directly into the document.
You can add parts of the transcript as quotes within the report to make it more memorable for the reader:
Edit the quote to make it stand out. The “Open note” link ensures that any viewer can go back to the underlying data (clip + transcript) and arrive at their own conclusions.
Use reports to share your narrative (backed by source data) with teams so they can use it to build out their own narratives or help you clarify your own. If someone looks through your report and disagrees with some of your assumptions, it sparks healthy debate. This is considered a ‘best practice’ of qualitative research and helps build team and organization-wide confidence in the conclusions made. Multiple pairs of eyes looking at the data, challenging your conclusions and assumptions, with you defending your stance, builds confident research.