TWE: Research: Checklist for Case Study

[pandanista]

Overview

We need to develop a step-by-step checklist for conducting case studies at HackforLA. This checklist will be used to analyze data collected from the RP017 research. This checklist can also be used as the basis for developing a Template for conducting a case study and the starting point for creating a Guide.

Action Items

• Review information collected in comment in TWE: Create Research Plan for RP017 #442
• Collect more case study literature (see this comment in this issue) and include them in the resources gathered section
• Based on information collected in the first two action items, write a comment in this issue that provides a step-by-step checklist for conducting a case study
• Link the comment you made to the resources gathered section below (Resources # 2.2)
• Have other researchers review the checklist and provide feedback
  • Make any needed changes to the checklist based on feedback
• Create a template of the Case Study Report Template if needed
  • Save template of Case Study Report Template to TWE: Research Template Folder (Resources # 1.3)
  • Place link to the template of Summary Template in Resources gathered section (Resources # 2.3)
    - [ ] Create a template of the Matrix (No need a matrix for case studies)
    - [ ] Save template of Matrix to google drive folder
    - [ ] Place link to template of Matrix in Resources/Instructions
• Share updated checklist with UX Lead for review
• Create a new issue titled below
  • TWE: Research: RP017: Data: Analyze (see TWE: Research: RP017: Data: Analyze #493)
  • Add links the same links to Resources/Instructions that are listed on this issue
  • Place in New Issue Approval column
    - [ ] Create a new issue in UI/UX Templates and Guides titled 'Create a Guide/Template: Case Study'
    - [ ] Add the same links to Resources/Instructions that are listed on this issue
    - [ ] Create a guide for case studies
    - [ ] Place in New Issue Approval column

Resources/Instructions

Resources needed to complete issue

1.1 Wiki Research Output Overview Page
1.2 Google Drive Research by Type Folder
1.3 TWE: Research Template Folder
1.4 HfLA Template on Making a Guide

Resources gathered during the completion of the issue

2.1 Case study and qualitative usability interview literature
2.1.1 Overview of writing a case study analysis in business;
2.1.2 Sample case study analysis;
2.1.3 Article that provides background/definition of case study - It also includes example case studies as well as a checklist for assessing the quality of a case study report;
2.1.4 Case Study Methodology of Qualitative Research: Key Attributes and Navigating the Conundrums in Its Application;
2.1.5 Case Study Method: A Step-by-Step Guide for Business Researchers;
2.1.6 Methodology or method? A critical review of qualitative case study reports;
2.1.7 UCSF libraries case studies research guide - It includes books, resources/examples, and latest articles on case studies as a research method;
2.1.8 Qualitative Usability Testing: Study Guide;
2.1.9 usability.gov's usability test results reporting;
2.1.10 Usability guide by Maze;
2.1.11 Usability Testing Report by U of Michigan;
2.1.12 Dept of Homeland Security's example
2.2 a step-by-step checklist for conducting a case study
2.3 Case Study Template

[pandanista]

Hi @adarosh, would you please construct a step-by-step checklist for conducting a case study based on your literature review in a comment below?

This comment in #338 is an example you can reference.

Let me know if you have any questions. Thank you!

[pandanista]

@adarosh Welcome back, and hope your 2024 is off to a good start.
Would you please provide an update?

1. Progress: "What is the current status of your project? What have you completed and what is left to do?" - we know you just came back from break, please just summarize the progress from before then
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.

[adarosh]

Hi Ying @pandanista - Hope you had a wonderful holiday season! I took the break, so the issue status remains the same as last discussed. I believe the plan is for Nino and I to work together on the case study once she is finished with transcription? I also think we agreed to create this checklist over the course of completing the case study? Let me know if that’s still the plan!

I have set this Friday (1/12) aside for admin. If you and/or Nino are able, I’d be happy to meet for a working session. Below are some resources that can guide next steps.

Overview of writing a case study analysis in business: https://writingcenter.uagc.edu/writing-case-study-analysis

Sample case study analysis: https://content.bridgepointeducation.com/curriculum/file/078d7964-9052-4248-8a8b-e85592ff572e/1/Sample%20Case%20Study%20Analysis.pdf

Article that provides background/definition of case study (includes example case studies as well as a checklist for assessing the quality of a case study report): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3141799/

[pandanista]

@adarosh - Thank you for the update, Angela!

As a follow-up to yesterday's meeting, I have set up and sent out a meeting invite for a work session on Friday, Jan 12. I invited and messaged Nino, but I am not sure if she will make it or not.

Thank you for gathering a few more resources! We can come up with a draft of a step-by-step case study checklist and set up the structure of the case study based on the resources we have accumulated so far.

[pandanista]

@adarosh Thank you for meeting with me today. We have made a lot of progress. :)

As a follow-up to our work session, I added all the case study and qualitative usability interview sources we found in the Resources gathered section (see the items listed under 2.1).

We have set up the case study report structure and created the template in the Research Template folder (see 2.3). Also, the RP017's case study report structure has been customized and is ready to go.

Let me know if you have any questions regarding drafting a step-by-step checklist for conducting a case study in a new comment in this issue.

Hope you have a speedy recovery.

[pandanista]

@adarosh Hi Angela, would you please provide an update?

1. Progress: "What is the current status of your project? What have you completed and what is left to do?" - we know you just came back from break, please just summarize the progress from before then
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.

[adarosh]

Hi Ying- see below for update. Thanks!

Progress: Met with Team Lead on 1/12 to review resources that were complied. Developed a structure for the case study report based on resources. Created template for report and saved to shared drive. Next steps are to finalize case study checklist for review. Can be available to meet with Nino as well if she will be joining issue. Will have checklist completed for Team Lead to review by next meeting.
Blockers: None
Availability: 1-2 hours
ETA: Next meeting
Pictures or links* (if necessary): N/A

[pandanista]

@adarosh If the step-by-step checklist draft for the case study is ready, please insert it in a new comment below. Thank you, Angela!

[adarosh]

@pandanista Please see below for a draft Case Study Checklist.

There are places where it might be helpful to add links to Google Drive, which I have emphasized with caps (see sections: Review resources on background/definition of case study, Draft case study report)- we can discuss these notes in the team meeting. Also, I had to adjust the formatting to add the checklist as a comment, but can transfer to Google Docs if that would help with clarity.

Let me know if you have any questions!

Case Study Checklist:

1. Develop a research plan

• Define problem statement (project overview and background)
• Identify objectives (what you want to learn)
• Choose appropriate research method

*Note: Section 2-6 don't have to happen linearly in most circumstances, so they are not sequential in order.

2. Review resources on background/definition of case study

• RP017-Proposed research methods (https://drive.google.com/drive/folders/1mNpReUG3ekhYMzhW77NDcIrxA7tfx5r9?usp=share_link)
  • This folder includes the case study and usability interview related literature, and it is in RP017's folder. We can move it to a more appropriate location once we are done with RP017.

3. Confirm that case study is an appropriate method

4. Recruit participant

5. Construct an interview guide

Introductory script

• Introduce yourself (name, role in project)
• Obtain informed consent (before asking any questions). Includes providing information on:
• Purpose of the study
• How data will be used/what products will result from study
• How the participant’s confidentiality will be protected
• Study contact information
• Emphasize participant's right to withdraw at any time
• Obtain signature of participant indicating informed consent
• You must have consent prior to asking any interview questions

Interview questions (semi-structured or structured)

• Probes (if interview is semi-structured)
• No probing is needed (if interview is structured)
  • In the case of structured interviews, the goal is to remove the influence interviewers have on the interview sessions so that all participants would have a similar experience by answering the same set of questions worded in the same manner.

Closing script

• Thank participant for their time
• Remind participant of study contact information

Include the interview guide as an appendix to the research plan

6. Develop an analysis plan

• Identify how/where you will store data (interview recording, transcription)
• Decide on which software tools to use (should use open source tools like Google Docs/Sheets at HfLA)
• Identify the research team (who will conduct the interview, transcribe interview, and write case study report)
• Estimate timeline for conducting the analysis (including when you will collect data and logic behind timing)
• Review the questions in the interview guide to confirm that they align with research objectives

7. Collect data/conduct interview(s)

• Determine if the interview(s) will be conducted remotely or in person
• Identify technology/tools to be used
• Test technology/tools prior to the interview(s)

8. Transcribe and read the interview(s)

• Deidentify and clean up the transcript (if necessary)

9. Review and examine the case (https://writingcenter.uagc.edu/writing-case-study-analysis)

• Identify relevant facts and key problems
• Consider why they exist and how they impact the organization or product
• Review outside research/resources to uncover possible solutions/changes
• Consider supporting evidence, pros/cons to solutions/changes
• Are possible solutions/changes realistic?

10. Draft case study report

• Create a case study report using the template with HfLA branding (https://docs.google.com/document/d/13GBl5vmR6eeB69XgtM-vz2hvwbS--L27DbSbQ7BrLx0/edit)
• Edit title and table of contents
• Complete sections of a case study report

The structure of a case study report

Background

• Provide a short summary about the significance of the research, rationale for conducting the research, and aims

Objective

• Present a brief overview of your specific research question(s) and objectives that address the problem

Study Design/Data Collection

• Discuss case study method and rationale for use, participant recruitment, protocol/development of interview guide

Data Analysis

• Outline your data analysis plan, note themes of interest for case study

Key Findings

• Provide your examination of the case, discuss findings as they relate to themes focused on during interview, present relevant facts and key problems identified

Recommendations

• Present possible solutions/changes that are realistic, noting possible pros/cons
• Support solution with outside evidence
• Discuss any specific strategies for accomplishing proposed solutions/changes

11. Complete peer review of the case study report

12. Finalize the case study report based on peer review feedback

[pandanista]

@adarosh I took another look at the checklist above, and added one bullet point in the section of "6. Develop an analysis plan":

• Review the questions in the interview guide to determine whether and how they will be able to help us verify our assumptions and achieve the research goals or objectives.

[pandanista]

@AndrewSalvatore Would you please do a peer review on the checklist in this comment?

Please provide your feedback in a new comment, and mark the action items below to indicate that you have accomplished them. Thank you. Let me know if you have any questions.

Action Items for Peer Review

• Review this issue
  • Read the issue overview to learn about why this document was created.
  • Locate the draft of the case study template document in the Resources.
  • Locate the folder of RP017-Proposed research methods (https://drive.google.com/drive/folders/1mNpReUG3ekhYMzhW77NDcIrxA7tfx5r9?usp=share_link) and check the content to see if they match with the literature listed in the Resources gathered section in the first comment of the issue.
  • Review the literature in RP017-Proposed research methods (https://drive.google.com/drive/folders/1mNpReUG3ekhYMzhW77NDcIrxA7tfx5r9?usp=share_link) to get familiar with case study
• Review the checklist in this comment
  • Preview the comment to get a sense of how the draft checklist is structured and the type of content.
  • Review the structure of the checklist to ensure that the headings are clearly identified.
  • Determine if the information is communicated in a way that is concise, precise, and free from jargon.
• Provide feedback
  • Add feedback in the form of (1) questions and/or (2) explicit instructions for how the content should be improved. Feedback should be given as a new comment in this issue. Include a screenshot as needed.
  • Determine the accuracy of the content for each section of the draft checklist. If you find an inaccuracy, error, or mis-categorization, explain how the information presented could be improved.
  • For each suggested improvement, provide a link or reference within your comment to justify your suggestion (if available).
• Once all the steps have been completed, reach out to the original owner of this issue to determine if they can complete the next steps in the issue.

[adarosh]

@pandanista Hi Ying- Thanks for the feedback! Below I have suggested alternative wording, since we don’t necessarily want to verify assumptions but do want to ensure that questions map onto research objectives:

• Review the questions in the interview guide to confirm that they align with research objectives

I also think the bullet would be more appropriate under the “Construct an interview guide” section, because that consideration should be part of the question development process.

Thanks again for reviewing!

[pandanista]

Thank you, @adarosh. I have updated the wording, and I've assigned the issue to Drew for a peer review.

@AndrewSalvatore Let us know your feedback and suggestions. Please follow the instructions in this comment. Thank you.

[ExperimentsInHonesty]

@pandanista added this issue to the ux lead/pm agenda #315 (comment)

[ExperimentsInHonesty]

@pandanista what are next steps for this issue?

[pandanista]

@adarosh and I will revise the checklist and keep it in a more narrow and specific scope.

We will review the new checklist after the revision.

[adarosh]

@pandanista Hi Ying- I reviewed the case study checklist. Based on your feedback regarding the checklist being meant for just this analysis, as opposed to how to conduct a case study from scratch, it looks like we could cut everything before part 9 (Review and examine the case). The remaining sections look like a great guide for the analysis. Let me know what you think. Thanks!

[pandanista]

@adarosh I agree. Let's take out the steps before step 9, and draft an updated checklist in a new comment.

[ExperimentsInHonesty]

@adarosh Please provide update

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.

[adarosh]

Hi @ExperimentsInHonesty and @pandanista - Below is an updated draft Case Study Checklist based on the revisions Ying and I discussed. Let me know if there are any other edits needed- thanks!

Case Study Checklist:

1. Review and examine the case (https://writingcenter.uagc.edu/writing-case-study-analysis)

• Identify relevant facts and key problems
• Consider why they exist and how they impact the organization or product
• Review outside research/resources to uncover possible solutions/changes
• Consider supporting evidence, pros/cons to solutions/changes
• Are possible solutions/changes realistic?

2. Draft case study report

• Create a case study report using the template with HfLA branding (https://docs.google.com/document/d/13GBl5vmR6eeB69XgtM-vz2hvwbS--L27DbSbQ7BrLx0/edit)
• Edit title and table of contents
• Complete sections of a case study report

The structure of a case study report

Background

• Provide a short summary about the significance of the research, rationale for conducting the research, and aims

Objective

• Present a brief overview of your specific research question(s) and objectives that address the problem

Study Design/Data Collection

• Discuss case study method and rationale for use, participant recruitment, protocol/development of interview guide

Data Analysis

• Outline your data analysis plan, note themes of interest for case study

Key Findings

• Provide your examination of the case, discuss findings as they relate to themes focused on during interview, present relevant facts and key problems identified

Recommendations

• Present possible solutions/changes that are realistic, noting possible pros/cons
• Support solution with outside evidence
• Discuss any specific strategies for accomplishing proposed solutions/changes

3. Complete peer review of the case study report

4. Finalize the case study report based on peer review feedback

[pandanista]

@sunannie27

• The checklist for case study has been finalized, please see this comment.
• Let me know if you have any questions.
• If the checklist makes sense, we can kick off the data analysis stage of RP017.

[sunannie27]

@pandanista Looks good to me!

[pandanista]

Thank you for reviewing this issue, @sunannie27. Closing this issue since we have copied the checklist into the action items in the next data analysis issue #493.