Analysis: Collecting Data on Issue Completion per Prework Author and Creating Looker Dashboards to Uncover Insights

[kimberlytanyh]

Dependency

• Create Project Board Looker Dashboard for Different Roles  #4921. Resume when the dashboard is ready

Overview

We need to collect data on the authors of all the prework issues in our repository to perform data analysis.

Action Items

• Find URL for GitHub REST API documentation and add it to the resources below #7758

• Read relevant sections in GitHub API documentation on retrieving data with REST APIs

• Search for other resources on platforms or libraries and syntax to use to retrieve data with GitHub REST APIs

• Download Postman to retrieve needed JSON data via GitHub REST API (based on online tutorials)

• Read documentation on rate limiting

• Retrieve data on all prework issues (date range from Nov 1, 2021 to now) using REST API in Jupyter Notebook

• Put JSON data in a tabular format and clean data

• Get distribution of issues completed by each complexity level for each prework author:
  Put data in columns: GitHub Handle, Date Prework Closed, No. of Good First Issues Completed, No. of Good Second Issues Completed, No. of Small Complexity Issues Completed, No. of Medium Complexity Issues Completed, No. of Large Complexity Issues Completed

• Export data as Excel file and add to Google Drive folder (GitHub Data Analysis)

• Manually check accuracy of numbers in dataset/spreadsheet)

• Write documentation on process and considerations (not complete yet)

• Duplicate data in another spreadsheet and perform following analysis:

1. Of the 202 people, how many people left the team?
2. How many people started and got to "Complexity: Large" issues (completed at least 2 for combined first and second good issue, and one of every other complexity type)?

• Perform above analysis again on only closed prework issues.

• Clean data and get number and percentage of closed large issues that were unassigned in Google Sheets

• Create Google spreadsheet with list of issues that have more than one complexity label and unassigned closed large issues.

• Perform cohort analysis on closed prework authors

1. Clean data from API and create dataset
2. Import into Google Drive and visualize data in Google Sheets

• Research how to connect data to Looker Studio in a way that new data can come in and Looker visualizations are automatically updated.

• Create new repository with Sophia and Chelsey's help that has GitHub Actions that perform cron job so that Python script can be run automatically daily for fresh data.

• Add automation components to Python script and verify data cleaning accuracy.

• Create Looker dashboard with data pulled in.

• Refine the Looker dashboard so that it is more intuitive

• Investigate correlation between number of issues available and cohort performance:

  • Design analysis and investigate where/how data can be obtained

Might be separated into another issue

• Get project board column data from GitHub and clean the data
• Set up data source and create Looker dashboard to show live number of issues available per role
• Create separate dashboard pages for developers (front end, back end, front and back end, and dev lead)
• Create documentation of process for GitHub class using Hack for LA template
• Set up automation of running of Python script so that dashbboard updates automatically

Resources/Instructions

1. GitHub API Documentation
2. GitHub Rate Limiting
3. Link to GitHub Data Analysis Folder
4. Spreadsheet with accurate numbers as of 03/26/2023
5. Link to process documentation
6. Using Google Sheets API to add and refresh dataframe in Python to Google Sheets:
   https://www.youtube.com/watch?v=sVURhxyc6jE
   https://medium.com/@jb.ranchana/write-and-append-dataframes-to-google-sheets-in-python-f62479460cf0
   https://www.youtube.com/watch?v=3wC-SCdJK2c
7. Slides documentation process from Python to GitHub

[kimberlytanyh]

After finish drafting this issue, add the label "Ready for Product".

[ExperimentsInHonesty]

@kimberlytanyh Add a step to add data to a google sheet on the Team Google Drive. Add a link to the folder it will go in, under the resources section.

[kimberlytanyh]

Weekly Update:

1. Progress: Retrieved data and calculated count of issues per complexity label. Left with converting rows to columns so that we can see the distribution in one row per assignee, and completing documentation.
2. Blockers: None
3. Availability: Mon - Fri, 12:00-5:00PM
4. ETA: ~21 hours. 1-3 hours for remaining deliverables.

[kimberlytanyh]

Weekly Update:

1. Progress: Adjusted data cleaning method and calculated count of issues per complexity label. Exported dataset as csv and uploaded to the drive. Manually checked accuracy of data. Working on data analysis now.
2. Blockers: None
3. Availability: Thurs-Saturday, Anytime
4. ETA: ~17 hours

[ExperimentsInHonesty]

@kimberlytanyh we are in the process of changing the labels on issues currently labeled Complexity: Good second issue to good first issue

Why?

• to improve current and future data analysis
• to make it easier for devs to know how many issues we need to make at a given time
• to make it easier for devs who are looking for their next issue to find one

What you need to know

• The issue where you can stay informed of the progress is here: Roll Out plan: Make issues required to change all the places that Good Second Issues appears with Good First Issue #4432
• The changes I made so far, are that I rolled out the change to all the closed issues and PRs. So there should be no closed Complexity: Good second issue Issue or PR
• You will have to revise your analysis to calculate if team members have done two good first issues instead of first grouping good first issue and Complexity: Good second issue together

[kimberlytanyh]

@ExperimentsInHonesty Thank you for the heads up! I will adjust my code for the next round of analysis rerun accordingly.

[kimberlytanyh]

Weekly Update:

Progress: Identified means for identifying pull requests in retrieved issues through GitHub API. Will re-perform all analyses done and try to improve accuracy of datasets.
Blockers: None
Availability: Saturday
ETA: ~6 hours

[kimberlytanyh]

@ExperimentsInHonesty As discussed in the Sunday Team Meeting, below are the labels to be added to prework/tracking issues for better data analysis:

Team Member Progression

• Join the team
• Set up development environment
• Self-assigning an issue and communicating availability
• Progress Report
• Good first issue (2nd)
• Small
• Issue making: Level 1
• Medium
• Issue making: Level 2
• Large
• Join the merge team
• Extra Large
• Issue making: Level 3
• Issue making: Level 4

[kimberlytanyh]

Progress: In the process of changing one more section of the code for automation and double checking accuracy of data after cleaning (need to improve accuracy of crediting the right amount of small issues for agenda issues that have multiple assignees). Next step is to add the Python script for automation and clean and create dataset for the live dashboard on number of issues available.

Blockers: None yet.
Availability: 6-8 hours
ETA: A few more weeks since it is an evolving and ongoing issue.

[github-actions]

@kimberlytanyh

Please add update using the below template (even if you have a pull request). Afterwards, remove the '2 weeks inactive' label and add the 'Status: Updated' label.

1. Progress: "What is the current status of your project? What have you completed and what is left to do?"
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures (optional): "Add any pictures of the visual changes made to the site so far."

If you need help, be sure to either: 1) place your issue in the developer meeting discussion column and ask for help at your next meeting, 2) put a "Status: Help Wanted" label on your issue and pull request, or 3) put up a request for assistance on the #hfla-site channel. Please note that including your questions in the issue comments- along with screenshots, if applicable- will help us to help you. Here and here are examples of well-formed questions.

_{You are receiving this comment because your last comment was before Tuesday, June 27, 2023 at 12:17 AM PST.}

[kimberlytanyh]

Progress: Completed documentation of process for live issue availability dashboard (for GitHub class). Left to do: Edit Python script to add in data from other columns, add it to repository for automation, and finish creating dashboard.
Blockers: None yet. Might have to consult Data Science COP about auto running automation script.
Availability: 21 hours next week Mon-Fri.
ETA: By next week or two.

[mayankt153]

• All the links in this issue will need to be checked to see if we have access
• https://github.com/hackforla/live-dashboard-automation/actions/workflows/schedule_run_to_update_data.yaml
  • Have access, its located in ____
  • public
• GitHub API Documentation
  • Have access, its located in ____
  • public
• GitHub Rate Limiting
  • Have access, its located in ____
  • public
• Link to GitHub Data Analysis Folder
  • Have access, its located in HackforLA.org website folder -> 5-Data Analytics -> Github Data Analysis
  • public
• Spreadsheet with accurate numbers as of 03/26/2023
  • Have access, its located in HackforLA.org website folder -> 5-Data Analytics -> Github Data Analysis -> Prework Analysis
  • public
• Link to process documentation
  • Have access, its located in ____
  • public
• Using Google Sheets API to add and refresh dataframe in Python to Google Sheets:
  https://www.youtube.com/watch?v=sVURhxyc6jE
  https://medium.com/@jb.ranchana/write-and-append-dataframes-to-google-sheets-in-python-f62479460cf0
  https://www.youtube.com/watch?v=3wC-SCdJK2c
  • Have access, its located in ____
  • public
• Slides documentation process from Python to GitHub
  • Have access, its located in ____
  • public