Add new function `calculate_stats()` #470

mrcaseb · 2024-07-02T16:21:57Z

This implements #445

the previous code unintentionally dropped some ids

mrcaseb · 2024-07-02T16:25:08Z

It turned out that some stats are hard to compute using the playstats because some stats require information of other players, e.g. receiving_air_yards.

The remaining stats probably need to be computed using pbp. I didn't start this yet

github-actions · 2024-07-02T16:30:17Z

🚀 Deployed on https://6730dacf8109c6f91e87a704--thirsty-brown-d38ddf.netlify.app

also invert yards here as it's the defense player stat

this adds off, def, and special teams info that can be used to create some stats

(only defense)

R/calculate_stats.R

jacobakaye · 2024-07-16T14:59:15Z

I installed the development version, and the old function 'calculate_player_stats' still worked while I got the error with the new function.

calculate_stats(pbp)
Error in calculate_stats(pbp) : could not find function "calculate_stats"

tanho63 · 2024-07-16T15:00:51Z

I installed the development version, and the old function 'calculate_player_stats' still worked while I got the error with the new function.

calculate_stats(pbp) Error in calculate_stats(pbp) : could not find function "calculate_stats"

Use

pak::pkg_install("nflverse/nflfastR#470")

to install the PR branch version. the development version refers to what is merged to main

- def_tds shouldn't count special teams tds - fumble recovery stats can happen in any unit, not only defense - same is true for penalties - fumble recovery tds need to be counted separately also added a misc yards column to catch remaining yards

mrcaseb · 2024-11-03T11:29:31Z

Fumble recovery stats now count all recoveries (not only defense). That's necessary to count fumble recoveries of offensive players, esp. when they result in a TD, e.g. Trey McBride 2024, wk2. cc: @alecglen
added new variable fumble_recovery_tds to account for the above
Changed variable names of fumble recovery and penalty stats to make clear they don't belong to defense only
documented the changes and improved variable explainer in nfl_stats_variables

…astR into new-stats-approach

mrcaseb · 2024-11-09T16:17:10Z

Changed variable names compared to the current stats approach are now documented here

nflfastR/data-raw/nfl_stats_variables.R

Lines 35 to 63 in 2d7a0e7

    
           # Differences to old offense stats ---------------------------------------- 
        
           # recent_team -> team (recent team in weekly data never made sense) 
        
           # interceptions -> passing_interceptions (all passing stats have the passing prefix) 
        
           # sacks -> sacks_suffered (to make clear it's not on defensive side) 
        
           # sack_yards -> sack_yards_lost (to make clear it's not on defensive side) 
        
           # dakota -> not implemented at the moment 
        
           setdiff(n_old_1, n2) 
        
           setdiff(n2, n_old_1) 
        
           # Differences to old defense stats ---------------------------------------- 
        
           # def_tackles -> there is def_tackles_solo and def_tackles_with_assist 
        
           # def_fumble_recovery_own -> fumble_recovery_own (it is not exclusive to defense) 
        
           # def_fumble_recovery_yards_own -> fumble_recovery_yards_own (it is not exclusive to defense) 
        
           # def_fumble_recovery_opp -> fumble_recovery_opp (it is not exclusive to defense) 
        
           # def_fumble_recovery_yards_opp -> fumble_recovery_yards_opp (it is not exclusive to defense) 
        
           # def_safety -> def_safeties (we use plural everywhere) 
        
           # def_penalty -> penalties (it is not exclusive to defense) 
        
           # def_penalty_yards -> penalty_yards (it is not exclusive to defense) 
        
           setdiff(n_old_2, n2) 
        
           setdiff(n2, n_old_2) 
        
           # Differences to old kicking stats ---------------------------------------- 
        
           # No differences 
        
           setdiff(n_old_3, n2) 
        
           setdiff(n2, n_old_3)

This is the final call for review

TheMathNinja · 2024-11-09T17:48:07Z

Changed variable names compared to the current stats approach are now documented here

nflfastR/data-raw/nfl_stats_variables.R

Lines 35 to 65 in cb83260

# Differences to old offense stats ----------------------------------------

# player_name -> player_display_name

# recent_team -> team (recent team in weekly data never made sense)

# interceptions -> passing_interceptions (all passing stats have the passing prefix)

# sacks -> sacks_suffered (to make clear it's not on defensive side)

# sack_yards -> sack_yards_lost (to make clear it's not on defensive side)

# dakota -> not implemented at the moment

setdiff(n_old_1, n2)

setdiff(n2, n_old_1)

# Differences to old defense stats ----------------------------------------

# player_name -> player_display_name

# def_tackles -> there is def_tackles_solo and def_tackles_with_assist

# def_fumble_recovery_own -> fumble_recovery_own (it is not exclusive to defense)

# def_fumble_recovery_yards_own -> fumble_recovery_yards_own (it is not exclusive to defense)

# def_fumble_recovery_opp -> fumble_recovery_opp (it is not exclusive to defense)

# def_fumble_recovery_yards_opp -> fumble_recovery_yards_opp (it is not exclusive to defense)

# def_safety -> def_safeties (we use plural everywhere)

# def_penalty -> penalties (it is not exclusive to defense)

# def_penalty_yards -> penalty_yards (it is not exclusive to defense)

setdiff(n_old_2, n2)

setdiff(n2, n_old_2)

# Differences to old kicking stats ----------------------------------------

# player_name -> player_display_name

setdiff(n_old_3, n2)

setdiff(n2, n_old_3)

This is the final call for review

Does this mean the new version is erasing the difference between solo tackles and tackles with assist? Or will it maintain the two categories and also offer a combined variable?

mrcaseb · 2024-11-09T17:50:08Z

Changed variable names compared to the current stats approach are now documented here

nflfastR/data-raw/nfl_stats_variables.R

Lines 35 to 65 in cb83260

# Differences to old offense stats ----------------------------------------

# player_name -> player_display_name

# recent_team -> team (recent team in weekly data never made sense)

# interceptions -> passing_interceptions (all passing stats have the passing prefix)

# sacks -> sacks_suffered (to make clear it's not on defensive side)

# sack_yards -> sack_yards_lost (to make clear it's not on defensive side)

# dakota -> not implemented at the moment

setdiff(n_old_1, n2)

setdiff(n2, n_old_1)

# Differences to old defense stats ----------------------------------------

# player_name -> player_display_name

# def_tackles -> there is def_tackles_solo and def_tackles_with_assist

# def_fumble_recovery_own -> fumble_recovery_own (it is not exclusive to defense)

# def_fumble_recovery_yards_own -> fumble_recovery_yards_own (it is not exclusive to defense)

# def_fumble_recovery_opp -> fumble_recovery_opp (it is not exclusive to defense)

# def_fumble_recovery_yards_opp -> fumble_recovery_yards_opp (it is not exclusive to defense)

# def_safety -> def_safeties (we use plural everywhere)

# def_penalty -> penalties (it is not exclusive to defense)

# def_penalty_yards -> penalty_yards (it is not exclusive to defense)

setdiff(n_old_2, n2)

setdiff(n2, n_old_2)

# Differences to old kicking stats ----------------------------------------

# player_name -> player_display_name

setdiff(n_old_3, n2)

setdiff(n2, n_old_3)

This is the final call for review

Does this mean the new version is erasing the difference between solo tackles and tackles with assist? Or will it maintain the two categories and also offer a combined variable?

It counts solo tackles, tackles with assist and tackle assists separately

TheMathNinja · 2024-11-09T19:26:17Z

Changed variable names compared to the current stats approach are now documented here

nflfastR/data-raw/nfl_stats_variables.R

Lines 35 to 65 in cb83260

# Differences to old offense stats ----------------------------------------

# player_name -> player_display_name

# recent_team -> team (recent team in weekly data never made sense)

# interceptions -> passing_interceptions (all passing stats have the passing prefix)

# sacks -> sacks_suffered (to make clear it's not on defensive side)

# sack_yards -> sack_yards_lost (to make clear it's not on defensive side)

# dakota -> not implemented at the moment

setdiff(n_old_1, n2)

setdiff(n2, n_old_1)

# Differences to old defense stats ----------------------------------------

# player_name -> player_display_name

# def_tackles -> there is def_tackles_solo and def_tackles_with_assist

# def_fumble_recovery_own -> fumble_recovery_own (it is not exclusive to defense)

# def_fumble_recovery_yards_own -> fumble_recovery_yards_own (it is not exclusive to defense)

# def_fumble_recovery_opp -> fumble_recovery_opp (it is not exclusive to defense)

# def_fumble_recovery_yards_opp -> fumble_recovery_yards_opp (it is not exclusive to defense)

# def_safety -> def_safeties (we use plural everywhere)

# def_penalty -> penalties (it is not exclusive to defense)

# def_penalty_yards -> penalty_yards (it is not exclusive to defense)

setdiff(n_old_2, n2)

setdiff(n2, n_old_2)

# Differences to old kicking stats ----------------------------------------

# player_name -> player_display_name

setdiff(n_old_3, n2)

setdiff(n2, n_old_3)

This is the final call for review

Does this mean the new version is erasing the difference between solo tackles and tackles with assist? Or will it maintain the two categories and also offer a combined variable?

It counts solo tackles, tackles with assist and tackle assists separately

Got it!

tanho63 · 2024-11-10T15:24:13Z

new_stats <- calculate_stats(2023, summary_level = "week", stat_type = "player")

old_stats <- dplyr::bind_rows(nflreadr::load_player_stats(2023,stat_type = "offense"),nflreadr::load_player_stats(2023,stat_type = "defense"),nflreadr::load_player_stats(2023,stat_type = "kicking"))

setdiff(names(old_stats), names(new_stats))
#> [1] "recent_team"                   "interceptions"                 "sacks"                        
#> [4] "sack_yards"                    "dakota"                        "def_tackles"                  
#> [7] "def_fumble_recovery_own"       "def_fumble_recovery_yards_own" "def_fumble_recovery_opp"      
#>[10] "def_fumble_recovery_yards_opp" "def_safety"                    "def_penalty"                  
#>[13] "def_penalty_yards"      

setdiff(names(new_stats),names(old_stats))
#> [1] "passing_interceptions"     "sacks_suffered"            "sack_yards_lost"           "passing_cpoe"             
#> [5] "def_safeties"              "misc_yards"                "fumble_recovery_own"       "fumble_recovery_yards_own"
#> [9] "fumble_recovery_opp"       "fumble_recovery_yards_opp" "fumble_recovery_tds"       "penalties"                
#> [13] "penalty_yards"             "punt_returns"              "punt_return_yards"         "kickoff_returns"          
#> [17] "kickoff_return_yards"

I see most things have equivalents, should we document these somewhere e.g. what new fields are avail + what fields are no longer avail? dakota and def_tackles for instance don't have direct equivs?

tests/testthat/test-calculate_stats.R

tanho63

helluva job tackling this! performance not superb but I fully support dplyr readability here over data.table performance/maintenance tradeoff since nflreadr will pull precalculated values 99% of the time. a few questions about field differences, I think.

mrcaseb · 2024-11-10T15:39:47Z

new_stats <- calculate_stats(2023, summary_level = "week", stat_type = "player")

old_stats <- dplyr::bind_rows(nflreadr::load_player_stats(2023,stat_type = "offense"),nflreadr::load_player_stats(2023,stat_type = "defense"),nflreadr::load_player_stats(2023,stat_type = "kicking"))

setdiff(names(old_stats), names(new_stats))
#> [1] "recent_team"                   "interceptions"                 "sacks"                        
#> [4] "sack_yards"                    "dakota"                        "def_tackles"                  
#> [7] "def_fumble_recovery_own"       "def_fumble_recovery_yards_own" "def_fumble_recovery_opp"      
#>[10] "def_fumble_recovery_yards_opp" "def_safety"                    "def_penalty"                  
#>[13] "def_penalty_yards"      

setdiff(names(new_stats),names(old_stats))
#> [1] "passing_interceptions"     "sacks_suffered"            "sack_yards_lost"           "passing_cpoe"             
#> [5] "def_safeties"              "misc_yards"                "fumble_recovery_own"       "fumble_recovery_yards_own"
#> [9] "fumble_recovery_opp"       "fumble_recovery_yards_opp" "fumble_recovery_tds"       "penalties"                
#> [13] "penalty_yards"             "punt_returns"              "punt_return_yards"         "kickoff_returns"          
#> [17] "kickoff_return_yards"

I see most things have equivalents, should we document these somewhere e.g. what new fields are avail + what fields are no longer avail? dakota and def_tackles for instance don't have direct equivs?

Done in the article that hosts the variables

mrcaseb · 2024-11-10T16:01:04Z

helluva job tackling this! performance not superb but I fully support dplyr readability here over data.table performance/maintenance tradeoff since nflreadr will pull precalculated values 99% of the time. a few questions about field differences, I think.

Yeah it's not really worth caring too much about performance here. It's also quite slow because we have to download both pbp and playstats before we do some have grouped aggregation

mrcaseb added 3 commits July 2, 2024 18:19

update stat_ids

d89f2f8

the previous code unintentionally dropped some ids

News and documentation

6cea069

start calculate_stats()

0a01a16

rushing/receiving yards and tds should include the stat ids for laterals

1b1f831

mrcaseb added 5 commits July 3, 2024 17:33

include half sacks and corresponding yards

6f614c2

also invert yards here as it's the defense player stat

add play information to stats df

58d13d0

this adds off, def, and special teams info that can be used to create some stats

implement special teams tds and defensive tds

7f01b16

try to avoid non matches because of different team abbrs

59b2e1b

implement fumble recoveries

f59b2e6

(only defense)

guga31bb reviewed Jul 3, 2024

View reviewed changes

R/calculate_stats.R Show resolved Hide resolved

This comment was marked as off-topic.

Sign in to view

mrcaseb mentioned this pull request Aug 1, 2024

[BUG] nflfastr::calculate_player_stats returns duplicate rows for defense and kicker #476

Closed

1 task

more stuff (I forgot what I did here tbh)

72884c3

mrcaseb mentioned this pull request Aug 5, 2024

Resolving inconsistencies between columns in load_player_stats() #454

Closed

1 task

mrcaseb and others added 11 commits October 15, 2024 18:16

add pbp stats, add documentation, combine all

b367fb4

Merge branch 'master' into new-stats-approach

7942ae7

News update

d3075b1

remove irrelevant comments

d807a30

fix pkgdown

a679cd3

add season_type and team info and fix yac stats

49aefe8

add cpoe and number of games

1d99b23

users should be able to do the rounding themselves

5c07c0a

deleted the dot

bb8e20c

implement game winning field goals

fba04c0

team variable naming consistency

627d6a6

mrcaseb enabled auto-merge (rebase) October 22, 2024 12:20

mrcaseb and others added 3 commits November 3, 2024 12:22

not all of these are def stats actually

60e1e4b

- def_tds shouldn't count special teams tds - fumble recovery stats can happen in any unit, not only defense - same is true for penalties - fumble recovery tds need to be counted separately also added a misc yards column to catch remaining yards

adjust variable explainer after def modifications

5401559

Merge branch 'master' into new-stats-approach

b0b9dd6

mrcaseb added 4 commits November 3, 2024 12:43

falsely counted team air yards of all game instead of play

d0f1485

Merge branch 'new-stats-approach' of https://github.com/nflverse/nflf…

5aa1d80

…astR into new-stats-approach

Add punt/kickoff returns and yardage

8d45b29

add timeouts

fd648dc

This was linked to issues Nov 3, 2024

Add lateral_return_yards variable? #355

Closed

Multiple Penalty Infractions on the Same Play Obscure the True Penalty Count and True Penalty Yardage #220

Closed

mrcaseb added 2 commits November 3, 2024 15:27

Version bump and news bullet for nfl_stats_variables

74094b2

document differences to old stats

cb83260

use player short name from playstats

2d7a0e7

deprecate old stats functions

457e5d6

tanho63 reviewed Nov 10, 2024

View reviewed changes

tests/testthat/test-calculate_stats.R Show resolved Hide resolved

tanho63 reviewed Nov 10, 2024

View reviewed changes

Describe variable name differences

ea1fa2c

snapshot test variable types

e1f357f

tanho63 approved these changes Nov 10, 2024

View reviewed changes

auto-merge was automatically disabled November 10, 2024 16:04
Rebase failed

mrcaseb merged commit bd7ab61 into master Nov 10, 2024
7 checks passed

mrcaseb deleted the new-stats-approach branch November 10, 2024 16:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new function `calculate_stats()` #470

Add new function `calculate_stats()` #470

mrcaseb commented Jul 2, 2024

mrcaseb commented Jul 2, 2024

github-actions bot commented Jul 2, 2024 •

edited

Loading

This comment was marked as off-topic.

This comment was marked as off-topic.

jacobakaye commented Jul 16, 2024

tanho63 commented Jul 16, 2024

mrcaseb commented Nov 3, 2024

mrcaseb commented Nov 9, 2024 •

edited

Loading

TheMathNinja commented Nov 9, 2024

mrcaseb commented Nov 9, 2024

TheMathNinja commented Nov 9, 2024

tanho63 commented Nov 10, 2024 •

edited

Loading

tanho63 left a comment

mrcaseb commented Nov 10, 2024

mrcaseb commented Nov 10, 2024

Add new function calculate_stats() #470

Add new function calculate_stats() #470

Conversation

mrcaseb commented Jul 2, 2024

mrcaseb commented Jul 2, 2024

github-actions bot commented Jul 2, 2024 • edited Loading

This comment was marked as off-topic.

This comment was marked as off-topic.

jacobakaye commented Jul 16, 2024

tanho63 commented Jul 16, 2024

mrcaseb commented Nov 3, 2024

mrcaseb commented Nov 9, 2024 • edited Loading

TheMathNinja commented Nov 9, 2024

mrcaseb commented Nov 9, 2024

TheMathNinja commented Nov 9, 2024

tanho63 commented Nov 10, 2024 • edited Loading

tanho63 left a comment

Choose a reason for hiding this comment

mrcaseb commented Nov 10, 2024

mrcaseb commented Nov 10, 2024

Add new function `calculate_stats()` #470

Add new function `calculate_stats()` #470

github-actions bot commented Jul 2, 2024 •

edited

Loading

mrcaseb commented Nov 9, 2024 •

edited

Loading

tanho63 commented Nov 10, 2024 •

edited

Loading