Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds disaggregation of two-region model objects #46

Draft
wants to merge 131 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
95d2169
Add initial model specs
MoLi7 May 3, 2022
51f9126
Merge branch 'dev' into disagg
bl-young May 23, 2022
d3d25ec
identify location for disaggregation
bl-young May 23, 2022
c8492f0
Added StateDisaggFunctions.R, a script to hold the disaggregation fun…
jvendries Jun 3, 2022
d6ceec6
added template disaggregation files for the utility sectors
jvendries Jun 3, 2022
8fa3769
added a template utility disaggregation yml file
jvendries Jun 4, 2022
f900401
added code block where calls to disaggregation functions will happen …
jvendries Jun 4, 2022
bda4070
Organization edits to StateDisaggFunctions.R
jvendries Jun 4, 2022
312a98e
added functionality to start disaggregating the state make tables
jvendries Jun 4, 2022
49a8e64
added functionality to start disaggregating the state make tables in …
jvendries Jun 4, 2022
bf6c5fd
edited the names of the Make, Use, and Env file included in this file…
jvendries Jun 4, 2022
3228cca
added note regarding formatting error currently being worked on that …
jvendries Jun 4, 2022
42b0305
formatted the state make tables to conform with useeior disaggregatio…
jvendries Jun 5, 2022
a1f1efe
removed old comments
jvendries Jun 5, 2022
e2f72d9
removed old comments from stateDisaggFunctions.R
jvendries Jun 10, 2022
8f5f673
moved formatting code for Make transactions to their own functions
jvendries Jun 10, 2022
dd7793b
added function to format MakeTransactions back to stateior formats af…
jvendries Jun 12, 2022
8f1d891
added lines in BuildModel.R to initiate the disaggregation of Full Us…
jvendries Jun 12, 2022
a944ac5
added functions in StateDisaggFunctions.R to split and format FullUse…
jvendries Jun 12, 2022
82f984b
added statements for assigning disaggregated model objects back to st…
jvendries Jun 12, 2022
6e2a2d0
added functions to
jvendries Jun 12, 2022
60446d4
began edits to disaggregate the US_Make and US_DomesticUse objects
jvendries Jun 17, 2022
b91f2b9
Edits in functions of the StateDisaggFunctions script where relevant …
jvendries Jun 17, 2022
843483c
Edited state disagg functions to handle disaggregation and formatting…
jvendries Jun 18, 2022
c23a42f
In StateDisaggFunctions.R, added:
jvendries Jun 18, 2022
9dfe168
In BuildModel.R, organized disaggregation code flow for objects that …
jvendries Jun 18, 2022
257f262
remove unneeded comments
jvendries Jun 20, 2022
0dd0fd5
simplify disaggregateStateSectorLists
bl-young Jun 20, 2022
4bc12d6
minor re-org of disagg code flow
bl-young Jun 20, 2022
9bc9def
disaggregate sector codes
bl-young Jun 20, 2022
7d56e3c
pass disagg into buildTwoRegionUseModel
bl-young Jun 20, 2022
12bf67c
consolidate functions to remove passed parameters
bl-young Jun 20, 2022
aae7919
initial disaggregation of SoI_DomesticUse
bl-young Jun 20, 2022
08b2498
fix typos
bl-young Jun 20, 2022
5952bb8
update handling of SoIDomesticUse
bl-young Jun 20, 2022
c6058f9
added Employment_ratios.csv to begin testing state-specific data in t…
jvendries Jun 20, 2022
ed6ae85
generalize model data to apply to states or national model
bl-young Jun 27, 2022
5e0b8f5
disaggregate domestic use table in state model
bl-young Jun 27, 2022
d995bdb
incorporate domestic use table into state disaggregation; pass disagg…
bl-young Jun 27, 2022
2085db5
use already disaggregated make and use
bl-young Jun 27, 2022
fd58780
replace NA with 0
bl-young Jun 28, 2022
ae1e3bd
handle ICF separately for disaggregated utilities
bl-young Jun 28, 2022
89f109e
make sure ICF order matches
bl-young Jun 28, 2022
ad02382
add disaggregated US_Use to model object
bl-young Jun 28, 2022
539183b
disaggregate overseas to ensure all are equal size
bl-young Jun 28, 2022
6a67224
reassign nonimport cols after disaggregation
bl-young Jun 28, 2022
07d8ca0
assign model objects initially if model object is present
bl-young Jun 28, 2022
bc2e61b
update documentation
bl-young Jun 28, 2022
91b30ff
ensure model runs without disaggregation
bl-young Jun 28, 2022
507b0b5
update documentation
bl-young Jun 28, 2022
adfbc7d
add temporary action for two region disagg
bl-young Jun 28, 2022
93cfaae
add temporary action for disaggregation
bl-young Jun 28, 2022
fc7f16c
temporary testing
bl-young Jun 28, 2022
d42d5f4
Merge branch 'dev' into disagg
bl-young Jul 7, 2022
baa0ee1
delete nonimport_cols remaining after merge commit
bl-young Jul 9, 2022
8337a6e
update actions to match changes in original two-region action
bl-young Jul 9, 2022
194902f
resolve conflicts in BuildModel, ensuring ver = model_ver is passed t…
bl-young Oct 25, 2022
f4f3d5e
Merge branch 'ver-0.2.1' into disagg
bl-young Oct 26, 2022
59a0e3e
update to 0.3.0
bl-young Oct 26, 2022
0629c6e
set IODataSource to blank to bypass sequential disaggregation within …
bl-young Mar 15, 2023
a665c7d
reference model specs for disaggregation, add fxn to read configurati…
bl-young Mar 22, 2023
92c3caf
use useeior config fxn, see https://github.com/USEPA/useeior/commit/f…
bl-young Mar 22, 2023
d9e76ae
consolidate actions and scripts to not differentiate disaggregated mo…
bl-young Mar 23, 2023
e55af30
point to linkstateior branch
bl-young Mar 23, 2023
4408663
modify handling of env variable
bl-young Mar 23, 2023
a1a8b9d
fix typo
bl-young Mar 23, 2023
137ec40
drop echo statement
bl-young Mar 23, 2023
130e7a9
drop env variable
bl-young Mar 23, 2023
8b3a24d
add `type: string` to avoid being passed as object
bl-young Mar 23, 2023
b55f38e
bypass useeior getConfiguration
bl-young Mar 23, 2023
85087d7
fix typo
bl-young Mar 23, 2023
3fca94e
package is required parameter
bl-young Mar 23, 2023
932a55d
Added function template to read in state-specific disaggregation para…
Mar 24, 2023
00d76b8
Added commented lines to test state-specific disaggregation paramters…
Mar 24, 2023
861a364
update packages for actions
bl-young Mar 25, 2023
cddf7b7
add value added data product
bl-young Mar 25, 2023
db991c2
Merge branch 'disagg' of https://github.com/USEPA/stateior into disagg
bl-young Mar 28, 2023
452733c
Merge branch 'master' into disagg
bl-young Mar 28, 2023
c8e793c
enable all years for two-region model build
bl-young Mar 29, 2023
9fce51f
add package to specs, fix typo
bl-young Mar 29, 2023
f42c216
add work folder to ignore list
Mar 30, 2023
45383f4
temporary changes to buildModel.R to debug VA 2020 disagg error
Apr 1, 2023
338772e
added a new yml disaggregation file to point to the employment ratios…
Apr 2, 2023
a590aa6
update getStateModelDisaggSpecs to save stateFile as dataframe that i…
Apr 2, 2023
7b33dcc
add comment for decision point on how to handle disaggregation by proxy
Apr 2, 2023
538ef44
Added dummy US employment ratios for testing in the Employment_ratios…
Apr 4, 2023
d49adae
deleted deprecated commented lines
Apr 4, 2023
29a16f1
Updated ratios file to include national ratios for the three detail l…
Apr 4, 2023
2d768f5
Added call to createDisaggFilesFromProxyData function to create Make,…
Apr 4, 2023
ad8a6fc
Added creareDisaggFilesFromProxyData function, which is intended to c…
Apr 4, 2023
4fb5ef7
In function createDisaggFilesFromProxyData:
Apr 5, 2023
86bbd68
Remove Employment ratios backup csv
Apr 5, 2023
3c3bbde
added testing files for testing the format of state use and Make DF t…
Apr 5, 2023
de88226
added new testing yml file to test useDf and MakeDF strctures that ne…
Apr 5, 2023
4bf5ac0
fixed errors that did not allow disaggregation to proceed with data d…
Apr 5, 2023
49029b0
added comment for next step in state disagg testing with proxy data.
Apr 5, 2023
cebc1ca
handle cases where there is no proxy data for disaggregation for a sp…
Apr 12, 2023
80b5ae8
add if statement to disaggregate model objects for each state using p…
Apr 12, 2023
bea3975
Merge branch 'dev' into disagg_state_data
Feb 5, 2024
d3d219f
require useeior v1.4.0
bl-young Feb 6, 2024
e1ed152
use make table for commodity output consistent with approach in build…
bl-young Feb 7, 2024
0b9f535
clean up spec file, env file not needed
bl-young Feb 7, 2024
0d4b915
move disaggregation specs from stateior to useeior see https://github…
bl-young Feb 29, 2024
58f7483
fix typos
bl-young Feb 29, 2024
13ddd22
point back to linkstateiorImportF branch
bl-young Feb 29, 2024
1a47335
add function call to useeior's balanceDisagg function
Mar 5, 2024
c46c8d5
need to update disagg for each individual state?
bl-young Mar 8, 2024
4f5a7ac
Merge branch 'disagg_state_data' of https://github.com/USEPA/stateior…
bl-young Mar 8, 2024
bcbb29c
Add if statement to get correct disaggregation percentages for each s…
Mar 21, 2024
518f31d
Remove unneeded code and comments
Mar 21, 2024
feb6185
Add placeholders for calling createDisaggFilesFromProxyData function …
Mar 21, 2024
765a729
Remove NAs when checking tolerance for q_SoI and q_RoUS
Mar 22, 2024
eb87da8
Add model spec for UtilityDisaggByProxy
Mar 25, 2024
2e58125
fix spec name
Mar 25, 2024
af078a9
add employment ratios for additional years
bl-young Mar 26, 2024
1a487a2
fix error in handling of useeior data objects resulting from renaming…
bl-young Apr 8, 2024
3e69c4f
replace `DisaggregatedSectorCodes` with `NewSectorCodes` to align wit…
bl-young Apr 9, 2024
009318f
don't add schema for select useeior objects
bl-young Apr 9, 2024
2350710
change call for createDisaggFilesFromProxyData function to useeior pa…
Apr 25, 2024
b786b92
change package to useeior
Apr 25, 2024
bcf707a
call create disagg files from proxy data function from useeior for st…
Apr 26, 2024
cfad0cd
Merge branch 'dev' into disagg
bl-young Oct 26, 2024
8b16e9b
temporarily drop new disaggregation approach with employment data
bl-young Oct 26, 2024
7a53ca2
drop additional file
bl-young Oct 26, 2024
2c18a58
Merge pull request #41 from USEPA/disagg_state_data
bl-young Oct 26, 2024
c2b395f
drop unneeded file (now in useeior), update specs and pointer to usee…
bl-young Oct 26, 2024
429dd8c
fix naming of objects
bl-young Oct 26, 2024
cb97a41
use alias for state model files
bl-young Dec 17, 2024
1200537
bump upload-artifact
bl-young Dec 17, 2024
3cac8ba
clean up line length
bl-young Dec 17, 2024
c51bcf6
update documentation
bl-young Dec 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions .github/workflows/save-two-region-model-data.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,13 @@ name: Save two-region data as .rds in parallel runs

on:
workflow_dispatch: # allow manual trigger
inputs:
model_spec:
description: "Model spec file (in quotes)"
required: true
default: "StateIOv1.2-shoofly"
type: string


jobs:
generate-save-data:
Expand Down Expand Up @@ -32,7 +39,7 @@ jobs:
# R_REMOTES_NO_ERRORS_FROM_WARNINGS: true

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3

- uses: r-lib/actions/setup-pandoc@v2

Expand All @@ -59,12 +66,13 @@ jobs:
run: |
devtools::load_all()
year <- ${{ matrix.data-year }}
model_spec <- ${{ github.event.inputs.model_spec }}
source("${{ matrix.script-run }}")
shell: Rscript {0}

# Upload .rds files
- name: Upload .rds files and prepare zip for manual download
uses: actions/upload-artifact@v2.3.0
uses: actions/upload-artifact@v4
with:
# Artifact name
name: data and metadata
Expand All @@ -75,7 +83,7 @@ jobs:

# Upload .json files
- name: Upload .json files and prepare zip for manual download
uses: actions/upload-artifact@v2.3.0
uses: actions/upload-artifact@v4
with:
# Artifact name
name: data and metadata
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@
.RData
.Ruserdata
inst/doc/**/*.html
work
6 changes: 3 additions & 3 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: stateior
Type: Package
Title: US State Input-Output (stateio) R modeling software
Version: 0.2.1
Version: 0.3.0
Date: 2022-10-20
Authors@R: c(
person("Mo", "Li", email="[email protected]", role="aut"),
Expand Down Expand Up @@ -40,10 +40,10 @@ Imports:
Depends:
R (>= 3.6)
Remotes:
github::USEPA/useeior@v1.1.0
github::USEPA/useeior@two_region_disagg
License: file LICENSE
Encoding: UTF-8
LazyData: true
RoxygenNote: 7.2.0
RoxygenNote: 7.3.1
Suggests:
testthat (>= 2.1.0)
225 changes: 174 additions & 51 deletions R/BuildModel.R

Large diffs are not rendered by default.

29 changes: 25 additions & 4 deletions R/InteregionalCommodityFlowFunctions.R
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,11 @@ calculateLocalandTradedRatios <- function(state, year, SoI = TRUE, ioschema, iol
#' sensitivity analysis on ICF, default is FALSE.
#' @param adjust_by A numeric value between 0 and 1 indicating the manual adjustment
#' to ICF if a sensitivity analysis is conducted, default is 0 due to no SA.
#' @param disagg optional, disaggregation specs
#' #' @return A data frame contains domestic 2 region ICFs.
generateDomestic2RegionICFs <- function(state, year, ioschema, iolevel,
ICF_sensitivity_analysis = FALSE,
adjust_by = 0) {
adjust_by = 0, disagg = NULL) {
# Specify BEA code
bea <- paste("BEA", ioschema, iolevel, "Code", sep = "_")
# Generate SoI-RoUS commodity flow ratios from FAF
Expand Down Expand Up @@ -142,14 +143,34 @@ generateDomestic2RegionICFs <- function(state, year, ioschema, iolevel,
# Merge ICF_2r_wide with complete BEA Commodity list
CommodityCodeName <- loadDatafromUSEEIOR(paste(iolevel,
"CommodityCodeName_2012",
sep = "_"))
sep = "_"),
appendSchema = FALSE)
# Update commodities from disaggregation
if (!is.null(disagg)){
disagg_df <- data.frame(BEA_2012_Summary_Commodity_Code = c("221100", "221200", "221300"),
BEA_2012_Summary_Commodity_Name = c("Elec", "NG", "Water"))
CommodityCodeName <- CommodityCodeName[CommodityCodeName$BEA_2012_Summary_Commodity_Code!="22",]
CommodityCodeName <- rbind(CommodityCodeName, disagg_df)
}

ICF <- merge(ICF_2r_wide, CommodityCodeName, by.x = bea,
by.y = paste("BEA", ioschema, iolevel, "Commodity_Code", sep = "_"),
all.y = TRUE)
if (iolevel == "Summary") {
# Adjust utilities
ICF[ICF[, bea] == "22", cols] <- calculateUtilitiesFlowRatios(state, year)[, cols]
ICF[ICF[, bea] == "22", "source"] <- "EIA"
if (is.null(disagg)){
ICF[ICF[, bea] == "22", cols] <- calculateUtilitiesFlowRatios(state, year)[, cols]
ICF[ICF[, bea] == "22", "source"] <- "EIA"
} else {
ICF[ICF[, bea] == "221100", cols] <- calculateElectricityFlowRatios(state, year)[, cols]
ICF[ICF[, bea] == "221100", "source"] <- "EIA"
ICF[ICF[, bea] %in% c("221200", "221300"), cols] <- data.frame("SoI2SoI" = 1,
"SoI2RoUS" = 0,
"RoUS2SoI" = 0,
"RoUS2RoUS" = 1)
ICF[ICF[, bea]%in% c("221200", "221300"), "source"] <- "Assuming no interregional trade"
}

# Adjust waste management and remediation services
ICF[ICF[, bea] == "562", cols] <- calculateWasteManagementServiceFlowRatios(state, year)[, cols]
ICF[ICF[, bea] == "562", "source"] <- "RCRAInfo and SMP"
Expand Down
Loading
Loading