Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AB scenario fails with copied file #1103

Closed
1 task done
marc-vdm opened this issue Nov 6, 2023 · 1 comment · Fixed by #1126
Closed
1 task done

AB scenario fails with copied file #1103

marc-vdm opened this issue Nov 6, 2023 · 1 comment · Fixed by #1126
Labels
bug Issues/PRs related to bugs scenarios Issues related to importing or calculating scenarios

Comments

@marc-vdm
Copy link
Member

marc-vdm commented Nov 6, 2023

Updating AB

  • Yes, I have updated AB and still experience this issue

What happened?

SDF file copied from AB, so errors are unlikely. Fails with the error below.

Files are confidential, will try to re-create later with non-confidential info.

Related:

Interestingly, not all files break:
Working version
princ-env-sci-db.xlsx
test_scenario.xlsx

Relevant errors

No data in scenario table 1, skipping
separator == ';'
Loading Scenario file. This may take a while for large files
<function ScenarioImportWidget.scenario_db_check at 0x000002D9E7620670> -- 0.0
<function SuperstructureManager.fill_empty_process_keys_in_exchanges at 0x000002D9E2AFBA60> -- 0.0010080337524414062
<function SuperstructureManager.verify_scenario_process_keys at 0x000002D9E2AFBB80> -- 0.01266932487487793
<function SuperstructureManager.check_duplicates at 0x000002D9E2AFBDC0> -- 0.0029997825622558594
<function SuperstructureManager.check_scenario_exchange_values at 0x000002D9E2AFBCA0> -- 0.0010004043579101562
<function SuperstructureManager.merge_flows_to_self at 0x000002D9E2AFB790> -- 0.003999948501586914
<function ScenarioImportWidget.sync_superstructure at 0x000002D9E7620550> -- 0.03170967102050781
<function ScenarioImportWidget.load_action at 0x000002D9E7620430> -- 5.103520154953003
Traceback (most recent call last):
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\layouts\tabs\LCA_results_tab.py", line 62, in generate_setup
    new_tab = LCAResultsSubTab(data, self)
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\layouts\tabs\LCA_results_tabs.py", line 136, in __init__
    self.setup_tabs()
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\layouts\tabs\LCA_results_tabs.py", line 143, in setup_tabs
    self._update_tabs()
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\layouts\tabs\LCA_results_tabs.py", line 154, in _update_tabs
    tab.update_tab()
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\layouts\tabs\LCA_results_tabs.py", line 634, in update_tab
    self.lca_scores_widget.update_tab()
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\layouts\tabs\LCA_results_tabs.py", line 687, in update_tab
    super().update_tab()
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\layouts\tabs\LCA_results_tabs.py", line 309, in update_tab
    self.update_plot()
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\layouts\tabs\LCA_results_tabs.py", line 693, in update_plot
    df = self.parent.mlca.get_results_for_method(method_index)
  File "C:\Users\meidemtvander\PycharmProjects\AB\activity_browser\bwutils\superstructure\mlca.py", line 214, in get_results_for_method
    return pd.DataFrame(
  File "C:\Users\meidemtvander\.conda\envs\ab_development_stable\lib\site-packages\pandas\core\frame.py", line 785, in __init__
    mgr = ndarray_to_mgr(
  File "C:\Users\meidemtvander\.conda\envs\ab_development_stable\lib\site-packages\pandas\core\internals\construction.py", line 336, in ndarray_to_mgr
    _check_values_indices_shape_match(values, index, columns)
  File "C:\Users\meidemtvander\.conda\envs\ab_development_stable\lib\site-packages\pandas\core\internals\construction.py", line 420, in _check_values_indices_shape_match
    raise ValueError(f"Shape of passed values is {passed}, indices imply {implied}")
ValueError: Shape of passed values is (6, 3), indices imply (5, 3)

Operating system

Windows 10

Conda environment

No response

@marc-vdm marc-vdm added bug Issues/PRs related to bugs scenarios Issues related to importing or calculating scenarios labels Nov 6, 2023
@marc-vdm
Copy link
Member Author

marc-vdm commented Nov 15, 2023

Update:

The problem seems to be in this section:

self.func_unit_translation_dict = {

The code here makes a dict (I don't know why, actually) of the functional units. I guess for easier reading, but the information is not shown to users??

In my error above, the code is reduced to 8 items instead of 9 as some FU have the same activity name (different product name though).
This also happens in non-scenario LCA, but apparently we don't use the variable self.func_key_list when that is the case as the calculation works as expected in that case.

I think we should review if we can get rid of it entirely (as the TODO states).

  • Review why self.func_key_list is needed for scenarios currently
  • Rewrite code so it is either not needed, or if not possible, re-write such that the keys are actually unique.

marc-vdm added a commit to marc-vdm/activity-browser that referenced this issue Nov 15, 2023
marc-vdm added a commit that referenced this issue Dec 13, 2023
* Delete .github/workflows/install-canary.yaml

* Remove unused code in montecarlo.py

* Resolve #1103

* GH action script for commenting on issues when related milestone is closed.

* update bw2io requirement

* Update readme install instructions

* Update meta.yaml

* Update README.md

* Change requirement to brightway2 instead of bw2io

* Update README.md

* Increases the flexibility of the excel importer so it can also handle the parameter scenario files.

* Update README.md

How to install using Mamba

* Set up signals infrascructure

* Implement dialog

* Implement finding of suitable alternatives and proper management of failing that.

* Implement actual relinking

* Minor documentation improvements + store exchanges to remove instead of to keep.

* Minor documentation + code improvements

* Resolve #782 + improve documentation of `Contributions` class

* Minor documentation + code improvements

* Fix type hinting error

* Update README.md

* Update README.md

* Update README.md

* Updates to the functions get_relevant_flows and get_relevant_activities (#1069)

* Updates to the functions from de Koning (get_relevant_flows and get_relevant_activities), avoiding use of pandas apply and using python map functionality for splitting pandas dataframes.

* Update the use of DataFrame.applymap to DataFrame.map in the excel file importer module.

* Multiple sdf update (#1083)

* Updates to the logging system to avoid the print statement. Merging with
commits for corrections to the uncertainty distributions that were
included within the same branch. Improves thread safety in logging
increasing stability in the multi-threaded processes.

* Changes the creation of log files. Amends the uncertainty wizard test to correct for use of logging over the standard console. Adds the .logs to .gitignore.

* 1) Simplifies the use of the python logging facility with QtThreads (as published on the python docs for logging), improving the stability of logging.\n2) Changes the generation of the logging object that is more compatible with pytest, supporting the existing test procedures.

* Provides an update to the Model for the calculation_setup Methods table, corrects for deletion of impact assessment methods. Updates the routine to the lca_setup module for calling the logger.

* Updates to the logger module, what was the ABLogger class is renamed to ABHandler. The module now contains the formats and settings for the Stream handlers. Calling of the methods in the other AB modules requires passing a logging.Logger instance and the name of the calling module. These are then incorporated into the wrapping routines in the class. Additions:\n 1) An error wrapper is provided and also explicitly provides a trace of the error.\n 2) A timestamp routine is provided and used for providing the file names for log files.\n 3) A standard location based on appdirs is provided and used for log file locations.

* Corrections to the setup of the logger in the test_uncertainty_wizard module

* Includes a change to the type used for the scenario columns when using multiple files with the combined (combinatoric) approach for the scenarios. Includes respective changes to the boolean tests applied to such Indexes.

* Minor corrections to local repository branch, to keep changes aligned

* Alterations to keep minor changes aligned with master branch and fork

---------

Co-authored-by: zoo <[email protected]>

* Use node16 actions for main pipeline

* Use node16 actions in release pipeline

* Use latest version of release-changelog-builder

* Use node16 action for install canary pipeline

* Remove the special build for arm architecture

This was added before brightway2 supported multi-arch builds on
conda-forge. It is not needed anymore, because the normal AB is now also
installable on arm arch.

* Install canary updates (#1093)

* Increase timeout to 30 min

Originally the idea was that installation should never take longer than
12 min, otherwise we should get a warning with the failed pipeline. But
unfortuantely conda currently takes longer than 12 minutes to solve
the environment.

* Add mamba install canary

* Download artifacts to start implementing env comparison

* Looks like on linux it can take more than 30 minutes :-(

* 60min not enough on linux, increasing to 120

* Completely remove timeout, default is 6 hours

* Use solver libmamba option

* Add diff step to compare installations

* Use node16 actions

* Re-add the 12 timeout

* Split diff into separate steps

* Yq action only runs on linux

* debug yq formatting step

* Simplify

* more fighting with yq action

* artifacts are apparently directories in this case

* Try again with while loop

* Run on all os, but only 3.9

* ignore diff exit code

* Run canary install for 3.8 and 3.9 again

---------

Co-authored-by: haasad <[email protected]>

* Adding error message if no scenario file is loaded in a scenario LCA (#1085)

* Updated and improved contributing file

* Improve text on dependencies

* Fix canary link

* Add good/bad examples to CONTRIBUTING.md for commit messages and PR titles

* Add more specific requirements on communicating work on issues in CONTRIBUTING.md

* Minor updates to CONTRIBUTING.md and README.md

* Minor improvements to CONTRIBUTING.md

* Fix formatting error in PULL_REQUEST_TEMPLATE.md

* Resolve #1105

* Update copyright and license information to only link to one place.

* Resolve 3 bugs for location linking (#1051): 1) database of own activity was not always added (in case none of the exchanges were in own database) 2) improve handling of multiple locations. 3) replaced exchanges were deleted from the old process instead of the new one.

* Improve choosing and handling of alternative locations for location choosing

* Resolve readonly database bug for activity duplication to new activity.

* Store AB version in logfile

* Remove deprecated channels from conda envs

* Remove experimental conda environment creation and upload

I started this mostly as an experimental alternative install method with
`conda env create`. However since the channel handling isn't such a pain
anymore now that everything is on conda-forge this is obsolete.

* Move conda-envs to .github directory

* Move changelog config to .github

* Move dev recipe to .github directory

* Move stable recipe to recipe/meta.yaml

* Update dev recipe

* Use micromamba instead of miniconda to speed up tests

* Update install instructions to use libmamba solver with conda

* Fix path for dev recipe

* Added tooltip for table content

* Added docstring for data method

* Cleaning-up documentation

* Update README.md

Resolve #1127

* Add - copy on impact category copy (#1136)

* Add - copy on impact category copy #1133

* Reorder impact category tree context menu

* Resolve bug with switching project on delete (#1138)

* Switch project on delete #1075

* Cleaning up ProjectController.change_project

* Resolving comments by marc-vdm

* GH action script for commenting on issues when related milestone is closed.

* GH action script for commenting on issues when related milestone is closed.

* GH action script for commenting on issues when related milestone is closed.

* review instructions bot

* review instructions bot

---------

Co-authored-by: bsteubing <[email protected]>
Co-authored-by: zoophobus <[email protected]>
Co-authored-by: Jonathan Kidner <[email protected]>
Co-authored-by: zoo <[email protected]>
Co-authored-by: haasad <[email protected]>
Co-authored-by: Adrian Haas <[email protected]>
Co-authored-by: Marin Visscher <[email protected]>
Co-authored-by: Marin Visscher <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issues/PRs related to bugs scenarios Issues related to importing or calculating scenarios
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant