Integrate TSAM mode in oemof.solph

[henhuy]

Closes #973

This PR wants to integrate timeseries aggregation into oemof.solph. Thereby, newly integrated multi-period mode for oemof.solph can be used as well.
The idea is, to run TSAM with all needed timeseries before setting up the energysystem and use outcome of aggregation to set up the energysystem. The energysystem components are build with regarding aggregated timeseries. Additionally, a tsa_parameters parameter is given to the energysystem, which is used to set up component equations and constraints to ensure TSAM-related component behaviour.
Following equations/constraints have to be adapted/are replaced in order to work correctly in TSAM mode:

• Model component needs six additional indexes in order to set up storage equations (see below)
• storage content equations for GenericStorageBlock and GenericInvestmentStorageBlock have to be adapted:
  • default storage content is replaced by two storage indexes, inter and intra storage index
  • minimum and maximum rule of default storage content are replaced by applying max and min rule to inter storage
  • default storage balance rule is replaced by two balance rules for intra and inter storage contents
  • default balanced storage rule is replaced by balanced storage rule for inter storage contents
• Variable cost have to be weighted related to occurrences of typical cluster periods
• Full time hours have to be weighted
• Others?

How does the API look?
In storage investment example v5 I set up an ES with aggregated timeseries and related TSA parameters for the energysystem.

The newly introduced parameter tsa_parameters should be set up as follows:

• a separate timeseries aggregation has to be performed for each period
• for each period (in a multi-period approach), a dictionary with outputs from TSAM aggregation has to be given to ES
• see example tsa_parameters and needed key-value pairs below:

tsa_parameters=[
    # First period
    {
        "timesteps_per_period": aggregation1.hoursPerPeriod,
        "order": aggregation1.clusterOrder,
        "occurrences": aggregation1.clusterPeriodNoOccur,
        "timeindex": aggregation1.timeIndex
    },
    # Second period
    {
        "timesteps_per_period": aggregation2.hoursPerPeriod,
        "order": aggregation2.clusterOrder,
        "occurrences": aggregation2.clusterPeriodNoOccur,
        "timeindex": aggregation2.timeIndex
    }
]

TODOS:

• Implement storage indexes (inter and intra) and equations/constraints for GenericStorageBlock
• Implement storage indexes (inter and intra) and equations/constraints for GenericInvestmentStorageBlock
• Implement initial SOC equation
• Implement weighted cost
• Implement weighted full time hours
• Add documentation for TSAM mode
• Add docstrings including Latex equations
• Check results with examples and tests

[pep8speaks]

Hello @henhuy! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file examples/tsam/invest_optimize_all_technologies_using_mp_and_tsam.py:

Line 32:80: E501 line too long (85 > 79 characters)
Line 52:80: E501 line too long (145 > 79 characters)
Line 56:80: E501 line too long (94 > 79 characters)
Line 62:80: E501 line too long (106 > 79 characters)

• In the file src/oemof/solph/constraints/storage_level.py:

Line 113:80: E501 line too long (92 > 79 characters)
Line 132:47: E241 multiple spaces after ','
Line 248:80: E501 line too long (91 > 79 characters)
Line 267:47: E241 multiple spaces after ','

• In the file src/oemof/solph/processing.py:

Line 639:1: E302 expected 2 blank lines, found 1
Line 648:17: E123 closing bracket does not match indentation of opening bracket's line
Line 650:80: E501 line too long (82 > 79 characters)
Line 654:1: E302 expected 2 blank lines, found 1
Line 670:80: E501 line too long (107 > 79 characters)
Line 672:80: E501 line too long (102 > 79 characters)
Line 688:50: E231 missing whitespace after ','
Line 690:80: E501 line too long (80 > 79 characters)

• In the file tests/test_scripts/test_solph/test_tsam/test_storage_invest_tsam_integration_without_mp.py:

Line 152:80: E501 line too long (84 > 79 characters)

• In the file tests/test_scripts/test_solph/test_tsam/test_storage_tsam_integration_with_segmentation_without_mp.py:

Line 55:13: E126 continuation line over-indented for hanging indent

Comment last updated at 2024-07-11 12:07:17 UTC

[henhuy]

You can already check storage equations if you want to help!
Check out my storage example (it's called storage investment - but in order to make testing simpler, I added a non-invest storage) - in order to check storage contents (intra&inter) it's useful to set a breakpoint at SOC calculation in postprocessing:

oemof-solph/src/oemof/solph/processing.py

Line 487 in 7684cde

for storage, soc in storages.items():

Other things which are very welcome, are:

• docstrings
• storage equations written in LaTex
• README section regarding TSAM mode and idea behind it

[Maxhi77]

I finished the implementation of weighted cost before I was on vacation. I will add that soon to the PR.

[henhuy]

I finished the implementation of weighted cost before I was on vacation. I will add that soon to the PR.

Nice! Could you create an extra branch for this and do a PR to my branch or show me your branch so that I can review?
I think it's easier to look at one change at a time..

[Maxhi77]

I would like to add an further todo. TSAM targets on the use of longterm storage. Right now, if tsam parameters are added to the energysystem each storage will have an inter and intra SOC. For short-term storages which are used on an daily base, f.e. batteries, that is not necessary and it would slow down the optimization. Therfore I think a state_variable should be added to the generic_storage. On basis of this variable it should be decided to add an inter and intra SOC or not.

Then it would be possible to optimize a long term storage and a short term storage with best possible performance.

[Maxhi77]

Linked to issue: #987 it would be handy to rename objective_weighting and add a tsam weighting. Right now, objective_weighting gets calculated during pre-processing and includes the period occurence of typical periods (f571759).

The idea would be:

• tsam_weighting gets generated based on tsa_paramets in energy_system.py. It inlcudes the occurence of typical periods.
• time_increment gets generated based on the input time_series. It includes the length of a timestep, compared to others.
• objective_weighting can be setted by the user to consider individual problems.

Variable cost would be:
variable_costs += ( m.flow[i, o, p, t] * m.tsam_weighting[t] * m.time_increment[t] * m.objective_weighting[t] * m.flows[i, o].variable_costs[t] * ((1 + m.discount_rate) ** -m.es.periods_years[p]) )

In total the idea is more intuitive for the user and easier to use than calculating the objective_weighting in a pre_processing, which includes time increment, period occurence and individual preferences.

What are your opinions?

[henhuy]

Things that might need changes/fixes:

• GenericInvestStorage always starts with initial SOC of zero, last SOC is free
• GenericStorage initial SOC is free or can be set by user, but must equal last SOC
• TSAM mode is only available in multi-period approach (but it is allowed to only have one period)
• Integrate segmentation

[nailend]

this is not needed and should be None, as you dont use multi-period, do you?

[henhuy]

No, it's the opposite indeed: Currently, TSAM-mode is only available in multi-period mode! But for those examples, I only use a single period to make things easier to test...

[henhuy]

I still wonder if multi-period should/could be activated ALWAYS, and an "old" default oemof.solph simulation simply uses only a single period... but this is another issue.

[nailend]

its basically translated into pyomo like this but a lot of extra constrains and variables are not necessary (e.g. for decommissioning etc)

[nailend]

I also found out, that the definition of ep_costs differ between those versions: single-period uses "annuities" and multi-period "total capacity costs" and calculates the annuities over the lifetime of the component

[henhuy]

@MaxHiDLR could you explain your latest commits?
As I understand it, storage_level constraint has not been set before?
Ans what are those multiplexer values in processing file?
THX
(As code in this PR has grown, I would suggest from now on we both start PRs on this branch whenever we add/fix something - this makes it much easier to review new PR and merge it into this branch, instead of merging directly and reviewing afterwards. I will start doing so as well!)

[Maxhi77]

@henhuy
The storage_level.py, which is used in example/storage_level_constraint, wasn't working with the additional time_grid of tsam. Therefore I added it. As well, I added to the processing function that the multiplexer added by storage_level.py can be processed.

We definitely can start to do PR to simplify the understanding of the changing history.

In the close future I will change the naming of the tsam "soc" to "storage content", to be more consistent with oemof wording.

[henhuy]

The storage_level.py, which is used in example/storage_level_constraint, wasn't working with the additional time_grid of tsam. Therefore I added it. As well, I added to the processing function that the multiplexer added by storage_level.py can be processed.

Good that you found this - I did not see an error? Or was it silently not working?

[Maxhi77]

Silently not working, since the storage_level doesn't get used in the usual operation of the storage. Therefore it might be good to add a test for this scenario as well.

[p-snft]

I'd like to talk about time indexing in a more general way. Of course, we should harmonize multi-period optimization and time series aggregation, but there is more. For example, there are different markets working of different time grids. So, if you offer you capacity, you have to do it for a full time slot. A generic multi-level time grid would also simplify modeling these kinds of things.

Would you have time for a video call in the near future?