Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding HVDC Line Implementation #111

Merged
merged 25 commits into from
Jul 29, 2017
Merged

Adding HVDC Line Implementation #111

merged 25 commits into from
Jul 29, 2017

Conversation

hakanergun
Copy link
Contributor

I saw that the DC line data was not written out to the dict.
In the coming days/weeks/ I'd like to work on the common AC/DC power flow / OPF implementation for powermodels.

Cheers,
Hakan

@ccoffrin
Copy link
Member

Thanks! This will be a great addition. We will need a small unit test for this. Maybe a variant of the 3 or 5 node case with a DC line added to it? At first it can be for testing IO, but later it can be used for testing correctness of the OPF.

@codecov-io
Copy link

codecov-io commented Jun 27, 2017

Codecov Report

Merging #111 into master will decrease coverage by 1.06%.
The diff coverage is 84.73%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #111      +/-   ##
==========================================
- Coverage   92.49%   91.42%   -1.07%     
==========================================
  Files          29       29              
  Lines        2530     2812     +282     
==========================================
+ Hits         2340     2571     +231     
- Misses        190      241      +51
Impacted Files Coverage Δ
test/output.jl 100% <ø> (ø) ⬆️
test/docs.jl 100% <ø> (ø) ⬆️
test/opf.jl 100% <100%> (ø) ⬆️
src/form/wrm.jl 100% <100%> (ø) ⬆️
src/prob/opf.jl 100% <100%> (ø) ⬆️
src/prob/ots.jl 100% <100%> (ø) ⬆️
src/core/constraint_template.jl 85.08% <100%> (+2.52%) ⬆️
src/prob/tnep.jl 100% <100%> (ø) ⬆️
src/prob/pf.jl 100% <100%> (ø) ⬆️
src/core/base.jl 99.12% <100%> (+0.18%) ⬆️
... and 19 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19b228a...ec90a9a. Read the comment docs.

@hakanergun
Copy link
Contributor Author

hakanergun commented Jun 27, 2017 via email

- New file acdcp.jl (as same file as acp.jl)
- Defining ACDCPPowerModel, instead of ACPPowerModel
- Include command in Powermodels.jl
For the to and From side of the DC Lines
@ccoffrin
Copy link
Member

ccoffrin commented Jul 23, 2017

Just a heads up, I had to do a quick fix for the bug in parsing of dc line data.

#132

hakanergun and others added 5 commits July 24, 2017 09:40
- Matpower OPF compoatible, in AC and DC OPF
- WR & WRM formulations updated and tested (no validation possible)
- DC Line output included
- Remove warning of not calculating DC lines
- Fix compatibility with OTS, PF, MISC
- Fix per unit conversion
- Distflow implementation, only for OPF
- added variable definition for current magnitude squared
@ccoffrin
Copy link
Member

Re the distflow models, In the interest of merging things sooner rather than later, it would be better if we break independent features into different branches each with with their own PRs. That way discussions can be focused around each specific feature.

For example, it looks like the proposed distflow formulation does not support all of the Matpower model parameters (see https://arxiv.org/abs/1506.04773) and could leverage some of the functions that are already around (e.g. https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/form/wr.jl#L528). These points are independent of support for DC lines.

@frederikgeth
Copy link
Collaborator

Ok, great idea to separate. Distflow implementation is definitely not ready, I'll be including variable transformation ratios.

Anyway, the scope of this DC line extension is to be matpower compatible.

I've tried to run runtests.jl to completion using a DC line with branch status 0, but there's still a few things which I didn't manage to fix in the OTS/TNEP formulations. I'll take a look today.

Then we'll also add a few new tests with operational dc lines which are numerically validated w.r.t. the matpower results.

Other things we have to take a look at before you consider merging?

By the way, eventually, we may consider also supporting different representations of dc lines. The matpower implementation is rather simple (linear: Pij + Pji == loss0 * br_status + loss1 * Pij ) and has some weird behavior (losses can be negative, e.g. when loss0=0, loss1>0, Pij<0). As already mentioned by @hakanergun, more advanced (HV)DC line steady-state models have been developed (cfr https://www.esat.kuleuven.be/electa/teaching/matacdc/MatACDCManual with filter, phase reactor and converter representations + separation of DC nodes, AC nodes, DC lines, AC lines and AC/DC converters to support meshed DC networks).
Furthermore, the matpower model, due to its linear nature in P and Q variables, is just duplicated for in the acp, wr and wrm files. It feels like this should be avoided.

Any suggestions how you would approach that? Do you see that as in-scope?

- All (pure ac model) tests now work as before supporting dc lines
- Tests of mixed ac-dc still to be developed
Copy link
Member

@ccoffrin ccoffrin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some preliminary suggestions. Over all looks pretty good.

@@ -44,6 +44,7 @@ function get_solution(pm::GenericPowerModel)
add_bus_voltage_setpoint(sol, pm)
add_generator_power_setpoint(sol, pm)
add_branch_flow_setpoint(sol, pm)
add_branch_flow_setpoint_dc(sol, pm)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the naming convention would be add_dc_line_flow_setpoint(sol, pm)

############## DC Lines ############################################
function variable_line_flow_dc(pm::GenericPowerModel; kwargs...)
variable_active_line_flow_dc(pm; kwargs...)
variable_reactive_line_flow_dc(pm; kwargs...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the naming convention would be variable_active_dc_line_flow(sol, pm)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented as suggested

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did this naming convention get updated? The diff still looks like the old version.

If you are touching the names, I was also thinking maybe _dcline_ would be more consistent than _dc_line_, your call.

pref[(l,i,j)] = pm.ref[:dcline][l]["pf"]
pref[(l,j,i)] = pm.ref[:dcline][l]["pt"]
loss0[(l,i,j)] = 0
loss0[(l,j,i)] = pm.ref[:dcline][l]["loss0"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems odd that losses would be directional.

f_idx = (i, f_bus, t_bus)
t_idx = (i, t_bus, f_bus)
br_status = dcline["br_status"]
loss0 = dcline["loss0"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that if pm.ref is setup correctly, then all components with status 0 are filtered out. So you do not need to consider the status at this point.

loss0[(l,j,i)] = pm.ref[:dcline][l]["loss0"]
br_status[(l,i,j)] = pm.ref[:dcline][l]["br_status"]
br_status[(l,j,i)] = pm.ref[:dcline][l]["br_status"]
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous note regarding br_status, these should all be 1.0 by the time we get here.

src/form/acp.jl Outdated
(1-loss1) * p_fr + (p_to - loss0 * br_status) == 0
```
"""
function constraint_ohms_yt_dc{T <: AbstractACPForm}(pm::GenericPowerModel{T}, f_bus, t_bus, f_idx, t_idx, br_status, loss0, loss1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A better name for this constraint would be something like constraint_dc_line. If you have a good taxonomy for different dc line formulations, maybe a few extra characters to indicate where this formulation fits in that taxonomy.

src/form/wrm.jl Outdated
p_fr + p_to == loss0 + loss1 * p_fr
```
"""
function constraint_ohms_yt_dc{T <: AbstractWRMForm}(pm::GenericPowerModel{T}, f_bus, t_bus, f_idx, t_idx, br_status, loss0, loss1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this constraint is identical across multiple formulations, you can follow the pattern for constraints like thermal limits. That is move the constraints to core/constraint.jl and give it a signature like, constraint_ohms_yt_dc{T}(pm::GenericPowerModel{T}, which will be used by any formulation, unless specialized further.

@@ -0,0 +1,85 @@
% Case to test adding data to matpower file
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to add a new file here, just fix the bus ids in the original case3.m

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still have added new DC files, where the br_status of the DC line is set to 1 as opposed to the original test files.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good, as long as we don't have two nearly identical copies of case3.m

@@ -0,0 +1,65 @@
% NESTA v0.6.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to make a case, specifically for testing the dc line model. For example one where flow on the dc line is quite high and significant losses are incurred. In the unit test, you might also want to check the amount of loss that occurs across the line.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case5_dc includes losses in the line model (case_3 is lossless) and is included in the pf.jl and opf.jl test scripts.

@ccoffrin
Copy link
Member

ccoffrin commented Jul 25, 2017

Re various problem formulations, it is ok if a feature is not supported by all models (e.g. DC lines in OTS / TNEP). The standard approach in PowerModels is to leave some combinations un-implemented, and wait for feature requests.

Re dc line formulation scope, PowerModels should focus on well established models, so probably good to stick to Matpower's formulation in the core, in the near future. I would encourage you to make an extension package (e.g. PowerModelsHVDC), tailored to exploring research questions on HVDC lines. FYI, I am also considering making a shared extensions package (e.g. PowerModelsAnnex), where we can store more obscure problem formulations and extensions. Your new-ish HVDC models would be welcome there.

Re duplication in the acp, wr and wrm files, use the Formulation type hierarchy to avoid this. See comment in the code review.

@ccoffrin
Copy link
Member

Also this maybe helpful with picking names of things, #26

frederikgeth and others added 3 commits July 25, 2017 18:28
- losses not considered in variable bounds
- but bounds are now derived according to matpower (see toggle_dcline.m)
- Added case3_dc, case5_dc with active DC lines
- Updated \test\opf.j and \test\pf.jl  and validated active DC lines cases against matpower
-  Warning that DC line costs are not supported
- Implemented Carleton's code review + additional clean-up
the case3_dc.m is already updated
@frederikgeth
Copy link
Collaborator

By the way, the dc line model is derived to be consistent with that of the ac branches. Namely, power always flows into the branch from a node and Pfrom + Pto = Ploss. However, this means that the dc line results have signs which are different from those in matpower.

Matpower and PowerModels' have the same sign for Pfrom, but the sign is reversed for Pto, Qfrom, Qto.

@ccoffrin
Copy link
Member

So if I understand correctly, in Matpower the definition of dc line flow values (on from and to sides of the line) is not consistent with the AC lines? Hence, you are making them consistent here?

@frederikgeth
Copy link
Collaborator

yes, (I think) I made them consisten in PowerModels. You can see the dcline (active power) model in the matpower doc http://www.pserc.cornell.edu/matpower/manual.pdf p 83. For the reactive power model you need to dig into the matpower code.

@ccoffrin
Copy link
Member

Is the flow of current/active power direction of a DC line typically fixed? Should an OPF consider a flexible DC flow direction or not?

@frederikgeth
Copy link
Collaborator

It is bidirectional in general, although rectifier-only HVDC is becoming a thing for connecting off-shore wind (e.g. https://www.siemens.com/press/pool/de/events/2015/energymanagement/2015-10-dc-grid/presentation-dc-grid-e.pdf). But with this matpower-style DC line representation you may end up with negative line losses. Also, the fixed converter losses (here: loss0) should rather be divided over both sides of the line, instead of just the 'to' side.

@ccoffrin
Copy link
Member

ccoffrin commented Jul 26, 2017

I read the Matpower spec in detail. Their idea of an HVDC line is uni-directional, and I think that is the only thing that is reasonable given the specified active power loss function. They propose modeling bi-directional HVDC lines as two separate parallel components. Not ideal but reasonable, as long as this is how the data comes in.

In PowerModels' internal data structure, lets have both a PMIN and PMAX on the from and to side of the line. This will help us setup the bounds on the variables and later on will be helpful for other HVDC models. During the data preparation phase, we can take Matpower's specified PMIN and PMAX and translate them to the other side, based on the linear loss function.

We will need to add a note about this data departure here, https://github.com/lanl-ansi/PowerModels.jl/blob/master/docs/src/network-data.md

In terms of the formulation, in the interest of consistency, my recommendation is that we follow the mathematical spec in Matpower. But we will add some additional checks in PowerModels to make sure the input data leads to a reasonable HVDC model (e.g. one with only positive losses).

Along the lines of (there may be some sign mistakes here),

  • PMINF <= 0
  • 0 <= LOSS1 <= 1
  • 0 <= LOSS0 <= -PMAXF + LOSS1*PMAXF

@frederikgeth
Copy link
Collaborator

Pmin and pmax have already been separated when reading in the data, cfr https://github.com/hakanergun/PowerModels.jl/blob/master/src/io/matpower.jl#L498

We'll add a note about the output data difference w.r.t. matpower.

I'm not sure about restricting to the conditions you listed with an assert for compatibility reasons, but a warning is appropriate I think.

Nevertheless if you model a bidirectional dc line as 2 separate unidirectional matpower dc lines you'll end up with another problem, namely the fact that these two lines together can effectively behave like a controllable load due circular flow from the line in one direction to the line in the other direction, each having positive losses. This shouldn't happen in most cases as you increase the dispatch cost by creating supplementary (virtual) losses / load, however, it can help improve/restore feasibility (e.g. line congestions can be solved by increasing load).

@ccoffrin
Copy link
Member

Re pmin/pmax, that's great! Solvers are more robust if we give variable bounds, so it would be preferable to give the opposite side of the line have values other than -inf/inf, if they can be determined.

This is a good place to add a check_dc_line_limits function,
https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/io/common.jl#L15
which can tighten any pmin/pmax values.

As for Matpower's bi-directional flow suggestion, I totally agree that is a not a good model. But if we replicate this behavior in PowerModels, it will be a perfect platform to suggest improvements, right? :-) With your help maybe both Matpower and PowerModels could update to a new HVDC line spec, together.

After reading the Matpower documentation, it is clear that it assumes the checks I have about lined above. Maybe they don't check for those properties, if so, that is a bug in Matpower; the mathematical model does not make sense outside of these conditions.

In PowerModels, we should have a function incheck_network_data that does one of two things if these conditions are not satisfied,

  • throw an error, asking the user to fix the issue in the data
  • update the data with some reasonable alternate value and throw a warning that the data was changed

In other similar cases I have gone with the later solution. For example see,

https://github.com/hakanergun/PowerModels.jl/blob/master/src/core/data.jl#L312

@frederikgeth
Copy link
Collaborator

Absolutely, replicating the matpower models - and behavior - is a great starting point for doing more interesting things ;)

We'll try to find some time to work on your further suggestions today.

frederikgeth and others added 6 commits July 27, 2017 09:42
- documentation on dc line output signs w.r.t. matpower
- checks on loss0, loss1
- Avoiding infinite bounds on pmin and pmax for DC lines
- Avoiding negative losses, by checking input data, overwriting bad values and throwing warnings
- problem definitions
- dcline result convention
Adapted test of parsing string with escape chars
@frederikgeth
Copy link
Collaborator

The tests run, bounds for pminf, pmint, pmaxf, pmaxt are derived, documentation is updated and warnings have been added.

Regarding the bidirectional lines with losses that are potentially negative, I think there are a few options.

  1. give warning that bidirectional + loss1>0 may lead to negative losses
  2. give warning that bidirectional + loss1>0 may lead to negative losses + change loss1 to 0 to avoid negative losses.
  3. give warning that bidirectional + loss1>0 may lead to negative losses + adapt flow bounds to allow only unidirectional flow
  4. give warning when observing negative losses in the result
  5. throw error when dc lines are defined bidirectional + loss1>0
  6. throw error when observing negative losses in the result

Option 1 and 4 are consistent with with the current matpower implementation; the others are not.
Options 2 and 3 give different results from matpower. Keep in mind that the direction in which the line is operated may still be the one with the positive losses, even when the line is bidirectional. I'm not sure that users coming from matpower will like this behavior.

Option 1 is currently implemented.

Copy link
Member

@ccoffrin ccoffrin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did another review, just a few minor comments.

############## DC Lines ############################################
function variable_line_flow_dc(pm::GenericPowerModel; kwargs...)
variable_active_line_flow_dc(pm; kwargs...)
variable_reactive_line_flow_dc(pm; kwargs...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did this naming convention get updated? The diff still looks like the old version.

If you are touching the names, I was also thinking maybe _dcline_ would be more consistent than _dc_line_, your call.

@@ -496,21 +492,54 @@ function parse_matpower_data(data_string::String)

elseif parsed_matrix["name"] == "dcline"
dclines = []
warn("DC Line costs are not considered")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this line occur in the block, elseif parsed_matrix["name"] == "dclinecost", that way, users will only see it if the data file includes a cost block?

1 2 1 10 10 25.9092634604825 -4.16425203581105 1.1 0.92617 10 900 -900 900 -900 900 0 0 0 0 0 0 0 0
]

% matpower data format extentions
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we still have case3.m, this file does not need to include these data format extensions.

@ccoffrin
Copy link
Member

ccoffrin commented Jul 27, 2017

Just did a final code review, once you have seen the suggestions, I am ready to merge this in.

One general question, is there any reason you did not yet implement the dclinecost block? I would be happy to add that functionality once this is merged in.

@ccoffrin
Copy link
Member

Also, please feel free to add your names to the Acknowledgment section of the README.md

- Consistent dcline naming
- DC line removed from case3
- Warning about DC line costs thrown if mpc.dclinecost is provided
@frederikgeth
Copy link
Collaborator

no particular reason that we haven't considered dc line costs so far.

by the way it seems the acp.jl function constraint_kcl_shunt_ne{T <: AbstractACPForm} can't easily be used or tested. Which solver do you use for that?

@ccoffrin
Copy link
Member

For Non-Convex NLP, the only option I am aware of is Bonmin. It is available via the CoinOptServices + AmplNLWriter packages. This is not included in the CI becouse it takes too long to build these packages. But the tests will run locally if you have these packages installed.

Here is an example,
https://github.com/lanl-ansi/PowerModels.jl/blob/master/test/tnep.jl#L73

Copy link
Member

@ccoffrin ccoffrin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constraint names in the docs are out of date.

@@ -31,6 +32,9 @@ for (i,branch) in pm.ref[:branch]
constraint_thermal_limit_from(pm, branch)
constraint_thermal_limit_to(pm, branch)
end
for (i,dcline) in pm.ref[:dcline]
constraint_dc_line(pm, dcline)
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is an example

- added dc line constraints
- updated quickguide with how to inspect line flow variables and losses
- specifications with renamed functions
@ccoffrin
Copy link
Member

This looks great! Thanks for so much for making this contribution and for taking care all of the small details!

@ccoffrin ccoffrin merged commit 7843133 into lanl-ansi:master Jul 29, 2017
@frederikgeth
Copy link
Collaborator

Glad to contribute! I hope we manage to do this more often

@hakanergun
Copy link
Contributor Author

hakanergun commented Jul 29, 2017 via email

@ccoffrin ccoffrin changed the title Write out DC line data to dict Adding HVDC Line Implementation Jul 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants