Adding SOC extended DistFlow (BFM) formulation #229

frederikgeth · 2018-03-25T11:12:26Z

I'm working on an implementation of the extended DistFlow formulation.

Before I continue, I'm wondering what you think of the following issues:

Adding a line index i to constraint_voltage_angle_difference(pm, n, *i*, f_bus, t_bus, buspair["angmin"], buspair["angmax"])
Name of the formulation, as it uses W variables but not WR/WI but squared current magnitude instead. Currently it is named AbstractWIForm, to refer to the 'W' voltage variables and current variables 'I', but I don't think it is an ideal name.
Place in the type hierarchy: AbstractWForms = Union{AbstractWRForms, AbstractWIForm}
As it is a BFM, one equation defines both the to and from side active/reactive power flow. So the call to constraint_ohms_yt_to(pm, i) in this formulation does nothing. Should the problem definitions be changed, where a call constraint_ohms(pm, i) for BIM-type models itself calls constraint_ohms_yt_from(pm, i) and constraint_ohms_yt_to(pm, i)?
The line power equations are prepared with support for asymmetric line shunts and shunt conductance.
There's still something wrong with constraint_voltage_angle_difference though, as the test case 24 (with SAD) returns an objective value which is too low, I'll take a deeper look hopefully sometime the coming week.

any other comments are welcome of course :)

ccoffrin

Looks like a good start. Detailed comments to follow.

ccoffrin · 2018-03-25T16:32:07Z

src/core/variable.jl

+
+
+"variable: `0 <= i[l] <= (Imax)^2` for `l` in `branch`es"
+function variable_series_current_magnitude_sqr(pm::GenericPowerModel, n::Int=pm.cnw; bounded = true)


I don't think you need this constraint. You can move this implementation into this shared variable space,

https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/form/wr.jl#L399

ok, great approach, hadn't noticed this before :)

ccoffrin · 2018-03-25T16:37:29Z

src/form/wi.jl

+"""
+Creates Ohms constraints (yt post fix indicates that Y and T values are in rectangular form)
+"""
+function constraint_ohms_yt_from(pm::GenericPowerModel{T}, n::Int, f_bus, t_bus, f_idx, t_idx, g, b, c, tr, ti, tm) where T <: AbstractWIForm


I agree this does not make a lot of sense. I think this formulation needs a new OPF formulation that better reflect its construction.

ok, I agree, made a proposal that still allows you to use BIM formulations in the BFM problem type and vice-versa.

ccoffrin · 2018-03-25T16:38:37Z

src/form/wi.jl

+    w_fr = pm.var[:nw][n][:w][f_bus]
+    w_to = pm.var[:nw][n][:w][t_bus]
+    l = f_idx[1]
+    i =    pm.var[:nw][n][:i][l]


Have a look at the QC model for variable naming conventions. I have been using cm so far.

ok, cm it is.

ccoffrin · 2018-03-25T17:14:52Z

Based on your recommendation lets adopt the name BranchFlow (i.e. bf) for this variant of the model.

Because the equations in the base formulation are quite different, I think we need to add a new problem name for this variant of the model, I am thinking opf_bf, pf_bf and so on. The primary differences will be the addition of branch current variables and different constraints that are stated on the branches; these should be called something like constraint_power_losses and constraint_voltage_magnitude_difference. In this case, we can re-use the existing formulation names (for example SOCWR and ACP) for different variants of this new problem, which will make re-using the shared functions simple.

Concerning constraint_voltage_angle_difference and constraint_voltage_magnitude_difference, for consistency we should make both of them branch-wise constraints, similar to https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/core/constraint_template.jl#L316. This means passing in arc_from in addition to f_bus and t_bus.

Concerning, the inconsistency on test case 24. My guess is that it is due to the fact that the BF formulation does not implement this constraint, https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/form/wr.jl#L516 To the best of my knowledge, the constraint has never been projected into the BF space. I would be happy to do the derivation, in due course. For now I would confirm that this is the issue and then integrate the case 24 test with a different objective value.

codecov · 2018-03-26T09:46:55Z

Codecov Report

Merging #229 into master will increase coverage by 0.21%.
The diff coverage is 97.87%.

@@            Coverage Diff             @@
##           master     #229      +/-   ##
==========================================
+ Coverage   93.53%   93.74%   +0.21%     
==========================================
  Files          36       39       +3     
  Lines        4115     4335     +220     
==========================================
+ Hits         3849     4064     +215     
- Misses        266      271       +5

Impacted Files	Coverage Δ
src/form/wr.jl	`99.43% <ø> (ø)`	⬆️
src/PowerModels.jl	`100% <ø> (ø)`	⬆️
src/form/acp.jl	`39.49% <ø> (ø)`	⬆️
src/form/acr.jl	`95.58% <ø> (ø)`	⬆️
src/form/shared.jl	`100% <ø> (ø)`	⬆️
src/core/constraint.jl	`92.5% <0%> (-2.38%)`	⬇️
src/core/constraint_template.jl	`93.35% <100%> (+0.85%)`	⬆️
src/form/act.jl	`100% <100%> (ø)`	⬆️
src/form/df.jl	`100% <100%> (ø)`
test/opf.jl	`100% <100%> (ø)`	⬆️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b33773e...fe44804. Read the comment docs.

frederikgeth · 2018-03-26T12:19:01Z

Thank you very much for your feedback. Further commits forthcoming.

I'll make the opf_bf, pf_bfproblem types. The suggested name constraint_power_losses makes a lot of sense. However, I'm not sure the formulation itself should be called bf as this is just an example of a BFM formulation. I'm partial to DistFlow (the nonconvex quadratic one in p, q, vm), convexified DistFlow (the SOC one in p, q, w, (cm)^2), linearized DistFlow (LP in p, q, w). What do you think? How would you abbreviate?

Concerning constraint_voltage_angle_difference and constraint_voltage_magnitude_difference, I think the passing arc_from is a good solution.

I will confirm soon that the difference between the current SOC-BIM and SOC-BFM is due to the missing lifted nonlinear cuts. For now I changed the objective value in the SOC-BFM tests.

- new problem type - code for using BFM in opf and BIM in opf_bf - new tests

frederikgeth · 2018-03-26T13:57:41Z

Updated the naming of the current variable, now also documented here for future reference. Is this what you expect cfr #26?

opf_pf is now defined, including some code for compatibility of formulations between opfand opf_bfin shared. pf_bfis still a todo.

The results of SOCWR and DistFlow for opf and opf_bf are not yet consistent for case 24, even after playing with inclusion/exclusion of constraint_voltage_angle_difference and cut_complex_product_and_angle_difference, I'll need to dig deeper to figure this out, sometime later this week.

ccoffrin · 2018-03-26T16:59:24Z

Concerning the problem name. We need a generic problem name which will then take different forms. As an example, the abstract problem opf_xyz will become DistFlow with APowerModel, ConvexDistFlow with BPowerModel, and LinDistFlow with CPowerModel.

I am inclined to call the abstract problem as opf_bf (for branch flow) because it sounds more formulation agnostic to me than DistFlow, but I am open to other proposals.

The variable name docs looks good to me. I like the cc notation for current magnitude squared. It is more consistent with standard.

todo: fix repeated warnings

frederikgeth · 2018-03-26T19:53:47Z

Ok, great, let me confirm. So, the problem types are named opf_bfand pf_bf, with corresponding filenames and function names.

Currently, the convex relaxation of the (balanced) DistFlow formulation (shorthand DF) is type SOCDFPowerModel, contained in df.jl. If we eventually implement the nonconvex formulation it could be called DFPowerModel and the linearized version could be LinDFPowerModel.

The BFM-equivalent constraints of BIM-style constraint_ohms_yt_from and constraint_ohms_yt_to are currently conceived as:

constraint_branch_flow_losses: defining the relationship between p_fr, p_to and the total line loss (and equivalent for q)
constraint_branch_kvl: defining the relationship between the complex voltages at the from and the to side, whether in normal (complex) variables, or squared variables
constraint_branch_current: defining how the branch current relates to the (complex) power flow and the (complex) voltage variables

ccoffrin

Looking better, comments inline.

ccoffrin · 2018-03-26T20:16:27Z

src/prob/opf_bf.jl

+        constraint_branch_kvl(pm, i)
+        constraint_branch_current(pm, i)
+
+        #constraint_voltage_angle_difference(pm, i)


Why is this commented out?

forgot to remove after trying to find the source of the issue with case24 optimality BIM vs BFM

ccoffrin · 2018-03-26T20:20:36Z

src/prob/opf_bf.jl

+function post_opf_bf(pm::GenericPowerModel)
+    variable_voltage(pm)
+    variable_generation(pm)
+    variable_branch_flow(pm)


We should add branch current variables as well, right?

current is also a flow variable, so I had put the initialization of the current variables at the same level as the calls to the active and reactive power initialization. will be addressed

ccoffrin · 2018-03-26T20:23:25Z

src/prob/opf_bf.jl

+    end
+
+    for i in ids(pm, :branch)
+        constraint_branch_flow_losses(pm, i)


constraint_branch feels redundant given that losses are only defined on branches. How about constraint_flow_losses?

ccoffrin · 2018-03-26T20:24:23Z

src/prob/opf_bf.jl

+
+    for i in ids(pm, :branch)
+        constraint_branch_flow_losses(pm, i)
+        constraint_branch_kvl(pm, i)


kvl is a property of cycles on the network, correct? I don't see how it connects to this constraint. I think voltage_magnitude_difference is accurate and consistent with the constraints already defined.

Maybe it should refer to Ohm's law, as it relates two voltages through impedance. But that may create confusion w.r.t. BIM-style OPF problem formulation.

ccoffrin · 2018-03-26T20:27:12Z

src/prob/opf_bf.jl

+    for i in ids(pm, :branch)
+        constraint_branch_flow_losses(pm, i)
+        constraint_branch_kvl(pm, i)
+        constraint_branch_current(pm, i)


This is probably the exception to the rule. Here I think it makes sense to keep branch explicit, because there are so many possibilities for different currents.

Also I will have to think about how to unify this with constraint_voltage, but I will leave that for future issue that is outside the scope of this PR.

ccoffrin · 2018-03-26T20:32:09Z

src/prob/opf_bf.jl

+
+    objective_min_fuel_cost(pm)
+
+    constraint_voltage(pm)


It is not immediately clear to me if this is needed in problem definition. It might help you to comment it out for now.

ok, removed

ccoffrin · 2018-03-26T20:38:44Z

src/form/shared.jl

 # https://docs.julialang.org/en/release-0.6/manual/style-guide/#Avoid-strange-type-Unions-1
 # and should be used with discretion.
 #
 # If you are about to add a union type,
 # first double check if a different type hierarchy can resolve the issue
 # instead.
 #
+AbstractBIMForms = Union{AbstractACPForm, AbstractACTForm, AbstractDCPForm, AbstractWRForm, AbstractWRMForm}


Note that type unions are discouraged in Julia. So for now, I would recommend removing this collection and simply say that the opf_bf problem does not yet support these forms and defining new forms just for BFMs.

Later on after this PR, I will consider unification of both forms.

ok, unification code removed

ccoffrin · 2018-03-26T20:41:24Z

src/core/constraint_template.jl

@@ -439,11 +439,12 @@ function constraint_voltage_angle_difference(pm::GenericPowerModel, n::Int, i::I
    branch = ref(pm, n, :branch, i)
    f_bus = branch["f_bus"]
    t_bus = branch["t_bus"]
+    arc_from = (i, f_bus, t_bus)
    pair = (f_bus, t_bus)
    buspair = ref(pm, n, :buspairs, pair)

    if buspair["branch"] == i


This might also be the issue of inconsistent on case 24. You might be to post this constraint for all branches in the BFM models.

In the end, this was not the problem, just a typo in the DistFlow equation for constraint_voltage_angle_difference

…dated test

frederikgeth · 2018-03-27T09:27:20Z

Results are now identical between SOCWRForm and SOCDFForm, for both opf and pf problem types, including for case24. Unit tests are added for opf_bf and pf_bf for SOCWRForm and problem definition documentation is updated.

ccoffrin

We are close now. Proposed a few ways to further simplify things and a few questions about the math.

ccoffrin · 2018-03-28T03:41:07Z

src/form/df.jl

+end
+
+""
+function variable_branch_flow(pm::GenericPowerModel{T}, n::Int=pm.cnw; kwargs...) where T <: AbstractDFForm


I think this function can be omitted. The most generic version works for all GenericPowerModels.

https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/core/variable.jl#L223

ccoffrin · 2018-03-28T03:43:11Z

src/form/df.jl

+end
+
+"do nothing, this model does not have complex voltage constraints"
+function constraint_voltage(pm::GenericPowerModel{T}, n::Int) where T <: AbstractDFForm


This implementation can be provided without any constraint on T and can be placed in core/constraint.jl

ccoffrin · 2018-03-28T03:47:58Z

src/form/df.jl

+
+    #define series flow expressions to simplify Ohm's law
+    p_fr_s = p_fr - g_sh_fr*(w_fr/tm^2)
+    q_fr_s = q_fr + b_sh_fr*(w_fr/tm^2)


It's not immediately obvious to me that this is correct. Did you double check? Until we have an asymmetrical network test this will not be checked by unit tests.

The derivation goes as follows. Branches are modeled with an ideal transformer and a pi-section in series.

As the ideal transformer is located at the from side, the voltage after the transformer, i.e. at the from side of the pi-element following it, is actually (w_fr/tm^2). Furthermore, as this transformer is ideal, it is lossless. Moreover, as we'll be doing the DistFlow derivation, which performs the angle relaxation, transformer phase shift does not influence the power flow (except for constraint_voltage_angle_difference.)

We observe that a a branch's power flow balance with asymmetrical shunts is composed of three loss contributions: p_fr + p_to = p_sh_loss_fr + p_s_loss + p_sh_loss_to, q_fr + q_to = q_sh_loss_fr + q_s_loss + q_sh_loss_to, where x_sh_loss_fr, x_sh_loss_to are the to from and to side shunt losses, and x_s_loss is the series flow loss. We can further define the flow only through the series impedance as p_fr_s + p_to_s = p_s_loss, q_fr_s + q_to_s = q_s_loss.

The pi-section loss terms, expressed in the w and ccm variables therefore are:

p_sh_loss_fr = g_sh_fr*(w_fr/tm^2)

q_sh_loss_fr = -b_sh_fr*(w_fr/tm^2)

p_s = r*ccm

p_s = x*ccm

p_sh_loss_to = g_sh_to*(w_to)

q_sh_loss_to = -b_sh_to*(w_to)

Ohm's law only relates the voltage magnitude and the series current and series impedance of the pi-element. Therefore, after performing the DistFlow derivation (v_j = v_i - z_ij *I_ij_s, with RHS and LHS multiplied by their complex conjugated) and the subsititution of the squared node voltage magnitude, (with w and squared branch series current magnitude, with ccm), you obtain expressions in the series complex power flow variables p_fr_s, q_fr_s. Furthermore, in these variables p_fr_s^2 +q_fr_s^2 == (w_fr/tm^2)*ccm is a valid expression, which is then relaxed.

I can add this somewhere in the documentation.

I haven't found the time to rederive constraint_voltage_angle_difference to confirm its current implementation though.

ccoffrin · 2018-03-28T03:49:35Z

src/form/df.jl

+
+    #define series flow expressions to simplify constraint
+    p_fr_s = p_fr - g_sh_fr*(w_fr/tm^2)
+    q_fr_s = q_fr + b_sh_fr*(w_fr/tm^2)


Same as above, it's not immediately obvious to me that this is correct. Did you double check? Especially because in my derivation the shunt value did not occur in this constraint.

ccoffrin · 2018-03-28T03:51:55Z

src/form/df.jl

+
+
+"variable: `0 <= i[l] <= (Imax)^2` for `l` in `branch`es"
+function variable_branch_series_current_magnitude_sqr(pm::GenericPowerModel, n::Int=pm.cnw; bounded = true)


Can we replace this function with the one that is already defined here, https://github.com/lanl-ansi/PowerModels.jl/blob/master/src/form/wr.jl#L399? If so, that function can move to core/variable.jl because it is generally applicable.

These constraints have different meaning: bounding series current in this BFM derivation vs bounding total current magnitude in QC. Anyway, I updated the series current upper bound with a compensation term for the current contribution of the shunts.

frederikgeth · 2018-03-28T14:07:57Z

docs/src/developer.md

+
+## DistFlow derivation
+
+### For an asymmetric pi section


here's the derivation, @ccoffrin

ccoffrin · 2018-03-28T17:04:36Z

The implementation looks reasonable. I would like to understand the math a bit better before we merge this in. Can we setup a VC to go over it?

ccoffrin · 2018-03-31T20:10:16Z

@frederikgeth, fixed merge conflict so that travis would run.

ccoffrin · 2018-04-03T16:19:38Z

@frederikgeth seems there are still a few updates needed to bring thus up to date with master.

# Conflicts: # src/form/shared.jl

ccoffrin · 2018-04-06T23:38:02Z

Thanks!

Adding SOC BIM / extended DistFlow formulation

fad2061

ccoffrin requested changes Mar 25, 2018

View reviewed changes

Rename of powermodel: WI -> DF + rename of current variable

9a43d6d

frederikgeth added 2 commits March 26, 2018 14:48

Updated current variable name + dev documentation for variable naming

cf15a85

defining opf_bf problem type

80e2aef

- new problem type - code for using BFM in opf and BIM in opf_bf - new tests

frederikgeth added 3 commits March 26, 2018 16:17

add arc_from to constraint_voltage_angle_difference

8555b37

add arc_from to constraint_voltage_angle_difference (2)

5b2fd7e

change signature of constraint_power_losses

4237e91

frederikgeth added 2 commits March 26, 2018 21:20

rename BFM constraints + add pf_bf problem type

7d21ad6

todo: fix repeated warnings

updated constraint names

ae19e18

ccoffrin requested changes Mar 26, 2018

View reviewed changes

frederikgeth added 5 commits March 27, 2018 10:00

Addressing code review part 1 + update docs

33981dc

Bugfix in SOCDF form + error for mixing prob/from + updated docs + up…

da27cac

…dated test

added test for pf_bf SOCDFPowerModel

dbe2dec

cleaning up naming / comments in df

799fb49

updated docstrings

c8c92c5

ccoffrin reviewed Mar 28, 2018

View reviewed changes

frederikgeth and others added 2 commits March 28, 2018 11:20

Derivation of distflow formulation with asymmetric shunts

750f74c

moved/removed functions + derive bound for series current

3223b83

frederikgeth commented Mar 28, 2018

View reviewed changes

ccoffrin approved these changes Mar 28, 2018

View reviewed changes

Complete DF derivation with asymmetric shunts, including conductance

eaffc76

Merge branch 'master' into master

29790ec

frederikgeth added 3 commits April 5, 2018 16:27

Merge remote-tracking branch 'lanl-ansi/master'

00272d7

# Conflicts: # src/form/shared.jl

Merge remote-tracking branch 'origin/master'

088a829

# Conflicts: # src/form/shared.jl

compatible with asymmetric pi-section + updated tests

fe44804

ccoffrin added this to the v0.6.0 milestone Apr 6, 2018

ccoffrin merged commit 57c3299 into lanl-ansi:master Apr 6, 2018



		"variable: `0 <= i[l] <= (Imax)^2` for `l` in `branch`es"
		function variable_series_current_magnitude_sqr(pm::GenericPowerModel, n::Int=pm.cnw; bounded = true)



		"variable: `0 <= i[l] <= (Imax)^2` for `l` in `branch`es"
		function variable_branch_series_current_magnitude_sqr(pm::GenericPowerModel, n::Int=pm.cnw; bounded = true)

Adding SOC extended DistFlow (BFM) formulation #229

Adding SOC extended DistFlow (BFM) formulation #229

Conversation

frederikgeth commented Mar 25, 2018

ccoffrin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ccoffrin commented Mar 25, 2018

codecov bot commented Mar 26, 2018 • edited Loading

Codecov Report

frederikgeth commented Mar 26, 2018

frederikgeth commented Mar 26, 2018

ccoffrin commented Mar 26, 2018

frederikgeth commented Mar 26, 2018 • edited Loading

ccoffrin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

frederikgeth commented Mar 27, 2018

ccoffrin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ccoffrin commented Mar 28, 2018

ccoffrin commented Mar 31, 2018

ccoffrin commented Apr 3, 2018

ccoffrin commented Apr 6, 2018

codecov bot commented Mar 26, 2018 •

edited

Loading

frederikgeth commented Mar 26, 2018 •

edited

Loading