Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New connetivity vector operations for improved networks #3635

Merged
merged 15 commits into from
Aug 6, 2024

Conversation

MatNif
Copy link
Collaborator

@MatNif MatNif commented Jul 18, 2024

The optimization algorithm manages to find converged solutions for thermal supply systems rather quickly, while the networks of the non-dominated solutions often look unrefined at best and completely counterintuitive at worst.

The changes introduced in this PR add a series of operations to the algorithm that can lead to better network structures faster. More explicitly, these new methods are:

  1. ConnectivityVector mutation and crossover algorithms that are based on a clustering of the buildings in the domain (done at the start of the run):
  • The new ClusterSwitch mutation switches all buildings in the same cluster to a random other connectivity value with a probability of mut_prob.
  • The new ClusterSwap crossover method recombines two connectivity vectors by exchanging the connectivity values of some of their clusters. This operation does not affect outliers.
  • The new ClusterAlignment crossover operation recombines two connectivity vectors by exchanging the connectivity values of the buildings where (a.) they deviate from the most prevalent connectivity value of the cluster they belong to and (b.) a swap of the connectivity value between the two connectivity vectors would align them with the most prevalent connectivity value of their cluster.

Preclustering of the buildings either follows:

  1. New ConnvectivityVector correction methods for network overlaps are also introduced. The user gets to select if ConnectivityVectors should be altered so that:
  • One of the overlapping networks gets deleted
  • Overlapping networks get merged
  • One of the networks gets cut on overlapping nodes (one randomly selected remaining piece of the network will be kept)
  • None of the above happens (this is mostly introduced to check for potential biases these operations might introduce)

Or if one of these four operations should be chosen at random.

The parameters for these new operations have been added to the config. Default parameters have been set to ClusterSwitch, ClusterSwap, Louvain and Random based on a first series of tests.

To test this branch

  1. Select a district of choice (ideally more than ten buildings)
  2. Run the optimisation with an old set of settings (e.g. "ShuffleIndexes", "Uniform", "Louvain", and "None", the clustering method will be ignored if the crossover and mutation algorithms are not based on a clustering)
  3. Run the optimisation with a set of new operations
  4. Compare the networks-graphs and objective function values of the two runs. While improvements in objective function values will likely be minor, there should be a clear visual improvement in the network graphs.

This update introduces a new pre-clustering approach for finding optimal networks more efficiently.
It effectively attempts to keep the networks more compact, which is assumed to be preferable to highly spread-out networks.
Alinging the naming of the newly introduced mutation and crossover methods to similar methods from the deap.tools library
The clustering based mutation and crossover operations were not fully implemented. This change offers an initial version of the operations.
The algorithm previously didn't reassign the cluster indexes of the individual buildings after clusters were subdivided. This change should fix that bug.
This new operation ensures that connectivity vectors that encode for the same connectivity state of a district are now forced to take on the same expression. This is done in an attempt to reduce the dimensionality of the problem, hopefully reducing the number of selected optimal solutions and speeding up convergence.
This introduces a second clustering method, which should take the network graph's layout into consideration more explicitly. The algorithm is based on the Louvain method for community detection.
Some duplicates could previously not be detected as some CapacityIndicatorVectors (CIV) held values of 0.0 and 1.0 (as floats) and others held 0 and 1 (as ints). With this change capacity indicator values of 0 and 1 will continuously be registered as integer values, allowing for proper comparisons and elimination of duplicates during the genetic optimisation.
Introducing methods to prevent network graphs from overlapping.
This introduces the chosen building clustering method as a parameter of the optimization script. The location where the latest network-modifying paramters are allocated is also slightly altered to align better with previous implementations for other class variables.

This commit also fixes a typo "helpercalsses" to "helperclasses"
Copy link

Changes detected in environment.yml but conda-lock.yml is not updated.

Changing how data type comparison is done to pass 'ruff' test
Copy link

Changes detected in environment.yml but conda-lock.yml is not updated.

@MatNif MatNif linked an issue Jul 18, 2024 that may be closed by this pull request
The envrionment.yml file used in this branch seems to have referred to an older version. This is a manual update of the changes that also includes the new operations required by the new clustering approaches.
The 'overlap_correction' method was briefly allocated to the 'Algorithm'-class but later added as a class variable of the connectivity vector class instead. This removes an artifact of the previous implementations.
Copy link

Changes detected in environment.yml but conda-lock.yml is not updated.

1 similar comment
Copy link

Changes detected in environment.yml but conda-lock.yml is not updated.

@ShiZhongming
Copy link
Collaborator

This PR works well. I tested it using a DH case in Zurich.
Just one thing - what did you mean by the network graph? @MatNif

@MatNif
Copy link
Collaborator Author

MatNif commented Jul 25, 2024

@ShiZhongming

By network-graph I mean the .geojson files output by the optimisation. They can be found in the individual "DCS_1XX/network" / "DHS_1XX/network" folders. The easiest way to visualise them currently is to upload the files to geojson.io.

Among other things, I will be working better visualisation toolsfor the dashboard in the coming months, so the visualisation should be possible in CEA itself soon.

@MatNif
Copy link
Collaborator Author

MatNif commented Aug 1, 2024

@martin-mosteiro, I guess you're quite busy. Have you had any time to give this a try?

If not, @ShiZhongming could we just merge the branch?

@ShiZhongming
Copy link
Collaborator

@MatNif Sorry that I missed this - Let's merge it!

@reyery reyery merged commit b9ebdf7 into master Aug 6, 2024
7 checks passed
@reyery reyery deleted the 3430-new-connetivity-vector-operations branch August 6, 2024 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Convergence to an optimal network is very slow
3 participants