Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve and complete JSON writing #482

Closed
5 tasks done
pierrepebay opened this issue Dec 5, 2023 · 5 comments · Fixed by #483
Closed
5 tasks done

Improve and complete JSON writing #482

pierrepebay opened this issue Dec 5, 2023 · 5 comments · Fixed by #483
Assignees
Labels
enhancement New feature or request

Comments

@pierrepebay
Copy link
Contributor

pierrepebay commented Dec 5, 2023

The goal is to be able to have the JSON outputted by LBAF (when using the internal data structures) to be identical to the input JSON.

Missing elements at this point are (non exhaustive):

  • Communications
  • subphases
  • index
  • collection_id
  • shared_node data

This issue should also deprecate the use of -1 to read/step through all phases (only explicit enumerations or ranges should be used).

@pierrepebay pierrepebay added the enhancement New feature or request label Dec 5, 2023
@cwschilly
Copy link
Contributor

cwschilly commented Dec 5, 2023

@lifflander @ppebay @pierrepebay
I have edited the original comment with more elements that are not being written out. For a couple of these, I am not sure what they represent.

Specifically, what do index and collection_id refer to in the following snippet of an input JSON file?

"id": 0,
      "tasks": [
        {
          "entity": {
            "collection_id": 7,
            "home": 0,
            "id": 3407875,
            "index": [
              12
            ],
            "migratable": true,
            "type": "object"
          },

@ppebay
Copy link
Contributor

ppebay commented Dec 11, 2023

@cwschilly I think these quantities are not relevant to LBAF, but I will let @lifflander confirm this (or not)

@ppebay
Copy link
Contributor

ppebay commented Dec 12, 2023

N.B.: the first id concerns the phase.

@cwschilly
Copy link
Contributor

cwschilly commented Dec 13, 2023

I am also not sure what the data in the shared_node block represents:

"metadata": {
  "rank": 0,
  "shared_node": {
    "id": 0,
    "num_nodes": 1,
    "rank": 0,
    "size": 4
  },
  "type": "LBDatafile"
}

For now I am using the approach discussed in the meeting of swallowing everything into a dict and passing it through to the end.

@ppebay
Copy link
Contributor

ppebay commented Dec 13, 2023

@pierrepebay regarding the other fields: used in vt-tv, or not?

cwschilly added a commit that referenced this issue Dec 14, 2023
cwschilly added a commit that referenced this issue Dec 14, 2023
cwschilly added a commit that referenced this issue Dec 19, 2023
cwschilly added a commit that referenced this issue Jan 3, 2024
cwschilly added a commit that referenced this issue Jan 9, 2024
cwschilly added a commit that referenced this issue Jan 9, 2024
* #482: write out unused params, subphases, and beginnings of communications

* #482: use mp manager to use dicts and lists correctly

* #482: add if statements and fix failing tests

* #482: add unit test for communications data

* #482: clean up

* #482: add flexibilty for multiple phases

* #482: add output folder for communications test

* #482: finalize code, fix failing tests

* #482: add new test data and work on supporting multiple phases

* #482: fix multi-phase test, deprecate -1 option

* #482: remove print statements and imbalance.txt

* #482: fix failing test

* #482: add extra if statement to guard against inconsistent inputs

* #482: remove unused function
github-actions bot pushed a commit that referenced this issue Jan 9, 2024
* #482: write out unused params, subphases, and beginnings of communications

* #482: use mp manager to use dicts and lists correctly

* #482: add if statements and fix failing tests

* #482: add unit test for communications data

* #482: clean up

* #482: add flexibilty for multiple phases

* #482: add output folder for communications test

* #482: finalize code, fix failing tests

* #482: add new test data and work on supporting multiple phases

* #482: fix multi-phase test, deprecate -1 option

* #482: remove print statements and imbalance.txt

* #482: fix failing test

* #482: add extra if statement to guard against inconsistent inputs

* #482: remove unused function
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants