Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Trace MPI events with PMPI wrappers when outside VT #900

Closed
lifflander opened this issue Jun 27, 2020 · 0 comments
Closed

Trace MPI events with PMPI wrappers when outside VT #900

lifflander opened this issue Jun 27, 2020 · 0 comments
Assignees
@pnstickne
Labels
type: task

Comments

@lifflander
Copy link
Collaborator

What Needs to be Done?

We currently track when we are outside VT. With the PMPI wrappers enabled, we can generate user events for MPI calls, like MPI_Send/MPI_Recv/MPI_Reduce/etc.
pnstickne added a commit that referenced this issue Jun 30, 2020
@pnstickne
#900 split generator into smaller methods
2b2bf0d 
- A bit easier to work with, perhaps.
pnstickne added a commit that referenced this issue Jun 30, 2020
@pnstickne
#900 add ability for custom actions in generator
c342e65 
- Now can map custom actions in the generator; currently a simple
  sub, and to be logging or some such.
pnstickne added a commit that referenced this issue Jun 30, 2020
@pnstickne
#900 add logging for MPI send+recv methods
7f9532b
pnstickne added a commit that referenced this issue Jun 30, 2020
@pnstickne
#900 trace- unify reporting to rank 0
7cfaf41 
- When a trace user ID is registered it is (often) reported
  up to rank 0 so it can be added to the STS definition.

  Unify the logic in one place to support future changes.

  Also fixes an edge-case where it may be possible for the same registration
  to be sent multiple times, which is wasteful.
pnstickne added a commit that referenced this issue Jun 30, 2020
@pnstickne
#900 trace- enqueue sending registration of user events
9b2c9c7 
- This unwinds the stack to avoid failing attempting to re-grant MPI
  access while running under a PMPI wrapper..
@nmm0 nmm0 mentioned this issue Jul 2, 2020
Closed
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 split generator into smaller methods
7549b5e 
- A bit easier to work with, perhaps.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 add ability for custom actions in generator
3dc83ca 
- Now can map custom actions in the generator; currently a simple
  sub, and to be logging or some such.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 add logging for MPI send+recv methods
b70c7e4
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 trace- unify reporting to rank 0
608a851 
- When a trace user ID is registered it is (often) reported
  up to rank 0 so it can be added to the STS definition.

  Unify the logic in one place to support future changes.

  Also fixes an edge-case where it may be possible for the same registration
  to be sent multiple times, which is wasteful.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 trace- enqueue sending registration of user events
0bc5326 
- This unwinds the stack to avoid failing attempting to re-grant MPI
  access while running under a PMPI wrapper..
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- add PMPI Component
ba3d2a7 
- Provides some API between auto-generated code and standard VT component
  registration and lifetime management.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- wrappers generate collective events
588153c
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- add basic runtime 'should log' control point
2fa95e3 
- This should (eventually) use option flags and/or the scheduler
  runtime depth information to determine if a PMPI call should
  be logged, regardless of overall tracing status.

  This also finishes up some component registration and exposure,
  as needed to be accessed from the free functions.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- add PMPI Component
945017d 
- Provides some API between auto-generated code and standard VT component
  registration and lifetime management.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- wrappers generate collective events
f1c2ab0
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- add basic runtime 'should log' control point
96d1d89 
- This should (eventually) use option flags and/or the scheduler
  runtime depth information to determine if a PMPI call should
  be logged, regardless of overall tracing status.

  This also finishes up some component registration and exposure,
  as needed to be accessed from the free functions.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- avoid invalid MPI access (timing)
3943774 
- Prevent issue where MPI_Wtime could be accessed prior to
  MPI_Init and crash program..
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- swtich from static to anon namespace
71ea0b7 
- Same effect, more modern.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- only trace outside of a VT event loop
5928230 
- Has to be outside of a loop, without any specific PMPI grants
  in scope. This can be improved once all usages use the newer
  loop structure..
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- add vt_trace_pmpi flag to enable tracing
4c31441 
- If nothing else, this allows tests to easily pass.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- exclude Wtime/Wtick universally
f9d12ca 
- These were individually excluded and are considered
  core infrastructure PMPI calls that have no need to be
  logged or guarded.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- even Wtime bypassed in TraceScopedEvent
e1a1a39 
- Passing 'no event' is used to create a NO-OP call to this type.
  Ensure that it truly is a NO-OP in this case.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 trace- unify reporting to rank 0
20fcb6c 
- When a trace user ID is registered it is (often) reported
  up to rank 0 so it can be added to the STS definition.

  Unify the logic in one place to support future changes.

  Also fixes an edge-case where it may be possible for the same registration
  to be sent multiple times, which is wasteful.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 trace- enqueue sending registration of user events
e498918 
- This unwinds the stack to avoid failing attempting to re-grant MPI
  access while running under a PMPI wrapper..
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- add PMPI Component
d0913e5 
- Provides some API between auto-generated code and standard VT component
  registration and lifetime management.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- update generators to handle tracing
49bce29 
- Splits generators into smaller pieces and prepares
  for future work.

- Wrappers generate collective events.
pnstickne added a commit that referenced this issue Jul 14, 2020
@pnstickne
#900 pmpi- add basic runtime 'should log' control point
d5e4522 
- This should (eventually) use option flags and/or the scheduler
  runtime depth information to determine if a PMPI call should
  be logged, regardless of overall tracing status.

  This also finishes up some component registration and exposure,
  as needed to be accessed from the free functions.
pnstickne added a commit that referenced this issue Jul 21, 2020
@pnstickne
#900 pmpi- even Wtime bypassed in TraceScopedEvent
4def551 
- Passing 'no event' is used to create a NO-OP call to this type.
  Ensure that it truly is a NO-OP in this case.
pnstickne added a commit that referenced this issue Jul 21, 2020
@pnstickne
#900 pmpi- add PMPI Component
a71b624 
- Provides some API between auto-generated code and standard VT component
  registration and lifetime management.

- This should (eventually) use option flags and/or the scheduler
  runtime depth information to determine if a PMPI call should
  be logged, regardless of overall tracing status.

  This also finishes up some component registration and exposure,
  as needed to be accessed from the free functions.
pnstickne added a commit that referenced this issue Jul 21, 2020
@pnstickne
#900 pmpi- update generators to handle tracing
f3d4cf7 
- Splits generators into smaller pieces and prepares
  for future work.

- Wrappers generate collective events.
pnstickne added a commit that referenced this issue Jul 21, 2020
@pnstickne
#900 pmpi- only trace outside of a VT event loop
1c4ae3b 
- Has to be outside of a loop, without any specific PMPI grants
  in scope. This can be improved once all usages use the newer
  loop structure..
pnstickne added a commit that referenced this issue Jul 21, 2020
@pnstickne
#900 pmpi- add vt_trace_pmpi flag to enable tracing
f4cfac0 
- If nothing else, this allows tests to easily pass.
pnstickne added a commit that referenced this issue Jul 21, 2020
@pnstickne
#900 pmpi- even Wtime bypassed in TraceScopedEvent
df4706e 
- Passing 'no event' is used to create a NO-OP call to this type.
  Ensure that it truly is a NO-OP in this case.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- add PMPI Component
e2db161 
- Provides some API between auto-generated code and standard VT component
  registration and lifetime management.

- This should (eventually) use option flags and/or the scheduler
  runtime depth information to determine if a PMPI call should
  be logged, regardless of overall tracing status.

  This also finishes up some component registration and exposure,
  as needed to be accessed from the free functions.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- update generators to handle tracing
393ed95 
- Splits generators into smaller pieces and prepares
  for future work.

- Wrappers generate collective events.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- only trace outside of a VT event loop
7429dcf 
- Has to be outside of a loop, without any specific PMPI grants
  in scope. This can be improved once all usages use the newer
  loop structure..
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- add vt_trace_pmpi flag to enable tracing
870402d 
- If nothing else, this allows tests to easily pass.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- even Wtime bypassed in TraceScopedEvent
e139532 
- Passing 'no event' is used to create a NO-OP call to this type.
  Ensure that it truly is a NO-OP in this case.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- add vt_trace_pmpi flag to enable tracing
d99d88a 
- If nothing else, this allows tests to easily pass.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- even Wtime bypassed in TraceScopedEvent
12a1692 
- Passing 'no event' is used to create a NO-OP call to this type.
  Ensure that it truly is a NO-OP in this case.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- add vt_trace_pmpi flag to enable tracing
faa3eea 
- If nothing else, this allows tests to easily pass.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- even Wtime bypassed in TraceScopedEvent
c36f6ab 
- Passing 'no event' is used to create a NO-OP call to this type.
  Ensure that it truly is a NO-OP in this case.
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- add license to generate_mpi_wrappers.pl
9773d04
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- add license to generate_mpi_wrappers.pl
49e85a0
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- add license to generate_mpi_wrappers.pl
7c88c6d
pnstickne added a commit that referenced this issue Jul 28, 2020
@pnstickne
#900 pmpi- update comment and guard MPI_Get
f1a6c85 
- MPI_Get is now guarded.

- Some minor comment updates.
pnstickne added a commit that referenced this issue Jul 31, 2020
@pnstickne
#900 pmpi- unify argument handling of --vt_trace_mpi
84dc911 
- Now uses an option instead of a flag. Specify the form of MPI to enable.
  In C++, the options should be separated with spaces:

      --vt_mpi_trace internal
      --vt_mpi_trace external
      --vt_mpi_trace internal externall

  'internal' is the few select internal calls, 'external' is PMPI overall.
lifflander pushed a commit that referenced this issue Aug 4, 2020
@pnstickne @lifflander
#900 pmpi- add PMPI Component
08e34f9 
- Provides some API between auto-generated code and standard VT component
  registration and lifetime management.

- This should (eventually) use option flags and/or the scheduler
  runtime depth information to determine if a PMPI call should
  be logged, regardless of overall tracing status.

  This also finishes up some component registration and exposure,
  as needed to be accessed from the free functions.
lifflander pushed a commit that referenced this issue Aug 4, 2020
@pnstickne @lifflander
#900 pmpi- update generators to handle tracing
fda5228 
- Splits generators into smaller pieces and prepares
  for future work.

- Wrappers generate collective events.
lifflander pushed a commit that referenced this issue Aug 4, 2020
@pnstickne @lifflander
#900 pmpi- only trace outside of a VT event loop
7dfb5e3 
- Has to be outside of a loop, without any specific PMPI grants
  in scope. This can be improved once all usages use the newer
  loop structure..
lifflander pushed a commit that referenced this issue Aug 4, 2020
@pnstickne @lifflander
#900 pmpi- add vt_trace_pmpi flag to enable tracing
c87c547 
- If nothing else, this allows tests to easily pass.
lifflander pushed a commit that referenced this issue Aug 4, 2020
@pnstickne @lifflander
#900 pmpi- even Wtime bypassed in TraceScopedEvent
96cde6b 
- Passing 'no event' is used to create a NO-OP call to this type.
  Ensure that it truly is a NO-OP in this case.
lifflander pushed a commit that referenced this issue Aug 4, 2020
@pnstickne @lifflander
#900 pmpi- add license to generate_mpi_wrappers.pl
16eede0
lifflander pushed a commit that referenced this issue Aug 4, 2020
@pnstickne @lifflander
#900 pmpi- update comment and guard MPI_Get
e951b42 
- MPI_Get is now guarded.

- Some minor comment updates.
lifflander pushed a commit that referenced this issue Aug 4, 2020
@pnstickne @lifflander
#900 pmpi- unify argument handling of --vt_trace_mpi
9619d8c 
- Now uses an option instead of a flag. Specify the form of MPI to enable.
  In C++, the options should be separated with spaces:

      --vt_mpi_trace internal
      --vt_mpi_trace external
      --vt_mpi_trace internal externall

  'internal' is the few select internal calls, 'external' is PMPI overall.
@lifflander lifflander mentioned this issue Nov 24, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pnstickne pnstickne

Labels
type: task
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pnstickne @lifflander