Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prototype testing of remote functionality #3817

Closed
wants to merge 14 commits into from

Conversation

eisenhauer
Copy link
Member

@pnorbert This is a first-cut at how we might test the Remote server functionality in CI.
If you build this and do something like:
ctest -V -R REMOTE
You'll see that the CMake FIXTURES stuff is setup so that it starts a remote server before it runs a set of tests that require it and shuts it down afterwards. The server itself has to go into the background and close all of its FDs in order for ctest to continue after starting it. The "shutdown" test sends a message to the server asking it to die and in response the server sends its statistics (files opened, bytes read and operation count, for both Get()-level and transport-level operations). These are output from the shutdown test (but you can only see them if you've turned on ctest verbosity:

test 648
    Start 648: serverCleanup

648: Test command: /Users/eisen/prog/ADIOS2/build/bin/remote_server "-kill_server"
648: Working Directory: /Users/eisen/prog/ADIOS2/build/Testing/adios2/engine/bp
648: Test timeout computed to be: 1500
648: Server final status: ADIOS files Opened: 10 (142 gets for 40.6 KB)  Simple files opened: 0 (0 reads for 0 B )
3/3 Test #648: serverCleanup ....................   Passed    0.02 sec

The following tests passed:
	serverSetup
	REMOTEWriteReadADIOS2stdio
	serverCleanup

Right now I've just set this up for a single test, but arguably we need to do this for all of the BP engine tests and maybe all of the staging-common tests too. This is just a first cut, so there are deficiencies and non-optimal stuff. For example, right now both server and client are looking at the same file, so if the client wasn't doing remote stuff but was just reading data directly we'd only know it by noticing that the output of the server didn't show operations. But it's a start.

@vicentebolea You may be off already, or close to it, but I wanted to ask if this looked OK from a CMake/Ctest point of view. Also, how much slack do we have on CI timing now? If we essentially double the number of BP tests we're running by doing them both as we have been and also with the server involved, are we doing to hit CI timeouts?

@vicentebolea
Copy link
Collaborator

@vicentebolea You may be off already, or close to it, but I wanted to ask if this looked OK from a CMake/Ctest point of view.

It looks great from the cmake/ctest perspective, I suppose that this will be linux/osx only? You might want to add these test is not WIN32

Also, how much slack do we have on CI timing now? If we essentially double the number of BP tests we're running by doing them both as we have been and also with the server involved, are we doing to hit CI timeouts?

If we were to double the BP tests, this can really hinder performance in our MPI enabled builds since they already take 20-40mins to pass. I suggest to add them to the non-mpi builds (serial) a good candidate would be: ubuntu20.04-gcc8-serial

@eisenhauer eisenhauer closed this Sep 28, 2023
@eisenhauer eisenhauer deleted the RemoteTesting branch September 28, 2023 02:00
@eisenhauer eisenhauer mentioned this pull request Sep 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants