remove all code that uses wreck and other removed flux-core interfaces

[grondo]

There is a PR pending in flux-core: flux-framework/flux-core#1988 which will remove the wreck execution system and many other interfaces and APIs that have been deprecated for awhile.

Before this is merged, perhaps, this will require a strategy for flux-sched, a few ideas that come to mind are:

1. Transition immediately to compatibility with flux-core master by removing all the code and tests that require the old exec system. This would probably mean removal of resrc, RDL, and most of the tests. At some point it will have to be done anyway.
2. branch off at the v0.7.0 tag for now, and tie testing and configure of that version to v0.11.0 of flux-core.
3. Back out the most recent change that makes flux-sched incompatible with flux-core-v0.11.0, keep working on master and tie master to flux-core-0.11.0.

[SteVwonder]

My vote is for Option 1. I think there is value in keeping flux-sched master building against flux-core@master. If we tie flux-sched to v0.11.0 now, I'm afraid that all the changes to become compatible with flux-core@master will end up in one massive PR rather than several smaller, easier to digest PRs. I think Option 1 would also make it easier to test out the new job manager / scheduler interfaces.

@dongahn, what are your thoughts?

[grondo]

Thanks @SteVwonder. Another benefit of option 1 is that flux-sched repo will be managed the same as flux-core, and we can trigger option 2 when needed. That being said, it will be a fair amount of work to make flux-sched work with flux-core + flux-framework/flux-core#1988.

[dongahn]

If we were to do this, we might as well do this now. Option 1. All-in will make our effort more focused.

It will take substantial effort and we need to make sure we have enough of @SteVwonder and my time going into this.

[grondo]

I have a very rough cut started to make sure the work I've done in flux-framework/flux-core#1988 doesn't remove something sched may require. I'll link that here once it is complete. It may give you a head start.

[grondo]

See https://github.com/grondo/flux-sched/tree/kill-wreck -- this version removes rdl, resrc and the simulator, along with associated tests, commands and scripts. It passes Travis-CI, but I'm not sure if I removed too much.

Edit: I'm happy to open this branch as a PR, but I wanted approval first -- this first pass may have been too heavy a hand.

[grondo]

@SteVwonder @dongahn, did either of you have a chance to review my branch posted above? Before we can merge the PR in flux-core that removes the old execution system, we wanted to make sure flux-sched@master can move forward.

[SteVwonder]

Yeah. It looks fine to me. The RDL and resrc need to get removed anyway since we are switching to resource. As far as sched, we can always go back into the git history if we need to reference the old sched code when implementing the new sched that interfaces with the resource-match service. And if we don't have a sched, then keeping the old simulator doesn't make much sense. So overall, 👍.

@dongahn, thoughts?

[grondo]

Do you want me to just open a PR? I wasn't sure if either you or @dongahn wanted to do that instead, using my branch perhaps as a reference..

[SteVwonder]

If you are offering to open a PR, I'm not going to turn you down 😆
If you would prefer one of us to do it, that's fine. I can do that.

[dongahn]

LGTM as well. BTW, what this means is we don't have a scheduler which works end to end. So we will want to make it a priority to catch up the flux-core's new execution system. Some of the major items:

• Put in an RFC for R, which should also define the version 1 format that resembles R_lite. I'm working it.

• When @garlick puts in the code for scheduler loop service within flux-core, make a copy of it to flux-sched and extend to hook up with the rest of the system

• Need lots lots of test codes

Although there are a lot of work to do, the beauty of this is we get to use our scheduler as early as possible :-)