OQS sub projects: Which ones to drop for good

[baentsch]

This is to initiate a discussion on OQS sub projects: There are many that do not receive any attention in terms of issue-creation, -resolution or other activity. We have declared some as "inactive" to no great(ly visible) disagreement.

Proposal:

• Designate 3 sub project tiers: Fully supported (at least 2 fully committed maintainers), best effort (maybe a part-time maintainer), inactive (none of us cares).
• Assign all projects to these tiers
• Only discuss/track fully supported projects going forward, e.g., in https://www.openquantumsafe.org/dashboard.html
• Integrate (cross-project) CI only for fully supported projects, incl. support of the same platforms & binaries/distro-packages
• Get input from PQCA members as to which projects they want (& fund)

Strawman proposal: Only liboqs, oqsprovider and possibly liboqs-python receive "fully supported" status.

Input/feedback/alternative suggestions welcome.

[baentsch]

Without feedback after a week and in line with our lazy consensus principle I consider there is no objection to change all sub projects' summary view (except the ones above, i.e., those with a dedicated maintainer) to read similar to libssh:

THIS PROJECT IS PRESENTLY NOT FULLY SUPPORTED. CONTRIBUTORS AND MAINTAINER(S) WANTED.

Such statement sets proper expectations by users hitting problems: They simply may get no response, let alone a fix. Most notably this will be applied to oqs-demos and profiling for which I personally currently have no propensity to look after any more.

This way we may be able to catch the attention of the community -- and make it clear that the takeover by Linux Foundation does not mean an automatic improvement in terms of actual people working on the project.

Personally, I'm afraid the contrary is true: With the establishment of PQCA "the world" may have been convinced that OQS is now fully supported and well-staffed by the corporate sponsors and thus cease to have an interest to actively contribute, possibly even expect reported issues to be resolved (more) quickly.

Explicitly tagging @dstebila as our "man in the GB" should this TSC "decision" (in the absence of other statements :-) not otherwise get flagged in his Inbox.

[christianpaquin]

I agree with the proposed 3 tiers, I think all 3 should get a different "message" (i.e., don't mark "best effort" and "inactive" with the same message, and we want to run some "best effort" cross-CI testing to "best effort" projects (could be prioritized based on available resources). Agreed that we need to urgently reflect the expected level of support in the project's documentation.

[dstebila]

Here's the PQCA's working document for their project lifecycle: https://docs.google.com/document/d/1NV-0vNgXWdc81oqT0jv0C-9Funb8dySS06u90ghF-X4/edit

[baentsch]

Here's the PQCA's working document for their project lifecycle: https://docs.google.com/document/d/1NV-0vNgXWdc81oqT0jv0C-9Funb8dySS06u90ghF-X4/edit

What concrete impact does this document have on this issue? Is there any timeline for this document to be completed (seems inactive)? Should completion of this document stop this issue from proceeding?

My concrete proposal:

• Suggest to @vsoftco to create GOVERNANCE.md files modeled after those of oqs-provider & liboqs for those projects he's maintaining.
• Drop all sub projects without GOVERNANCE.md files (and hence maintainers) from the list of actively maintained OQS sub projects, e.g., on the Web site (as already done for ssh&openssl-111) as well as CI, incl. dashboard.

[dstebila]

Here is a table with my first draft attempt at classifying all our subprojects/repositories against the classification Michael listed at the top of this issue (Fully supported / Best effort / Inactive) and the PQCA project lifecycle (Experimentation / Labs / Incubation / Growth / Impact / Emeritus). My classification here is really a first gut reaction based on where I think we have the most activity and momentum. I wasn't sure about some of the language wrappers, such as .NET/Go/Java, for example -- just a gut feeling that they get less attention than C++/Python/Rust, but that could be mistaken.

I don't think either system is perfect. In terms of helping us make operational decisions (e.g., how to prioritize CI) I think the simpler classification is useful.

Please contribute your own thoughts, both about the classification systems and on any particular subprojects.

Subproject	Comment	Classification as in tsc/issue/2	Classification per PQCA lifecycle
boringssl	Periodically updated	Best effort
ci-containers	Support repository, not sub-project	Fully supported	N/A
liboqs		Fully supported	Impact stage
liboqs-cpp		Best effort? Fully supported?	Growth stage?
liboqs-dotnet		Best effort?	Incubation stage?
liboqs-go		Best effort?	Incubation stage?
liboqs-java		Best effort? Inactive?	??
liboqs-python		Best effort? Fully supported?	Growth stage?
liboqs-rust		Best effort? Fully supported?	Growth stage?
libssh		Inactive	Emeritus
openssh	Interest in reviving this?	Inactive -> Best effort?	Emeritus -? Incubation stage?
openssl	1.1.1	Inactive	Emeritus
oqs-demos		Best effort	Experimentation stage?
oqs-provider		Fully supported	Impact stage
profiling	Intended to be revived	Inactive	Experimentation stage

[baentsch]

Thanks for taking a stab at this summary, @dstebila.

I completely agree with columns 1-3 with one request for change: The term "Fully supported" should be changed to "Fully github supported" for oqs-provider in the third column as the former may imply support on all channels PQCA has chosen to create (email lists, DM tooling, ...) but that I cannot follow and hence, cannot promise to provide support on. Tagging @thb-sb for commenting on this: If you'd follow and interact with the PQCA comms channels (and keep acting as oqs-provider maintainer) (?), it may be OK to leave the entry as-is.

Regarding the forth column, I have to absolutely and adamantly protest the labelling of oqs-provider as (LF/PQCA) "Impact Stage": That project is NOT receiving financial nor any other support from PQCA (rather the opposite if I compare how OQS operates before and after the LF take-over: Contributions and procedures have surely become more difficult and cumbersome). Most notably, oqs-provider (well, I guess myself as the 95% contributor/maintainer of it) is accordingly not willing to "to cross promote the foundation along with their activities" as per the demand placed on such project according to the PQCA project classification --> please change over to "Incubation stage" if you need to apply this classification -- that I personally consider pretty much completely unsuitable to OQS:

To justify the latter statement, see these two pretty much contradictory statements:

Document that it is being used successfully in production by at least two independent end users which, in the TAC’s judgment, are of adequate quality and scope.
(from PQCA project classification)

WE DO NOT CURRENTLY RECOMMEND RELYING ON THIS LIBRARY IN A PRODUCTION ENVIRONMENT OR TO PROTECT ANY SENSITIVE DATA
(from liboqs usage advice).

and one unresolved issue #1 to document that labelling liboqs "Impact Stage" as being completely unjustifiable; according to the above, liboqs isn't even "Growth Stage". Now, if we were to correct this, not a single OQS sub project would be anywhere else than "Incubation stage" -- pretty much leading the whole PQCA classification ad absurdum -- which is why I'd suggest dropping that column outright (also automatically resolving my initial comments regarding the PQCA classification of oqs-provider :-).

Here's btw my practical, non-PQCA-process driven suggestion as to how to handle oqs-demos now: open-quantum-safe/oqs-demos#275

[christianpaquin]

WRT OpenSSH, I discussed with my AWS peers with whom we do interop testing in the NCCoE project, but there doesn't seem to have bandwidth to take maintenance ownership (@brian-jarvis-aws can confirm)

[brian-jarvis-aws]

WRT OpenSSH, I discussed with my AWS peers with whom we do interop testing in the NCCoE project, but there doesn't seem to have bandwidth to take maintenance ownership (@brian-jarvis-aws can confirm)

That’s correct regarding maintenance, however I do still plan to allocate time for one of my team members to bring the project up to date with the latest upstream OpenSSH version for continued interop testing.

[baentsch]

@dstebila Here's another voice arguing that liboqs should be rated in "an early lifecycle stage": @hartm writes in open-quantum-safe/liboqs#1832 (comment):

API changes are better the earlier they are in a project's lifecycle. As the project grows, it becomes harder and harder to make these kinds of changes. So if it's something that's needed, it's better to do now rather than later.

I fully support this view -- as long as we then also agree that liboqs cannot be rated at LinuxFoundation rating level "Impact" or even "Growth" (regardless of whether or not the project is 8 years old :-)

My rationale for that "low" rating was that OQS does not have a "healthy" contributor or maintainer base (using the term from the LF/PQCA lifecycle document they want to apply to OQS) -- but API stability obviously is another matter in that consideration.

[baentsch]

One more criterion helping to define to a sub project's maturity level: Security incident handling.