Multi-steplib zis proposal

[1000TurquoisePogs]

As requested in #703, this ticket is to discuss an implementation proposal

Goal: have zwe components install -o /the/extension[.pax] go beyond its current task of just copying unix content into the extensions directory, but also to update the ZIS STC steplib list.

Idea 1

zwe components install -o /the/extension[.pax] -d THE.DS1,THE.DS2
... Have -d serve as a list of datasets to append to the steplib.

Problems:

• zwe components upgrade and zwe components uninstall exist. How do we know which steplibs to REMOVE? How can we track which extension added which steplib entry? a file in the workspace directory? something in zowe.yaml?

• I don't like this being manual. How can we detect the datasets so that users do not need to be prompted, and cannot do something wrong? I'd like to query SMPE's knowledge somehow.

Idea 2

No new -d parameter, just manifest info & SMPE lookup.
... How can I query what datasets on a system are associated with an FMID? I think that's superior. If a Zowe extension manifest could have a field for stating its FMID, like:

name: angular-sample
id: org.zowe.zlux.sample.angular
version: 2.16.0
fmid: AZWE999

Then, can I get a list of datasets associated with it via something involving https://www.ibm.com/docs/en/zos/3.1.0?topic=command-listing-by-fmid-fmidset ?

And, if there were multiple datasets, how do I know which one goes in the steplib?
Would we need a suffix pattern to follow, such as:

name: angular-sample
id: org.zowe.zlux.sample.angular
version: 2.16.0
fmid: AZWE999 # Proposed new attribute. Used with steplibLLQ to find exact dataset.
zisPlugins:
  - id: plugin1
    steplibLLQ: ZISPLUG1 # Proposed new pattern. No copy occurs, but steplib entry added
  - id: plugin2
    steplibLLQ: ZISPLUG2
  - id: plugin3
    path: "/zisServer" # Current zowe pattern. it copies from here to the zis plugin ds

[JoeNemo]

Idea 1 needs to gather everything that is currently needed from Zowe base and all components (if i understand correctly) and remove what is no longer needed and add STEPLIB entries for everything that is needed. While not impossible, this sounds pretty complex.

[JoeNemo]

In Idea 2, what is the requirement that SMP/E actually know all STEPLIBs? Did we state this as a requirement for component developers?

[1000TurquoisePogs]

Idea 1 needs to gather everything that is currently needed from Zowe base and all components (if i understand correctly) and remove what is no longer needed and add STEPLIB entries for everything that is needed. While not impossible, this sounds pretty complex.

I know how to write it, but it sounds accident-prone. Users will just type the wrong thing and erase or add the wrong steplib.
If there was some eyecatcher in a zis plugin it would help. but is there?

[P-E-J]

Great to see you want to keep it automated but a STEPLIB change by hand is very common. So if I need to install an extra plugin and the manual state 'add this library to the ZWESLSTC steplib' it will work perfectly. I would make it clearer to me, especially while I have to do this anyway in our version control program in a place where we keep all proclibs together. From this central place we have a secure pipeline to distrubute all stc to all our sysplexes (10x).

[1000TurquoisePogs]

@P-E-J about automation:

Although the discussion was originally about datasets, the command zwe components install today also includes unix ZFS operations.

To avoid running that command at all, you would need to copy or symbolic link an extension's ZFS content into the "zowe.extensionDir" directory manually, and ensure one of your YAML lists the extension as "enabled: true". Some products seem to ship with a YAML that says this and request users to append that YAML to their YAML config list, but the ZFS directory copy has so far been automated. Is that a good enough solution for you to perform install/upgrade manually or do you have other ideas?