extend staging capabilities for newer GAE runtimes under development

[ludoch]

We are testing the current Maven GAE plugin for new scenarios for App Engine Standard future runtimes...

First of all, they are for Standard (even if the current logs I see shows that Flex staging is somewhat used)
Secondly, we need to be able to put new abritrary files under the src/main/appengine/ directory so that all those files end up in the staging directory.
Currently, it seems that the staging phase only put the app.yaml from src/main/appengine/ but would not copy a src/main/appengine/foo file.
Examples for foo (i.e random files) would be a new package,json that will trigger Cloud builder steps, as well as new Cloud Builder specific files defined in a gcp-build field (PRD is intenral, but I can give internal links to Googlers).
So the only ask is that a stage phase of the plugins (Maven, Gradle, Eclipse, Idea) would copy all the files from src/main/appengine in the staging directory.
The rule to follow if some files are already in the staging directory is TBD:
1/ Warning, and let src/main/appengine/ content win?
2/ Error?
3/ do not override.
I would prefer 1 as the staging directory is specific to App Engine, so App Engine configuration should win over conflicts.

[patflynn]

I'm open to supporting the usecase, but I'm doubtful whether we want to copy everything in src/main/appengine into the staging dir. The issue is that we had to do some special magic to avoid serving app.yaml in flex compat. The appengine directory is really for config that's passed to gcloud and not to be included in the deployment artifacts.

Anything that needs to be included at serving runtime should be configured as deployment artifacts somehow.

Supporting standard apps using app.yaml and bypassing standard staging is not part of our current design, so if that is going to be the flow required for new standard java runtimes, some re-work is going to be in the cards.

In the mean time I'm not against allowing flex scenarios to specify more than one deployment file.

[ludoch]

One thing to look at is why the flex staging is possibly used when an app.yaml does not content env:flex
The runtime id check should be irrelevant as this is a java tooling, so the user intent is to use java, regardless of the runtime id, and the new java runtimes may have all sorts of new runtimeId, including container, and in some funny cases, nodejs8 sometimes.

[ludoch]

"Anything that needs to be included at serving runtime should be configured as deployment artifacts somehow." how?
For java, the main deployment artifact is a jar file, possibly resource files that could be outside this jar files, and configuration files for deployment.
Up to now, config file was only app.yaml, but with Cloud Builder extra steps that need to be configured, including these config scripts which name can be random, I still believe that tooling should treat src/main/appengine/* and sub directory as valid content to be added to a staging area.

[patflynn]

@ludoch you would end up serving those files in flex compat.

[ludoch]

No, if env is not set in appengine-web or if env is not set to flex/flexible, then the app is always deployed to standard.
The optional env field in the generated app.yaml is the critical parameter there.

[ludoch]

Friendly ping

[patflynn]

@ludoch ideally we'd get in the room with you and dnorsdtrom to discuss how we'd like to update our toolchain to natively support app.yaml standard apps.

[elharo]

Is this done?

[JoeWang1127]

close as not planned