-
Notifications
You must be signed in to change notification settings - Fork 595
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create API for GCE Metadata #917
Comments
For reference, the Go implementation: |
So far, our APIs assume you're talking to the googleapis.com API, regardless of how you auth. We have recently landed support for Resource Manager and are working on Logging, when things get more complex. Specifically, a portion of our API-- creating a Project (RM) or a Sink (Logging)-- requires user auth, as opposed to from a service account. This can quickly confuse our users, since it's up to our docs to explain how, when, & where to use auth and run our code. So this method is another example where we have a method that may or may not work when actually called. Are there other actions that can be performed only when a user is running on a GCE instance? Possibly enough that could warrant creating a new module that has a dream-suite of utilities for someone running inside GCE? |
Not that I know of, but it's entirely possible. This will be a common thread though for the compute environment based APIs. For example, there are App Engine APIs we need to build veneers for that dig out App Engine specific information that only work when running inside of App Engine: https://cloud.google.com/appengine/docs/python/refdocs/google.appengine.api.modules.modules I think those use standard auth mechanisms though based on this: |
There's a library that looks like it was built to be a GAE suite of tools:
It doesn't look like it's been updated recently, so I'm not sure if it has any plans for expansion in sight. However, that seems like the right approach, as it's obvious to the user that their GAE app will benefit from this GAE-only toolkit. That could set the precedent as well for GCE, even if it starts as just a module with one function, "getMetadata". Heck, I'll write it! // @jgeewax any thoughts? |
Yeah - we really need to take appengine-nodejs down (we don't want people using that). It's talking to the App Engine v1 APIs with node, which was never really intended. The App Engine APIs I'm talking about are available via a One Platform API: |
The API seems pretty straightforward, so I wouldn't mind writing a library for it. Just to review, my proposal here is:
The whole point being, where confusion is possible, simply narrowing the scope of what a user can expect from a given library we offer. |
Interesting - why separate these specifically into their own NPM modules? Splitting this up seems kind of confusing, unless we choose to split up the whole module. Instead, I could see adding something like: let gcloud = require('gcloud');
gcloud.appEngine.isAvailable(function(err, isAvailable) {
if (!err && isAvailable) {
gcloud.appEngine.getModules(function(err, modules) {
console.log(modules);
});
} else {
console.log('not running in app engine!');
}); That way I can write code to specifically handle the case where I'm not in a particular environment. Same for GCE: let gcloud = require('gcloud');
gcloud.computeEngine.isAvailable(function(err, isAvailable) {
if (!err && isAvailable) {
gcloud.computeEngine.getMetadata(path, function(err, res) {
console.log(res);
});
} else {
console.log('not running in compute engine!');
}
}); If isAvailable doesn't sound right, getExecutionEnvironment is another option: let gcloud = require('gcloud');
gcloud.getExecutionEnvironment(function(err, env) {
if (!err && env === gcloud.environments.appEngine) {
console.log("I'm in app engine!");
} else {
console.log('not running in app engine!');
}
}); |
@jmdobry and @jonparrott interested in your thoughts here too. |
|
One reason is simply to shrink the overhead of this repo. The bigger this library grows, it's simply harder to maintain and put out a new release. If we stabilize at 1.0, then catch a breaking change in one of (smallest possible number here) APIs we support, we have to force a major increment upgrade (+1.0) on all of our users to have them benefit from smaller releases with features and fixes. That aside, I prefer this split to avoid confusion. I think a user should be able to install a library and have everything work. Then, if they need something more specific to their application, they can install another library. I mentioned earlier the Logging and Resource Manager API muddying things up a bit earlier, but in those cases, we at least give the user the majority of the API regardless of what environment they're in. With these extra APIs for App and Compute Engine, I think the separation will make it more clear for a user to understand what to use given their environment. Also, it will help users who may only need the GAE/GCE functionality by offering that library as separately installable. |
Why is that?
Basically, the user shouldn't need this behavior because they're the ones deploying their app, so they should know where it's at? |
Because from my point of view, App Engine is not a special snowflake in our hosting options.
I'm just saying that at most, we should have
Ideally, there should be little need to distinguish between GAE, GCE, and GKE. Even if that is desired, the metadata API is the wrong place to do it. |
That makes sense to me. The idea of being able to stop/delete a module programmatically sounded cool, though. But, if all we want is a "try to get metadata" method, this is fine with me?: var gcloud = require('gcloud');
gcloud.getMetadata(function(err, metadata) {
if (/*not in a place we can get metadata*/) {
err.message = 'Not in a place we can get metadata.';
}
}); |
FYI, I believe the ideal API surface for the metadata server should look something like this: var metadata = require('gcloud').metadata();
// get instance tags.
metadata.instance.tags(function(err, tags) {
console.log(tags);
// ['one', 'two', 'blue', 'shoe']
});
// Get project-level custom metadata.
metadata.project.custom(function(err, kv) {
console.log(kv);
// {color: "red", active: "yes"}
});
// watch for changes to the instance tags.
metadata.instance.tags.watch(function(err, tags) {
//...
}); |
That should be done in the with the App Engine Admin API. :)
Yep, or just have all of the methods of metadata do one of the following:
FYI, IMO the timeout for hitting the metadata server should be set relatively low. |
I think your examples show more than I thought was involved with my "just try to get the metadata" understanding. I still would do this separate if it was my own project, but I can see the importance of the docs all being in one place.
I definitely wouldn't have thought of anything like that myself. Any other things we should support? |
That's a good start. Writing metadata is also a useful thing to have. See https://cloud.google.com/compute/docs/metadata Let me know if there's anything I can do to help. As a heads up, please use |
Definitely, thanks as always! |
Hey on the App Engine API bit - we absolutely need to have an App Engine Management API available. There is an entire REST API surface of commands customers will use: When I say App Engine API - I don't mean things like mail and datastore, I'm talking about APIs that:
That merits an SDK in the same way VM creation and management on our platform needs a client library. |
Is this being worked on? If not, I'm happy to submit a PR. I think the API surface @jonparrott described is a good place to start. |
Thanks for offering! Just for the sake of mentioning, I whipped up this quick API that has a basic, similar surface recently: https://www.npmjs.com/package/gcp-metadata. It's lacking features that @jonparrott originally proposed, such as a watcher. And it's missing all of the stuff @JustinBeckwith suggested. Since this issue was originally discussed, we've modularized our repo and deprecated
And if the answer to #1 is 👍 , then a PR to get the ball rolling would be awesome 💃 @lukesneeringer for all of the answers. |
Another thing that has changed since the issue was discussed is that we can auto-generate GAPICs now. This sounds like a thing that should be auto-gen only so we do not have as high of a maintenance burden on it. |
This issue was moved to googleapis/nodejs-compute#12 |
Re-opening, as this wasn't really an issue about the Compute API. Is there anything to do here? @alexander-fenster ? |
👋 I think gcp-metadata is covering this quite nicely. @stephenplusplus we can probably close this out :) |
…tion context (#917) * feat: added option to configure the number of sentences in the suggestion context PiperOrigin-RevId: 420101041 Source-Link: googleapis/googleapis@549b567 Source-Link: googleapis/googleapis-gen@7d095a8 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiN2QwOTVhOGM2YTYyMjNkZDRlZTc3MmMyODUxMTI0ODNjMmFmYjFjMCJ9 * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
🤖 I have created a release *beep* *boop* --- ## [4.7.0](googleapis/nodejs-dialogflow@v4.6.0...v4.7.0) (2022-03-25) ### Features * added export documentation method ([35a77f4](googleapis/nodejs-dialogflow@35a77f4)) * added option to configure the number of sentences in the suggestion context ([#917](googleapis/nodejs-dialogflow#917)) ([17aa7d6](googleapis/nodejs-dialogflow@17aa7d6)) * removed OPTIONAL for speech model variant docs: added more docs for speech model variant and improved docs format for participant ([#906](googleapis/nodejs-dialogflow#906)) ([7df7d27](googleapis/nodejs-dialogflow@7df7d27)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
... chore: update gapic-generator-ruby to the latest commit chore: release gapic-generator-typescript 1.5.0 Committer: @miraleung PiperOrigin-RevId: 380641501 Source-Link: googleapis/googleapis@076f7e9 Source-Link: googleapis/googleapis-gen@27e4c88
🤖 I have created a release \*beep\* \*boop\* --- ### [2.3.4](https://www.github.com/googleapis/nodejs-vision/compare/v2.3.3...v2.3.4) (2021-06-22) ### Bug Fixes * make request optional in all cases ([#917](https://www.github.com/googleapis/nodejs-vision/issues/917)) ([d301b79](https://www.github.com/googleapis/nodejs-vision/commit/d301b79dee83948bada4b16e33048933d454c051)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
There's a great metadata service for getting access to information on the VM. It would be great to have a structured API for this:
https://cloud.google.com/compute/docs/metadata?hl=en
The text was updated successfully, but these errors were encountered: