Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically set project-id and zone #13618

Closed
dadoonet opened this issue Sep 16, 2015 · 4 comments · Fixed by #33721
Closed

Automatically set project-id and zone #13618

dadoonet opened this issue Sep 16, 2015 · 4 comments · Fixed by #33721
Labels
:Distributed Coordination/Discovery-Plugins Anything related to our integration plugins with EC2, GCP and Azure >enhancement help wanted adoptme v6.5.0 v7.0.0-beta1

Comments

@dadoonet
Copy link
Member

The project id can be retrieved from the running instance by calling http://metadata.google.internal/computeMetadata/v1/project/project-id.

Same for the default zone: http://metadata.google.internal/computeMetadata/v1/project/attributes/google-compute-default-zone

No need then to force them in elasticsearch.yml.

@clintongormley clintongormley changed the title [cloud-gce] automatically set project-id and zone Automatically set project-id and zone Sep 16, 2015
@clintongormley clintongormley added :Distributed Coordination/Discovery-Plugins Anything related to our integration plugins with EC2, GCP and Azure and removed :Plugin Discovery GCE labels Feb 13, 2018
@DaveCTurner
Copy link
Contributor

@dadoonet could you clarify what needs doing here? The first point sounds like that a value for cloud.gce.project_id could be obtained from the GCE metadata service (presumably still allowing a manual override), right? I don't understand the second point: I see that there's a cloud.gce.zone setting, but this seems to want to be a list of all zones to search for nodes, not just the default zone?

@dadoonet
Copy link
Member Author

The first point sounds like that a value for cloud.gce.project_id could be obtained from the GCE metadata service (presumably still allowing a manual override), right?

Yes.

I see that there's a cloud.gce.zone setting, but this seems to want to be a list of all zones to search for nodes, not just the default zone?

Indeed but if not set we can just fill that list with the current zone only which is I believe what users are doing most of the time I guess: deploying in one single zone.

@DaveCTurner
Copy link
Contributor

Ah right. I was thinking more about AWS availability zones of which it's best to use more than 1, but if that's a common way to use GCE then that makes sense.

@vladimirdolzhenko
Copy link
Contributor

vladimirdolzhenko commented Sep 14, 2018

com.google.cloud.ServiceOptions#getDefaultProjectId tries to resolve project based on

  • system property GOOGLE_CLOUD_PROJECT (or legacy system property GCLOUD_PROJECT)
  • environment variable GOOGLE_CLOUD_PROJECT or GCLOUD_PROJECT
  • from http://metadata.google.internal/computeMetadata/v1/project/project-id

ES assumes system properties as unsafe while environment variables could be predefined (e.g. for ECS Task IAM profile credentials in repository-s3 plugin) on the environment (could it be different from value returned from http://metadata.google.internal/computeMetadata/v1/project/project-id? )

Are we going to use only last method ?

vladimirdolzhenko pushed a commit to vladimirdolzhenko/elasticsearch that referenced this issue Sep 14, 2018
vladimirdolzhenko added a commit that referenced this issue Oct 3, 2018
Fetch default values for project-id and zone from metadata server

Closes #13618
vladimirdolzhenko added a commit that referenced this issue Oct 3, 2018
Fetch default values for project-id and zone from metadata server

Closes #13618

(cherry picked from commit a7f62ee)
kcm pushed a commit that referenced this issue Oct 30, 2018
Fetch default values for project-id and zone from metadata server

Closes #13618
@jimczi jimczi removed the v7.0.0 label Feb 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Discovery-Plugins Anything related to our integration plugins with EC2, GCP and Azure >enhancement help wanted adoptme v6.5.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants