chore: remove unused code #1119

mdelapenya · 2021-05-04T09:57:18Z

What does this PR do?

It removes all references in code to boostrapping the fleet-server, as it's already provided by runtime dependencies declared by docker-compose.

Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose.

Why is it important?

It removes not-used code, simplifying methods

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have run the Unit tests for the CLI, and they are passing locally
I have run the End-2-End tests for the suite I'm working on, and they are passing locally
I have noticed new Go dependencies (run make notice in the proper directory)

How to test this PR locally

Related issues

Relates Auto bootstrap fleet during initialize scenario #1116

Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose.

elasticmachine · 2021-05-04T10:33:41Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #1119 updated
Start Time: 2021-05-04T18:53:44.032+0000
Duration: 23 min 39 sec
Commit: 06a2bf9

Test stats 🧪

Test	Results
Failed	0
Passed	152
Skipped	0
Total	152

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	152
Skipped	0
Total	152

We do not need it anymore, as the fleet server is already bootstrapped

internal/kibana/fleet.go

mergify · 2021-05-04T17:20:33Z

This pull request is now in conflict. Could you fix it @mdelapenya? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b remove-fleet-server-leftovers upstream/remove-fleet-server-leftovers
git merge upstream/master
git push upstream remove-fleet-server-leftovers

mdelapenya · 2021-05-04T19:23:54Z

internal/kibana/policies.go

@@ -87,7 +87,8 @@ func (c *Client) DeleteAllPolicies() {
 	}
 	for _, pkgPolicy := range packagePolicies {
 		// Do not remove the fleet server package integration otherwise fleet server fails to bootstrap
-		if !strings.Contains(pkgPolicy.Name, "fleet_server") && pkgPolicy.PolicyID == "system" {
+		if !strings.Contains(pkgPolicy.Name, "fleet_server") && !strings.Contains(pkgPolicy.Name, "system") {


🤦‍♀️ good catch

* chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]> (cherry picked from commit 82380ce) # Conflicts: # e2e/_suites/fleet/fleet.go # e2e/_suites/fleet/stand-alone.go # internal/kibana/fleet.go

* chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]> (cherry picked from commit 82380ce) # Conflicts: # e2e/_suites/fleet/fleet.go # e2e/_suites/fleet/fleet_server.go # e2e/_suites/fleet/stand-alone.go # internal/installer/elasticagent.go # internal/installer/tar.go # internal/kibana/fleet.go # internal/kibana/policies.go

* master: chore: update mergify's titles (elastic#1142) chore: remove unused pipelines (elastic#1143) feat: use Docker copy to transfer binaries to containers (elastic#1136) chore: abstract image pulling (elastic#1137) Support multiple deployment backends (elastic#1130) chore: remove unused code (elastic#1119) Unify fleet and stand-alone suites (elastic#1112) Pull fresh docker images before suite (elastic#1123) chore: initialise timeout factor next to the declaration (elastic#1118) chore: match Go version with Beats (elastic#1120) feat: support running k8s autodiscover suite for Beats PRs and local repositories (elastic#1115) Auto bootstrap fleet during initialize scenario (elastic#1116) Kubernetes Deployment (elastic#1110)

* chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]>

* chore: initialise timeout factor next to the declaration (#1118) * chore: initialise timeout factor on its own package * chore: reuse timeout factor from common * chore: remove unused code (#1119) * chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]> * Support multiple deployment backends (#1130) * Abstract out deployment Provides ability to plugin different deployment backends for use in testing. Current deployment backends supported are "docker" and "kubernetes" * remove unused import * remove unsetting of fleet server hostname as it's not needed * add deployer support to stand-alone * add elastic-agent to k8s deployment specs * Update internal/docker/docker.go Signed-off-by: Adam Stokes <[email protected]> Co-authored-by: Manuel de la Peña <[email protected]> * chore: abstract process checks to the deployer (#1156) * chore: abstract process checks to the deployer * chore: rename variable in log entry * docs: improve comment * fix: go-fmt * feat: simplify the initialisation of versions (#1159) * chore: use fixed version in shell scripts * chore: move retry to utils We could move it to its own package, but at this moment it's very small * chore: initialise stackVesion at one single place * chore: initialise agent version base at one single place * chore: initialise agent version at one single place * chore: reduce the number of requests to Elastic's artifacts endpoint * chore: rename AgentVersionBase variable to BeatVersionBase * chore: rename AgentVersion variable to BeatVersion * chore: use Beat version in metricbeat test suite * chore: check if the version must use the fallback after coming from a Git SHA * feat: support flavours in services, specially in the elastic-agent (#1162) * chore: move compose to deploy package * feat: use a ServiceRequest when adding services * feat: add service flavour support * chore: remove unused centos/debian services * fixup: add service flavour * chore: move docker client to the deploy package We will need another abstraction to represent the Docker client operations, as it's clear what is a deployment and what is an operation in the deployment. Maybe a Client struct for each provider will help out in differenciate it * chore: use ServiceRequest everywhere * chore: run agent commands with a ServiceRequest * chore: use ServiceRequest in metricbeat test suite * chore: pass flavours to installers * chore: add a step to install the agent for the underlying OS * chore: always add flavour * fix: use installer for fleet_mode when removing services at the end of the scenario * fix: update broken references in metricbeat test suite * fix: update broken references in helm test suite * fix: standalone does not have an installer * fix: use service instead of image to get a service request for the agent * feat: support for scaling services in compose * fix: run second agent using compose scale option * fix: update kibana's default Docker namespace * fix: there are 2 metricbeat instances * fix: wait for 2 filebeat instances * fix: fleet backend processes count * chore: use 2 instances for background processes Co-authored-by: Adam Stokes <[email protected]>

* chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]>

* Move kubernetes/kubectl/kind code to internal project layout (#1092) This is mainly a cleanup to keep all internal related code that could be reusable in our `internal` directory layout. Next steps would be to take what's in `internal/kubectl` and merge with this code. Signed-off-by: Adam Stokes <[email protected]> * feat: bootstrap fleet-server for the deployment of regular elastic-agents (#1078) * chore: provide a fleet-server base image based on centos/debian with systemd * WIP * fix: remove duplicated fields after merge conflicts * fix: update method call after merge conflicts * chore: extract service name calculation to a method * chore: extract container name calculation to a method * chore: refactor get container name method * chore: refactor method even more * chore: use installer state to retrieve container name * chore: use installer when calculating service name * fix: adapt service names for fleet server * chore: enrich log when creating an installer * fix: use fleet server host when creating fleet config * fix: use https when connecting to fleet-server It's creating its own self-signed certs * feat: bootstrap a fleet server before a regular agent is deployed to fleet It will define the server host to be used when enrolling agents * fix: use fleet policy for agents, not the server one * fix: get different installers for fleet-server and agents * fix: use the old step for deploying regular agents * chore: rename variable with consistent name * chore: rename fleet-server scenario * fix: use proper container name for standalone mode * chore: save two variables * chore: rename standalone scenario for bootstrapping fleet-server * chore: rename bootstrap methods * chore: encapsulate bootstrap fleet-server logic * Update fleet.go * chore: remove Fleet Server CI parallel execution * chore: remove feature file for fleet-server * chore: boostrap fleet server only once We want to have it bootstrapped for the entire test suite, not for each scenario * fix: an agent was needed when adding integrations Co-authored-by: Adam Stokes <[email protected]> * apm-server tests (#1083) * some tests for apm-server * clean op dir on init instead of after * fix agent uninstall (#1111) * Kubernetes Deployment (#1110) * Kubernetes Deployment Signed-off-by: Adam Stokes <[email protected]> * Expose hostPort for kibana, elasticsearch, fleet without needing ingress This is nice for local development where you don't need an ingress and are relatively sure that the host system has the required ports available to bind to. Signed-off-by: Adam Stokes <[email protected]> * Auto bootstrap fleet during initialize scenario (#1116) Signed-off-by: Adam Stokes <[email protected]> Co-authored-by: Manuel de la Peña <[email protected]> * feat: support running k8s autodiscover suite for Beats PRs and local repositories (#1115) * chore: add license * chore: initialise configurations before test suite * chore: use timeout_factor from env * fix: tell kind to skip pulling beats images * chore: add a method to load images into kind * feat: support running k8s autodiscover for Beats PRs or local filesystem * chore: add license header * chore: expose logger and use it, simplifying initialisation * fix: only run APM services for local APM environment * Revert "chore: expose logger and use it, simplifying initialisation" This reverts commit a89325c. * chore: log scenario name * fix: always cache beat version for podName * chore: reduce log level Co-authored-by: Adam Stokes <[email protected]> * chore: initialise timeout factor next to the declaration (#1118) * chore: initialise timeout factor on its own package * chore: reuse timeout factor from common * Unify fleet and stand-alone suites (#1112) * fix agent uninstall * unify fleet and stand alone suites * move things around a bit more * fixe bad merge * simplify some things * chore: remove unused code (#1119) * chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]> * Support multiple deployment backends (#1130) * Abstract out deployment Provides ability to plugin different deployment backends for use in testing. Current deployment backends supported are "docker" and "kubernetes" * remove unused import * remove unsetting of fleet server hostname as it's not needed * add deployer support to stand-alone * add elastic-agent to k8s deployment specs * Update internal/docker/docker.go Signed-off-by: Adam Stokes <[email protected]> Co-authored-by: Manuel de la Peña <[email protected]> * fix: bump stale agent version to 7.12-snapshot * chore: abstract process checks to the deployer (#1156) * chore: abstract process checks to the deployer * chore: rename variable in log entry * docs: improve comment * fix: go-fmt * feat: simplify the initialisation of versions (#1159) * chore: use fixed version in shell scripts * chore: move retry to utils We could move it to its own package, but at this moment it's very small * chore: initialise stackVesion at one single place * chore: initialise agent version base at one single place * chore: initialise agent version at one single place * chore: reduce the number of requests to Elastic's artifacts endpoint * chore: rename AgentVersionBase variable to BeatVersionBase * chore: rename AgentVersion variable to BeatVersion * chore: use Beat version in metricbeat test suite * chore: check if the version must use the fallback after coming from a Git SHA * feat: support flavours in services, specially in the elastic-agent (#1162) * chore: move compose to deploy package * feat: use a ServiceRequest when adding services * feat: add service flavour support * chore: remove unused centos/debian services * fixup: add service flavour * chore: move docker client to the deploy package We will need another abstraction to represent the Docker client operations, as it's clear what is a deployment and what is an operation in the deployment. Maybe a Client struct for each provider will help out in differenciate it * chore: use ServiceRequest everywhere * chore: run agent commands with a ServiceRequest * chore: use ServiceRequest in metricbeat test suite * chore: pass flavours to installers * chore: add a step to install the agent for the underlying OS * chore: always add flavour * fix: use installer for fleet_mode when removing services at the end of the scenario * fix: update broken references in metricbeat test suite * fix: update broken references in helm test suite * fix: standalone does not have an installer * fix: use service instead of image to get a service request for the agent * feat: support for scaling services in compose * fix: run second agent using compose scale option * fix: update kibana's default Docker namespace * feat: make a stronger verification of fleet-server being bootstrapped (#1164) * fix: resolve issues in k8s-autodiscover test suite (#1171) * chore: use timeout factor when tagging docker images * fix: resolve alias version in k8s-autodiscover test suite * fix: use common versions for k8s-autodiscover * fix: update background processes to 2 instances Co-authored-by: Adam Stokes <[email protected]> Co-authored-by: Juan Álvarez <[email protected]>

* chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]>

… backport for 7.16 (#2032) * chore: remove unused code (#1119) * chore: remove unused code * chore: remove all references to fleet server hostname Because we assume it's a runtime dependency, provided by the initial compose file, we do not need to calculate service names, or URIs for the fleet-service endpoint. Instead, we assume it's listening in the 8220 port in the "fleet-server" hostname, which is accessible from the network created by docker-compose. * fix: use HTTP to connect to fleet-server * chore: remove fleet server policy code We do not need it anymore, as the fleet server is already bootstrapped * chore: remove all policies but system and fleet_server * Update policies.go * Update fleet.go * Update stand-alone.go Co-authored-by: Adam Stokes <[email protected]> * fix: wrong resolve conflicts * fix: wrong resolve conflicts Co-authored-by: Adam Stokes <[email protected]>

mdelapenya added 2 commits May 4, 2021 11:36

chore: remove unused code

ff8e2d9

mdelapenya self-assigned this May 4, 2021

mdelapenya requested a review from a team May 4, 2021 09:57

cachedout approved these changes May 4, 2021

View reviewed changes

mdelapenya marked this pull request as ready for review May 4, 2021 10:27

fix: use HTTP to connect to fleet-server

3388517

cachedout approved these changes May 4, 2021

View reviewed changes

chore: remove fleet server policy code

f85e088

We do not need it anymore, as the fleet server is already bootstrapped

mdelapenya commented May 4, 2021

View reviewed changes

internal/kibana/fleet.go Show resolved Hide resolved

chore: remove all policies but system and fleet_server

b16b0b5

adam-stokes added 4 commits May 4, 2021 13:24

Merge branch 'master' into remove-fleet-server-leftovers

0e81f86

Update policies.go

af3e7a9

Update fleet.go

b240ee2

Update stand-alone.go

06a2bf9

adam-stokes merged commit 82380ce into elastic:master May 4, 2021

mdelapenya commented May 4, 2021

View reviewed changes

mdelapenya added v7.13.0 v7.14.0 labels May 4, 2021

mergify bot mentioned this pull request May 4, 2021

chore: remove unused code (backport #1119) #1128

Closed

mergify bot mentioned this pull request May 4, 2021

chore: remove unused code (backport #1119) #1129

Closed

mdelapenya deleted the remove-fleet-server-leftovers branch May 6, 2021 06:25

mdelapenya mentioned this pull request May 15, 2021

fix: update backports for 7.x #1172

Merged

8 tasks

mdelapenya mentioned this pull request May 17, 2021

chore: backports for 7.13.x branch #1178

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: remove unused code #1119

chore: remove unused code #1119

mdelapenya commented May 4, 2021

elasticmachine commented May 4, 2021 •

edited

Loading

Build stats

Test stats 🧪

Trends 🧪

Test stats 🧪

mergify bot commented May 4, 2021

mdelapenya May 4, 2021

chore: remove unused code #1119

chore: remove unused code #1119

Conversation

mdelapenya commented May 4, 2021

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

Related issues

elasticmachine commented May 4, 2021 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

mergify bot commented May 4, 2021

mdelapenya May 4, 2021

Choose a reason for hiding this comment

elasticmachine commented May 4, 2021 •

edited

Loading