Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend k8s schema validation #2929

Merged
merged 4 commits into from
Feb 4, 2022

Conversation

atsikham
Copy link
Contributor

As discussed with @seriva, there is no possibility for now to check additional fields and that's done by purpose and such behavior was requested before.

So this PR is only about schema extension with a basic validation.

seriva
seriva previously approved these changes Jan 26, 2022
erzetpe
erzetpe previously approved these changes Jan 26, 2022
@atsikham atsikham dismissed stale reviews from erzetpe and seriva via ed955d8 February 4, 2022 08:26
@przemyslavic
Copy link
Collaborator

/azp run

@przemyslavic
Copy link
Collaborator

przemyslavic commented Feb 4, 2022

@atsikham validation for min and max values for certificate expiration_days doesn't seem to work, looks like only type is verified.

          expiration_days:
            title: expiration_days schema
            "$id": "#/properties/specification/properties/advanced/properties/certificates/properties/expiration_days"
            description: |-
              Days to make certificate be valid for.
              For more explanation about the limit navigate to
              https://groups.google.com/g/mailing.openssl.users/c/3kK_f0ywCZQ.
            type: integer
            minimum: 1
            maximum: 24855
            default: 365

@przemyslavic
Copy link
Collaborator

przemyslavic commented Feb 4, 2022

@seriva do you have any idea why it works a bit weird?
When I change the data type for expiration_date to string or whatever, it will fail right away:

epiuser@685a39ead66c:/shared$ epicli apply -f /shared/k8svalidate.yml --vault-password x
12:23:35 INFO cli.engine.schema.DefaultMerger - Starting step
12:23:35 INFO cli.engine.schema.DefaultMerger - Merging: epiphany-cluster name: default
12:23:35 INFO cli.engine.schema.DefaultMerger - Merging: configuration/kubernetes-master name: default
12:23:35 INFO cli.engine.schema.DefaultMerger - Step finished in: 0.04s
12:23:35 ERROR epicli - Types of key "expiration_days" are different: <class 'int'>, <class 'str'>. Unable to merge.

and when setting a number outside the range 1-24855 it fails after terraform 😨

epiuser@685a39ead66c:/shared$ epicli apply -f /shared/k8svalidate.yml --vault-password x
12:25:21 INFO cli.engine.schema.DefaultMerger - Starting step
12:25:21 INFO cli.engine.schema.DefaultMerger - Merging: epiphany-cluster name: default
12:25:21 INFO cli.engine.schema.DefaultMerger - Merging: configuration/kubernetes-master name: default
12:25:21 INFO cli.engine.schema.DefaultMerger - Step finished in: 0.04s
12:25:21 INFO cli.engine.schema.SchemaValidator - Starting step
12:25:21 INFO cli.engine.schema.SchemaValidator - Validating: epiphany-cluster - k8svalidate
12:25:21 INFO cli.engine.schema.SchemaValidator - Step finished in: 0.04s
12:25:21 INFO cli.engine.providers.aws.InfrastructureBuilder - Starting step
12:25:23 INFO cli.engine.providers.aws.InfrastructureBuilder - Step finished in: 1.79s
12:25:23 INFO cli.engine.schema.SchemaValidator - Starting step
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/public-key - ubuntu
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/vpc - qa-k8svalidate-vpc
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/default-security-group - default-security-group
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/resource-group - qa-k8svalidate-rg
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/internet-gateway - qa-k8svalidate-internet-gateway
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/route-table - qa-k8svalidate-route-table
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/subnet - qa-k8svalidate-kubernetes-master-subnet-0
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/security-group - qa-k8svalidate-kubernetes-master-security-group-0
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/route-table-association - qa-k8svalidate-kubernetes-master-0-route-association
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/subnet - qa-k8svalidate-kubernetes-master-subnet-1
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/security-group - qa-k8svalidate-kubernetes-master-security-group-1
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/route-table-association - qa-k8svalidate-kubernetes-master-1-route-association
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/virtual-machine - qa-k8svalidate-kubernetes-master-asg-0
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/launch-configuration - qa-k8svalidate-kubernetes-master-launch-config
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/subnet - qa-k8svalidate-repository-subnet-0
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/security-group - qa-k8svalidate-repository-security-group-0
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/route-table-association - qa-k8svalidate-repository-0-route-association
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/virtual-machine - qa-k8svalidate-repository-asg-0
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/launch-configuration - qa-k8svalidate-repository-launch-config
12:25:23 INFO cli.engine.schema.SchemaValidator - Validating: infrastructure/efs-storage - qa-k8svalidate-efs
12:25:23 INFO cli.engine.schema.SchemaValidator - Step finished in: 0.26s
12:25:23 INFO cli.engine.terraform.TerraformTemplateGenerator - Starting step
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: epiphany-cluster ---> 000_k8svalidate.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/public-key ---> 001_ubuntu.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/vpc ---> 002_qa-k8svalidate-vpc.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/default-security-group ---> 003_default-security-group.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/resource-group ---> 004_qa-k8svalidate-rg.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/internet-gateway ---> 005_qa-k8svalidate-internet-gateway.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/route-table ---> 006_qa-k8svalidate-route-table.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/subnet ---> 007_qa-k8svalidate-kubernetes-master-subnet-0.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/security-group ---> 008_qa-k8svalidate-kubernetes-master-security-group-0.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/route-table-association ---> 009_qa-k8svalidate-kubernetes-master-0-route-association.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/subnet ---> 010_qa-k8svalidate-kubernetes-master-subnet-1.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/security-group ---> 011_qa-k8svalidate-kubernetes-master-security-group-1.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/route-table-association ---> 012_qa-k8svalidate-kubernetes-master-1-route-association.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/virtual-machine ---> 013_qa-k8svalidate-kubernetes-master-asg-0.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/launch-configuration ---> 014_qa-k8svalidate-kubernetes-master-launch-config.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/subnet ---> 015_qa-k8svalidate-repository-subnet-0.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/security-group ---> 016_qa-k8svalidate-repository-security-group-0.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/route-table-association ---> 017_qa-k8svalidate-repository-0-route-association.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/virtual-machine ---> 018_qa-k8svalidate-repository-asg-0.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/launch-configuration ---> 019_qa-k8svalidate-repository-launch-config.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Generating: infrastructure/efs-storage ---> 020_qa-k8svalidate-efs.tf
12:25:24 INFO cli.engine.terraform.TerraformTemplateGenerator - Step finished in: 0.26s
12:25:24 INFO cli.engine.terraform.TerraformFileCopier - Starting step
12:25:24 INFO cli.engine.terraform.TerraformFileCopier - Step finished in: 0.00s
12:25:24 INFO cli.engine.terraform.TerraformCommand - Running: "terraform -chdir=/shared/build/k8svalidate/terraform/ init -no-color"
12:25:24 INFO cli.engine.terraform.TerraformCommand -
12:25:24 INFO cli.engine.terraform.TerraformCommand - Initializing the backend...
12:25:26 INFO cli.engine.terraform.TerraformCommand -
12:25:26 INFO cli.engine.terraform.TerraformCommand - Initializing provider plugins...
12:25:26 INFO cli.engine.terraform.TerraformCommand - - Reusing previous version of hashicorp/aws from the dependency lock file
12:25:28 INFO cli.engine.terraform.TerraformCommand - - Using previously-installed hashicorp/aws v3.71.0
12:25:28 INFO cli.engine.terraform.TerraformCommand -
12:25:28 INFO cli.engine.terraform.TerraformCommand - Done running "terraform -chdir=/shared/build/k8svalidate/terraform/ init -no-color"
12:25:28 INFO cli.engine.terraform.TerraformCommand - Terraform has been successfully initialized!
12:25:28 INFO cli.engine.terraform.TerraformRunner - Starting step
12:25:28 INFO cli.engine.terraform.TerraformCommand -
12:25:28 INFO cli.engine.terraform.TerraformCommand - Running: "terraform -chdir=/shared/build/k8svalidate/terraform/ apply -state=/shared/build/k8svalidate/terraform//terraform.tfstate -auto-approve -no-color"
12:25:28 INFO cli.engine.terraform.TerraformCommand - You may now begin working with Terraform. Try running "terraform plan" to see
12:25:28 INFO cli.engine.terraform.TerraformCommand - any changes that are required for your infrastructure. All Terraform commands
12:25:28 INFO cli.engine.terraform.TerraformCommand - should now work.

terraform part

12:25:45 INFO cli.engine.terraform.TerraformCommand -
12:25:45 INFO cli.engine.terraform.TerraformCommand - Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
12:25:45 INFO cli.engine.terraform.TerraformCommand - Done running "terraform -chdir=/shared/build/k8svalidate/terraform/ apply -state=/shared/build/k8svalidate/terraform//terraform.tfstate -auto-approve -no-color"
12:25:45 INFO cli.engine.terraform.TerraformRunner - Step finished in: 17.56s
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Starting step
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/feature-mapping
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/shared-config
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/helm
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/applications
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/node-exporter
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/filebeat
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/firewall
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/repository
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Adding: configuration/image-registry
12:25:45 INFO cli.engine.schema.ConfigurationAppender - Step finished in: 0.19s
12:25:45 INFO cli.engine.schema.SchemaValidator - Starting step
12:25:45 INFO cli.engine.schema.SchemaValidator - Validating: configuration/feature-mapping
12:25:45 INFO cli.engine.schema.SchemaValidator - Validating: configuration/shared-config
12:25:45 INFO cli.engine.schema.SchemaValidator - Validating: epiphany-cluster - k8svalidate
12:25:45 INFO cli.engine.schema.SchemaValidator - Validating: configuration/kubernetes-master
12:25:45 ERROR cli.engine.schema.SchemaValidator - Failed validating: configuration/kubernetes-master
12:25:45 ERROR cli.engine.schema.SchemaValidator - 24856 is greater than the maximum of 24855

Failed validating 'maximum' in schema['properties']['specification']['properties']['advanced']['properties']['certificates']['properties']['expiration_days']:
    {'$id': '#/properties/specification/properties/advanced/properties/certificates/properties/expiration_days',
     'default': 365,
     'description': 'Days to make certificate be valid for.\n'
                    'For more explanation about the limit navigate to\n'
                    'https://groups.google.com/g/mailing.openssl.users/c/3kK_f0ywCZQ.',
     'maximum': 24855,
     'minimum': 1,
     'title': 'expiration_days schema',
     'type': 'integer'}

On instance['specification']['advanced']['certificates']['expiration_days']:
    24856
12:25:45 INFO cli.engine.schema.SchemaValidator - Step finished in: 0.11s

OK I get it, when having different data types it fails on merging, and all the validation is performed later.

@przemyslavic
Copy link
Collaborator

przemyslavic commented Feb 4, 2022

However, I still believe that all validation should be executed before the terraform part.

@atsikham atsikham merged commit 265d0d9 into hitachienergy:develop Feb 4, 2022
cicharka added a commit that referenced this pull request Feb 8, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
rafzei pushed a commit to rafzei/epiphany that referenced this pull request Feb 8, 2022
erzetpe added a commit to erzetpe/epiphany that referenced this pull request Feb 9, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit that referenced this pull request Feb 9, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit to erzetpe/epiphany that referenced this pull request Feb 10, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit to erzetpe/epiphany that referenced this pull request Feb 24, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit to erzetpe/epiphany that referenced this pull request Feb 24, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit to erzetpe/epiphany that referenced this pull request Feb 24, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit to erzetpe/epiphany that referenced this pull request Feb 24, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit that referenced this pull request Feb 24, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit that referenced this pull request Mar 1, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit that referenced this pull request Mar 3, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit that referenced this pull request Mar 8, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit that referenced this pull request Mar 22, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit that referenced this pull request Apr 4, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
cicharka pushed a commit to cicharka/epiphany that referenced this pull request Apr 11, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
cicharka pushed a commit to cicharka/epiphany that referenced this pull request Apr 11, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
cicharka pushed a commit to cicharka/epiphany that referenced this pull request Apr 11, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
cicharka pushed a commit to cicharka/epiphany that referenced this pull request Apr 11, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
cicharka pushed a commit to cicharka/epiphany that referenced this pull request Apr 11, 2022
* K8s improvements (hitachienergy#2918)

* Removal of Logstash component - next steps hitachienergy#2833 (hitachienergy#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (hitachienergy#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (hitachienergy#2929)

* Extend deprecation note with Ignite removal steps (hitachienergy#2949)

* Escalate permissions for ip command (hitachienergy#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (hitachienergy#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: hitachienergy#1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag hitachienergy#959
- Updated DoD for bugs to reflect changes made for hitachienergy#2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
cicharka pushed a commit that referenced this pull request Apr 11, 2022
* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
erzetpe added a commit that referenced this pull request Apr 20, 2022
* Define additional disks with defined sizes to VMs for Azure (#2953)

* Additional disks for azure DRAFT

* Simplify config

* add changelog note

* Change starting index value in names

* Change value of lun attribute to alling with disks indexing

* Merge 'develop' into 'feature/rook-ceph-storage' (#2959)

* K8s improvements (#2918)

* Removal of Logstash component - next steps #2833 (#2941)

* Add information about manual package removal that
  needs to be performed by cluster admin
* include information about inventory file

* Fix for ansible_default_ipv4 empty value (#2922)

* Fix for ansible_default_ipv4 empty value

* Fix after ToBar review

* Add check if there is only one default routing config

* Add check for different metrics value in case more than one default routing

* Review fixes

* Update fail_msg

Co-authored-by: to-bar <[email protected]>

* Extend k8s schema validation (#2929)

* Extend deprecation note with Ignite removal steps (#2949)

* Escalate permissions for ip command (#2952)

* Changed autoscaling_group to plain EC2 VMs on AWS. (#2939)

- Replaced AWS auto_scaling_groups with plain EC2 VM creation.
- Added proper host sorting how it was implemented for the any and azure providers: #1076
- Sync up features with Azure Terraform implementation
-  Added support for use_network_security_groups flag #959
- Updated DoD for bugs to reflect changes made for #2832

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>

* Add Ansible Rook role (#2960)

* Add initial ansible rook role

* Fix issue with no yet existing template

* Add Rook schema related files

* Remove already setup vars, add missing enters

* Fix issue with feature mapping

* Add configuration support

* Move kubeconfig to rook role yaml

* Update changelog

* Add started dashes to yaml defaults for Rook

* Add RH fix suggested by cicharka

* Add rook cluster helm chart installation

* Add separate chart values definitions

* Limit fact checking in rook role

* Add initial rook/ceph documentation

* Update documentation for Rook

* Update download requirements

* Persistent Storage: Note about disks on AWS (#2998)

* Updated changelog

* Rearrange and update documentation

* Update components

* Fix after review

* Limit hosts where apply is specified

* Add comment about repository facts gathering

* Make rook namespace configurable

* Add linter comments to disable false positive

* Add requested comments

* Change shell to command module

* Change mode to preserve for temp configuration data

* Update documentation

* Update ansible/playbooks/roles/repository/files/download-requirements/requirements/x86_64/images.yml

Co-authored-by: przemyslavic <[email protected]>

* Update ansible/playbooks/roles/repository/files/download-requirements/requirements/x86_64/images.yml

Co-authored-by: przemyslavic <[email protected]>

* Update ansible/playbooks/roles/repository/files/download-requirements/requirements/x86_64/images.yml

Co-authored-by: przemyslavic <[email protected]>

* Update ansible/playbooks/roles/repository/files/download-requirements/requirements/x86_64/images.yml

Co-authored-by: przemyslavic <[email protected]>

* Update ansible/playbooks/roles/repository/files/download-requirements/requirements/x86_64/images.yml

Co-authored-by: przemyslavic <[email protected]>

* Update ansible/playbooks/roles/repository/files/download-requirements/requirements/x86_64/images.yml

Co-authored-by: przemyslavic <[email protected]>

* Update ansible/playbooks/roles/repository/files/download-requirements/requirements/x86_64/images.yml

Co-authored-by: przemyslavic <[email protected]>

* Update ansible/playbooks/roles/repository/files/download-requirements/requirements/x86_64/images.yml

Co-authored-by: przemyslavic <[email protected]>

* use newer version of rook (#3053)

* Feature/rook ceph storage additions (#3064)

* rook/ceph: add default values in epiphany configuration

* rook/ceph (#3068)

* add default values in epiphany configuration
* bump up verison of rook

* Revert schema defaults (#3074)

* Revert "rook/ceph (#3068)"

This reverts commit 989be85.

* Revert "Feature/rook ceph storage additions (#3064)"

This reverts commit 17c49af.

* minor fixes for schema and docs

* add rook feature_mapping for single and custom

* rook: modify tests and schema for image registry

* rook: offline mode

* enahnce playbook to allow chart files use images from
  local image registry
* remove unused condition for chart and operator specification
  which is always defined now
* modify path for rook/ceph image in order to be compliant
  with values.yml

Signed-off-by: cicharka <[email protected]>

* rook: add missing csiaddons for offline mode

Signed-off-by: cicharka <[email protected]>

Co-authored-by: Anatoli Tsikhamirau <[email protected]>
Co-authored-by: Irek Głownia <[email protected]>
Co-authored-by: to-bar <[email protected]>
Co-authored-by: Luuk van Venrooij <[email protected]>
Co-authored-by: cicharka <[email protected]>
Co-authored-by: przemyslavic <[email protected]>
Co-authored-by: cicharka <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants