-
Notifications
You must be signed in to change notification settings - Fork 15
MGMT-1233 - support v4.6, use official openshift-installer extracted … #19
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,61 +1,60 @@ | ||
### This is a image for generating ignition manifests & kubeconfig | ||
1) Dockerfile.assisted-ignition-generator - dockerfile for building the assisted-ignition-generator image | ||
2) installer_dir - testing directory that includes all the files needed for local testing (on laptop) | ||
- install-config.yaml.baremetal - install-config for baremetal environment | ||
- test_env.txt - environment variables for testing | ||
- test_hosts_list.yaml - simulated output of Get Host from assisted-service command.Used as an input for testing | ||
|
||
1) Dockerfile (deprecated) - dockerfile for building the assisted-ignition-generator image with openshift-installer platform none | ||
2) Dockerfile.baremetal - dockerfile for building the assisted-ignition-generator image with openshift-installer platform baremetal | ||
3) installer_dir/install-config.yaml - example of install-config.yaml for none platform | ||
4) installer_dir/install-config.yaml.baremetal - example of install-config.yaml for baremetal platform. | ||
5) openshift-install - executable that produces ignition files. It is copied from installer repository before issuing docker build command. Should be build in accordance with the platform we are running on | ||
a) none platform - should be build with command <TAGS="none" hack/build.sh> | ||
b) baremetal platform - should be build with command <TAGS="baremetal" hack/build.sh>. Prior to that the following files in the installer source code should be changed: | ||
- hack/build.sh - CGO_ENABLED flag should be enable in case of baremetal platform also: <if (echo "${TAGS}" | grep -q 'libvirt\|baremetal')> | ||
- pkg/types/baremetal/validation/libvirt.go - build tag should be changed from baremetal to libvirt ( to avoid validations via libvirt) | ||
|
||
Environment variables: | ||
----------------------------- | ||
|
||
Since this image will be run as a job initiated by assisted-service, all input parameters are passed as environment variables: | ||
|
||
1) WORK_DIR - directory inside the container where we run all our code. Default: /data | ||
2) INSTALLER_CONFIG - directory where the install-config.yaml is created/set and where the output ( ignitions) are created. Default: /data/installer_dir | ||
3) CLUSTER_ID - input parameter for the job | ||
4) INVENTORY_ENDPOINT - url that defines how python client connects to assisted-service. | ||
5) S3_ENDPOINT_URL - S3 endpoint. results of the job will be uploaded to that S3 | ||
6) S3_BUCKET - the S3 bucket to upload to | ||
7) aws_access_key_id - AWS access key id | ||
8) aws_secret_access_key - AWS secret access key | ||
9) OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE - the OCP release image that we are working on | ||
|
||
Building: | ||
---------------- | ||
1) currently we use our own openshift-install executable. We save it into the container using Dockerfile.installer-image and pushing it to a repository ( make sure to make it public. Currently | ||
we use quay.io/yshnaidm/openshift-installer. In case we recompile openshift-install, we need to update this image: | ||
docker build -f Dockerfile.installer-image -t <repository> . | ||
Example: docker build -f Dockerfile.installer-image -t quay.io/yshnaidm/openshift-installer:latest . | ||
2) we are using the image built in step 1 to get the needed openshift-install. Once we start using openshift-install from the release, we can update Dockerfile.baremetal accordingly | ||
docker build -f Dockerfile.baremetal . -t quay.io/ocpmetal/assisted-ignition-generator:stable | ||
--------------------------------------- | ||
1) Build quay.io/ocpmetal/assisted-ignition-generator - image for running ignition generation job. Uses assisted-service container to install assisted-service python client. Install OC client by wget appropriate package | ||
Note: beforebuilding make sure that you deleted all the files under installer_dir that were created during testing: | ||
|
||
cd installer_dir && sudo rm -rf auth bootstrap.ign install-config.yaml master.ign metadata.json worker.ign .openshift_install.log .openshift_install_state.json | ||
|
||
make build | ||
docker tag <image id> quay.io/ocpmetal/assisted-ignition-generator | ||
|
||
Testing: | ||
--------------- | ||
Testing can be done in 2 stages: | ||
|
||
1) test generation of the ignition files , locally on your laptop. | ||
a) copy install-config.yaml.baremetal to installer-config.yaml in installer_dir. | ||
b) run assisted-ignition-generator image that you previously created. | ||
|
||
docker run -v $(pwd)/installer_dir:/data/installer_dir -it assisted-ignition-generator:ad6939c67c115cef7877ab7d06d72f2d06cebe0 | ||
Testing: | ||
------------------------------------- | ||
|
||
if no error is printed, then the ignition files are generated in the installer_dir | ||
Testing can be done in 2 stages: | ||
|
||
2) test specific manipulations on generated ignition. Currently only BMH annotations generations is checked. This stage must be run only after first stage | ||
a) change permissions of the file generated in the first stage. from installer_dir run: | ||
sudo chmod -R 777 auth bootstrap.ign master.ign metadata.json worker.ign .openshift_install.log .openshift_install_state.json | ||
b) run: | ||
skipper run python3 test_bmh_annotations.py | ||
1) Test generation of the ignition files , locally on your laptop. | ||
|
||
a) Copy install-config.yaml.baremetal to installer-config.yaml in installer_dir. | ||
|
||
b) Run assisted-ignition-generator immage that you previously created. | ||
Since 4.6 we use release image to extract installer.The value of release-image is set as environment value by bm-envtory. when testing we use env file installer-dir/test_env.txt | ||
If no error is printed, then test was successsful and the ignition files are generated in the installer_dir | ||
|
||
docker run -v $(pwd)/installer_dir:/data/installer_dir --env-file $(pwd)/installer_dir/test_env.txt -it assisted-ignition-generator:<hash> | ||
|
||
Usage: | ||
``` | ||
usage: process-ignition-manifests-and-kubeconfig.py [-h] | ||
[--file_name FILE_NAME] | ||
[--s3_endpoint_url S3_ENDPOINT_URL] | ||
[--s3_bucket S3_BUCKET] | ||
2) Test specific manipulations on generated ignition. Currently only BMH annotations generations is checked. This stage must be run only after first stage | ||
|
||
Generate ignition manifest & kubeconfig | ||
a) Change permissions of the file generated in the first stage. | ||
From installer_dir run: | ||
|
||
sudo chmod -R 777 auth bootstrap.ign master.ign metadata.json worker.ign .openshift_install.log .openshift_install_state.json | ||
|
||
optional arguments: | ||
-h, --help show this help message and exit | ||
--file_name FILE_NAME | ||
output directory name | ||
--s3_endpoint_url S3_ENDPOINT_URL | ||
s3 endpoint url | ||
--s3_bucket S3_BUCKET | ||
s3 bucket | ||
``` | ||
b) Test code that changes the ignitions: | ||
|
||
skipper run python3 test_bmh_annotations.py |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -69,5 +69,4 @@ platform: | |
bootMode: UEFI | ||
hardwareProfile: unknown | ||
|
||
pullSecret: '{"auths":{"cloud.openshift.com":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGV2K3lzaG5haWRtMXkxdTZhempjaGJ0cm5hZDFkb3U1Z3FtcnliOlVBNjU0SVZFQzZPMFFBM0ZJTDA2WjY1WDRSSjM0WElNM0JYRVkzVFBCVU9IUkZESEhRSkkxMFFJNUI5MEY1Tks=","email":"[email protected]"},"quay.io":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGV2K3lzaG5haWRtMXkxdTZhempjaGJ0cm5hZDFkb3U1Z3FtcnliOlVBNjU0SVZFQzZPMFFBM0ZJTDA2WjY1WDRSSjM0WElNM0JYRVkzVFBCVU9IUkZESEhRSkkxMFFJNUI5MEY1Tks=","email":"[email protected]"},"registry.connect.redhat.com":{"auth":"NTMxOTg0MDl8dWhjLTFZMXU2QXpqY0hidHJOYUQxZE91NUdxbVJZQjpleUpoYkdjaU9pSlNVelV4TWlKOS5leUp6ZFdJaU9pSmpPREUzWVRZMk1qQXhOR00wTlRFMFlqWXlPR1ppTkROaE9UazNNek0xTUNKOS5temRzdWFYTHY1VWMzQ0Y5OVRsT2lCNlJ1bnhOMmNlaTJGTmZsZ2RKTG5WWEdqQVVSMDlqd3FiNXlJXzV2blhYUjlydVAxem5MaEh3NU1acG1iMDA3cl9jVllNWk53WHVtZGZDLWQxdlVhY1NmMjR1Yl9yM0RRT04tQ3Q2U2FQWnc3a3ZtLTU3anlvNXU4YUhYSlNWZ0hrRlJielFFMF9oNHF3bVJBZ0R6RmRNQ2V5dU5ST2MxaGZmWHM2QzdYSjg4QVlFSlZPUi1xeVZIVXo5VTdCRl9Pcm1nX1hYajQyLVAzZUxZSkhqUk9FSEdfTDhxTEJsS2R3T0hvQ2owR2dIaXgxelotd0ExU0hiY25BNE5aWGNUa1dMUTFsM0tTVUM2ODVYbWFxbDBCT3ZEVXpaRnZramhPOVBVTXRZUUF4bXZZMG1oTGg1RzhhdXJkRi1QdjRRcXhtMkdHbEpEYmlITUZjbGxzamFtbnR5b0loaFZwcURMYzJxLTZ4UXVub0FiZno0Zk5uNWdEekRNemVZWVNEMjZUakVYWi1DN0lJZncwdE92b2tSZGk2QTB6LTE2cEc2MHh5ZFRkX1JISEZwM0sxeXJLdlBCZXZLRVRNalJPamxPbE1zYVJKM3lITkczWV9OYkRWSEVHWnRFYmtIS3Rfel9BeV9zSE1YYUx5b0FwUVMwNmNDUUc0VHU4S3Q2dHpOLXg0QnJ6bTJqcGhYU1Y5ZFpoTEhjTUpSNGRHTkVBQnIteG9BNFdadFlPQ3RvWEpRZEQzYUc4Y2l3R1ptN0VNb1lLR0ZWSUcycmhTQnZfeUxsQXRzbTVpRTdLaGhoQUtqak1TdE9BZlRnSEswWFVDczhGWE9PWGNrdEl4ZWhsVHgwVnIteUZoY0FGcmdhU1NBMWhycFdlTQ==","email":"[email protected]"},"registry.redhat.io":{"auth":"NTMxOTg0MDl8dWhjLTFZMXU2QXpqY0hidHJOYUQxZE91NUdxbVJZQjpleUpoYkdjaU9pSlNVelV4TWlKOS5leUp6ZFdJaU9pSmpPREUzWVRZMk1qQXhOR00wTlRFMFlqWXlPR1ppTkROaE9UazNNek0xTUNKOS5temRzdWFYTHY1VWMzQ0Y5OVRsT2lCNlJ1bnhOMmNlaTJGTmZsZ2RKTG5WWEdqQVVSMDlqd3FiNXlJXzV2blhYUjlydVAxem5MaEh3NU1acG1iMDA3cl9jVllNWk53WHVtZGZDLWQxdlVhY1NmMjR1Yl9yM0RRT04tQ3Q2U2FQWnc3a3ZtLTU3anlvNXU4YUhYSlNWZ0hrRlJielFFMF9oNHF3bVJBZ0R6RmRNQ2V5dU5ST2MxaGZmWHM2QzdYSjg4QVlFSlZPUi1xeVZIVXo5VTdCRl9Pcm1nX1hYajQyLVAzZUxZSkhqUk9FSEdfTDhxTEJsS2R3T0hvQ2owR2dIaXgxelotd0ExU0hiY25BNE5aWGNUa1dMUTFsM0tTVUM2ODVYbWFxbDBCT3ZEVXpaRnZramhPOVBVTXRZUUF4bXZZMG1oTGg1RzhhdXJkRi1QdjRRcXhtMkdHbEpEYmlITUZjbGxzamFtbnR5b0loaFZwcURMYzJxLTZ4UXVub0FiZno0Zk5uNWdEekRNemVZWVNEMjZUakVYWi1DN0lJZncwdE92b2tSZGk2QTB6LTE2cEc2MHh5ZFRkX1JISEZwM0sxeXJLdlBCZXZLRVRNalJPamxPbE1zYVJKM3lITkczWV9OYkRWSEVHWnRFYmtIS3Rfel9BeV9zSE1YYUx5b0FwUVMwNmNDUUc0VHU4S3Q2dHpOLXg0QnJ6bTJqcGhYU1Y5ZFpoTEhjTUpSNGRHTkVBQnIteG9BNFdadFlPQ3RvWEpRZEQzYUc4Y2l3R1ptN0VNb1lLR0ZWSUcycmhTQnZfeUxsQXRzbTVpRTdLaGhoQUtqak1TdE9BZlRnSEswWFVDczhGWE9PWGNrdEl4ZWhsVHgwVnIteUZoY0FGcmdhU1NBMWhycFdlTQ==","email":"[email protected]"}}}' | ||
sshKey: 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD14Gv4V5DVvyr7O6/44laYx52VYLe8yrEA3fOieWDmojRs3scqLnfeLHJWsfYA4QMjTuraLKhT8dhETSYiSR88RMM56+isLbcLshE6GkNkz3MBZE2hcdakqMDm6vucP3dJD6snuh5Hfpq7OWDaTcC0zCAzNECJv8F7LcWVa8TLpyRgpek4U022T5otE1ZVbNFqN9OrGHgyzVQLtC4xN1yT83ezo3r+OEdlSVDRQfsq73Zg26d4dyagb6lmrryUUAAbfmn/HalJTHB73LyjilKiPvJ+x2bG7AeiqyVHwtQSpt02FCdQGptmsSqqWF/b9botOO38eUsqPNppMn7LT5wzDZdDlfwTCBWkpqijPcdo/LTD9dJlNHjwXZtHETtiid6N3ZZWpA0/VKjqUeQdSnHqLEzTidswsnOjCIoIhmJFqczeP5kOty/MWdq1II/FX/EpYCJxoSWkT/hVwD6VOamGwJbLVw9LkEb0VVWFRJB5suT/T8DtPdPl+A0qUGiN4KM= [email protected]' | ||
pullSecret: '{"auths": ...}' | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why we have ... ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this file is in github (part of testing), so we cannot keep real pull secret there. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,3 @@ | ||
OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=quay.io/openshift-release-dev/ocp-release@sha256:029d56f7cf4a0de420f84f9178e030a3cfb83a480f3d086a7f5ed31b62bc08d0 | ||
OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE=registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-08-31-220837 | ||
CLUSTER_ID=36f9da71-dbed-4cba-a974-7e103285fe55 | ||
DHCP_ALLOCATION_FILE=whatever |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,6 +19,7 @@ | |
INSTALL_CONFIG = "install-config.yaml" | ||
INSTALL_CONFIG_BACKUP = "backup-install-config.yaml" | ||
SERVICE_CONFIG = "services-config.yaml" | ||
REGISTRY_CONFIG = "registry-config.json" | ||
|
||
|
||
def get_s3_client(s3_endpoint_url, aws_access_key_id, aws_secret_access_key): | ||
|
@@ -71,11 +72,9 @@ def update_bmh_files(ignition_file, cluster_id, inventory_endpoint, token, | |
with open(ignition_file, "r") as file_obj: | ||
data = json.load(file_obj) | ||
storage_files = data['storage']['files'] | ||
# since we don't remove file for now, we don't need to iterate through copy | ||
# for file_data in storage_files[:]: | ||
for file_data in storage_files: | ||
# if file_data['path'] == '/etc/motd': | ||
# storage_files.remove(file_data) | ||
for file_data in storage_files[:]: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Now that you remove a file you should iterate a copy There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also why do we need to remove it? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
if file_data['path'] == '/etc/motd' or 'baremetal-provisioning-config' in file_data['path']: | ||
storage_files.remove(file_data) | ||
if bmh_utils.is_bmh_cr_file(file_data['path']): | ||
bmh_utils.update_bmh_cr_file(file_data, hosts_list) | ||
|
||
|
@@ -126,11 +125,8 @@ def backup_restore_install_config(config_dir): | |
|
||
def generate_installation_files(work_dir, config_dir): | ||
with backup_restore_install_config(config_dir=config_dir): | ||
# [TODO] - uncomment this line when moving to 4.6, and comment the next one | ||
# command = "OPENSHIFT_INSTALL_INVOKER=\"assisted-installer\" %s/openshift-baremetal-install create ignition-configs --dir %s" \ | ||
# % (work_dir, config_dir) | ||
command = "OPENSHIFT_INSTALL_INVOKER=\"assisted-installer\" %s/openshift-install create " \ | ||
"ignition-configs --dir %s" % (work_dir, config_dir) | ||
command = "OPENSHIFT_INSTALL_INVOKER=\"assisted-installer\" %s/openshift-baremetal-install create ignition-configs --dir %s" \ | ||
% (work_dir, config_dir) | ||
try: | ||
logging.info("Generating installation files") | ||
subprocess.check_output(command, shell=True, stderr=sys.stdout) | ||
|
@@ -154,18 +150,22 @@ def pull_secret(config_dir): | |
return yaml.safe_load(yaml_file)['pullSecret'] | ||
|
||
|
||
def set_pull_secret(config_dir): | ||
with open('/root/.docker/config.json', 'w+') as config_file: | ||
config_file.write(pull_secret(config_dir)) | ||
def set_pull_secret(work_dir, config_dir): | ||
registry_file_path = os.path.join(work_dir, REGISTRY_CONFIG) | ||
with open(registry_file_path, 'w+') as registry_file: | ||
registry_file.write(pull_secret(config_dir)) | ||
return registry_file_path | ||
|
||
|
||
# def prepare_generation_data(work_dir, config_dir, install_config, openshift_release_image): | ||
def prepare_generation_data(config_dir, install_config): | ||
def prepare_generation_data(work_dir, config_dir, install_config, openshift_release_image): | ||
# set instal-config.yaml | ||
prepare_install_config(config_dir, install_config) | ||
# [TODO] - part of 4.6 , must be solved as part of MGMT-1816 | ||
# set_pull_secret(config_dir) | ||
# [TODO] - remove comment after fixing subsystem | ||
# oc_utils.extract_baremetal_installer(work_dir, openshift_release_image) | ||
# set pull secret in a file | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not sure why comments are needed, u just comment function name :) |
||
registry_config_file = set_pull_secret(work_dir, config_dir) | ||
# extract openshift-installer | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i think better to comment on the function itself if it is complicated There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. in any case, once we move to 4.6, we will move the code to Golang in assisted-service... The same applies for comment above |
||
oc_utils.extract_baremetal_installer(work_dir, openshift_release_image, registry_config_file) | ||
# prepare data for futher use by assistedd-istaller | ||
create_services_config(work_dir, config_dir, openshift_release_image, registry_config_file) | ||
|
||
|
||
def create_config_dir(work_dir): | ||
|
@@ -179,8 +179,8 @@ def openshift_token(config_dir): | |
return secret["auths"]["cloud.openshift.com"]["auth"] | ||
|
||
|
||
def create_services_config(work_dir, config_dir, openshift_release_image): | ||
mco_image = oc_utils.get_mco_image(work_dir, openshift_release_image) | ||
def create_services_config(work_dir, config_dir, openshift_release_image, registry_config_file): | ||
mco_image = oc_utils.get_mco_image(work_dir, openshift_release_image, registry_config_file) | ||
config_data = {'mco_image': mco_image} | ||
with open(os.path.join(config_dir, SERVICE_CONFIG), "w+") as yaml_file: | ||
yaml.dump(config_data, yaml_file) | ||
|
@@ -201,7 +201,7 @@ def main(): | |
bucket = os.environ.get('S3_BUCKET', args.s3_bucket) | ||
aws_access_key_id = os.environ.get("AWS_ACCESS_KEY_ID", "accessKey1") | ||
aws_secret_access_key = os.environ.get("AWS_SECRET_ACCESS_KEY", "verySecretKey1") | ||
# openshift_release_image = os.environ.get("OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE") | ||
openshift_release_image = os.environ.get("OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. and if not set? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the job will fail on extracting openshift-installer, which is good |
||
skip_cert_verification = os.environ.get('SKIP_CERT_VERIFICATION', False) | ||
ca_cert_path = os.environ.get('CA_CERT_PATH') | ||
|
||
|
@@ -212,16 +212,11 @@ def main(): | |
config_dir = create_config_dir(work_dir=work_dir) | ||
|
||
# prepare all the data(files) needed by opeshift-installer | ||
# prepare_generation_data(work_dir, config_dir, install_config, openshift_release_image) | ||
prepare_generation_data(config_dir, install_config) | ||
prepare_generation_data(work_dir, config_dir, install_config, openshift_release_image) | ||
|
||
# run openshift installer to produce ignitions and kubeconfig | ||
generate_installation_files(work_dir=work_dir, config_dir=config_dir) | ||
|
||
# create service config otput | ||
# [TODO] - remove after fixing subsystem | ||
# create_services_config(work_dir, config_dir, openshift_release_image) | ||
|
||
# update BMH configuration in boostrap ignition | ||
update_bmh_files("%s/bootstrap.ign" % config_dir, cluster_id, inventory_endpoint, openshift_token(config_dir), | ||
skip_cert_verification, ca_cert_path) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why small case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will fix
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
?