Skip to content

Commit

Permalink
Merge offline into develop. (#603)
Browse files Browse the repository at this point in the history
* Added download role

* Kubernetes install packages, dependencies list added

* Loading images from download cache- Kubernetes role

* Docker installation from offline

* Added download role

* Commit to store prototype (temporary changes)

* Updated prototype

* sync -> copy + improvements

* The same package for multiple OS distros handled

* Grafana offline mode install, docker offline fixes

* ELK, HAProxy, Postgresql moved to offline installation

* RabbitMQ package installation offline mode

* HAProxy experimental binary install (offline mode)

* RabbitMQ offline, HAProxy offline binary install (#473)

* RabbitMQ package installation offline mode

* HAProxy experimental binary install (offline mode)

* Added links to kibana, haproxy, filebeat, elasticsearch, grafana, docker packages (#474)

* Added links to haproxy_exporter and rabbitmq packages (#476)

* Offline mode for (file install): elacticsearch-curator, haproxy_exporter, jmx_exporter, kafka, node_exporter

* Fixes for downloading packages and installing Docker on Ubuntu (#480)

* Updated variable names for dest dirs

* Fixes for downloading packages

* Prometheus, AlertManager and ZooKeeper move to offline mode

* Tasks for downloading all files and improvements (#412)

* Added links to postgresql, zookeeper, kafka, prometheus, exporter packages (#484)

* Exact version match (#485)

* Exact version match (#412)

* Simplified setting properties (#412)

* Fix: Prevent docs duplication in manifest

* Remote_src fix in unarchive, haproxy and haproxy_exporter validated

* [Offline-mode] ELK stack updated

* Fixes for offline mode with packages and files

* Kubernetes installation of deb package fix

* Download all packages + fix for admin_user (#504)

* Feature/offline-mode - added image download capability (#486)

* image download capability added
* multiple fixes and multi-arch support added

* Added links to Kubernetes packages (#502)

* Removing the test environment destruction function

* Added links to kibana, haproxy, filebeat, elasticsearch, grafana, docker packages

* Added links to haproxy_exporter and rabbitmq packages

* Added links to haproxy_exporter and rabbitmq packages

* Added links to postgresql packages

* Added links to postgresql packages

* Added links to postgresql packages

* Added links to zookeeper, kafka, exporter packages

* Added links to Kubernetes packages

* replaced sha512 with file_name

* More fixes to offline mode (#503)

* Fix: Prevent docs duplication in manifest

* Remote_src fix in unarchive, haproxy and haproxy_exporter validated

* [Offline-mode] ELK stack updated

* Fixes for offline mode with packages and files

* Kubernetes installation of deb package fix

* Removed test data (#412) (#505)

* Feature/offline-mode - image download tweaks, added configuration example (#507)

* change url to full docker registry path with version tag, added configuration examples

* removal of unnecessary quotations, silenced ansible warning

* Kafka, Kafka exporter, postgresql and install role

* Add offline-mode flag to epicli

* Add offline mode to AnsibleVarsGenerator

* Add offline flag for epicli (#510)

* Add offline-mode flag to epicli

* Add offline mode to AnsibleVarsGenerator

* Installing packages + files + images using artifacts repository

* Installing packages + files + images using artifacts repository (#512)

* Rewrote: postgress, haproxy and haproxy_exporter.

* Prometheus, Grafana, Exporters, Kafka, Zookeeper to use private repo

* Kafka exporter moved to private repo

* Rewrote: postgress, haproxy and haproxy_exporter. (#514)

* Missing flannel image definition in K8s configuration

* Download role cleanup

* Install role cleanup

* Add stub of repositories role

* Fixed feature mapping yaml

* Updated elk, Kibana, Filebeat, elk-curator.

* Added file_name for haproxy-exporter

* Roles to use private repo (#515)

* Installing packages + files + images using artifacts repository

* Prometheus, Grafana, Exporters, Kafka, Zookeeper to use private repo

* Kafka exporter moved to private repo

* Missing flannel image definition in K8s configuration

* Download role cleanup

* Install role cleanup

* Fixed feature mapping yaml

* Feature/offline mode (#516)

- Updated elk, Kibana, Filebeat, elk-curator
- Added file_name for haproxy-exporter

* Add setup and teardown to Epiphany role

* Add integration to repository role with epicli

* Change repository configuration to take files from kubernetes master server

* Fixed remove of node_exporter endpoint accessibility on kubernetes master for default configuration

* Add offline/online mode for Red Hat (#519)

* Added prepare command to prepare offline repo scripts.

* - Updated RabbitMQ roll to pull stuff from repo.

* Add scopeo copying (#521)

* Add offline/online mode for Red Hat

* Add skopeo copy and stub of task for Debian

* Copy scopeo to upload dir

* Add new structure of files to repository role (#522)

* Add offline/online mode for Red Hat

* Add skopeo copy and stub of task for Debian

* Copy scopeo to upload dir

* Changed repository script structure

* Move repository scripts

* Add directories copy for new data structure

* Fixes for Red Hat script

* Refactor of repository role.

* Refactored stuff.

* download-requirements.sh for CentOS-7 (#513) (#526)

download-requirements.sh for CentOS-7

* Started refactoring  repository role. (#527)

* Refactor of repository role.
* Refactored stuff.

* Run download-requirements.sh from any location (#528)

Run download-requirements.sh from any location

* Move Skopeo for copying to repo host and add message for long running repo task.

* Feature/offline mode (#529)

- Move Skopeo for copying to repo host and add message for long running repo task.

* Offline progress

- Changed offline_mode to offline_requirements accepting a path param.
- Removed old offline mode code
- Removed useless checks for AnsibleVarsGenerator.py

* Minor message update.

* Feature/offline mode (#530)

- Changed offline_mode to offline_requirements accepting a path param.
- Removed old offline mode code
- Removed useless checks for AnsibleVarsGenerator.py

* Offline progress

- Fix for local epi-repo path.
- Cleaned up old code.

* Feature/offline mode (#531)

- Fix for local epi-repo path.
- Cleaned up old code.

* Refactoring repository role (#532)

- Refactoring repository role

* Changes in repository role (#533)

* Redirect STDERR to STDOUT for logging

* Fix for packages containing 'error' in name

* Added Perl as dependency for vim

* Fix for 'ERROR 403: Forbidden' on AWS

* Added python-firewall for firewalld

* Added dependencies for vim

* - Minor fixes after merge with develop

* Fixed version numbers.

* More dependencies for vim (#538)

* Merged develop into feature/offline + some fixes (#539)

* Updated documentation

- Added changelog
- Added versions to for components
- Minor documentation updates
- Removed unused documentation

* Fixed links.

* Fixed changelog.

* Added node_exporter port known issue.

* Feature/skopeo (#475)

* Updated documentation

- Added changelog
- Added versions to for components
- Minor documentation updates
- Removed unused documentation

* Fixed links.

* Fixed changelog.

* Added node_exporter port known issue.

* - Added Skopeo minor update to devcontainer.

* subnets, network interfaces, security rules, ansible inventory (#469)

Fixes for subnets
Fixed versions for Terraform providers for both Azure and AWS
Added VMs
Added network interfaces
Added security rules
Added ansible inventory generation

* File generating hashes for directory (#477)

* Item: #422 Desc: Add missing ports for prometheus and grafana

* Item: #0000 Desc: File hash generator for directory

* Item: #0000 Desc: Moved folder to new structure

* Testruns (#506)

- Added running of python unit tests via debug config in VSCode
- Added running of serverspec tests via debug config in VSCode
- Documentation

* Release prep part 1 (#501)

* Updated documentation

- Added changelog
- Added versions to for components
- Minor documentation updates
- Removed unused documentation

* Fixed links.

* Fixed changelog.

* Added node_exporter port known issue.

* Preparation for release.

* Feature/azure-k8n-storage (#487)

- Added shared storage for k8n

* Azure rehat fixes (#511)

Fix for container-selinux package on redhat.
Fixes for HAProxy and Posgress on Azure.
Fix for running tests from VSCode.

* Fix typo rabbitmq (#518)

* Fix/rabbitmq error (#523)

- Fixed typo for rabbitmq machine

* Fixed link to container-selinux package (legacy) (#525)

* Fixed link to container-selinux package

* Refactor of repository role.

* Refactored stuff.

* Move Skopeo for copying to repo host and add message for long running repo task.

* Offline progress

- Changed offline_mode to offline_requirements accepting a path param.
- Removed old offline mode code
- Removed useless checks for AnsibleVarsGenerator.py

* Minor message update.

* Offline progress

- Fix for local epi-repo path.
- Cleaned up old code.

* - Minor fixes after merge with develop

* Fixed version numbers.

* Debian scripts for download

* Reverted temp changes

* Review changes

* Changed scripts fore Debian packages (#540)

* Installing packages + files + images using artifacts repository

* Prometheus, Grafana, Exporters, Kafka, Zookeeper to use private repo

* Kafka exporter moved to private repo

* Missing flannel image definition in K8s configuration

* Download role cleanup

* Install role cleanup

* Fix for dependencies to be downloaded (#544)

* uniq -> 'sort --unique'

* Updated package dependencies

* Remove trailing spaces

* java-1.8.0-openjdk -> java-1.8.0-openjdk-headless

* Refactored repository role, changes for RHEL (#547)

* Refactored repository role, changes for RHEL #536

* Changed directories structure for tasks #536

* Added repository role to feature mappings #536

* Fix for removing flag file if expired #536

* Added wget #536

* Minor fix for prepare when output dir is not supplied.

* Minor fix for prepare paths (#550)

- Minor fix for prepare when output dir is not supplied.

* Offline mode - progress for RedHat (#552)

* Added --log-to-journal option #536

* download-requirements script logs to journal #536

* Added python-slip-dbus for firewalld on AWS #536

* Disabled repository role in feature mappings #536

* Static version for docker-ce #536

* download-reqs: yum makecache fast -> yum makecache #536

* Do not install epel-release #536

* Download latest versions of dependencies #536

* Optimizations and better naming #536

* Colons -> hyphens in file names of images #536

* jmx_prometheus_javaagent v0.12.0 #536

* Better task names in zookeeper role

* Offline mode progress (#555)

* yum makecache -> yum makecache fast #536

* Added versions for docker-ce & docker-ce-cli #536

* Added property 'download_done_flag_expire_minutes' #536

* enabled-system-repos.txt in /var/tmp #536

* Clean up temporary files #536

* Added execution time #536

* Fixes for Azure in repository role #536

* Fixes for centos offline mode

* Fixes for offline centos  (#577)

* Installing packages + files + images using artifacts repository

* Prometheus, Grafana, Exporters, Kafka, Zookeeper to use private repo

* Kafka exporter moved to private repo

* Missing flannel image definition in K8s configuration

* Download role cleanup

* Install role cleanup

* Fixed feature mapping yaml

* Debian scripts for download

* Reverted temp changes

* Review changes

* Fixes for centos offline mode

* Fixes for offline mode for RedHat after testing (#579)

* Improvements after testing #536

* Rabbitmq and Erlang in fixed version #536

* Added --cacheonly to yum repolist #536

* Removed old structure #536

* Local docker registry for controlplane images

* Change comment for custom_image_registry

Co-Authored-By: to-bar <[email protected]>

* Skip downloading when image exists #536

* Fix in repository teardown #536

* Missing coredns images, flannel installation fix

* WIP: Feature/offline Ubuntu part (#580)

* offline repo for ubuntu - merge scripts into repository role - initial support

* requirements and repo creation fixes

* ubuntu offline-online installation works

* apt-cache policy not needed here

* cleanups

* added missing bash error handling

* added removal of 3rd party repos, other cleanups, install wget and gpg for minimal OS

* fix for running script from different location (#583)

* Fixes for offline mode for RedHat and CentOS (#585)

* Fix for hosts without yum-utils package #536

* Fixes for repo prereqs and image permissions  #536

* Offline mode Ubuntu part - fixes (#587)

* fix for running script from different location

* fixed skopeo .tar permissions to 644

* added missing dependencies for jq

* Fix/offline ubu (#588)

* fix for running script from different location

* fixed skopeo .tar permissions to 644

* added missing dependencies for jq

* bootstrap apache and dpkg-dev installation in air-gap mode

* Requirements and repo creation for centos

* Added missing file

* Requirements and repo creation for centos (#589)

* Requirements and repo creation for centos

* Added missing file

* Added NSG rules for image repository and package repository

* Added security rules to AWS and fixed on Azure

* Added security rules to AWS and fixed on Azure (#591)

* Offline mode Ubuntu part - fixes for air-gap mode, install libdpkg-perl (#592)

* fixes for air-gap mode, install libdpkg-perl

* unified stderr to stdout bash pipe with rhel version of this script

* convoluted comments are convoluted

* added missing dependencies

* Change name for RedHat filebeat

* File name typo fix (#593)

* Changed name to fix again...

* Fixed name for RedHat in rabbitmq tasks.

* Epicli validation and RabbitMQ and Postgress cluster fixes. (#595)

* Ensure httpd is running

* Ensure httpd is running (#597)

* Ensure httpd is running

* Added killing of the previous instance for Ansible

* Set up sysctl params for K8s

* Fix/offline ubu (#599)

* fixes for air-gap mode, install libdpkg-perl

* unified stderr to stdout bash pipe with rhel version of this script

* convoluted comments are convoluted

* added missing dependencies

* added dependencies for postgresql-10 and gpg/gnupg

* Fixed default HAProxy config

* Fix/offline last minute improvements (#601)

* fixes for air-gap mode, install libdpkg-perl

* unified stderr to stdout bash pipe with rhel version of this script

* convoluted comments are convoluted

* added missing dependencies

* added dependencies for postgresql-10 and gpg/gnupg

* restore system repos if missing, logging improvements

* Fix in enable-system-repos.sh for RHEL (#604)

* Merged develop into offline. (#605)

* Fix/aws ports and naming (#576)

* FIxes:

- Init issues for different providers
- Link in docs

* Added ports for clustering Postgres and RabbitMQ

* RabbitMQ test fix for AWS hostnames (#578)

* Test fix - added regexp to eliminate false negatives in particular cases (#586)

* Feature/doc updates (#584)

* - Updated changelog

* Added article for up/downscaling, clustering components.

* Added article for offline installation.

* removal of unneeded code from Ubuntu 16.04 era (#606)
  • Loading branch information
seriva authored Oct 11, 2019
1 parent ab173f8 commit 43a1e50
Show file tree
Hide file tree
Showing 140 changed files with 3,643 additions and 1,055 deletions.
2 changes: 1 addition & 1 deletion core/src/epicli/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ share/python-wheels/
*.egg
MANIFEST

requirements.txt
/requirements.txt

# PyInstaller
# Usually these files are written by a python script from a template
Expand Down
4 changes: 2 additions & 2 deletions core/src/epicli/cli/engine/BuildEngine.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def process_configuration_docs(self):

def collect_infrastructure_config(self):
with provider_class_loader(self.cluster_model.provider, 'InfrastructureConfigCollector')(
[*self.input_docs, *self.configuration_docs, *self.infrastructure_docs]) as config_collector:
[*self.configuration_docs, *self.infrastructure_docs]) as config_collector:
config_collector.run()

def validate(self):
Expand Down Expand Up @@ -107,7 +107,7 @@ def apply(self):
self.collect_infrastructure_config()

# Merge all the docs
docs = [*self.input_docs, *self.configuration_docs, *self.infrastructure_docs]
docs = [*self.configuration_docs, *self.infrastructure_docs]

# Save docs to manifest file
save_manifest(docs, self.cluster_model.specification.name)
Expand Down
53 changes: 53 additions & 0 deletions core/src/epicli/cli/engine/PrepareEngine.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
import os
import stat
import inspect
import shutil
from os.path import dirname

from cli.helpers.Step import Step
from cli.helpers.data_loader import DATA_FOLDER_PATH
from cli.helpers.Config import Config
from cli.helpers.build_saver import copy_files_recursively


class PrepareEngine(Step):
PREPARE_PATH = DATA_FOLDER_PATH + '/common/ansible/playbooks/roles/repository/files/download-requirements'

def __init__(self, input_data):
super().__init__(__name__)
self.os = input_data.os

def __enter__(self):
super().__enter__()
return self

def __exit__(self, exc_type, exc_value, traceback):
super().__exit__(exc_type, exc_value, traceback)

def prepare(self):
prepare_src = os.path.join(self.PREPARE_PATH, self.os)
skopeo_src = os.path.join(dirname(dirname(inspect.getfile(os))), 'skopeo_linux')
prepare_dst = os.path.join(Config().output_dir, 'prepare_scripts')

if not os.path.exists(prepare_src):
supported_os = os.listdir(self.PREPARE_PATH)
raise Exception(f'Unsupported OS: {self.os}. Currently supported: {supported_os}')

if not os.path.exists(skopeo_src):
raise Exception('Skopeo dependency not found')

# copy files to output dir
copy_files_recursively(prepare_src, prepare_dst)
shutil.copy(skopeo_src, prepare_dst)

# make sure the scripts and skopeo are executable
self.make_file_executable(os.path.join(prepare_dst, 'skopeo_linux'))
self.make_file_executable(os.path.join(prepare_dst, 'download-requirements.sh'))

self.logger.info(f'Prepared files for downloading the offline requirements in: {prepare_dst}')
return 0

@staticmethod
def make_file_executable(file):
executable_stat = os.stat(file)
os.chmod(file, executable_stat.st_mode | stat.S_IEXEC)
24 changes: 16 additions & 8 deletions core/src/epicli/cli/engine/ansible/AnsibleRunner.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
import inspect
import os
import time
import shutil
from os.path import dirname

from cli.engine.ansible.AnsibleCommand import AnsibleCommand
from cli.engine.ansible.AnsibleInventoryCreator import AnsibleInventoryCreator
Expand All @@ -8,10 +11,11 @@
from cli.helpers.build_saver import get_inventory_path, get_ansible_path, copy_files_recursively
from cli.helpers.naming_helpers import to_role_name
from cli.helpers.data_loader import DATA_FOLDER_PATH
from cli.helpers.Config import Config


class AnsibleRunner(Step):
ANSIBLE_PLAYBOOKS_PATH = DATA_FOLDER_PATH + "/common/ansible/playbooks/"
ANSIBLE_PLAYBOOKS_PATH = DATA_FOLDER_PATH + '/common/ansible/playbooks/'

def __init__(self, cluster_model, config_docs):
super().__init__(__name__)
Expand Down Expand Up @@ -42,21 +46,25 @@ def run(self):

copy_files_recursively(AnsibleRunner.ANSIBLE_PLAYBOOKS_PATH, get_ansible_path(self.cluster_model.specification.name))

# todo: install packages to run ansible on Red Hat hosts
self.ansible_command.run_task_with_retries(hosts="all", inventory=inventory_path, module="raw",
args="cat /etc/lsb-release | grep -i DISTRIB_ID | grep -i ubuntu && "
"sudo apt-get update && sudo apt-get install -y python-simplejson "
"|| echo 'Cannot find information about Ubuntu distribution'", retries=5)
# copy skopeo so Ansible can move it to the repositry machine
if not Config().offline_requirements:
shutil.copy(os.path.join(dirname(dirname(inspect.getfile(os))), 'skopeo_linux'), '/tmp')

self.ansible_vars_generator.run()


self.logger.info('Setting up repository for cluster provisioning. This will take a while...')
self.ansible_command.run_playbook_with_retries(inventory=inventory_path,
playbook_path=self.playbook_path('common'),
playbook_path=self.playbook_path('repository_setup'),
retries=5)

self.ansible_command.run_playbook(inventory=inventory_path,
playbook_path=self.playbook_path('common'))

enabled_roles = self.inventory_creator.get_enabled_roles()

for role in enabled_roles:
self.ansible_command.run_playbook(inventory=inventory_path,
playbook_path=self.playbook_path(to_role_name(role)))

self.ansible_command.run_playbook(inventory=inventory_path,
playbook_path=self.playbook_path('repository_teardown'))
18 changes: 11 additions & 7 deletions core/src/epicli/cli/engine/ansible/AnsibleVarsGenerator.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ def populate_group_vars(self, ansible_dir):
main_vars = ObjDict()
main_vars = self.add_admin_user_name(main_vars)
main_vars = self.add_validate_certs(main_vars)
main_vars = self.add_shared_config(main_vars)
main_vars = self.add_offline_requirements(main_vars)

vars_dir = os.path.join(ansible_dir, 'group_vars')
if not os.path.exists(vars_dir):
Expand All @@ -59,22 +61,24 @@ def populate_group_vars(self, ansible_dir):
vars_file_name = 'all.yml'
vars_file_path = os.path.join(vars_dir, vars_file_name)

with open(vars_file_path, 'w') as stream:
with open(vars_file_path, 'a') as stream:
dump(main_vars, stream)

def add_admin_user_name(self, document):
if document is None:
raise Exception('Config is empty for: ' + 'group_vars/all.yml')

document['admin_user'] = self.cluster_model.specification.admin_user
return document

def add_validate_certs(self, document):
if document is None:
raise Exception('Config is empty for: ' + 'group_vars/all.yml')

document['validate_certs'] = Config().validate_certs
return document

def add_offline_requirements(self, document):
document['offline_requirements'] = Config().offline_requirements
return document

def add_shared_config(self, document):
shared_config_doc = select_first(self.config_docs, lambda x: x.kind == 'configuration/shared-config')
document.update(shared_config_doc.specification)
return document

def add_provider_info(self, document):
Expand Down
23 changes: 14 additions & 9 deletions core/src/epicli/cli/engine/schema/ConfigurationAppender.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@


class ConfigurationAppender(Step):
REQUIRED_DOCS = ['configuration/feature-mapping', 'configuration/shared-config', 'epiphany-cluster']

def __init__(self, input_docs):
super().__init__(__name__)
self.cluster_model = select_single(input_docs, lambda x: x.kind == 'epiphany-cluster')
Expand All @@ -14,22 +16,25 @@ def __init__(self, input_docs):
def run(self):
configuration_docs = []

for document_kind in ConfigurationAppender.REQUIRED_DOCS:
doc = select_first(self.input_docs, lambda x: x.kind == document_kind)
if doc is None:
doc = load_yaml_obj(types.DEFAULT, 'common', document_kind)
self.logger.info("Adding: " + doc.kind)
configuration_docs.append(doc)
else:
configuration_docs.append(doc)

for component_key, component_value in self.cluster_model.specification.components.items():
if component_value.count < 1:
continue

features_map = select_first(self.input_docs, lambda x: x.kind == 'configuration/feature-mapping')
if features_map is None:
features_map = select_first(configuration_docs, lambda x: x.kind == 'configuration/feature-mapping')

if features_map is None:
features_map = load_yaml_obj(types.DEFAULT, 'common', 'configuration/feature-mapping')
self.logger.info("Adding: " + features_map.kind)
configuration_docs.append(features_map)

features_map = select_first(configuration_docs, lambda x: x.kind == 'configuration/feature-mapping')
config_selector = component_value.configuration
for feature_key in features_map.specification.roles_mapping[component_key]:
config = select_first(self.input_docs, lambda x: x.kind == 'configuration/' + feature_key and x.name == config_selector)
if config is not None:
configuration_docs.append(config)
if config is None:
config = select_first(configuration_docs, lambda
x: x.kind == 'configuration/' + feature_key and x.name == config_selector)
Expand Down
25 changes: 25 additions & 0 deletions core/src/epicli/cli/epicli.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from cli.engine.PatchEngine import PatchEngine
from cli.engine.DeleteEngine import DeleteEngine
from cli.engine.InitEngine import InitEngine
from cli.engine.PrepareEngine import PrepareEngine
from cli.helpers.Log import Log
from cli.helpers.Config import Config
from cli.version import VERSION
Expand Down Expand Up @@ -60,6 +61,7 @@ def main():
backup_parser(subparsers)
recovery_parser(subparsers)
delete_parser(subparsers)
prepare_parser(subparsers)

# check if there were any variables and display full help
if len(sys.argv) < 2:
Expand All @@ -78,6 +80,8 @@ def main():
config.log_type = args.log_type
config.log_count = args.log_count
config.validate_certs = True if args.validate_certs == 'true' else False
if 'offline_requirements' in args and not args.offline_requirements is None:
config.offline_requirements = args.offline_requirements
config.debug = args.debug
config.auto_approve = args.auto_approve

Expand Down Expand Up @@ -114,6 +118,8 @@ def apply_parser(subparsers):
help='File with infrastructure/configuration definitions to use.')
sub_parser.add_argument('--no-infra', dest='no_infra', action="store_true",
help='Skip infrastructure provisioning.')
sub_parser.add_argument('--offline-requirements', dest='offline_requirements', type=str,
help='Path to the folder with pre-prepared offline requirements.')

def run_apply(args):
adjust_paths_from_file(args)
Expand Down Expand Up @@ -198,11 +204,30 @@ def run_recovery(args):
sub_parser.set_defaults(func=run_recovery)


def prepare_parser(subparsers):
sub_parser = subparsers.add_parser('prepare', description='Creates a folder with all prerequisites to setup the offline requirements to install a cluster offline.')
sub_parser.add_argument('--os', type=str, required=True, dest='os',
help='The OS to prepare the offline requirements for.')

def run_prepare(args):
adjust_paths_from_output_dir()
with PrepareEngine(args) as engine:
return engine.prepare()

sub_parser.set_defaults(func=run_prepare)


def experimental_query():
if not query_yes_no('This is an experimental feature and could change at any time. Do you want to continue?'):
sys.exit(0)


def adjust_paths_from_output_dir():
if not Config().output_dir:
Config().output_dir = os.getcwd() # Default to working dir so we can at least write logs.
dump_config(Config())


def adjust_paths_from_file(args):
if not os.path.isabs(args.file):
args.file = os.path.join(os.getcwd(), args.file)
Expand Down
22 changes: 18 additions & 4 deletions core/src/epicli/cli/helpers/Config.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import os


class Config:
class __ConfigBase:
def __init__(self):
Expand All @@ -18,6 +19,7 @@ def __init__(self):
self._validate_certs = True
self._debug = False
self._auto_approve = False
self._offline_requirements = ''

@property
def docker_cli(self):
Expand Down Expand Up @@ -93,17 +95,29 @@ def debug(self):
@debug.setter
def debug(self, debug):
if not debug is None:
self._debug = debug
self._debug = debug

@property
def auto_approve(self):
return self._auto_approve

@auto_approve.setter
def auto_approve(self, auto_approve):
if not auto_approve is None:
self._auto_approve = auto_approve

self._auto_approve = auto_approve

@property
def offline_requirements(self):
return self._offline_requirements

@offline_requirements.setter
def offline_requirements(self, offline_requirements):
if not offline_requirements is None:
# To make sure Ansible copies the content of the folder the the repository host.
if not offline_requirements.endswith('/'):
offline_requirements = f'{offline_requirements}/'
self._offline_requirements = offline_requirements

instance = None

def __new__(cls):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,26 @@ specification:
destination_port_range: "9100"
source_address_prefix: "10.1.0.0/20"
destination_address_prefix: "0.0.0.0/0"
- name: image_registry
description: Allow image registry traffic
priority: 303
direction: Inbound
access: Allow
protocol: Tcp
source_port_range: "*"
destination_port_range: "80"
source_address_prefix: "10.1.0.0/20"
destination_address_prefix: "0.0.0.0/0"
- name: package_repository
description: Allow package repository traffic
priority: 304
direction: Inbound
access: Allow
protocol: Tcp
source_port_range: "*"
destination_port_range: "5000"
source_address_prefix: "10.1.0.0/20"
destination_address_prefix: "0.0.0.0/0"
- name: out
description: Allow out
priority: 101
Expand Down Expand Up @@ -167,7 +187,7 @@ specification:
source_port_range: "*"
destination_port_range: "0"
source_address_prefix: "0.0.0.0/0"
destination_address_prefix: "0.0.0.0/0"
destination_address_prefix: "0.0.0.0/0"
---
kind: infrastructure/virtual-machine
version: 0.4.0
Expand Down Expand Up @@ -243,6 +263,16 @@ specification:
destination_port_range: "22"
source_address_prefix: "0.0.0.0/0"
destination_address_prefix: "0.0.0.0/0"
- name: repository
description: Allow repository traffic
priority: 302
direction: Inbound
access: Allow
protocol: Tcp
source_port_range: "*"
destination_port_range: "80"
source_address_prefix: "10.1.0.0/20"
destination_address_prefix: "0.0.0.0/0"
- name: node_exporter
description: Allow node_exporter traffic
priority: 302
Expand Down
Loading

0 comments on commit 43a1e50

Please sign in to comment.