Feat/refac_core (#10)

* refac(hcledit): Move bin in subdirectory

* refac(terraform_utils): Move in own directory.

* refac(terraform_utils): Move to own directory

* feat(vscode): Add some project settings.

* feat(pip): Add py-markdown-table as requirement.

* refac(terraform_utils): Add init file.

* feat(providers): Add Providers.

* feat(provider_handler): Add provider_handler and builder.

* refac(models): Update versioned resources models.

* feat(composition): Remove composition.

* feat(models): Add statistics model.

* feat(cli): Switch to provider implementation.

* refac(builder): Change secret input from file path to dict.

* refac(action): Switch to provider implementation.

* ruff auto-fix

* feat(vscode): Add task to auto-fix with ruff.

* doc(README): Update doc.

* fix(provider): Fix get markdown table

* feat(Statistics): Add function to get markdown table.

* feat(action): Add report to pr body.

* fix(upgrade_resources): Add catch to handle unsuccesfull upgrades

* ruff format

* fix(terraform_provider): remove exception handling.

* refac(exception): Remove base exception.

* fix(cicd): Change error text.

* fix(hcledit_cli): Check if binary exists.

* fix(hcledit): Make binary executable.

* fix(hcledit_cli): REmove exist check.

* Revert "fix(hcledit_cli): REmove exist check."

This reverts commit c4a29a2.

* fix(setup): Change package_data path.

* fix(Statistics): Fix markdown table.

* fix(Statistics): Change markdown input to list.

* feat(vscode): Disable justmycode in launch.json

* doc(README): Add supported platforms.

* fix(pr_body): Add missing newline after provider table.

* fix(pr_body): Remove second newline.

* fix(pr_body): Format tables.

* fix(pr_body): Add quotes.

* refac(markdown_table): Switch to new library for markdown tables.

* fix(pip): Fix pytablewriter version

* fix(terraform_provider): Set table_name

* feat(action): Implement handling of existing PRs

* fix(action): Save upgradable resources instead of all.

.github/workflows/cli_integration_test.yml

@@ -70,7 +70,7 @@ jobs:
             throw "Failed to get resources"
           }
           if ( -not ( $report.resources_patched -gt 3 ) ) {
-              throw "No resources should be patched"
+              throw "At least 3 resources should be patched"
           }
           if ( $report.errors -gt 0 ) {
               throw "Errors have been detected"

.vscode/launch.json

@@ -10,7 +10,7 @@
                 "--debug",
                 "report"
             ],
-            "justMyCode": true
+            "justMyCode": false
         },
         {
             "name": "InfraPatch CLI: Update",
@@ -21,15 +21,15 @@
                 "--debug",
                 "update"
             ],
-            "justMyCode": true
+            "justMyCode": false
         },
         {
             "name": "InfraPatch CLI: custom",
             "type": "python",
             "request": "launch",
             "module": "infrapatch.cli",
             "args": "${input:custom_args}",
-            "justMyCode": true
+            "justMyCode": false
         }
     ],
     "inputs": [

.vscode/settings.json

@@ -34,6 +34,7 @@
     "source.unusedImports",
     "source.convertImportFormat"
   ],
+  "python.analysis.extraPaths": [],
   "python.analysis.typeCheckingMode": "basic",
   "python.analysis.diagnosticMode": "workspace",
   "editor.guides.indentation": false,
@@ -48,5 +49,9 @@
   "python.testing.pytestEnabled": true,
   "files.exclude": {
     "**/__pycache__": true
+  },
+  "python.analysis.autoSearchPaths": false,
+  "[json]": {
+    "editor.defaultFormatter": "vscode.json-language-features"
   }
 }

.vscode/tasks.json

@@ -27,6 +27,20 @@
             },
             "problemMatcher": []
         },
+        {
+            "label": "ruff auto-fix",
+            "type": "shell",
+            "command": "ruff",
+            "args": [
+                "check",
+                ".",
+                "--fix"
+            ],
+            "presentation": {
+                "reveal": "always"
+            },
+            "problemMatcher": []
+        },
         {
             "label": "pip install all dependencies",
             "type": "shell",

README.md

@@ -7,6 +7,10 @@ The CLI works by scanning your .tf files for versioned providers and modules and
 ## CLI
 The follwoing chapter describes the CLI usage.
 
+## Supported Platforms
+
+Currently, the CLI supports only MacOS and Linux.
+
 ### Installation
 
 Before installing the CLI, make sure you have Python 3.11 or higher installed.
@@ -102,6 +106,23 @@ jobs:
 
 ```
 
+#### Providers
+
+InfraPatch supports individual providers to detect and patch versions. Currently, the following providers are available:
+| Name                | Description                            |
+| ------------------- | -------------------------------------- |
+| terraform_modules   | Provider to patch Terraform Modules.   |
+| terraform_providers | Provider to patch Terraform Providers. |
+
+Per default, all providers are enabled. You can only enable specific providers by specifying the provider names as comma separated list in the input `enabled_providers`:
+
+  ```yaml
+    - name: Run in update mode
+      uses: Noahnc/infrapatch@main
+      with:
+        enabled_providers: terraform_modules,terraform_providers
+  ```
+
 #### Report only Mode
 
 By default, the Action will create a Branch with all the changes and opens a PR to Branch for which the Action was triggered.

action.yml

@@ -26,6 +26,10 @@ inputs:
     description: "Only report new versions. Do not update files. Defaults to false"
     default: "false"
     required: true
+  enabled_providers:
+    description: "Comma separated list of provider names to enable. Defaults to terraform_modules,terraform_providers"
+    default: "terraform_modules,terraform_providers"
+    required: true
   registry_secrets:
     description: "Registry secrets to use for private registries. Needs to be a newline separated list of secrets in the format <registry_domain>:<secret_name>. Defaults to empty"
     required: false
@@ -93,6 +97,7 @@ runs:
         REPORT_ONLY: ${{ inputs.report_only }}
         REGISTRY_SECRET_STRING: ${{ inputs.REGISTRY_SECRETS }}
         WORKING_DIRECTORY_RELATIVE: ${{ inputs.working_directory_relative }}
+        ENABLED_PROVIDERS: ${{ inputs.enabled_providers }}
 
         # Calculated config from other steps
         HEAD_BRANCH: ${{ steps.branch.outputs.head }}

infrapatch/action/__main__.py

@@ -1,14 +1,15 @@
 import logging as log
-
+from typing import Union
 
 import click
 from github import Auth, Github, GithubException
 from github.PullRequest import PullRequest
-from infrapatch.action.config import ActionConfigProvider
+from github.Repository import Repository
 
-from infrapatch.core.composition import build_main_handler
+from infrapatch.action.config import ActionConfigProvider
 from infrapatch.core.log_helper import catch_exception, setup_logging
-from infrapatch.core.models.versioned_terraform_resources import get_upgradable_resources
+from infrapatch.core.provider_handler import ProviderHandler
+from infrapatch.core.provider_handler_builder import ProviderHandlerBuilder
 from infrapatch.core.utils.git import Git
 
 
@@ -25,7 +26,19 @@ def main(debug: bool):
     github_repo = github.get_repo(config.repository_name)
     github_head_branch = github_repo.get_branch(config.head_branch)
 
-    main_handler = build_main_handler(default_registry_domain=config.default_registry_domain, credentials_dict=config.registry_secrets)
+    if len(config.enabled_providers) == 0:
+        raise Exception("No providers enabled. Please enable at least one provider.")
+
+    builder = ProviderHandlerBuilder(config.working_directory)
+    builder.with_git_integration()
+    if "terraform_modules" in config.enabled_providers or "terraform_providers" in config.enabled_providers:
+        builder.add_terraform_registry_configuration(config.default_registry_domain, config.registry_secrets)
+    if "terraform_modules" in config.enabled_providers:
+        builder.with_terraform_module_provider()
+    if "terraform_providers" in config.enabled_providers:
+        builder.with_terraform_provider_provider()
+
+    provider_handler = builder.build()
 
     git.fetch_origin()
 
@@ -34,50 +47,75 @@ def main(debug: bool):
     except GithubException:
         github_target_branch = None
 
+    upgradable_resources_head_branch = None
+    pr = None
     if github_target_branch is not None and config.report_only is False:
+        pr = get_pr(github_repo, config.head_branch, config.target_branch)
+        if pr is not None:
+            upgradable_resources_head_branch = provider_handler.get_upgradable_resources()
         log.info(f"Branch {config.target_branch} already exists. Checking out...")
         git.checkout_branch(config.target_branch, f"origin/{config.target_branch}")
 
         log.info(f"Rebasing branch {config.target_branch} onto origin/{config.head_branch}")
         git.run_git_command(["rebase", "-Xtheirs", f"origin/{config.head_branch}"])
         git.push(["-f", "-u", "origin", config.target_branch])
 
-    resources = main_handler.get_all_terraform_resources(config.working_directory)
+    provider_handler.print_resource_table(only_upgradable=True, disable_cache=True)
 
     if config.report_only:
-        main_handler.print_resource_table(resources)
         log.info("Report only mode is enabled. No changes will be applied.")
         return
 
-    upgradable_resources = get_upgradable_resources(resources)
-
-    if len(upgradable_resources) == 0:
-        log.info("No upgradable resources found.")
+    if provider_handler.check_if_upgrades_available() is False:
+        log.info("No resources with pending upgrade found.")
         return
 
     if github_target_branch is None:
         log.info(f"Branch {config.target_branch} does not exist. Creating and checking out...")
         github_repo.create_git_ref(ref=f"refs/heads/{config.target_branch}", sha=github_head_branch.commit.sha)
         git.checkout_branch(config.target_branch, f"origin/{config.head_branch}")
 
-    main_handler.update_resources(upgradable_resources, True, config.working_directory, config.repository_root, True)
-    main_handler.dump_statistics(upgradable_resources, save_as_json_file=True)
+    provider_handler.upgrade_resources()
+    if upgradable_resources_head_branch is not None:
+        log.info("Updating status of resources from previous branch...")
+        provider_handler.set_resources_patched_based_on_existing_resources(upgradable_resources_head_branch)
+    provider_handler.print_statistics_table()
+    provider_handler.dump_statistics()
 
     git.push(["-f", "-u", "origin", config.target_branch])
 
-    create_pr(config.github_token, config.head_branch, config.repository_name, config.target_branch)
+    body = get_pr_body(provider_handler)
 
+    if pr is not None:
+        pr.edit(body=body)
+        return
+    create_pr(github_repo, config.head_branch, config.target_branch, body)
 
-def create_pr(github_token, head_branch, repository_name, target_branch) -> PullRequest:
-    token = Auth.Token(github_token)
-    github = Github(auth=token)
-    repo = github.get_repo(repository_name)
+
+def get_pr_body(provider_handler: ProviderHandler) -> str:
+    body = ""
+    markdown_tables = provider_handler.get_markdown_tables()
+    for table in markdown_tables:
+        body += table.dumps()
+        body += "\n"
+
+    body += provider_handler._get_statistics().get_markdown_table().dumps()
+    body += "\n"
+    return body
+
+
+def get_pr(repo: Repository, head_branch, target_branch) -> Union[PullRequest, None]:
     pull = repo.get_pulls(state="open", sort="created", base=head_branch, head=target_branch)
     if pull.totalCount != 0:
         log.info(f"Pull request found from '{target_branch}' to '{head_branch}'")
         return pull[0]
-    log.info(f"No pull request found from '{target_branch}' to '{head_branch}'. Creating a new one.")
-    return repo.create_pull(title="InfraPatch Module and Provider Update", body="InfraPatch Module and Provider Update", base=head_branch, head=target_branch)
+    log.debug(f"No pull request found from '{target_branch}' to '{head_branch}'.")
+    return None
+
+
+def create_pr(repo: Repository, head_branch: str, target_branch: str, body: str) -> PullRequest:
+    log.info(f"Creating new pull request from '{target_branch}' to '{head_branch}'.")
+    return repo.create_pull(title="InfraPatch Module and Provider Update", body=body, base=head_branch, head=target_branch)
 
 
 if __name__ == "__main__":

infrapatch/action/config.py

@@ -15,6 +15,7 @@ class ActionConfigProvider:
     repository_name: str
     default_registry_domain: str
     working_directory: Path
+    enabled_providers: list[str]
     repository_root: Path
     report_only: bool
     registry_secrets: dict[str, str]
@@ -25,6 +26,7 @@ def __init__(self) -> None:
         self.target_branch = _get_value_from_env("TARGET_BRANCH")
         self.repository_name = _get_value_from_env("REPOSITORY_NAME")
         self.repository_root = Path(os.getcwd())
+        self.enabled_providers = _get_value_from_env("ENABLED_PROVIDERS", default="").split(",")
         self.working_directory = self.repository_root.joinpath(_get_value_from_env("WORKING_DIRECTORY_RELATIVE", default=""))
         self.default_registry_domain = _get_value_from_env("DEFAULT_REGISTRY_DOMAIN")
         self.registry_secrets = _get_credentials_from_string(_get_value_from_env("REGISTRY_SECRET_STRING", secret=True, default=""))

infrapatch/cli/__main__.py

@@ -2,70 +2,87 @@
 from typing import Union
 
 import click
+from infrapatch.core.credentials_helper import get_registry_credentials
+from infrapatch.core.provider_handler import ProviderHandler
+from infrapatch.core.provider_handler_builder import ProviderHandlerBuilder
 
 from infrapatch.cli.__init__ import __version__
-from infrapatch.core.composition import MainHandler, build_main_handler
 from infrapatch.core.log_helper import catch_exception, setup_logging
+from infrapatch.core.utils.terraform.hcl_edit_cli import HclEditCli
+from infrapatch.core.utils.terraform.hcl_handler import HclHandler
 
-main_handler: Union[MainHandler, None] = None
+provider_handler: Union[ProviderHandler, None] = None
 
 
 @click.group(invoke_without_command=True)
 @click.option("--debug", is_flag=True, help="Enable debug logging.")
 @click.option("--version", is_flag=True, help="Prints the version of the tool.")
+@click.option("--working-directory-path", default=None, help="Working directory to run. Defaults to the current working directory")
 @click.option("--credentials-file-path", default=None, help="Path to a file containing credentials for private registries.")
-@click.option("--default_registry_domain", default="registry.terraform.io", help="Default registry domain for resources without a specified domain.")
+@click.option("--default-registry-domain", default="registry.terraform.io", help="Default registry domain for resources without a specified domain.")
 @catch_exception(handle=Exception)
-def main(debug: bool, version: bool, credentials_file_path: str, default_registry_domain: str):
+def main(debug: bool, version: bool, working_directory_path: str, credentials_file_path: str, default_registry_domain: str):
     if version:
         print(f"You are running infrapatch version: {__version__}")
         exit(0)
     setup_logging(debug)
-    global main_handler
+
+    global provider_handler
     credentials_file = None
+    working_directory = Path.cwd()
+
+    if working_directory_path is not None:
+        working_directory = Path(working_directory_path)
+        if not working_directory.exists() or not working_directory.is_dir():
+            raise Exception(f"Project root '{working_directory.absolute().as_posix()}' does not exist.")
+
     if credentials_file_path is not None:
         credentials_file = Path(credentials_file_path)
-    main_handler = build_main_handler(default_registry_domain, credentials_file)
+        if not credentials_file.exists() or not credentials_file.is_file():
+            raise Exception(f"Credentials file '{credentials_file}' does not exist.")
+    credentials = get_registry_credentials(HclHandler(HclEditCli()), credentials_file)
+    provider_builder = ProviderHandlerBuilder(working_directory)
+    provider_builder.add_terraform_registry_configuration(default_registry_domain, credentials)
+    provider_builder.with_terraform_module_provider()
+    provider_builder.with_terraform_provider_provider()
+    provider_handler = provider_builder.build()
 
 
 # noinspection PyUnresolvedReferences
 @main.command()
-@click.option("--project-root-path", default=None, help="Root directory of the project. If not specified, the current working directory is used.")
 @click.option("--only-upgradable", is_flag=True, help="Only show providers and modules that can be upgraded.")
 @click.option("--dump-json-statistics", is_flag=True, help="Creates a json file containing statistics about the found resources and there update status as json file in the cwd.")
 @catch_exception(handle=Exception)
-def report(project_root_path: str, only_upgradable: bool, dump_json_statistics: bool):
+def report(only_upgradable: bool, dump_json_statistics: bool):
     """Finds all modules and providers in the project_root and prints the newest version."""
-    if project_root_path is None:
-        project_root = Path.cwd()
-    else:
-        project_root = Path(project_root_path)
-    global main_handler
-    if main_handler is None:
-        raise Exception("main_handler not initialized.")
-    resources = main_handler.get_all_terraform_resources(project_root)
-    main_handler.print_resource_table(resources, only_upgradable)
-    main_handler.dump_statistics(resources, dump_json_statistics)
+    if provider_handler is None:
+        raise Exception("provider_handler not initialized.")
+    provider_handler.print_resource_table(only_upgradable)
+    provider_handler.print_statistics_table()
+    if dump_json_statistics:
+        provider_handler.dump_statistics()
 
 
 @main.command()
-@click.option("--project-root-path", default=None, help="Root directory of the project. If not specified, the current working directory is used.")
 @click.option("--confirm", is_flag=True, help="Apply changes without confirmation.")
 @click.option("--dump-json-statistics", is_flag=True, help="Creates a json file containing statistics about the updated resources in the cwd.")
 @catch_exception(handle=Exception)
-def update(project_root_path: str, confirm: bool, dump_json_statistics: bool):
+def update(confirm: bool, dump_json_statistics: bool):
     """Finds all modules and providers in the project_root and updates them to the newest version."""
-    if project_root_path is None:
-        project_root = Path.cwd()
-    else:
-        project_root = Path(project_root_path)
-    global main_handler
-    if main_handler is None:
+    global provider_handler
+    if provider_handler is None:
         raise Exception("main_handler not initialized.")
 
-    resources = main_handler.get_all_terraform_resources(project_root)
-    main_handler.update_resources(resources, confirm, project_root, project_root)
-    main_handler.dump_statistics(resources, dump_json_statistics)
+    provider_handler.print_resource_table(only_upgradable=True)
+    if not confirm:
+        if not click.confirm("Do you want to apply the changes?"):
+            print("Aborting...")
+            return
+
+    provider_handler.upgrade_resources()
+    provider_handler.print_statistics_table()
+    if dump_json_statistics:
+        provider_handler.dump_statistics()
 
 
 if __name__ == "__main__":