Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

📢 Migration of Components to a New GitHub Organization complete! #1177

Open
osterman opened this issue Nov 5, 2024 · 10 comments
Open

📢 Migration of Components to a New GitHub Organization complete! #1177

osterman opened this issue Nov 5, 2024 · 10 comments

Comments

@osterman
Copy link
Member

osterman commented Nov 5, 2024

Hello, Cloud Posse Community!

We’re excited to announce that starting on November 12, 2024, we will begin migrating each component in the cloudposse/terraform-aws-components repository to individual repositories under a new GitHub organization. This change aims to improve the stability, maintainability, and usability of our components.

Why This Migration?

Our goal is to make each component easier to use, contribute to, and maintain. This migration will allow us to:

  • Leverage terratest automation for better testing.
  • Implement semantic versioning to clearly communicate updates and breaking changes.
  • Improve PR review times and accelerate community contributions.
  • Enable Dependabot automation for dependency management.
  • And much more!

What to Expect Starting November 12, 2024

  • Migration Timeline: The migration will begin on November 12 and is anticipated to finish by the end of the following week.
  • Code Freeze: Starting on November 12, this repository will be set to read-only mode, marking the beginning of a code freeze. No new pull requests or issues will be accepted here after that date.
  • New Contribution Workflow: After the migration, all contributions should be directed to the new individual component repositories.
  • Updated Documentation: To support this transition, we are updating our documentation and cloudposse-component updater.
  • Future Archiving: In approximately six months, we plan to archive this repository and transfer it to the cloudposse-archives organization.

Frequently Asked Questions

  • Does this affect Terraform modules? No, only the terraform-aws-components repository is affected. Our Terraform modules will remain where they are.

We are committed to making this transition as seamless as possible. If you have any questions or concerns, please feel free to post them in this issue. Your feedback is important to us, and we appreciate your support as we embark on this new chapter!

Thank you,
The Cloud Posse Team

@osterman osterman changed the title Test 📢 Upcoming Migration of Components to a New GitHub Organization (CODE FREEZE 11/12 - 11/17) Nov 5, 2024
@osterman osterman pinned this issue Nov 5, 2024
@mtb-xt
Copy link
Contributor

mtb-xt commented Nov 6, 2024

Will there be any guidance on how to update our reference architectures to use this new setup?

@goruha
Copy link
Member

goruha commented Nov 6, 2024

@mtb-xt for sure
We prepared PR to component-updater GHA that will migrate components to the new org smoothly.
But we will also create documentation on how you can migrate manually.

@osterman
Copy link
Member Author

osterman commented Nov 6, 2024

@mtb-xt just to clarify what @goruha said, we've updated the component-updater GitHub Action to automatically rewrite the sources in the vendor and component manifest files when it opens the PRs. This should make the upgrade transparent and painless.

@goruha goruha changed the title 📢 Upcoming Migration of Components to a New GitHub Organization (CODE FREEZE 11/12 - 11/17) 📢 Migration of Components to a New GitHub Organization (CODE FREEZE 11/12 - 11/17) Nov 13, 2024
@goruha goruha mentioned this issue Nov 13, 2024
@nitrocode
Copy link
Member

Very cool. Looking forward to seeing all the components tested.

  1. Will all the same components be available after the migration?

  2. Will components still be open source?

  3. What will the new organization be named?

    Looked through the source code of the component updater and saw this https://github.com/cloudposse-terraform-components

  4. Will the component updater be updated to allow overriding the above org to use different sources if needed?

  5. Would you folks consider opening up the codeowners for components once they are all in their own repositories like you folks do with terraform modules ?

Thanks!

@osterman
Copy link
Member Author

@nitrocode thanks for raising these questions.

  1. Will all the same components be available after the migration?
  • Yes, we're migrating the components as is, bit-for-bit, to facilitate a switch. However, we anticipate promptly doing major releases on many of the components after that point, to introduce new functionality and improve brownfield infra (a large driver for the initiative). Those breaking changes will likely require local changes in your configurations. That won't happen immediately, but is the reason we're doing all this.
  1. Will components still be open source?

Yes, we have no plans to change license.

  1. What will the new organization be named?
    Looked through the source code of the component updater and saw this https://github.com/cloudposse-terraform-components

Correct, you found it!

https://github.com/cloudposse-terraform-components

As part of our transition to GitHub Enterprise (GHE), we are reorganizing our open-source projects into more purpose-built organizations. This allows us to better manage repository rulesets, GitHub Apps, and other configurations specific to each organization's purpose. This approach enhances our security posture and improves discoverability. Additionally, keeping our components separate from less opinionated child modules avoids confusion and ensures clearer organization.

  1. Will the component updater be updated to allow overriding the above org to use different sources if needed?

The component updater uses the sources as define in the vendor and component manifests. Thus, that's supported today.

One thing we've added to the component updater to make this switch less painful, is the ability for it to rewrite the sources to their new homes. So if it see's references to components in cloudposse/terraform-aws-components it will rewrite those to the new locations.

  1. Would you folks consider opening up the codeowners for components once they are all in their own repositories like you folks do with terraform modules ?

Yes, so we'll be able to accept more contributions of components and delegate ownership of components with this move. Note, CODEOWNERS only works with paid GitHub seats, so I think we'll continue to look for solutions that work better for non-org members, such as "allow lists" that we've implemented elsewhere.

@mergify mergify bot mentioned this issue Nov 19, 2024
@goruha
Copy link
Member

goruha commented Nov 19, 2024

We’re excited to announce that the migration of components to individual repositories under the new GitHub organization is now complete! 🎉
cloudposse/terraform-aws-components repository has been set to read-only mode.
All future updates, contributions, and issues should be directed to the respective component repositories in the new organization.

Component New repository
access-analyzer cloudposse-terraform-components/aws-access-analyzer
account cloudposse-terraform-components/aws-account
account-map cloudposse-terraform-components/aws-account-map
account-quotas cloudposse-terraform-components/aws-account-quotas
account-settings cloudposse-terraform-components/aws-account-settings
acm cloudposse-terraform-components/aws-acm
alb cloudposse-terraform-components/aws-alb
amplify cloudposse-terraform-components/aws-amplify
api-gateway-account-settings cloudposse-terraform-components/aws-api-gateway-account-settings
api-gateway-rest-api cloudposse-terraform-components/aws-api-gateway-rest-api
athena cloudposse-terraform-components/aws-athena
aurora-mysql cloudposse-terraform-components/aws-aurora-mysql
aurora-mysql-resources cloudposse-terraform-components/aws-aurora-mysql-resources
aurora-postgres cloudposse-terraform-components/aws-aurora-postgres
aurora-postgres-resources cloudposse-terraform-components/aws-aurora-postgres-resources
auth0/app cloudposse-terraform-components/aws-auth0-app
auth0/connection cloudposse-terraform-components/aws-auth0-connection
auth0/tenant cloudposse-terraform-components/aws-auth0-tenant
aws-backup cloudposse-terraform-components/aws-backup
aws-config cloudposse-terraform-components/aws-config
aws-inspector cloudposse-terraform-components/aws-inspector
aws-inspector2 cloudposse-terraform-components/aws-inspector2
aws-saml cloudposse-terraform-components/aws-saml
aws-shield cloudposse-terraform-components/aws-shield
aws-sso cloudposse-terraform-components/aws-identity-center
aws-ssosync cloudposse-terraform-components/aws-ssosync
aws-teams cloudposse-terraform-components/aws-teams
aws-team-roles cloudposse-terraform-components/aws-team-roles
bastion cloudposse-terraform-components/aws-bastion
cloudmap-namespace cloudposse-terraform-components/aws-cloudmap-namespace
cloudtrail cloudposse-terraform-components/aws-cloudtrail
cloudtrail-bucket cloudposse-terraform-components/aws-cloudtrail-bucket
cloudwatch-logs cloudposse-terraform-components/aws-cloudwatch-logs
cognito cloudposse-terraform-components/aws-cognito
config-bucket cloudposse-terraform-components/aws-config-bucket
datadog-configuration cloudposse-terraform-components/aws-datadog-credentials
datadog-integration cloudposse-terraform-components/aws-datadog-integration
datadog-lambda-forwarder cloudposse-terraform-components/aws-datadog-lambda-forwarder
datadog-logs-archive cloudposse-terraform-components/aws-datadog-logs-archive
datadog-monitor cloudposse-terraform-components/aws-datadog-monitor
datadog-private-location-ecs cloudposse-terraform-components/aws-datadog-private-location-ecs
datadog-synthetics cloudposse-terraform-components/aws-datadog-synthetics
datadog-synthetics-private-location cloudposse-terraform-components/aws-datadog-synthetics-private-location
dms/endpoint cloudposse-terraform-components/aws-dms-endpoint
dms/iam cloudposse-terraform-components/aws-dms-iam
dms/replication-instance cloudposse-terraform-components/aws-dms-replication-instance
dms/replication-task cloudposse-terraform-components/aws-dms-replication-task
dns-delegated cloudposse-terraform-components/aws-dns-delegated
dns-primary cloudposse-terraform-components/aws-dns-primary
documentdb cloudposse-terraform-components/aws-documentdb
dynamodb cloudposse-terraform-components/aws-dynamodb
ec2-client-vpn cloudposse-terraform-components/aws-ec2-client-vpn
ec2-instance cloudposse-terraform-components/aws-ec2-instance
ecr cloudposse-terraform-components/aws-ecr
ecs cloudposse-terraform-components/aws-ecs
ecs-service cloudposse-terraform-components/aws-ecs-service
efs cloudposse-terraform-components/aws-efs
eks/actions-runner-controller cloudposse-terraform-components/aws-eks-actions-runner-controller
eks/alb-controller cloudposse-terraform-components/aws-eks-alb-controller
eks/alb-controller-ingress-class cloudposse-terraform-components/aws-eks-alb-controller-ingress-class
eks/alb-controller-ingress-group cloudposse-terraform-components/aws-eks-alb-controller-ingress-group
argocd-repo cloudposse-terraform-components/aws-argocd-github-repo
eks/argocd cloudposse-terraform-components/aws-eks-argocd
eks/aws-node-termination-handler cloudposse-terraform-components/aws-eks-node-termination-handler
eks/cert-manager cloudposse-terraform-components/aws-eks-cert-manager
eks/cluster cloudposse-terraform-components/aws-eks-cluster
eks/datadog-agent cloudposse-terraform-components/aws-eks-datadog-agent
eks/echo-server cloudposse-terraform-components/aws-eks-echo-server
eks/external-dns cloudposse-terraform-components/aws-eks-external-dns
eks/external-secrets-operator cloudposse-terraform-components/aws-eks-external-secrets-operator
eks/github-actions-runner cloudposse-terraform-components/aws-eks-github-actions-runner
eks/idp-roles cloudposse-terraform-components/aws-eks-idp-roles
eks/karpenter cloudposse-terraform-components/aws-eks-karpenter-controller
eks/karpenter-node-pool cloudposse-terraform-components/aws-eks-karpenter-node-pool
eks/keda cloudposse-terraform-components/aws-eks-keda
eks/loki cloudposse-terraform-components/aws-eks-loki
eks/metrics-server cloudposse-terraform-components/aws-eks-metrics-server
eks/prometheus-scraper cloudposse-terraform-components/aws-eks-prometheus-scraper
eks/promtail cloudposse-terraform-components/aws-eks-promtail
eks/redis cloudposse-terraform-components/aws-eks-redis
eks/redis-operator cloudposse-terraform-components/aws-eks-redis-operator
eks/reloader cloudposse-terraform-components/aws-eks-reloader
eks/storage-class cloudposse-terraform-components/aws-eks-storage-class
eks/spacelift-worker-pool-controller cloudposse-terraform-components/aws-eks-spacelift-worker-pool-controller
eks/spacelift-worker-pool cloudposse-terraform-components/aws-eks-spacelift-worker-pool
eks/tailscale cloudposse-terraform-components/aws-eks-tailscale
elasticache-redis cloudposse-terraform-components/aws-elasticache-redis
elasticsearch cloudposse-terraform-components/aws-elasticsearch
eventbridge cloudposse-terraform-components/aws-eventbridge
github-action-token-rotator cloudposse-terraform-components/aws-github-action-token-rotator
github-oidc-provider cloudposse-terraform-components/aws-github-oidc-provider
github-oidc-role cloudposse-terraform-components/aws-github-oidc-role
github-runners cloudposse-terraform-components/aws-github-runners
github-webhook cloudposse-terraform-components/aws-github-webhook
global-accelerator cloudposse-terraform-components/aws-global-accelerator
global-accelerator-endpoint-group cloudposse-terraform-components/aws-global-accelerator-endpoint-group
glue/catalog-database cloudposse-terraform-components/aws-glue-catalog-database
glue/catalog-table cloudposse-terraform-components/aws-glue-catalog-table
glue/connection cloudposse-terraform-components/aws-glue-connection
glue/crawler cloudposse-terraform-components/aws-glue-crawler
glue/iam cloudposse-terraform-components/aws-glue-iam
glue/job cloudposse-terraform-components/aws-glue-job
glue/registry cloudposse-terraform-components/aws-glue-registry
glue/schema cloudposse-terraform-components/aws-glue-schema
glue/trigger cloudposse-terraform-components/aws-glue-trigger
glue/workflow cloudposse-terraform-components/aws-glue-workflow
guardduty cloudposse-terraform-components/aws-guardduty
iam-role cloudposse-terraform-components/aws-iam-role
iam-service-linked-roles cloudposse-terraform-components/aws-iam-service-linked-roles
ipam cloudposse-terraform-components/aws-ipam
kinesis-stream cloudposse-terraform-components/aws-kinesis-stream
kms cloudposse-terraform-components/aws-kms
lakeformation cloudposse-terraform-components/aws-lakeformation
lambda cloudposse-terraform-components/aws-lambda
macie cloudposse-terraform-components/aws-macie
managed-grafana/api-key cloudposse-terraform-components/aws-managed-grafana-api-key
managed-grafana/dashboard cloudposse-terraform-components/aws-managed-grafana-dashboard
managed-grafana/data-source/loki cloudposse-terraform-components/aws-managed-grafana-data-source-loki
managed-grafana/data-source/managed-prometheus cloudposse-terraform-components/aws-managed-grafana-data-source-managed-prometheus
managed-grafana/workspace cloudposse-terraform-components/aws-managed-grafana-workspace
managed-prometheus/workspace cloudposse-terraform-components/aws-managed-prometheus-workspace
memorydb cloudposse-terraform-components/aws-memorydb
mq-broker cloudposse-terraform-components/aws-mq-broker
msk cloudposse-terraform-components/aws-msk
mwaa cloudposse-terraform-components/aws-mwaa
network-firewall cloudposse-terraform-components/aws-network-firewall
opsgenie-team cloudposse-terraform-components/aws-opsgenie-team
philips-labs-github-runners cloudposse-terraform-components/aws-philips-labs-github-runners
rds cloudposse-terraform-components/aws-rds
redshift cloudposse-terraform-components/aws-redshift
redshift-serverless cloudposse-terraform-components/aws-redshift-serverless
route53-resolver-dns-firewall cloudposse-terraform-components/aws-route53-resolver-dns-firewall
runs-on cloudposse-terraform-components/aws-runs-on
s3-bucket cloudposse-terraform-components/aws-s3-bucket
security-hub cloudposse-terraform-components/aws-security-hub
ses cloudposse-terraform-components/aws-ses
sftp cloudposse-terraform-components/aws-sftp
site-to-site-vpn cloudposse-terraform-components/aws-site-to-site-vpn
snowflake-account cloudposse-terraform-components/aws-snowflake-account
snowflake-database cloudposse-terraform-components/aws-snowflake-database
sns-topic cloudposse-terraform-components/aws-sns-topic
spa-s3-cloudfront cloudposse-terraform-components/aws-spa-s3-cloudfront
spacelift/admin-stack cloudposse-terraform-components/aws-spacelift-admin-stack
spacelift/spaces cloudposse-terraform-components/aws-spacelift-spaces
spacelift/worker-pool cloudposse-terraform-components/aws-spacelift-worker-pool-asg
sqs-queue cloudposse-terraform-components/aws-sqs-queue
ssm-parameters cloudposse-terraform-components/aws-ssm-parameters
sso-saml-provider cloudposse-terraform-components/aws-sso-saml-provider
strongdm cloudposse-terraform-components/aws-strongdm
tfstate-backend cloudposse-terraform-components/aws-tfstate-backend
tgw/cross-region-hub-connector cloudposse-terraform-components/aws-tgw-hub-connector
tgw/hub cloudposse-terraform-components/aws-tgw-hub
tgw/spoke cloudposse-terraform-components/aws-tgw-spoke
vpc cloudposse-terraform-components/aws-vpc
vpc-flow-logs-bucket cloudposse-terraform-components/aws-vpc-flow-logs-bucket
vpc-peering cloudposse-terraform-components/aws-vpc-peering
waf cloudposse-terraform-components/aws-waf
zscaler cloudposse-terraform-components/aws-zscaler

@goruha goruha changed the title 📢 Migration of Components to a New GitHub Organization (CODE FREEZE 11/12 - 11/17) 📢 Migration of Components to a New GitHub Organization complete! Nov 19, 2024
@nitrocode
Copy link
Member

nitrocode commented Nov 21, 2024

@osterman @goruha One thing I noticed is that the git history for each component was lost. It's probably too late now to see if that can be retained, but I wanted to point it out in case others were looking for older history, then it would be good to archive terraform-aws-components repo for posterity.

Edit: oh right just reviewed the post again and saw this

Future Archiving: In approximately six months, we plan to archive this repository and transfer it to the cloudposse-archives organization

@goruha
Copy link
Member

goruha commented Nov 27, 2024

@nitrocode I re-run the migration script and copied the git history.
It was impossible to preserve commit sha, but we got history, references to PRs, and commit authorship .

You can check this repo as an example https://github.com/cloudposse-terraform-components/aws-bastion/commits/main/

goruha added a commit to cloudposse/github-action-atmos-component-updater that referenced this issue Nov 27, 2024
## what
* Automatically migrate existing components to the new GitHub organization

## why
* Makes migration to new component structure simple and stable

## Refs
* cloudposse/terraform-aws-components#1177
@nitrocode
Copy link
Member

Awesome, I see the history now. Thank you!

@nitrocode
Copy link
Member

Last nitpick, I see the references in the readme for each component often link back to this repo which is soon to be archived.

E.g. https://github.com/cloudposse-terraform-components/aws-iam-role?tab=readme-ov-file#references

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants