Supported the atmos.d, allowing automatic inclusion of configuration files from the atmos.d directory #808
Conversation
haitham911
requested review from
johncblandii,
RoseSecurity,
aknysh and
osterman
📝 WalkthroughWalkthroughThe pull request introduces comprehensive enhancements to the Atmos CLI configuration management system. The changes focus on improving configuration loading, supporting flexible import mechanisms, and establishing more robust environment variable handling. Key modifications include expanding configuration discovery methods, adding support for the Changes
Assessment against linked issues
Possibly related PRs
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🧹 Outside diff range and nitpick comments (7)
examples/demo-atmos.d/atmos.yaml (3)
4-11: Document security-sensitive settingsCritical settings like
apply_auto_approveandauto_generate_backend_filewould benefit from inline documentation explaining their security implications.Consider adding comments:
components: terraform: base_path: "components/terraform" + # Disable auto-approve for safety in production environments apply_auto_approve: false deploy_run_init: true init_run_reconfigure: true + # Disable auto-generation of backend files to prevent unauthorized state access auto_generate_backend_file: false
12-19: Consider expanding name pattern flexibilityThe current name pattern only supports {stage}. Consider supporting additional variables for better organization.
Consider expanding:
- name_pattern: "{stage}" + name_pattern: "{org}-{stage}-{component}"
20-29: Clean up formatting and enhance documentationThe vendor configuration contains helpful examples but needs formatting cleanup.
Apply these changes:
-vendor: +vendor: # Single file - base_path: "./vendor.yaml" + base_path: "./vendor.yaml" # Default configuration # Directory with multiple files #base_path: "./vendor" # Absolute path #base_path: "vendor.d/vendor1.yaml"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 20-20: trailing spaces
(trailing-spaces)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 26-26: trailing spaces
(trailing-spaces)
examples/demo-atmos.d/custom-import/atmos.yaml (2)
26-35: Consider moving configuration examples to documentationWhile the commented examples are helpful, they might be better suited in the documentation, keeping the configuration file cleaner.
- # Single file base_path: "./vendor.yaml" - - # Directory with multiple files - #base_path: "./vendor" - - # Absolute path - #base_path: "vendor.d/vendor1.yaml"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
36-38: Consider environment-specific log configurationsThe current logging setup is good for development, but you might want to add environment-specific configurations for production use.
Example addition:
logs: file: "${ATMOS_LOG_FILE:-/dev/stderr}" level: "${ATMOS_LOG_LEVEL:-Info}"pkg/schema/schema.go (1)
29-29: Add field documentation.Consider adding a documentation comment to describe the purpose and usage of the
Importfield, following Go's documentation conventions.Add this documentation above the field:
+ // Import specifies a list of paths from which to import additional configurations. + // Supports local files, directories (using glob patterns), and remote URLs. Import []string `yaml:"import" json:"import" mapstructure:"import"`pkg/config/config.go (1)
231-232: Combine the return statement for clarityThe return statement is unnecessarily split across two lines.
Consider writing it on a single line:
-return cliConfig, - err +return cliConfig, err
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (10)
examples/demo-atmos.d/atmos.d/configs.d/config1.yaml(1 hunks)examples/demo-atmos.d/atmos.d/configs.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/atmos.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/config1.yaml(1 hunks)examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/custom-import/extra-config.yaml(1 hunks)pkg/config/config.go(3 hunks)pkg/schema/schema.go(1 hunks)pkg/utils/glob_utils.go(2 hunks)
✅ Files skipped from review due to trivial changes (5)
- examples/demo-atmos.d/atmos.d/configs.d/config1.yaml
- examples/demo-atmos.d/atmos.d/configs.d/sub/config2.yaml
- examples/demo-atmos.d/custom-import/configs.d/config1.yaml
- examples/demo-atmos.d/custom-import/configs.d/sub/config2.yaml
- examples/demo-atmos.d/custom-import/extra-config.yaml
🧰 Additional context used
🪛 yamllint (1.29.0-1)
examples/demo-atmos.d/atmos.yaml
[error] 20-20: trailing spaces
(trailing-spaces)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 26-26: trailing spaces
(trailing-spaces)
[warning] 38-38: wrong indentation: expected 2 but found 0
(indentation)
[warning] 41-41: wrong indentation: expected 4 but found 2
(indentation)
examples/demo-atmos.d/custom-import/atmos.yaml
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
[warning] 44-44: wrong indentation: expected 2 but found 0
(indentation)
[warning] 47-47: wrong indentation: expected 4 but found 2
(indentation)
🔇 Additional comments (6)
examples/demo-atmos.d/atmos.yaml (1)
30-33: Add validation for log levels
Consider adding validation to ensure only valid log levels (Debug, Info, Warning, Error) are accepted.
examples/demo-atmos.d/custom-import/atmos.yaml (3)
7-9: Consider adding URL validation for remote configurations
While loading from remote URLs provides flexibility, it could pose security risks. Consider implementing URL validation and HTTPS enforcement.
Would you like me to propose a validation pattern for secure URL handling?
18-24: Strong stack configuration, warrior! 💪
The stack configuration is well-structured with clear inclusion/exclusion patterns and a simple naming convention.
11-16: Verify backend file management strategy
Setting auto_generate_backend_file: false means you'll need to manage backend configurations manually. This is a valid choice but requires careful consideration.
Run this check to ensure consistent backend configuration across components:
✅ Verification successful
Setting auto_generate_backend_file: false is appropriate for this setup
The configuration choice is valid as there's no existing Terraform component directory at the specified path, indicating this is likely a new project setup. Manual backend management is a reasonable approach when starting fresh, allowing for more controlled and explicit backend configuration as the project evolves.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check for existing backend configurations
fd -e tf -e hcl . components/terraform -x grep -l 'backend'
Length of output: 167
Script:
#!/bin/bash
# First, let's verify if the components/terraform directory exists and check the directory structure
fd --type d terraform
# Then, let's search for backend configurations across all Terraform files in the repository
fd -e tf -e hcl . -x grep -l 'backend'
# Also check for any Terraform component references in the atmos configuration
fd -g "*.yaml" -x grep -A 5 -B 5 "terraform:"
Length of output: 33961
Script:
#!/bin/bash
# Let's check for backend configurations in the actual Terraform components
fd -t f -e tf . examples/demo-atmos.d/custom-import/components/terraform -x grep -l 'backend'
# Also check for any backend configuration files
fd -t f -g "*.backend.tf" -g "backend.tf" examples/demo-atmos.d/custom-import/components/terraform
# Check if there are any existing backend configurations in the component directory
ls -la examples/demo-atmos.d/custom-import/components/terraform/
Length of output: 709
pkg/utils/glob_utils.go (2)
7-7: LGTM! Clean import addition, warrior!
The sort package import is well-placed and necessary for the new functionality.
Line range hint 19-48: Verify caching behavior for dynamic configurations, brave one!
The caching mechanism using sync.Map could prevent picking up new configuration files added to atmos.d during runtime. Additionally, we should verify that the error handling aligns with the requirement to log non-existent paths as warnings.
Let's verify the implementation:
✅ Verification successful
Cache invalidation is not a concern, brave warrior!
The caching mechanism in GetGlobMatches is pattern-based, which means:
- Each unique glob pattern gets its own cache entry
- When a new file is added that matches an existing pattern, it will be discovered on the next process restart
- The cache is process-scoped (using
sync.Map), so it's automatically cleared between runs - Error handling correctly returns both the error and logs the non-existent paths
The implementation aligns perfectly with the requirements:
- Cache is scoped to the current process, preventing stale state between runs
- Error handling returns descriptive errors for non-existent paths:
failed to find a match for the import '%s' ('%s' + '%s')
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for warning logs related to non-existent paths and cache invalidation
# Search for warning log implementations
rg -A 2 "failed to find a match for the import" .
# Check if there's any cache invalidation mechanism
rg -A 5 "getGlobMatchesSyncMap" .
# Look for any warning level log statements
rg -A 2 "Warning|WARN|warn" .
Length of output: 47400
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (2)
pkg/config/config.go(3 hunks)pkg/schema/schema.go(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- pkg/schema/schema.go
🔇 Additional comments (1)
pkg/config/config.go (1)
231-232: LGTM! Clean error handling.
The multi-line error return improves readability.
…correct command structure
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (3)
examples/demo-atmos.d/custom-import/atmos.yaml (2)
26-34: Clean up trailing spacesRemove trailing spaces from lines 26, 29, and 32 for better maintainability.
-vendor: +vendor: # Single file -base_path: "./vendor.yaml" +base_path: "./vendor.yaml" # Directory with multiple files -#base_path: "./vendor" +#base_path: "./vendor"🧰 Tools
🪛 yamllint (1.29.0-1)
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
36-38: Logging configuration is good but could be enhancedCurrent configuration is solid. Consider adding:
- Log rotation settings
- Format specification
- Additional log destinations for production environments
pkg/config/config.go (1)
401-444: Consider adding rate limiting and path validationThe import processing is robust, but consider these enhancements:
- Add rate limiting for remote downloads to prevent abuse
- Validate import paths before processing to ensure they're safe
func processImports(cliConfig schema.CliConfiguration, v *viper.Viper) error { + // Add rate limiting for remote downloads + rateLimiter := time.NewTicker(time.Second) + defer rateLimiter.Stop() + for _, importPath := range cliConfig.Import { if importPath == "" { continue } + // Basic path validation + if strings.Contains(importPath, "..") { + u.LogWarning(cliConfig, fmt.Sprintf("Warning: skipping potentially unsafe import path '%s'", importPath)) + continue + }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (4)
examples/demo-atmos.d/atmos.d/config1.yaml(1 hunks)examples/demo-atmos.d/atmos.d/sub/config2.yaml(1 hunks)examples/demo-atmos.d/custom-import/atmos.yaml(1 hunks)pkg/config/config.go(3 hunks)
✅ Files skipped from review due to trivial changes (2)
- examples/demo-atmos.d/atmos.d/config1.yaml
- examples/demo-atmos.d/atmos.d/sub/config2.yaml
🧰 Additional context used
🪛 yamllint (1.29.0-1)
examples/demo-atmos.d/custom-import/atmos.yaml
[error] 26-26: trailing spaces
(trailing-spaces)
[error] 29-29: trailing spaces
(trailing-spaces)
[error] 32-32: trailing spaces
(trailing-spaces)
🔇 Additional comments (7)
examples/demo-atmos.d/custom-import/atmos.yaml (4)
11-16: Terraform security settings look good!
The Terraform component configuration follows security best practices:
apply_auto_approve: falseprevents unintended infrastructure changesauto_generate_backend_file: falseensures explicit backend configuration
18-24: Stack configuration structure is well-organized!
Good use of:
- Explicit path inclusions/exclusions
- Clear naming pattern
- Logical directory structure
43-47: Command configuration looks good!
The test command is well-defined with clear description and steps.
7-9: Verify security measures for remote configuration imports
The remote URL import capability could pose security risks. Ensure that:
- HTTPS is enforced for remote URLs
- URL validation is implemented
- Content verification is performed before applying configurations
Consider implementing:
- URL allowlist/blocklist
- SSL certificate validation
- Content integrity checks (e.g., checksums)
pkg/config/config.go (3)
231-232: Clean error handling format!
The multi-line error return improves readability while maintaining functionality.
239-250: Validate path to prevent directory traversal
While the implementation correctly handles the atmos.d directory existence, consider adding path validation as suggested in the past review to prevent potential path traversal attacks.
446-464: Add safeguards to remote downloads
The download function needs the safeguards mentioned in the past review:
- Context timeout
- File size limits
- Temporary directory cleanup
…onfig and processImports
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (4)
pkg/config/config.go (4)
231-232: Consider using idiomatic Go error handling.While splitting the return statement improves readability, consider using the more idiomatic Go style:
- return cliConfig, - err + return cliConfig, fmt.Errorf("failed to unmarshal config: %w", err)
239-256: Strong implementation with robust security measures!The atmos.d directory support is well-implemented with proper path validation and error handling. The path traversal prevention is particularly noteworthy.
Consider adding more context to the error message:
- return cliConfig, fmt.Errorf("invalid atmos.d path: attempted directory traversal") + return cliConfig, fmt.Errorf("invalid atmos.d path '%s': attempted directory traversal outside base path '%s'", atmosDPath, basePath)
257-269: Consider adding debug logging for configuration merging.The import processing logic is sound, but adding debug logs would help troubleshoot configuration issues.
err = v.Unmarshal(&cliConfig) if err != nil { + u.LogTrace(cliConfig, fmt.Sprintf("Failed to unmarshal merged config: %v", err)) return cliConfig, err } + u.LogTrace(cliConfig, "Successfully merged and unmarshaled imported configurations")
407-459: Consider parallel processing for multiple imports.The implementation is robust with proper error handling and path validation. However, processing imports sequentially might be slow for many imports.
Consider using a worker pool for parallel processing:
func processImports(cliConfig schema.CliConfiguration, v *viper.Viper) error { + type importResult struct { + path string + err error + } + workers := make(chan struct{}, 4) // Limit concurrent downloads + results := make(chan importResult) + for _, importPath := range cliConfig.Import { - if importPath == "" { - continue - } + go func(imp string) { + workers <- struct{}{} // Acquire worker + defer func() { <-workers }() // Release worker + + if imp == "" { + results <- importResult{"", nil} + return + } + // ... rest of the import processing logic ... + }(importPath) }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
pkg/config/config.go(3 hunks)
🔇 Additional comments (1)
pkg/config/config.go (1)
461-479: Past review comment about remote download safeguards is still applicable.
The previous review comment suggesting context timeout, file size limits, and temporary directory cleanup remains valid.
…andle cleanup on error
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Outside diff range and nitpick comments (1)
pkg/config/config.go (1)
232-233: Combine the return statement into a single lineThe return statement is split across two lines. Merging it into one line enhances readability.
Apply this diff:
- return cliConfig, - err + return cliConfig, err
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
pkg/config/config.go(3 hunks)
🔇 Additional comments (4)
pkg/config/config.go (4)
240-257: Good handling of the 'import' key and path security
The logic for setting the default Import path when it's empty is well-implemented. Ensuring the path doesn't escape the intended directory adds a valuable security check.
259-270: Import processing added effectively
The addition of import processing and re-unmarshalling is correctly handled. Error checks are appropriately placed.
417-426: Consider validating remote URLs for security
Currently, any URL starting with "http://" or "https://" is accepted. Validating these URLs against an allowlist can prevent unintended or malicious imports.
464-484: Add a file size limit check in downloadRemoteConfig
While the context timeout is set, the function lacks a file size limit check. This could allow the download of excessively large files, leading to resource exhaustion.
|
Update documentation for Atmos CLI Configuration schema and examples |
|
Update documentation for Atmos CLI Configuration |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (4)
website/docs/cli/configuration/configuration.mdx (2)
31-31: Remove unnecessary comma before "if".The comma before "if" in this sentence is grammatically incorrect.
-When the `base_path` is explicitly defined, it will always be resolved to the absolute path relative to the current working directory, if an absolute path was not already set. +When the `base_path` is explicitly defined, it will always be resolved to the absolute path relative to the current working directory if an absolute path was not already set.🧰 Tools
🪛 LanguageTool
[typographical] ~31-~31: Usually, there’s no comma before “if”.
Context: ...elative to the current working directory, if an absolute path was not already set. ...(IF_NO_COMMA)
44-45: Consider clarifying "deep-merged" behavior.While technically accurate, it would be helpful to briefly explain what deep merging means in this context, especially for users who might be unfamiliar with the term.
Consider adding a brief explanation:
-The Atmos (CLI) configuration is loaded from the following locations, in order of precedence, and deep-merged: configurations listed earlier have higher priority +The Atmos (CLI) configuration is loaded from the following locations, in order of precedence, and deep-merged (recursively combined at all levels): configurations listed earlier have higher priority🧰 Tools
🪛 LanguageTool
[misspelling] ~45-~45: The word “deep” is an adjective and doesn’t fit in this context. Did you mean the adverb “deeply”?
Context: ...gle command; the configurations will be deep merged in the order you provide them. T...(ADJECTIVE_ADVERB)
examples/demo-env/stacks/catalog/station.yaml (1)
3-8: Component structure is well-formed, consider adding inline documentation.The component structure follows the Atmos manifest schema correctly. Since this is in the examples directory and serves as a demonstration for the new atmos.d feature, consider adding inline comments to explain the purpose of each section.
components: terraform: example: + # Metadata section defines component-specific attributes metadata: component: test + # Variables section for component configuration (empty for demo purposes) vars: {}examples/demo-env/stacks/deploy/staging.yaml (1)
1-12: STRONG CONFIGURATION! Consider template optimization.The staging configuration mirrors dev.yaml perfectly, with the appropriate stage value. However, consider implementing a base template to reduce configuration duplication across environments.
Example approach using a base template:
# base.yaml # yaml-language-server: $schema=https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json import: - catalog/example components: terraform: example: vars: {} # Then in environment files: # staging.yaml vars: stage: staging import: - ./base.yaml
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
examples/demo-env/stacks/catalog/station.yaml(1 hunks)examples/demo-env/stacks/deploy/dev.yaml(1 hunks)examples/demo-env/stacks/deploy/prod.yaml(1 hunks)examples/demo-env/stacks/deploy/staging.yaml(1 hunks)website/docs/cli/configuration/configuration.mdx(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- examples/demo-env/stacks/deploy/prod.yaml
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/configuration/configuration.mdx
[grammar] ~27-~27: Please add a punctuation mark at the end of paragraph.
Context: ...f the components, and so forth. ## Base Path The base path defines the root directo...
(PUNCTUATION_PARAGRAPH_END)
[typographical] ~31-~31: Usually, there’s no comma before “if”.
Context: ...elative to the current working directory, if an absolute path was not already set. ...
(IF_NO_COMMA)
[misspelling] ~45-~45: The word “deep” is an adjective and doesn’t fit in this context. Did you mean the adverb “deeply”?
Context: ...gle command; the configurations will be deep merged in the order you provide them. T...
(ADJECTIVE_ADVERB)
🔇 Additional comments (6)
website/docs/cli/configuration/configuration.mdx (2)
27-41: LGTM! Clear and comprehensive base path documentation.
The base path section effectively explains the configuration options, resolution strategy, and behavior. The precedence order is well-documented and the examples are helpful.
🧰 Tools
🪛 LanguageTool
[grammar] ~27-~27: Please add a punctuation mark at the end of paragraph.
Context: ...f the components, and so forth. ## Base Path The base path defines the root directo...
(PUNCTUATION_PARAGRAPH_END)
[typographical] ~31-~31: Usually, there’s no comma before “if”.
Context: ...elative to the current working directory, if an absolute path was not already set. ...
(IF_NO_COMMA)
75-79: LGTM! Comprehensive security warnings for remote imports.
The security considerations for remote imports are well-documented and provide essential guidance for secure usage.
examples/demo-env/stacks/catalog/station.yaml (1)
1-2: LGTM! Schema declaration looks solid.
The schema reference ensures proper validation and IDE support. Let's verify the schema URL is accessible.
✅ Verification successful
Schema declaration is valid and consistently used across the codebase
The schema URL is valid and returns a proper JSON object. The same schema reference is consistently used across all Atmos manifest files in the codebase, which confirms this is the correct schema to use.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify the schema URL is accessible and valid
curl -s -f -o /dev/null "https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json" || echo "Schema URL is not accessible"
Length of output: 98
Script:
#!/bin/bash
# Let's try to verify the schema content by downloading and checking its format
curl -s "https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json" | jq -r 'type'
# Let's also check if there are any other schema references in the codebase for comparison
rg -g '*.{yaml,yml}' '^\s*#.*\$schema.*atmos' -A 1
Length of output: 33022
examples/demo-env/stacks/deploy/dev.yaml (3)
1-1: STRONG WORK! Schema validation is properly configured.
The schema reference ensures your configuration maintains structural integrity, warrior!
3-4: EXCELLENT! Environment variables are clearly defined.
The stage variable is properly set to "dev", making the environment purpose crystal clear.
9-12: SOLID STRUCTURE! Component definition is clean and precise.
The terraform component follows the recommended format with the empty vars block.
| import: | ||
| - catalog/example |
There was a problem hiding this comment.
💡 Codebase verification
The import statement catalog/example doesn't match the actual catalog structure. The catalog directory contains only station.yaml, but no example.yaml file.
- The import statement in
examples/demo-env/stacks/deploy/dev.yamlreferencescatalog/example - The actual catalog contains only
station.yaml
Consider either:
- Updating the import to
catalog/stationto match the existing file - Creating the missing
example.yamlfile in the catalog directory
🔗 Analysis chain
WELL STRUCTURED! Let's verify the catalog import.
The import statement follows the new atmos.d convention. However, let's verify the catalog structure.
The previous command failed because the catalog directory wasn't found. Let's first check the directory structure and then look for any example configuration files to understand the correct path.
We found multiple catalog directories, but we need to focus on the one in demo-env since that's where our import statement is located. Let's verify its contents.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Verify the existence and structure of the catalog
fd -t f "example.yaml" -p "catalog/"
Length of output: 130
Script:
#!/bin/bash
# Let's try a broader search to understand the catalog structure
fd "catalog" --type directory
fd "example" --type file
Length of output: 649
Script:
#!/bin/bash
# Let's check the content of the catalog directory in demo-env specifically
ls -la examples/demo-env/stacks/catalog/
# And look for any yaml files that might be our target
fd -t f ".yaml$" -p "examples/demo-env/stacks/catalog/"
Length of output: 314
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
website/docs/cli/configuration/configuration.mdx (2)
31-31: Minor grammatical improvement neededRemove the comma before "if" to improve readability.
-When the `base_path` is explicitly defined, it will always be resolved to the absolute path relative to the current working directory, if an absolute path was not already set. +When the `base_path` is explicitly defined, it will always be resolved to the absolute path relative to the current working directory if an absolute path was not already set.🧰 Tools
🪛 LanguageTool
[typographical] ~31-~31: Usually, there’s no comma before “if”.
Context: ...elative to the current working directory, if an absolute path was not already set. ...(IF_NO_COMMA)
45-45: Improve clarity in merge descriptionConsider using "deeply merged" instead of "deep merged" for better grammatical accuracy.
-the configurations will be deep merged in the order you provide them +the configurations will be deeply merged in the order you provide them🧰 Tools
🪛 LanguageTool
[misspelling] ~45-~45: The word “deep” is an adjective and doesn’t fit in this context. Did you mean the adverb “deeply”?
Context: ...gle command; the configurations will be deep merged in the order you provide them. T...(ADJECTIVE_ADVERB)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
examples/demo-env/components/terraform/test/main.tf(1 hunks)website/docs/cli/configuration/configuration.mdx(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- examples/demo-env/components/terraform/test/main.tf
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/configuration/configuration.mdx
[typographical] ~31-~31: Usually, there’s no comma before “if”.
Context: ...elative to the current working directory, if an absolute path was not already set. ...
(IF_NO_COMMA)
[misspelling] ~45-~45: The word “deep” is an adjective and doesn’t fit in this context. Did you mean the adverb “deeply”?
Context: ...gle command; the configurations will be deep merged in the order you provide them. T...
(ADJECTIVE_ADVERB)
🔇 Additional comments (1)
website/docs/cli/configuration/configuration.mdx (1)
59-81: Well-structured import documentation with security considerations
The import functionality is well documented with clear examples and important security warnings for remote imports. The section effectively covers:
- Different types of imports
- Configuration examples
- Security best practices
|
Important Cloud Posse Engineering Team Review RequiredThis pull request modifies files that require Cloud Posse's review. Please be patient, and a core maintainer will review your changes. To expedite this process, reach out to us on Slack in the |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (8)
examples/demo-env/components/terraform/example/main.tf (1)
1-4: STRONG WORK, WARRIOR! Your environment variable configuration stands ready for battle!The data source is well-structured and the filter pattern correctly captures all ATMOS-prefixed variables. Consider enhancing the comment to list the specific variables being captured (ATMOS_CLI_CONFIG_PATH, ATMOS_BASE_PATH) for better documentation.
# Fetch the required environment variables using the `environment_variables` data source data "environment_variables" "required" { - filter = "ATMOS_.*" # Fetches all variables starting with "ATMOS_" + filter = "ATMOS_.*" # Fetches ATMOS_CLI_CONFIG_PATH, ATMOS_BASE_PATH, and other ATMOS_ variables }examples/demo-configuration/atmos.d/tools/stack.yaml (2)
3-6: Path patterns follow glob syntax best practices.The inclusion/exclusion rules are well structured:
deploy/**/*captures all files under the deploy directory recursively**/_defaults.yamlproperly excludes default configuration filesConsider documenting these patterns in the project's README to help other team members understand the file organization strategy.
Would you like me to help draft documentation for these path patterns?
1-7: Strong foundation for atmos.d directory convention implementation.The configuration structure effectively implements the atmos.d directory convention mentioned in the PR objectives. The combination of:
- Clear base path definition
- Flexible path inclusion/exclusion rules
- Environment-aware naming patterns
Creates a robust foundation for managing stack configurations. This approach will scale well as the number of configuration files grows.
examples/demo-configuration/atmos.d/commands.yaml (1)
3-4: Consider a more specific command name and description.The current command name "test" is quite generic. Consider making it more specific to its purpose, such as "describe-config-test" or "validate-config". Additionally, the description could better explain what configuration aspects are being tested.
- name: "test" - description: "Run all tests with custom command" + name: "describe-config-test" + description: "Validate and display the current Atmos configuration"examples/demo-configuration/atmos.d/tools/helmfile.yml (1)
16-17: Remove extra blank lines.Consider removing the extra blank lines at the end of the file.
examples/demo-configuration/atmos.yaml (2)
1-2: Enhance documentation to reflect new features.The current description could be more comprehensive. Consider adding details about:
- The atmos.d directory convention
- Configuration merging behavior
- Supported file patterns
-# Description: Configuration file for the Atmos CLI -# default import path is ./atmos.d import .yaml files from the directory and merge them +# Description: Configuration file for the Atmos CLI +# +# Features: +# - Automatically imports and merges .yaml files from ./atmos.d directory +# - Processes files in lexicographical order +# - Supports glob patterns for file matching +# - Enables deep merging of configurations
3-3: Document the purpose of base_path.The
base_pathsetting would benefit from a comment explaining its role in configuration resolution.+# Base path for resolving relative paths in configurations base_path: "./"examples/demo-configuration/atmos.d/tools/terraform.yaml (1)
13-15: Enhance the path documentation, noble scribe! 📜Consider adding examples for both absolute and relative paths, similar to the command examples above.
# Can also be set using 'ATMOS_COMPONENTS_TERRAFORM_BASE_PATH' ENV var, or '--terraform-dir' command-line argument # Supports both absolute and relative paths + # Examples: + # base_path: "/absolute/path/to/components/terraform" + # base_path: "../../components/terraform" base_path: "components/terraform"
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
.github/workflows/test.yml(1 hunks)examples/demo-configuration/atmos.d/commands.yaml(1 hunks)examples/demo-configuration/atmos.d/logs.yaml(1 hunks)examples/demo-configuration/atmos.d/tools/helmfile.yml(1 hunks)examples/demo-configuration/atmos.d/tools/stack.yaml(1 hunks)examples/demo-configuration/atmos.d/tools/terraform.yaml(1 hunks)examples/demo-configuration/atmos.yaml(1 hunks)examples/demo-env/components/terraform/example/main.tf(1 hunks)examples/demo-env/components/terraform/example/outputs.tf(1 hunks)examples/demo-env/stacks/catalog/example.yaml(1 hunks)website/docs/cli/configuration/configuration.mdx(4 hunks)
✅ Files skipped from review due to trivial changes (1)
- examples/demo-env/stacks/catalog/example.yaml
👮 Files not reviewed due to content moderation or server errors (3)
- examples/demo-configuration/atmos.d/logs.yaml
- .github/workflows/test.yml
- website/docs/cli/configuration/configuration.mdx
🧰 Additional context used
📓 Learnings (1)
examples/demo-configuration/atmos.d/tools/helmfile.yml (1)
Learnt from: osterman
PR: cloudposse/atmos#808
File: examples/demo-atmos.d/atmos.d/tools/helmfile.yml:10-10
Timestamp: 2024-12-12T15:17:45.245Z
Learning: In `examples/demo-atmos.d/atmos.d/tools/helmfile.yml`, when suggesting changes to `kubeconfig_path`, ensure that the values use valid Go template syntax.
🪛 LanguageTool
website/docs/cli/configuration/configuration.mdx
[style] ~22-~22: A comma is missing here.
Context: ...vior) you'd specify the command to run (e.g. [opentofu](/core-concepts/projects/co...
(EG_NO_COMMA)
[style] ~119-~119: This word has been used in one of the immediately preceding sentences. Using a synonym could make your text more interesting to read, unless the repetition is intentional.
Context: ...### Stage 3: Apply User Preferences :::important If the --config flag is used, this st...
(EN_REPEATEDWORDS_IMPORTANT)
[grammar] ~181-~181: Did you mean “is” or “can be”?
Context: ... supported (unlike stacks). :::warning Be Careful with Remote Imports - Always us...
(BE_IS)
[style] ~818-~818: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...
(IN_ORDER_TO_PREMIUM)
🪛 yamllint (1.35.1)
examples/demo-configuration/atmos.yaml
[error] 5-5: trailing spaces
(trailing-spaces)
🔇 Additional comments (12)
examples/demo-env/components/terraform/example/outputs.tf (2)
2-10: EXCELLENT FORMATION, STRATEGIST! Your ATMOS outputs are battle-ready!
The outputs for atmos_cli_config_path and atmos_base_path are well-structured with clear descriptions and proper data source references.
12-15: HOLD THE LINE! Let's verify the EXAMPLE variable's existence!
The EXAMPLE environment variable is referenced but not mentioned in the PR objectives. We should verify its existence and purpose.
Let's search for its definition in the codebase:
✅ Verification successful
STAND EASY, WARRIOR! The EXAMPLE variable is well-defined!
The EXAMPLE environment variable is properly defined in examples/demo-env/stacks/catalog/example.yaml with a value of "test". This is a demo environment setup, and the variable is being used appropriately in the terraform output.
- Variable definition:
examples/demo-env/stacks/catalog/example.yaml - Variable usage:
examples/demo-env/components/terraform/example/outputs.tf
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for EXAMPLE variable definition in yaml files
rg -l 'EXAMPLE:' --type yaml
# Search for any reference to this variable in terraform files
rg 'EXAMPLE' --type terraform
Length of output: 138
Script:
#!/bin/bash
# Let's check the content of the example.yaml file
cat examples/demo-env/stacks/catalog/example.yaml
# Search for EXAMPLE using ripgrep with .tf extension
rg 'EXAMPLE' -g '*.tf'
Length of output: 426
examples/demo-configuration/atmos.d/tools/stack.yaml (2)
1-2: Base path configuration looks solid!
The base_path: "stacks" setting establishes a clear root directory for stack definitions, which aligns well with the conventional structure for Atmos projects.
7-7: Verify template variable resolution for name pattern.
The name_pattern: "{dev}" suggests environment-based templating. Let's ensure this pattern is properly documented and verify its resolution behavior.
Run this script to check for documentation and usage patterns:
examples/demo-configuration/atmos.d/commands.yaml (1)
5-6: Verify the command execution.
The step executes atmos describe config. Let's verify this command exists and is properly documented.
✅ Verification successful
The atmos describe config command is properly implemented and documented
Based on the codebase analysis, the command is valid and properly implemented:
- The command is registered in
cmd/describe_config.gowith proper command definition and execution logic - It's implemented in
internal/exec/describe_config.gowithExecuteDescribeConfigCmdfunction - The command is used in multiple example configurations, including
examples/demo-configuration/atmos.d/commands.yaml - It's part of the core
describecommand group that shows configuration for CLI, stacks and components
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify the existence and documentation of the 'atmos describe config' command
# Search for command documentation
rg -A 5 "describe config" "docs/"
# Search for command implementation
ast-grep --pattern 'describe config'
Length of output: 72
Script:
#!/bin/bash
# Let's try a broader search for 'describe' command and its variations
# Search for describe command in yaml files
rg -t yaml "describe" -A 5
# Search for describe command in go files
rg -t go "describe" -A 5
# Look for command definitions or registrations
ast-grep --pattern 'Command{
$$$
Use: "describe"
$$$
}'
# Search for any atmos command definitions
rg "atmos.*describe" -A 5
Length of output: 69253
examples/demo-configuration/atmos.d/tools/helmfile.yml (4)
1-5: LGTM! Clear and well-documented base configuration.
The base configuration is well-structured with good documentation about override options. The relative path approach for base_path promotes portability.
9-10: Review security implications and template syntax.
Two considerations for the kubeconfig_path:
- Using
/dev/shmfor kubeconfig storage might expose sensitive data in shared memory. - Per previous feedback, ensure the path supports Go template syntax for dynamic configuration.
Consider using a more secure default location or documenting the security implications of using shared memory.
11-14: Verify pattern variable resolution.
The naming patterns use placeholders that need to be resolved at runtime. Let's verify the pattern resolution logic exists.
✅ Verification successful
The previous pattern didn't yield results, so I'm broadening the search to:
- Find any Go functions handling patterns
- Find references to these specific pattern variables
- Look for code handling similar template substitutions
This will help us verify if and how these patterns are processed.
I found references to ReplaceContextTokens in the previous search, let's examine its implementation and any validation logic to ensure these patterns are properly handled.
Pattern resolution is properly implemented
The codebase has a robust implementation for resolving the pattern variables in pkg/config/utils.go. The ReplaceContextTokens function handles all the required placeholders:
- {namespace}
- {tenant}
- {environment}
- {stage}
- And additional tokens like {component}, {region}, etc.
The patterns in your configuration will be correctly processed using this implementation.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for pattern resolution implementation
ast-grep --pattern 'func $_(pattern string) {
$$$
{namespace}
$$$
}'
Length of output: 76
Script:
#!/bin/bash
# Let's search for pattern resolution in Go files
rg -t go "func.*pattern.*string" -A 5
# Also search for any references to these specific patterns
rg "helm_aws_profile_pattern|cluster_name_pattern" -A 3
# Look for template/pattern substitution logic
rg "{namespace}.*{tenant}.*{environment}.*{stage}" -A 3
Length of output: 20619
Script:
#!/bin/bash
# Let's look for the pattern substitution implementation in the config package
rg -A 10 "ReplaceContextTokens" pkg/config/
# Also search for any validation of these patterns
rg -A 5 "ValidateConfig.*helmfile"
Length of output: 870
6-8: Verify EKS availability in deployment environment.
The use_eks: true setting assumes EKS availability. Ensure proper error handling exists in the codebase for scenarios where EKS is not available.
✅ Verification successful
EKS configuration is properly handled with fallback options
The codebase shows robust handling of EKS configuration:
- Documentation clearly states that
use_ekscan be disabled via configuration - Default value is actually
falseaccording to the core concepts documentation - When disabled, the system falls back to using
kubeconfig_pathfor cluster authentication - Demo examples demonstrate the flexibility by showcasing
use_eks: falseconfiguration
EKS configuration is properly handled with robust error checking
The codebase implements thorough validation and error handling for EKS configuration:
- When
use_eksis true, the system validates required configurations:- Checks for valid
kubeconfig_path - Validates
helm_aws_profile_pattern - Ensures
cluster_name_patternis provided
- Checks for valid
- Clear error messages guide users when configurations are missing
- Automatic kubeconfig setup via
aws eks update-kubeconfigwhen EKS is enabled - Graceful fallback to standard kubeconfig when EKS is disabled
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for EKS-related error handling in the codebase
rg -A 5 "use_eks.*false|!use_eks"
Length of output: 3418
Script:
#!/bin/bash
# Let's verify if there's proper handling in the code implementation
ast-grep --pattern 'if $COND.use_eks {
$$$
}'
Length of output: 49
Script:
#!/bin/bash
# Let's check the implementation with a broader search pattern
rg -A 5 "\.use_eks" --type go
Length of output: 1187
Script:
#!/bin/bash
# Let's check the helmfile-related code implementation
rg -A 10 "helmfile" --type go | grep -A 10 -B 2 "eks\|kubeconfig"
Length of output: 10753
examples/demo-configuration/atmos.d/tools/terraform.yaml (3)
1-2: Clean component structure, warrior! 🛡️
The YAML structure follows proper hierarchical organization.
16-23: Well-fortified defaults, commander! 🛡️
Excellent security-conscious configuration:
apply_auto_approve: falseprevents accidental deployments- Other defaults promote efficient workflows while maintaining safety
3-12: Verify command path validation, brave one! ⚔️
The configuration allows arbitrary command paths. Let's ensure there's proper validation in place.
✅ Verification successful
Let me gather more specific information about command validation.
Let me gather more specific information about command execution safety.
Command configuration is secure and well-implemented, brave one! 🛡️
The command configuration is properly handled with multiple layers of validation and safety:
- Command execution is centralized through a single
ExecuteShellCommandfunction that provides controlled execution - Command configuration follows a clear precedence order:
- Environment variables (ATMOS_COMPONENTS_TERRAFORM_COMMAND)
- Command-line arguments (--terraform-command)
- Configuration file settings
- The command is executed in a controlled environment with:
- Working directory validation
- Environment variable sanitization
- Proper error handling and logging
The arbitrary command paths are a feature, not a security concern, as they allow for:
- Version-specific terraform binaries
- Alternative implementations (e.g., OpenTofu)
- Custom wrappers for enterprise environments
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for command path validation in the codebase
rg -A 5 "command.*validation|validateCommand|command.*path"
# Look for command execution patterns
ast-grep --pattern 'exec($$$)'
Length of output: 27400
Script:
#!/bin/bash
# Search for terraform command validation
rg -A 10 "terraform.*command.*validate|validateTerraform|terraform.*exec"
# Look for command execution safety checks
rg -A 10 "command.*check|exec.*safe|validate.*path"
# Search for terraform command configuration handling
ast-grep --pattern 'Components.Terraform.Command'
Length of output: 65935
Script:
#!/bin/bash
# Search for command execution implementation
rg -A 10 "ExecuteShellCommand|exec.Command|os.Exec"
# Look for command validation in Go files
rg -A 10 "command.*validate|validateCommand" --type go
# Search for command configuration handling in Go files
rg -A 10 "Components.Terraform.Command.*=|setCommand" --type go
Length of output: 13860
| # Description: Configuration file for the Atmos CLI | ||
| # default import path is ./atmos.d import .yaml files from the directory and merge them | ||
| base_path: "./" | ||
|
|
||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Add configuration examples for atmos.d features.
The file should demonstrate the new features mentioned in the PR objectives, particularly:
- Custom atmos.d path configuration
- Import configurations
- URL imports
Here's a suggested structure:
# Description: Configuration file for the Atmos CLI
#
# Features:
# - Automatically imports and merges .yaml files from ./atmos.d directory
# - Processes files in lexicographical order
# - Supports glob patterns for file matching
# - Enables deep merging of configurations
# Base path for resolving relative paths in configurations
base_path: "./"
# Custom atmos.d directory path (optional)
atmos_d_path: "./atmos.d"
# Additional import configurations (optional)
import:
# Local files and directories
- path: "./configs/*.yaml"
# Remote configurations
- url: "https://example.com/shared-config.yaml"🧰 Tools
🪛 yamllint (1.35.1)
[error] 5-5: trailing spaces
(trailing-spaces)
|
|
||
|
|
There was a problem hiding this comment.
Clean up file structure.
Remove trailing spaces and normalize empty lines.
base_path: "./"
-
- 🧰 Tools
🪛 yamllint (1.35.1)
[error] 5-5: trailing spaces
(trailing-spaces)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
website/docs/cli/configuration/configuration.mdx (5)
23-23: Add missing comma after "e.g."Add a comma after "e.g." for proper punctuation.
-then [this is where](/cli/configuration/components#terraform-component-behavior) you'd specify the command to run (e.g. [`opentofu`](/core-concepts/projects/configuration/opentofu)) +then [this is where](/cli/configuration/components#terraform-component-behavior) you'd specify the command to run (e.g., [`opentofu`](/core-concepts/projects/configuration/opentofu))🧰 Tools
🪛 LanguageTool
[style] ~23-~23: A comma is missing here.
Context: ...vior) you'd specify the command to run (e.g. [opentofu](/core-concepts/projects/co...(EG_NO_COMMA)
31-31: Add security note about configuration file permissionsConsider adding a security note about protecting sensitive configuration files with appropriate file permissions.
Add this note after the paragraph:
+:::warning Security Note +Ensure that configuration files containing sensitive information have appropriate file permissions: +- On Unix-like systems: `chmod 600 atmos.yaml` +- On Windows: Restrict access to authorized users only +:::
151-151: Add note about import cycle detectionConsider adding information about how Atmos handles import cycles to prevent infinite recursion.
Add this note after the introduction paragraph:
+:::note Import Cycles +Atmos automatically detects and prevents import cycles. If a circular dependency is detected in the import chain, Atmos will fail with an error message indicating the cycle path. +:::
195-195: Add examples of absolute and relative base pathsConsider adding concrete examples to illustrate the difference between absolute and relative base paths.
Add these examples after the paragraph:
+For example: +```yaml +# Absolute path +base_path: "/opt/atmos/project" + +# Relative path (resolved relative to current working directory) +base_path: "./project" +```
Line range hint
785-813: Improve table formatting for better readabilityConsider improving the table formatting by:
- Aligning column widths
- Using consistent punctuation in descriptions
- Grouping related variables together
Also, consider adding a note about environment variable precedence:
+:::tip Environment Variable Precedence +When the same configuration is specified in both the YAML file and environment variables, the environment variable takes precedence. This allows for easy overrides without modifying the configuration file. +:::🧰 Tools
🪛 LanguageTool
[style] ~818-~818: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...(IN_ORDER_TO_PREMIUM)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
website/docs/cli/configuration/configuration.mdx(4 hunks)
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/configuration/configuration.mdx
[style] ~23-~23: A comma is missing here.
Context: ...vior) you'd specify the command to run (e.g. [opentofu](/core-concepts/projects/co...
(EG_NO_COMMA)
[style] ~119-~119: This word has been used in one of the immediately preceding sentences. Using a synonym could make your text more interesting to read, unless the repetition is intentional.
Context: ...### Stage 3: Apply User Preferences :::important If the --config flag is used, this st...
(EN_REPEATEDWORDS_IMPORTANT)
[grammar] ~181-~181: Did you mean “is” or “can be”?
Context: ... supported (unlike stacks). :::warning Be Careful with Remote Imports - Always us...
(BE_IS)
[style] ~818-~818: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...
(IN_ORDER_TO_PREMIUM)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
pkg/config/config.go (2)
264-283: Consider using filepath.Clean before path comparison.While the directory traversal check is good, using
filepath.Cleanbefore the prefix check would provide additional safety against path manipulation.-atmosDPath := filepath.Join(basePath, "atmos.d") +atmosDPath := filepath.Clean(filepath.Join(basePath, "atmos.d")) -if !strings.HasPrefix(atmosDPath, basePath) { +cleanBasePath := filepath.Clean(basePath) +if !strings.HasPrefix(atmosDPath, cleanBasePath) {
516-535: Consider adding context cancellation cleanup.While the timeout is good, ensure resources are properly cleaned up on context cancellation.
func downloadRemoteConfig(url string) (string, string, error) { tempDir, err := os.MkdirTemp("", "atmos-import-*") if err != nil { return "", "", err } + // Ensure cleanup on error + var cleanupOnce sync.Once + cleanup := func() { + cleanupOnce.Do(func() { + os.RemoveAll(tempDir) + }) + } + // Cleanup on context cancellation + go func() { + <-ctx.Done() + cleanup() + }() + defer cleanup() tempFile := filepath.Join(tempDir, "atmos.yaml") ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel()website/docs/cli/configuration/configuration.mdx (2)
23-23: Add a comma after "e.g."For better readability and correct punctuation:
-you'd specify the command to run (e.g. [`opentofu`] +you'd specify the command to run (e.g., [`opentofu`]🧰 Tools
🪛 LanguageTool
[style] ~23-~23: A comma is missing here.
Context: ...vior) you'd specify the command to run (e.g. [opentofu](/core-concepts/projects/co...(EG_NO_COMMA)
836-838: Simplify the wording for better clarity.The current wording is a bit verbose. Consider:
-spawn an interactive shell with certain environment variables set, in order to enable the user to use other tools +(in the case of `atmos terraform shell`, the Terraform or Tofu CLI) natively, +spawn an interactive shell with certain environment variables set to enable using other tools +(in the case of `atmos terraform shell`, the Terraform or Tofu CLI) natively,🧰 Tools
🪛 LanguageTool
[style] ~836-~836: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...(IN_ORDER_TO_PREMIUM)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
.github/workflows/test.yml(1 hunks)internal/exec/helmfile.go(1 hunks)internal/exec/terraform.go(1 hunks)pkg/config/config.go(6 hunks)pkg/config/config_test.go(1 hunks)pkg/schema/schema.go(1 hunks)pkg/utils/glob_utils.go(2 hunks)website/docs/cli/configuration/configuration.mdx(4 hunks)
🚧 Files skipped from review as they are similar to previous changes (6)
- internal/exec/helmfile.go
- pkg/schema/schema.go
- .github/workflows/test.yml
- pkg/utils/glob_utils.go
- internal/exec/terraform.go
- pkg/config/config_test.go
🧰 Additional context used
📓 Learnings (1)
pkg/config/config.go (2)
Learnt from: osterman
PR: cloudposse/atmos#808
File: pkg/config/config.go:478-483
Timestamp: 2024-12-02T21:26:32.337Z
Learning: In the 'atmos' project, when reviewing Go code like `pkg/config/config.go`, avoid suggesting file size checks after downloading remote configs if such checks aren't implemented elsewhere in the codebase.
Learnt from: osterman
PR: cloudposse/atmos#887
File: internal/exec/stack_processor_utils.go:380-380
Timestamp: 2024-12-25T20:28:47.526Z
Learning: Windows path handling often requires `filepath.Join` to ensure correct separators and comparisons. Insufficient tests can break cross-platform compatibility, so migrating from `path.Join` to `filepath.Join` needs thorough testing on Windows before merging.
🪛 LanguageTool
website/docs/cli/configuration/configuration.mdx
[style] ~23-~23: A comma is missing here.
Context: ...vior) you'd specify the command to run (e.g. [opentofu](/core-concepts/projects/co...
(EG_NO_COMMA)
[style] ~119-~119: This word has been used in one of the immediately preceding sentences. Using a synonym could make your text more interesting to read, unless the repetition is intentional.
Context: ...### Stage 3: Apply User Preferences :::important If the --config flag is used, this st...
(EN_REPEATEDWORDS_IMPORTANT)
[grammar] ~181-~181: Did you mean “is” or “can be”?
Context: ... supported (unlike stacks). :::warning Be Careful with Remote Imports - Always us...
(BE_IS)
[style] ~836-~836: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...
(IN_ORDER_TO_PREMIUM)
🔇 Additional comments (3)
pkg/config/config.go (2)
Line range hint
103-141: LGTM! Clear and well-structured configuration loading logic.The new configuration loading order is well-documented and follows a clear precedence:
- ATMOS_CLI_CONFIG_PATH check
- System directory check
- User-specific configuration check
The implementation correctly handles the mutual exclusivity of ATMOS_CLI_CONFIG_PATH with other paths.
451-458: LGTM! Proper URL validation.The URL validation is thorough:
- Checks for HTTP/HTTPS scheme
- Validates URL parsing
- Provides clear error messages
website/docs/cli/configuration/configuration.mdx (1)
Line range hint
803-831: LGTM! Comprehensive environment variables documentation.The environment variables table is well-structured and provides clear descriptions for each variable. The mapping between YAML paths and environment variables is particularly helpful for users.
🧰 Tools
🪛 LanguageTool
[style] ~836-~836: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...(IN_ORDER_TO_PREMIUM)
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
website/docs/cli/configuration/configuration.mdx (3)
27-29: Add security considerations for configuration loading.While the configuration loading process is well documented, consider adding security implications when loading configurations from different sources, especially for CI/CD environments.
Add a security note after line 45:
You can pass the `--config` flag multiple times in a single command; the configurations will be deep merged in the order you provide them. + :::warning Security Note + When using multiple configuration files, especially in CI/CD environments: + - Validate the source and permissions of configuration files + - Consider using checksums for critical configurations + - Be cautious with configuration files from untrusted sources + :::Also applies to: 39-45
191-196: Add examples for path resolution.Consider adding practical examples to illustrate the difference between absolute and relative path resolution.
Add examples after line 196:
When `base_path` is explicitly set in the Atmos configuration, it is always resolved to an absolute path based on the current working directory from which `atmos` is executed, unless an absolute path is already provided. + For example: + ```yaml + # Relative path (resolved relative to current working directory) + base_path: "./infrastructure" + + # Absolute path (used as-is) + base_path: "/opt/atmos/infrastructure" + ```
Line range hint
839-867: Add clarification about environment variable precedence.The environment variables table is comprehensive, but it would be helpful to explicitly state the precedence rules between environment variables and YAML configuration.
Add a note before the table:
+ :::note Environment Variable Precedence + When both environment variables and YAML configuration settings are present: + 1. Command-line arguments take highest precedence + 2. Environment variables override YAML configuration + 3. YAML configuration serves as the default + :::
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
website/docs/cli/configuration/configuration.mdx(4 hunks)
🔇 Additional comments (2)
website/docs/cli/configuration/configuration.mdx (2)
13-13: LGTM! Clear and well-structured introduction.The addition of the Admonition component and the improved introduction provide better context for users about the CLI configuration's purpose and scope.
Also applies to: 21-23
179-180: LGTM! Clear documentation about templating limitations.The explicit note about templating limitations in imports helps prevent confusion and potential issues.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (4)
website/docs/cli/configuration/configuration.mdx (4)
13-23: Add missing comma in the example.A comma should be added after "e.g." for proper punctuation.
-...vior) you'd specify the command to run (e.g. [`opentofu`](/core-concepts/projects/co... +...vior) you'd specify the command to run (e.g., [`opentofu`](/core-concepts/projects/co...🧰 Tools
🪛 LanguageTool
[style] ~23-~23: A comma is missing here.
Context: ...vior) you'd specify the command to run (e.g. [opentofu](/core-concepts/projects/co...(EG_NO_COMMA)
149-187: Consider adding examples for each import type.While the imports section is well-documented, it would be more helpful to have specific examples for each import type (Remote URL, Specific Path, Wildcard globs).
Consider adding examples like this:
import: # Remote URL example - "https://raw.githubusercontent.com/org/repo/main/config.yaml" # Specific Path example - "/absolute/path/to/config.yaml" - "./relative/path/config.yaml" # Wildcard glob examples - "configs/*.yaml" # Match all YAML files in configs directory - "stacks/**/*.yaml" # Match all YAML files recursively in stacks directory🧰 Tools
🪛 LanguageTool
[grammar] ~181-~181: Did you mean “is” or “can be”?
Context: ... supported (unlike stacks). :::warning Be Careful with Remote Imports - Always us...(BE_IS)
Line range hint
842-887: Environment variables table needs alignment fix.The environment variables table has some formatting inconsistencies in the markdown.
- Lines 842 and 866 have extra newlines that break table formatting
- Consider using consistent column widths for better readability
Also, consider adding a note about the case sensitivity of these environment variables.
🧰 Tools
🪛 LanguageTool
[style] ~875-~875: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...(IN_ORDER_TO_PREMIUM)
875-877: Simplify the wording for better clarity.The sentence is wordy and can be simplified.
-spawn an interactive shell with certain environment variables set, in order to enable the user to use other tools +(in the case of `atmos terraform shell`, the Terraform or Tofu CLI) +spawn an interactive shell with certain environment variables set to enable using other tools🧰 Tools
🪛 LanguageTool
[style] ~875-~875: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...(IN_ORDER_TO_PREMIUM)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
website/docs/cli/configuration/configuration.mdx(4 hunks)
🧰 Additional context used
🪛 LanguageTool
website/docs/cli/configuration/configuration.mdx
[style] ~23-~23: A comma is missing here.
Context: ...vior) you'd specify the command to run (e.g. [opentofu](/core-concepts/projects/co...
(EG_NO_COMMA)
[style] ~119-~119: This word has been used in one of the immediately preceding sentences. Using a synonym could make your text more interesting to read, unless the repetition is intentional.
Context: ...### Stage 3: Apply User Preferences :::important If the --config flag is used, this st...
(EN_REPEATEDWORDS_IMPORTANT)
[grammar] ~181-~181: Did you mean “is” or “can be”?
Context: ... supported (unlike stacks). :::warning Be Careful with Remote Imports - Always us...
(BE_IS)
[style] ~875-~875: Consider a shorter alternative to avoid wordiness.
Context: ...with certain environment variables set, in order to enable the user to use other tools (in ...
(IN_ORDER_TO_PREMIUM)
🔇 Additional comments (2)
website/docs/cli/configuration/configuration.mdx (2)
27-146: Well-structured configuration stages documentation!The three-stage configuration loading process is clearly explained with proper precedence rules and examples. The use of admonitions for important notes about the
--configflag is particularly helpful.🧰 Tools
🪛 LanguageTool
[style] ~119-~119: This word has been used in one of the immediately preceding sentences. Using a synonym could make your text more interesting to read, unless the repetition is intentional.
Context: ...### Stage 3: Apply User Preferences :::important If the--configflag is used, this st...(EN_REPEATEDWORDS_IMPORTANT)
189-215: Excellent base path documentation with clear precedence rules.The base path section effectively explains:
- How the base path is determined
- The order of precedence
- The relationship with other paths
|
💥 This pull request now has conflicts. Could you fix it @haitham911? 🙏 |
what
Supported the atmos.d convention for atmos.yaml configuration, allowing automatic inclusion of configuration files from the atmos.d directory.
Made the path to atmos.d configurable within atmos.yaml, enabling users to specify custom directories for additional configurations.
Implemented deep merging of configurations in lexicographical order, recursively processing files within specified directories to ensure consistent and predictable configuration outcomes.
Added support for the import key inside atmos.yaml, allowing users to define a list of locations (local files, directories using glob patterns, and remote URLs) to import configurations from.
expose env variable ATMOS_CLI_CONFIG_PATH and ATMOS_BASE_PATH before run terraform and helm cmd
demos :
why
Simplifies configuration management by adopting the atmos.d directory convention, making it easier to include additional configurations without explicit declarations.
details
Load Config
flowchart TD A["Load Configuration File"] --> B{"Import Section Exists?"} B -- Yes --> C["Process Imports in Order"] C --> D{"Import Type?"} D --> E["Remote URL"] D --> F["Specific Path"] D --> G["Wildcard Globs"] E --> H["Fetch Config from Remote URL"] F --> I["Read Config from Filesystem"] G --> I["Read Config from Filesystem"] H --> J["Call Load Configuration File (Recursively)"] I --> J["Call Load Configuration File (Recursively)"] J --> L["Deep Merge with Current Config in Memory"] L --> K{"More Imports to Process?"} K -- Yes --> C K -- No --> M["Configuration Processing Complete"] %% Loopback for recursion J -.-> A %% Styling for clarity style A fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style B fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style C fill:#457B9D,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style D fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style E fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style F fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style G fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style H fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style I fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style J fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style L fill:#457B9D,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style K fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style M fill:#1D3557,stroke:#1D3557,stroke-width:2px,color:#FFFFFF classDef recursion stroke-dasharray: 5 5;3 Stages
--- config: layout: fixed --- flowchart TD A["Start Configuration Process"] --> Z1["Load Atmos Schema Defaults"] Z1 --> Z2["Load // go:embed cloudposse/atmos/atmos.yaml"] Z2 --> Z3["Deep Merge Schema Defaults and Embedded Config"] Z3 --> B{"Is --config Provided?"} %% If --config is provided B -- Yes --> C["Stage 1: Load Explicit Configurations via --config"] C --> C1{"Load each --config path in order"} C1 --> C2["Update ATMOS_CLI_CONFIG_PATH with loaded config absolute paths (separated by delimiter)"] C2 --> D["Process Imports and Deep Merge"] D --> E["Final Merged Config"] E --> F["Output Final Configuration"] %% If --config is not provided B -- No --> G["Stage 1: Load System Configurations"] G --> G1{"Check System Paths"} G1 -- Found --> G2["Load First Found Config: %PROGRAMDATA%/atmos.yaml (Windows), /usr/local/etc/atmos.yaml, or /etc/atmos.yaml"] G2 --> G3["Update ATMOS_CLI_CONFIG_PATH"] G3 --> H["Process Imports and Deep Merge"] G1 -- Not Found --> H["Process Imports and Deep Merge"] H --> I["Stage 2: Discover Additional Configurations"] I --> I1{"Check ATMOS_CLI_CONFIG_PATH"} I1 -- Found --> I2["Load First Found Config: atmos.yaml or atmos.d/**/* from ATMOS_CLI_CONFIG_PATH"] I2 --> I4["Update ATMOS_CLI_CONFIG_PATH with loaded absolute paths"] I4 --> J["Process Imports and Deep Merge"] I1 -- Not Found --> I5{"Check Git Repository Root"} I5 -- Found --> I6["Load First Found Config: atmos.yaml, .atmos.yaml, atmos.d/**/*, .atmos.d/**/*, or .github/atmos.yaml from Git Repo Root"] I6 --> I4 I5 -- Not Found --> I12{"Check Current Working Directory"} %% New branch for Current Working Directory (note it's not identical to repo root) I12 -- Found --> I13["Load First Found Config: atmos.yaml, .atmos.yaml, atmos.d/**/*, .atmos.d/**/* from CWD"] I13 --> I4 I12 -- Not Found --> I18["No configuration found in Stage 2"] I18 --> K["Stage 3: Apply User Preferences"] J --> K["Stage 3: Apply User Preferences"] K --> K1{"Check $XDG_CONFIG_HOME/atmos.yaml"} K1 -- Found --> K2["Load First Found Config: $XDG_CONFIG_HOME/atmos.yaml"] K2 --> K3["Update ATMOS_CLI_CONFIG_PATH with $XDG_CONFIG_HOME/atmos.yaml"] K3 --> L["Process Imports and Deep Merge"] K1 -- Not Found --> K4{"Check User's Home Directory"} K4 -- Found --> K5["Load First Found Config: %LOCALAPPDATA%/atmos/atmos.yaml (Windows), ~/.config/atmos/atmos.yaml, or ~/.atmos/atmos.yaml (Linux/macOS)"] K5 --> K3 K4 -- Not Found --> K7["No configuration found in Stage 3"] K7 --> M["Final Merged Config"] L --> M["Final Merged Config"] M --> F["Output Final Configuration"] %% Styling for clarity style A fill:#457B9D,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style Z1 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style Z2 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style Z3 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style B fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style C fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style C1 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style C2 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style D fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style E fill:#457B9D,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style F fill:#1D3557,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style G fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style G1 fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style G2 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style G3 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style H fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I1 fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I2 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I4 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style J fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I5 fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I6 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I12 fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I13 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I18 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style K fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style K1 fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style K2 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style K3 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style K4 fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style K5 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style K7 fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style L fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style M fill:#457B9D,stroke:#1D3557,stroke-width:2px,color:#FFFFFFBase Path
flowchart TD A["Load Atmos Configuration"] --> B["Start Computing Base Path"] B --> C{"Is --base-path Provided?"} %% If --base-path is provided C -- Yes --> D["Set base_path to --base-path (resolve absolute if relative)"] D --> E["Update ATMOS_BASE_PATH with absolute path"] %% If --base-path is not provided C -- No --> G{"Is ATMOS_BASE_PATH Set?"} G -- Yes --> H["Set base_path to ATMOS_BASE_PATH (resolve absolute if relative)"] H --> E G -- No --> I{"Is base_path Set in Configuration?"} I -- Yes --> J{"Is base_path Absolute?"} J -- Yes --> K["Set base_path as is"] K --> E J -- No --> L["Resolve base_path relative to current working directory"] L --> K I -- No --> M["Infer base_path"] M --> M1{"Check CWD for atmos.yaml, .atmos.yaml, atmos.d/**/*, .github/atmos.yaml"} M1 -- Found --> M2["Set base_path to absolute path containing directory"] M1 -- Not Found --> M3{"Is Git Repository Detected?"} M3 -- Yes --> M4["Set base_path to Git Repository Root"] M3 -- No --> M5["Set base_path to absolute path of ./"] M2 --> E M4 --> E M5 --> E E --> F["Base Path Computed"] %% Styling for clarity style A fill:#457B9D,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style B fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style C fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style D fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style E fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style F fill:#457B9D,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style G fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style H fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style I fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style J fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style K fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style L fill:#A8DADC,stroke:#1D3557,stroke-width:2px,color:#000000 style M fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style M1 fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style M2 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style M3 fill:#F4A261,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style M4 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFF style M5 fill:#E63946,stroke:#1D3557,stroke-width:2px,color:#FFFFFFreferences
Summary by CodeRabbit
Release Notes
New Features
Configuration Improvements
ATMOS_CLI_CONFIG_PATHandATMOS_BASE_PATHenvironment variables.Documentation
Bug Fixes