Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for using Ansible's constructed inventory plugin to dynamically group hosts from AWX inventories #13448

Merged
merged 33 commits into from
Mar 29, 2023

Conversation

relrod
Copy link
Member

@relrod relrod commented Jan 19, 2023

SUMMARY

Constructed inventory is a separate "kind" of inventory, along-side of normal (manual) inventories and "smart" inventories. The functionality overlaps with smart inventory, and it is intended that smart inventory is sunsetted and will be eventually removed.

For more information about the underlying mechanism, see https://docs.ansible.com/ansible/latest/collections/ansible/builtin/constructed_inventory.html

Demo Problem

This is branched from original demo at:

https://github.com/AlanCoding/Ansible-inventory-file-examples/tree/master/issues/AWX371

Consider that we have 2 original "source" inventories named "East" and "West".

# East inventory original contents
host1 account_alias=product_dev
host2 account_alias=product_dev state=shutdown
host3 account_alias=sustaining
# West inventory original contents
host4 account_alias=product_dev
host6 account_alias=product_dev state=shutdown
host5 account_alias=sustaining state=shutdown

The user's intent is to operate on shutdown hosts in the product_dev group. So these are two AND conditions that we want to filter on.

To accomplish this, the user will create a constructed inventory with the following properties.

source_vars =

plugin: constructed
strict: true
use_vars_plugins: true  # https://github.com/ansible/ansible/issues/75365
groups:
  shutdown: resolved_state == "shutdown"
  shutdown_in_product_dev: resolved_state == "shutdown" and account_alias == "product_dev"
compose:
  resolved_state: state | default("running")

limit = "shutdown_in_product_dev"

Then when running a job template against the constructed inventory, it should act on host2 and host6, because those are the two hosts that fit the criteria.

Mechanic

The constructed inventory contents will be materialized by an inventory update which runs via ansible-inventory. This is always configured to update-on-launch before a job, but the user can still select a cache timeout value in case this takes too long.

When creating a constructed inventory, the API enforces that it always has 1 inventory source associated with it. All inventory updates have an associated inventory source, and the fields needed for constructed inventory (source_vars and limit) are fields on the inventory source model normally.

Capabilities

In addition to filtering on hostvars, users will be able to filter based on facts, which are prepared before the update in the same way as for jobs.

For filtering on related objects in the database, users will need to use "meta" vars that are automatically prepared by the server. These have names such as:

  • awx_inventory_name
  • awx_inventory_id

Best Practices

It is very important to set the strict parameter to True so that users can debug problems with their templates, because these can get complicated. If the template fails to render, users will get an error in the associated inventory update for that constructed inventory.

When encountering errors, it may be prudent to increase verbosity to get more details.

ISSUE TYPE
  • New or Enhanced Feature
COMPONENT NAME
  • API
  • UI
  • Collection
  • CLI
  • Docs
  • Other

@TheRealHaoLiu TheRealHaoLiu changed the title [DO NOT MERGE] constructed inventory [WIP] constructed inventory Jan 24, 2023
@github-actions github-actions bot added component:awx_collection issues related to the collection for controlling AWX component:cli component:docs labels Jan 26, 2023
@CFSNM CFSNM force-pushed the feature_constructed-inventory branch from 0866bfc to 8e24252 Compare February 14, 2023 18:53
@TheRealHaoLiu TheRealHaoLiu force-pushed the feature_constructed-inventory branch from 8e24252 to 0866bfc Compare February 15, 2023 15:03
@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug B 1 Bug
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

No Coverage information No Coverage information
4.2% 4.2% Duplication

@relrod relrod force-pushed the feature_constructed-inventory branch 4 times, most recently from 7c3e401 to 6c89367 Compare March 23, 2023 15:00
@relrod relrod changed the title [WIP] constructed inventory Allow for using Ansible's constructed_inventory inventory plugin to dynamically group hosts from AWX inventories Mar 27, 2023
@relrod relrod marked this pull request as ready for review March 27, 2023 15:58
@relrod relrod changed the title Allow for using Ansible's constructed_inventory inventory plugin to dynamically group hosts from AWX inventories Allow for using Ansible's constructed inventory plugin to dynamically group hosts from AWX inventories Mar 27, 2023
AlanCoding and others added 12 commits March 28, 2023 11:17
Add in required setting about empty groups
move limit field from InventorySourceSerializer to InventorySourceOptionsSerializer (#13464)

InventorySourceOptionsSerializer is the parent for both InventorySourceSerializer and InventoryUpdateSerializer

The limit option need to be exposed to both inventory_source and inventory_update

Co-Authored-By: Hao Liu <[email protected]>
Make the GET function work at most basic level

Basic functionality of updating working

Add functional test for the GET and PATCH views

Add constructed inventory list view for direct creation

Add limit field to constructed inventory serializer
…y and hack temporary image (#13474)

* Use control plane EE for constructed inventory and hack temporary image

* Update page registry to work with new endpoints
…ield updates (#13487)

* Add constructed inventory docs and do minor field updates

Add verbosity field to the constructed views

automatically set update_on_launch for the auto-created constructed inventory source
Also rename source_inventories to input_inventories
- add kind 'constructed' to inventory module
- add 'input_inventories' field to inventory module

Co-authored-by: Rick Elrod <[email protected]>
Signed-off-by: Rick Elrod <[email protected]>
 * Add constructed inventory API model
 * Add constructed inventory detail view
 * Add util to switch inventory url based on "kind"
- prevent constructed inventory host,group,inventory_source creation
- disable deleting constructed inventory hosts
- remove the ability to add constructed inventory sources
- remove ability to add constructed inventories to constructed inventories
- block updates to constructed source type
- added tests for group/host/source creation
marshmalien and others added 16 commits March 28, 2023 11:20
Including changes to our custom Ordered m2m field which previously broke
if the source and target model was the same.

Signed-off-by: Rick Elrod <[email protected]>
Co-authored-by: Alan Rominger <[email protected]>
* initial functional filter-on-facts functionality

* Move facts to its own module to make interface more coherent

* Update test
…es (#13727)

- When updating, we need the original object so we can make sure we
  aren't changing things we shouldn't be.
- We want to allow source_vars and limit, but not much else.
- We want to block everything else (at least, if it doesn't match what
  is in the original object...to allow the collection to work properly).
- Add two functional tests.

Signed-off-by: Rick Elrod <[email protected]>
…o both hosts (#13718)

* Backlink events to real hosts and summaries to both hosts

* Prevent error when original host is deleted during job run

* No duplicate entries, review suggestion from Rick

* Change word tense in help text, dict style adjustments

From code review

Co-authored-by: Rick Elrod <[email protected]>

* Back out new variable for constructed host id

---------

Co-authored-by: Rick Elrod <[email protected]>
* Save facts on model for original host

Redirect to original host for ansible facts

Use current inventory hosts for facts instance_id filter
Thanks for Gabe for identifying this bug

* Fix spelling of queryset

Co-authored-by: Rick Elrod <[email protected]>

* Fix sign error with facts expiry - from review

---------

Co-authored-by: Rick Elrod <[email protected]>
@relrod relrod force-pushed the feature_constructed-inventory branch from 0304a06 to 77743ef Compare March 28, 2023 16:21
for input_inventory in inventory_update.inventory.input_inventories.all():
args.append('-i')
script_params = dict(hostvars=True, towervars=True)
source_inv_path = self.write_inventory_file(input_inventory, private_data_dir, f'hosts_{input_inventory.id}', script_params)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion to name this based on input_inventory.name instead of input_inventory.id. Valid feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants