Roles 2.0

[bcoca]

Proposal: Roles2.0

Author: bcoca

Date: 2021--13-32

• Status: New
• Proposal type: core design
• Targeted release: future release
• Associated PR:
• Estimated time to implement: 3 weeks

Motivation

Most users are constantly surprised bye the behavior of Roles, specially when it comes to variable management and handlers

Problems

What problems exist that this proposal will solve?

• Variables pollute the play and other instances of the same role or other roles
• Handlers also pollute the play and may overwrite other handlers or be overwritten themselves when they have specific function to the role instance itself.

Solution proposal

New role execution:

• semantic versioning (should work just like collections)

• private by default (may need a few values/considerations for vars/handlers and to control even explicit exports)
  import_role: name=yolo private=false

• allow meta declare 2.0 pragma (role author says it works in 2.0 style!)

• allow include_role/import_role to force 2.0 execution (play author gambles it MIGHT work in 2.0 style!)

   - include_role:  
       name: newstyle
       role_spec: '2.0'
     vars:   # these would only be available inside this role and contained tasks (including other roles called)
       var1: val1 
  

• remove 'inline params' as they are confusing and can/will overlap with keywords

• create 'always export' facility for vars and handlers (role's purpose is to provide vars or handlers to play)
  {vars|default|handlers}/export.yml

• no support for meta dependencies , use explicit meta/requirements.yml file for installing dependencies
  I expect lots of pushback here, but this is to avoid confusion on inheritance since it is reversed but most people expect it
  otherwise, while include/import_role are very clear and explicit on how inheritance works.

• magic var with 'role_vars' to get access to definitions in roles w/o extra vars or other changes
  _role['defaults']['varname']

• role's handlers execute at role end by default (currently we do after pre_tasks/tasks/post_tasks)

    # current code that adds implied meta: flush_handlers to plays
    block_list.extend(self.pre_tasks)
    block_list.append(flush_block)
    block_list.extend(self._compile_roles())
    block_list.extend(self.tasks)
    block_list.append(flush_block)
    block_list.extend(self.post_tasks)
    block_list.append(flush_block)

Testing (optional)

LOTS!

Documentation (optional)

Attach to role argspec/documentation expansion

[geerlingguy]

Main thing I'd like to see with no support for meta dependencies is support for a role depending on a collection. E.g. if my role requires community.general, it would be nice if galaxy could install that at install time.

[bcoca]

the requirements file should be able to handle that since now it supports 'dual format' to include roles and collections

[sivel]

Main thing I'd like to see with no support for meta dependencies is support for a role depending on a collection. E.g. if my role requires community.general, it would be nice if galaxy could install that at install time.

That is a business decision that would have to be over turned. It's purposeful lack of cross type deps due to non-technical decisions. As of now, I haven't heard of any plans to change that behavior.

[geerlingguy]

@sivel - In that case, what's the point of supporting roles further. IMO that's about the most essential step moving forward if roles are going to continue existing in any form.

Right now we're back to the pre-Ansible-Galaxy days of telling people "you're going to have to manually install some things if you want to use this role."

IMO it's either make roles 2.0 a thing and make it pleasant, or just abandon it and pour a ton of energy into making role authors happier to migrate to collections (and making single-role collections easier to get going and maintain).

What I'd really hate to see is 'roles 2.0' be a half-dedicated effort to make roles better but in reality it just means we fragment the roles out in the world so it's even harder to write Ansible tasks that work across 2.9, 2.10, 3.0 etc. (not to mention ansible-core vs ansible).

[sivel]

In that case, what's the point of supporting roles further

This proposal doesn't attempt to address those concerns. The concerns that this proposal targets apply equally to stand alone roles and roles in a collection. This proposals target isn't to solve the problems you are concerned with in relation to interoperability between stand alone roles and collection based roles.

[geerlingguy]

Ah... so roles just in the context of Ansible itself—not the 'ecosystem roles'. In that case, I'll keep fighting those battles elsewhere.

For this proposal, a couple minor notes I have:

create 'always export' facility for vars and handlers

Handlers are often useful across many different places (I often rely on roles providing them since I architect playbooks to stash all handlers inside roles, e.g. a webserver role would handle webserver restarts, reloads, etc., and I call them from my playbook all the time). Would handlers by default not be global anymore? I don't think I'd lump handlers in the same boat as vars in this case—vars are often very role-specific and don't affect the rest of a playbook, but handlers (IMO) are on a different level.

role's handlers execute at role end by default

This honestly seems like it could be the most tricky change. There are some apps and playbooks that depend on the restart happening at the right place in a process, and sometimes I rely on that happening at the end of the entire play / section, other times I call meta: flush_handlers in a role.

What I really would love, instead, is the ability to do something like:

  - meta: flush_handlers
    handlers:
      - my_handler_here
      - another_handler

Or, alternatively, have it default to flushing only the current role's handlers. That is often necessary in the middle of a role, and sometimes I don't really want to flush all play handlers, everywhere, but that's the only option really.

[nkakouros]

How will the magic var _role work exactly?

• Will it be somehow publicly available or only inside the role itself? Will it be affected by the public/private setting?
• Will the 'defaults' key be a representation of the defaults/main.yml file? What about other keys, will they automatically track other files under the defaults/ directory?
• Will the magic var be writable? What if a value is changed? Will it be reset on subsequent executions of the role?
• Is this the start of a broader introspection mechanism for roles?

[bcoca]

@geerlingguy there are many requests for handlers to be 'exclusive to the role', this proposal was trying to add the most flexible version of that.

About specific handler execution, that is outside the scope of roles, it is a core engine and I still think it is more appropriate to have tasks with when: x is changed

@nkakouros

• private to the role (was my first thought), not sure if valuable outside of it or we get to 'meta' but then would have to deal with 'role name' and dupes
• of 'loaded defaults', so most of the time defaults/main.yml but would really represent defaults_from, same for vars
• no, it is trying to represent 'what the role originally had' so making it writable misses the mark. changed values are accessible directly, not via this variable
• possibly, not sure how much more we want to expose and how much is useful to do so

[nkakouros]

@bcoca These are probably out of scope for this proposal, but some thoughts are:

• The _role variable could help solve Intelligent role defaults resolution #171 in a clean way if it did not just provided access to the loaded defaults but to all defaults under defaults/. An idea could be to have _role['defaults']['main'], _role['defaults']['other'] provide access to the various defaults files and have _role['loaded_defaults'] or sth similar provide the effective defaults.
• It would also make sense to provide access to the meta of the role, for instance to check supported OSes, print the author name or contact email for informative messages, etc.

[reyjrar]

Here's a real world example of me stubbing my toe on the current roles limitations.

I created a role base for configuring a base system with just the essential services. Initially, this was a list of tasks to be included, some of which were include_roles. We start to notice long application times, upwards of 10 minutes per playbook.

Come to find that include_roles always re-executes every role the included role depends on. We thought we were relying on the ansible dependency tracking so we'd skip re-applying common roles. So, I reworked everything to remove include_roles as much as a possible. This isn't possible everywhere, because some services allow multiple instances. This drops the execution time down to about 2 minutes on avg.

However, I'm still applying this massive base role with a lot of include_roles because those roles need to execute after tasks in the base role are run. So I go to rework the base role such that it contains only a meta/main.yaml that depends on sub-components in the correct order, and a tasks/main.yaml that just flushes handlers so any roles run after base during server provisioning have a sane view of life.

To make the migration easier, I came up with a plan. There are a lot of files and templates in the base role, so I thought: just move those to base/common/{files,templates} and then create base/component/auth and move the configure_server_auth.yaml task in there, setting dependencies: ["base/common"] on the base/component/auth role.

NOPE. I can now notify handlers defined in base/common from base/component/auth, but the search path is the opposite of what I expected. So, to accomplish this, there are 3 paths forward:

1. Move the {files,templates} into the relevant sub components, this is the most correct fix, but also makes finding the template a little more tricky. It also adds significant overhead to my project as I have to move a bunch of files around.
2. Symlink {files,templates} from base/common into the sub components. This feels dirty, I hate it. You have dependencies but not really in a way that someone coming from a programming or from another CM system would expect.
3. (What I chose for now) moved {files,templates} back into the base role.. because the subroles are dependencies of base, the files are found. HOWEVER the subroles are not able to be executed as stand-alones, they don't have any {files,templates} in them. This prevents me from requesting a deployment of just a subcomponent, say: deploy --only-role base/component/auth

OK, so, yes, I'm doing things wrong (for someone's definition of wrong, not mine, mind you), but the fact that handlers are visible when I use dependencies infers that the search paths might be able to move in that direction. There are A LOT of these surprises dealing with roles and variables in Ansible.

I agree with the @bcoca, Roles need to be better supported and provide the potential to solve the problem I'm fighting. Other CM Systems have clearer, more consistent behavior with dependencies and variable scoping.