-
Notifications
You must be signed in to change notification settings - Fork 662
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Drop or replace drivers with ansible content #3919
Comments
I'm not convinced it'll be a "win" in the end. The driver glue/code is small and usually the issues are the playbooks (the notable exception being vagrant and it's vagrant plugin). Converting them to module/filter is going to add extra failure paths and maintainance imho. Moreover, when I think how handling collection search paths and things like that seems to lead to troubles in -lint, I don't see how moving everything to a collection is an improvement. Side-question: I'm not sure to understand how it'll work in details. For instance, how 'molecule login' will work after this change ? Do you have more details ? |
@apatard The problem is that the driver glue/code is code. However small and trivial it may be to a python programmer, it is constraining to those who are not. Most Ansible content creators are not programmers so this is a "win" in that it will let all content creators more effectively test their content. I will have to leave your side question to @ssbarnea. |
I might not have complete understanding of your suggestion.
You can do that already. Just create another collection and move content from playbook https://github.com/ansible-community/molecule-plugins/blob/main/src/molecule_plugins/docker/playbooks/create.yml to playbook in that collection. Let's call new collection molecule.docker, playbook will be called create (for create, destroy for destroy, etc). Remove contents of file https://github.com/ansible-community/molecule-plugins/blob/main/src/molecule_plugins/docker/playbooks/create.yml and add following
This way you decoupled driver and playbooks to manage docker infra. You can test driver separately, and molecule.docker collection separately. Other content that is required to provision docker infra also has to be moved to collection. For instance https://github.com/ansible-community/molecule-plugins/tree/main/src/molecule_plugins/docker/playbooks/filter_plugins. All what should be left in the driver is - driver name and cookiecutter templates to call for external collection content (playbook, playbooks will use other content). This way it is much more simple to create driver and test it. Same with molecule.docker (in this example) collection - this is collection you can test it more easily compared when it was in cookiecutter template. Important note - I am not saying that molecule.docker collection should be the same as Community.Docker collection. It should depend on it, but the interface is completely different. Usually in collections roles are the interface, for molecule collections playbooks are the interface. Playbooks use molecule_yml dictionary to create all the infrastructure.
This can be as easy as agreeing that all playbooks that start with molecule_* are molecule playbooks and can be used in molecule. For instance molecule.docker.molecule_create. Also these playbooks can be added to community.docker collection (in this example). So no need to create another collection for molecule. But this might force to add additional dependencies to community.docker collection - investigation is required. |
No doubt @kksat. One can use the delegated (not the best name admittedly) driver to do that. The point is to make using Ansible itself the undisputed preferred means of provisioning a test environment in a way that is accessible and as straightforward and easy for any content developer (especially new ones) to pick up and use. My hope is that there are a plethora of reusable options made available, but given this new found freedom of not having to write python code I have a feeling that a lot of content creators will opt to create their own tailored to their specific needs, preferences and standards.
But that assumes that a driver (provisioner) will not be shared beyond the collection it is embedded. I know you could in theory use the molecule content embedded with another collection, but don't you think managing that will get messy? |
My point is that you can do that already with any molecule driver. All the actual provisioner code in the driver can be moved to separate collection and imported from collection in the driver. create.yml is just a playbook and one can import playbook from any collection. See for example https://github.com/redhat-sap/molecule-driver-azure/blob/main/molecule_driver_azure/cookiecutter/%7B%7Bcookiecutter.molecule_directory%7D%7D/%7B%7Bcookiecutter.scenario_name%7D%7D/create.yml We created molecule driver for azure once and when we update only respective collection to change drivers behaviour. Because driver delegates all provisioning to ansible collection while still doing its tasks.
I think there is a misunderstanding. If all the provisioned code is in ansible collection, that exactly means it can be shared and reused outside of molecule driver. As soon as there is data similar to platforms list in molecule.yml, collection playbooks can be used to provision instances without molecule driver with ansible-playbook command.
I am not sure what do you mean here by "use the molecule content embedded with another collection" we included playbooks from ansible collection to driver. That was not messy to change ansible collection that driver uses, that was actually simple, because collection development is well understood (as well as testing and sharing content). But how to test cookie cutter templates inside driver? - this is a challenge from my point of view.
I believe we still need python code, because driver (python code) does something, that ansible collection cannot do. For instance installing python dependencies, running sanity checks, providing schema for the molecule.yml, etc. But we can certainly make python code much easier to create. In our driver we only changed package name and driver class name from community driver - This is what needed in most of the cases to create driver, if other features needed - python development (extended molecule use case) TLDR My point of view |
What I'm getting it that we are agreement on the principle, but differ on the implementation. Our concern is that multiple drivers written in python is additional (and significant) maintenance and support responsibility we want to remove in order to streamline future development and reduce what any user has to learn in addition to their core Ansible skills.
My understanding of what you wrote was that the molecule driver (provisioner) would be in the collection you are testing. Did I misunderstand what you meant?
I've not been a coder for a few years now. I'll leave @ssbarnea and @cidrblock to discuss in more detail. Could you be more specific though? Is it just that python is far better suited for programming logic or is there something else you have in mind here? I'm just not seeing why consolidating all playbook driven provisioning cannot be consolidated into one default driver that everyone can learn and use.
That I agree. Again, I think the difference here is implementation details. The current proposal is one default driver that implements all of the provision and deprovisioning tasks as Ansible content. What I get from your suggestion is that there should be some provision/deprovision content in Ansible but executed by an environment specific driver like docker, podman, aws, azure and so on. Is that an accurate summarization? |
I fully agree with this statement. My point is that you do not need to create V6 for that and cause a lot of disruption. This is completely doable with current version.
Not, not like this. Molecule driver is separate - https://github.com/redhat-sap/molecule-driver-azure Collection that molecule driver uses is separate - https://github.com/redhat-sap/molecule.driver
For instance installing python dependencies, running sanity checks, providing schema for the molecule.yml, etc. These are all tasks that drivers do. I do not see how it is possible to implement them with pure ansible. Maybe I am missing something.
Yes, but it is only because driver does something that ansible cannot do - installing pip dependencies, schema, etc. |
Please check #3947 which is a full example of using podman directly. While this makes the test body little bit better it should provide a much better stability over time, as it would not need a driver that needs to be updated to keep up with all podman collection changes. Podman and docker modules are known to change their arguments quite often, so they will benefit a lot from a pure ansible approach. Other benefits are that users will be able to mix multiple ways of provisioning hosts, not being limited to a single one. |
@ssbarnea Sorry Sorin, I do not see how PR #3947 is different from current approach for the podman driver. Looks like a lot of functionality been stripped and with this approach it will be only possible to reference container image, not more. But there are a lot of parameters in podman collection. Ok, podman driver is not referenced anywhere, is this a benefit your are looking for? But now to add functionality to the driver, molecule itself has to be changed. And how podman itself will be installed, because it is needed for molecule with this approach, molecule will depend on podman? It one wants to add parameter Compare your approach with https://github.com/redhat-sap/molecule.driver |
@kksat As the entire content of The idea of not using a driver is that we don't need to maintain the feature pairing between the driver and the 3rd party collection module(s) that are effectively doing the work. Obviously that collection authors could write modules that would make the use of molecule easier (less verbose playbooks). |
How should the plugin rewrite/replacement be done? And even though there might not be 3rd party modules, the (vagrant|docker|podman|...) "glue" still needs to be maintained. |
We are going to make every effort to preserve driver functionality for version 6. An emphasis on using the default driver shouldn't need to include completely breaking the exisiting drivers :) |
IIUC, the idea is that after this issue is fixed, there is no Previously, if someone wants to test an Ansible role, they have to use a specific The molecule plugin makes the interface unified across different collections(e.g. docker, vagrant and podman). We just need to define In the future, it's end user's responsibility to handle these logic which provides more flexibility, but it introduces more code in end user's code base.
IMHO, the answer is yes.
It depends on how collection handle this issue. If one collection requires some tools, with/without
For this specific scenario, you can write your own Please correct me if I say anything wrong @ssbarnea |
Partial: #3919 @ssbarnea Would you kindly review this PR? Signed-off-by: Jack Zhang <[email protected]>
That clarifies it. Thank you |
We may or may not look into this in the future. It really depends on the uptake of ansible as the provionioner and if examples/references exist that meet or exceed the capabilties and quality of the drivers. |
In order to make it easier to write tests, we will investigate dropping current molecule drivers and replacing them with pure ansible collections.
Please note that even now, anyone using delegated driver can already do this. The difference is that we will try to a way to facility the migration from someone using a particular driver with minimal effort, so they would not have to rewrite their entire test.
The text was updated successfully, but these errors were encountered: