Download current environment configuration

[tatomir146]

For initial work with Dynatrace Monitoring as Code, customers would like to be able to download all the current configuration into a project structure.

For example, when running MaC with a flag (e.g.: download), all the supported configurations are downloaded and stored in a new project folder with the entire structure in place, as well as an initial .yaml file to apply. Optionally, the user can specify a list of configuration types that they want to download (e.g.: if they only want management-zones and auto-tag).

This will help existing customers to get started with MaC and start migrating their existing config over.

[patrickpichler]

Wouldn't it make more sense to remodel the cli to feature commands instead of just flags?
Otherwise we would need to explicitly handle that only a certain number of flags is active, as
some of those flags are mutually exclusive.

So instead of monaco -d it should be monaco dry-run.
For download it should probably look like monaco download.

[UnseenWizzard]

Yeah, as mentioned somewhere all flags that would fit a download are used already.
I personally still like the idea of a second executable for this, mainly to call it 'cheese' but also to not bloat monaco/main even more.

Changing invocation to something like monaco upload & monaco download would work as well, but I find it strange that some of the flags will be the same, some will differ between those two invocations. then again that's probably normal.

--dry-run I'd actually argue is a valid option that either commands should have. I might want to see how many configs I hit, before I pull them to my file system - e.g. when making the possibly bad choice to download everything from a big environment.

[patrickpichler]

To be honest we could easily refactor the current functionality of main out as its own command and only to the args parsing/deciding what to call in main. That way one could even reuse single commands in other code projects
if needed.

As for the second binary, I'm do not quite like this to be honest. I don't want to remember the name of two tools ^^

You are right, --dry-run should probably stay a flag to tell the user what would happen. But to be honest I would
still promote the current functionality of --dry-run to command level and call it something like validate.

[IgnacioGoldman]

Hi,

I have created another tool that works in a similar way of the Monitoring as Code, configurations in a single yaml file, but it’s able to pull configurations from tenants.

How it works?

1. An initial configuration is required to pass the tenants, tokens, configuration endpoints to touch. (The other 3 are to make the tool useful for Managed customers via MC)
   
2. Execute the python and pull the configuration from the tenants. The configuration will be stored in JSON and the tool will create yamls for each tenant, to easily move the configuration between them:
   
   3.Then it’s just a matter of copying and pasting configuration from one tenant to the other in case we need to create or edit a specific configuration. Let’s suppose that a new tag with many rules is created in the tenant xxu26128. The rule will be visible under the respective yaml file and can be moved to the remaining tenants. This is how a config yaml file looks like:
   

I would like to include this functionality within MaC but i feel it will deviate in the way you're thinking the project:

• It doesn't work with a JSON model and a YAML config file (such as MaC). The .yaml contains the full configuration (i'll start applying filters to reduce the lines of config)
• The structure of the directories are different. I find mine really useful since i just go wherever i want to propagate the configuration, i open all the files and with a copy and paste it's solved
• Written in Python (not a problem but still another point to work in)

Not sure if i'm commenting this thread in the right way and i hope i can help 👍
here's the link to my MaC version: https://github.com/IgnacioGoldman/Dynatrace/tree/master/saveConfigAPI

[UnseenWizzard]

Hi @IgnacioGoldman,
thanks for reaching out, I'm excited for your contribution!

Is your https://github.com/IgnacioGoldman/Dynatrace private though?
I get a 404 on that link :(

From what you describe the structure does sound very different indeed.
Our tool was designed around the idea to have one set of configuration, that can be modified for each environment via templating.

One file per environment which is copy pasted seems somewhat error prone and a lot of effort once you go past a handful of environments?

[IgnacioGoldman]

Hi Nico,

Yes indeed, it's private but i just sent you an invitation.

Just to clarify, it's one file per environment depending on the confiugration. For example, if yo want to propagate auto-tags in 4 different environemtns then you have 4 files, one for each. Then for Managemetn Zones, the files will be different. The yamls turn out more complex, i agree with you, but i see my approach faster and more flexible since the yamls are not dependant with the JSONs (you don't need to provide the structure with the JSON), so moving configuration between tenants is faster.

I have been using monaco for automatically create alerting profiles and notification systems and it works well. But when it comes to tags or configurations that contains many rules, then it will become complex by itself.

Coming back to my solution, i was going to start creating filters to reduce the information to display in the yamls. Then my approach would reduce the complexity.

[JACONDE]

Can I get an invitation as well to your project @IgnacioGoldman ?

[IgnacioGoldman]

I sent an invitation to "JACONDE" if that's correct

[JACONDE]

Just received. Thank you !

[kristofre]

@JACONDE @IgnacioGoldman we are picking up this issue and will be developing an "Environment Download" solution that will look like this for a first release.

Monaco should be able to:

1. Have a flag to allow downloading all configurations of a tenant (e.g.: -download or something)
2. Create the folder structure in a folder that is specified in the command line (as a subfolder of the projects directory), e.g.: -dt=mytenantname
3. Iterate through all the configurations and
   b. Download all the JSONs into individual files
   c. Create 1 YAML file that contains a list of all configurations and their matching JSONs
   d. Follow dependencies and replace them with Monaco references (this capability should be able to be turned off with a flag, e.g. -no-dependencies

[JACONDE]

That will be very good @kristofre . Do you have any ETA by when this new feature will be avaialble?

[IgnacioGoldman]

Great to hear that @kristofre, let me know if i can help with anything else.

[kristofre]

That will be very good @kristofre . Do you have any ETA by when this new feature will be avaialble?

@JACONDE development work will start this or next week with a PR targeted for the end of the calendar year or beginning of next

[ghost]

I'll take care of the implementation

[patrickpichler]

I guess we can close this issue, or is still something missing?
@didiladi @UnseenWizzard @diegorqc

[didiladi]

I would really like to see an integration test for the functionality before closing this one. 🚀

[UnseenWizzard]

download feature exists for a while, further automated tests covered in #423 -> closing