Add JSON Output Format to terraform plan

[evankroske]

Terraform Version

Terraform v0.8.6

Affected Resource

N/A

Terraform Configuration Files

N/A

Debug Output

N/A

Panic Output

N/A

Expected Behavior

terraform plan -out plan.json -format json should create a file plan.json containing the execution plan in JSON format.

Actual Behavior

flag provided but not defined: -format

Use Cases

Unit Testing

A Terraform user could write a test that parses the execution plan to ensure that their configuration produced the desired resources.

Today, users have to parse the output of terrraform plan to test their configurations, which makes their tests fragile. I'm doing this now. This practice will become more common as HCL becomes more dynamic.

In addition, a change to the output format of terraform plan might break users' tests, keeping them from upgrading to the latest version of Terraform. Providing a documented output format for terraform plan would keep users on the upgrade train and reduce the burden of supporting older versions of Terraform.

Third-Party Provisioners

Adding support for generating an execution plan in a documented, machine-readable format would allow users to write their own tools to apply execution plans.

References

• Support use cases with conditional logic #1604

[ghost]

+1

[ghost]

Hi Guys,
Any updates on this one?
Thank You

[apparentlymart]

Hi @evankroske! Sorry for the silence here.

We do intend to make machine-readable plan output eventually, but we are intentionally postponing this for now since we don't feel ready to commit to a stable JSON format for plans. Forthcoming Terraform versions will change how diffs are represented internally, which will necessitate changes to the plan structure.

Once the structure is more stabilized we do intend to provide a machine-readable option, which would then be considered a compatibility constraint for future versions.

I'm going to close this for now to represent the fact that no work is planned right now. We'll open one or more new issues/PRs later when we're ready to implement this.

Thanks for the feature request!

[shreychen]

I'm looking forward this feature, too.

[mahieddine]

This is absolutely a most needed feature

[btilford]

This would be really useful for debugging. Debug output really doesn't give you very much useful information very often.

[pvcon13]

I am using an api (in front of) to control Terraform instance creation (autoscale) would benefit greatly from JSON returns to provide a useful restful payload and to parse and store return values. If you change the format later we can change parsing: no big deal, and in the case of a restful payload no changes would be needed.

[repl-raymond-bolander]

Can we re-open this issue? Many larger teams could benefit from this capability for compliance and audit purposes.

Even some sort of stop-gap would be better than not having it.

Given the cost/benefits, it would be good to make some progress on this. If and when changes happen that require the format to change, we can accommodate that as it comes with a compatibility / versioning schema. And I say this with the appreciation that it is generally wise to limit extraneous work and it is good to limit the introduction of scenarios where we force breaking changes on developers.

[apparentlymart]

We do not plan to merge any features into Terraform Core for this right now because it will increase the maintenance burden as we implement new features.

However, those who need this in the mean time and are willing to deal with breaking changes moving forward may wish to investigate making a Go program that uses the ReadPlan function to obtain the internal representation of the plan and work with that.

This API will change in future releases, and changes to the plan format and internal APIs are not reported as breaking changes in the changelog, but if you "vendor" into your program the code corresponding to your current Terraform version it should be possible to load a plan file from disk using this function and then do arbitrary operations with the result.

We are aware that some users are already doing such things, accepting the compatibility caveats. I believe some of these are open source, and so while I can't recommend any specific implementations (having no personal experience with them) you may find some starting-point code via web search.

[repl-raymond-bolander]

@apparentlymart Thanks for taking the time to share. I'll take a look at those options.

To frame everyone's expectations, how substantial are the planned API changes? Should the community be expecting a state of flux for a while? I don't see a milestones list in github - so what should the community be referring to in order to better understand the possible blockers on this issue?

[apparentlymart]

The format of both the state and plan are currently based around flat maps from strings to strings, dating back to Terraform's early days when all values were strings.

Terraform has grown support for other types over time, but they are always forced through this "flatmap" layer, which causes lossiness and confusing behavior on some edge-cases, described in many different GitHub issues now. The most significant forthcoming change, then, is to change how these two formats represent user-supplied (and remote-API-supplied) values so that we can use a unified type system throughout Terraform and avoid these strange edge-cases.

Another thing that will cause significant changes to the plan and state formats is the solution to #953, for similar reasons: the current formats assume only one instance of each module, so they will need to be changed to support lists of module instances.

We are hoping to get all of these necessary breaking changes done in a single release, but due to the broad scope we don't know yet if this will be possible.

These big changes aside, we do also regularly make more minor changes to the formats to address bugs and add small features. For example, the handling of the "provider" attribute on a resource in state changed in 0.11.0 as part of the changes to how provider configurations interact with modules, and there were some changes to the plan format through the 0.10 releases in order to deal with the provider auto-install mechanism. Programs calling Terraform's packages as a library to read these formats would not have been affected by these changes unless they depended specifically on these aspects (unlikely).

Since we plan iteratively we can't share detailed roadmap/schedule information (such detailed information does not exist beyond the current phase of work), but I hope the above serves as some useful context about why we're being reserved about creating additional dependencies on these structures.

[philidem]

FYI, after exploring some options, my team decided to parse the terraform plan stdout output. I open-sourced the tool that we created: https://github.com/lifeomic/terraform-plan-parser

Please try it out and let me know if you have any feedback. I would still prefer if terraform produced this JSON output natively but this library is a good stop-gap solution.

[jstoner]

My team has been parsing terraform plan output for a while--since last summer, in fact. We have administrative tools that run on top of Terraform that make hundreds of invocations, making changes across our company. We haven't done a fancy formal parser, just line-by line putting together a simple data structure.

It's not a huge problem, except when we upgrade Terraform and the format changes. Which happens occasionally.

We would like to register our strong interest in JSON formatted plan output, whenever you guys get to it.

[giorgiosironi]

In my use case I wouldn't need to rely on a format, I am wrapping terraform plan commands from another process and would like an output to show that doesn't contain:

The plan command received a saved plan file as input. This command
will output the saved plan. This will not modify the already-existing
plan. If you wish to generate a new plan, please pass in a configuration
directory as an argument.

as this doesn't apply to my context.

[docwhat]

Converting an existing -out=plan.tfplan to a known schema suitable for automation would be fine by me. The textual output is useful for humans and then you wouldn't have to worry about the format of the plan.tfplan file.

[jufemaiz]

Does v0.12 have any updates on this issue?

[apparentlymart]

Machine-readable plan output is not in scope for Terraform v0.12.0.

[jufemaiz]

Thanks @apparentlymart for the update (sad though it is)

[karthikeayan]

What about the json format is similar to aws api response?

[joestump]

Not sure when this was added or if it'll address all use cases people are looking to support, but terraform show appears to support this for plan files.

[pa-yourserveradmin-com]

Not sure when this was added or if it'll address all use cases people are looking to support, but terraform show appears to support this for plan files.

With a small, but important note:

This format is available in Terraform 0.12 and later.

So, until 0.12 is not yet released - there is still no chance to see the plan in JSON using only built-in Terraform functional 😞

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.