Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data sources with indexed references to managed resources #26458

Merged
merged 1 commit into from
Oct 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 14 additions & 3 deletions terraform/context_plan_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6346,17 +6346,28 @@ func TestContext2Plan_dataReferencesResource(t *testing.T) {
m := testModuleInline(t, map[string]string{
"main.tf": `
locals {
x = "value"
x = "value"
}

resource "test_resource" "a" {
value = local.x
value = local.x
}

// test_resource.a.value can be resolved during plan, but the reference implies
// that the data source should wait until the resource is created.
data "test_data_source" "d" {
foo = test_resource.a.value
foo = test_resource.a.value
}

// ensure referencing an indexed instance that has not yet created will also
// delay reading the data source
resource "test_resource" "b" {
count = 2
value = local.x
}

data "test_data_source" "e" {
foo = test_resource.b[0].value
}
`})

Expand Down
64 changes: 38 additions & 26 deletions terraform/transform_reference.go
Original file line number Diff line number Diff line change
Expand Up @@ -328,30 +328,8 @@ func (m ReferenceMap) dependsOn(g *Graph, depender graphNodeDependsOn) ([]dag.Ve

refs := depender.DependsOn()

// For data sources we implicitly treat references as depends_on entries.
// If a data source references a resource, even if that reference is
// resolvable, it stands to reason that the user intends for the data
// source to require that resource in some way.
if n, ok := depender.(GraphNodeConfigResource); ok &&
n.ResourceAddr().Resource.Mode == addrs.DataResourceMode {
for _, r := range depender.References() {

// We don't need to wait on referenced data sources. They have no
// side effects, so our configuration reference should suffice for
// proper ordering.
var resAddr addrs.Resource
switch s := r.Subject.(type) {
case addrs.Resource:
resAddr = s
case addrs.ResourceInstance:
resAddr = s.Resource
}

if resAddr.Mode == addrs.ManagedResourceMode {
refs = append(refs, r)
}
}
}
// get any implied dependencies for data sources
refs = append(refs, m.dataDependsOn(depender)...)

// This is where we record that a module has depends_on configured.
if _, ok := depender.(*nodeExpandModule); ok && len(refs) > 0 {
Expand Down Expand Up @@ -391,10 +369,44 @@ func (m ReferenceMap) dependsOn(g *Graph, depender graphNodeDependsOn) ([]dag.Ve
return res, fromModule || fromParentModule
}

// Return extra depends_on references if this is a data source.
// For data sources we implicitly treat references to managed resources as
// depends_on entries. If a data source references a managed resource, even if
// that reference is resolvable, it stands to reason that the user intends for
// the data source to require that resource in some way.
func (m ReferenceMap) dataDependsOn(depender graphNodeDependsOn) []*addrs.Reference {
var refs []*addrs.Reference
if n, ok := depender.(GraphNodeConfigResource); ok &&
n.ResourceAddr().Resource.Mode == addrs.DataResourceMode {
for _, r := range depender.References() {

var resAddr addrs.Resource
switch s := r.Subject.(type) {
case addrs.Resource:
resAddr = s
case addrs.ResourceInstance:
resAddr = s.Resource
r.Subject = resAddr
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More a question: reading this movement from the previous block, this appears to be the addition that has changed things (changing the subject). What happened here? Reading the PR summary, I was looking for some change to reading ManagedResourceMode but that behavior hasn't changed (but please correct my reading!): you go from checking == for ManagedResourceMode to != and a continue ... so this behavior is the same as the earlier block.

Copy link
Contributor

@pselle pselle Oct 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the relevant bit in the PR summary?

we need to re-target that reference to the containing resource for planning

But I'm also interested in this change from == to != ... is it doing anything I'm not seeing? (not saying to revert this part of the change, this continue might make things make more logical sense to future readers)

Copy link
Member Author

@jbardin jbardin Oct 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the r.Subject = resAddr is the key change here. The problem was that when the reference was to an indexed resource instance, that wouldn't match up with the resource addresses used for depends_on when building the plan graph, since there are no individual instances at that point. This prevented the data source from being connected to the resource during planning, and allowing the data source to be read earlier than expected.

You correct that the resAddr.Mode != addrs.ManagedResourceMode logic didn't change. I believe I had some more code in here that made a difference during the refactor (and made it longer, instigating moving it to another method), but the append logic did end up being the same in the end.

}

if resAddr.Mode != addrs.ManagedResourceMode {
// We only want to wait on directly referenced managed resources.
// Data sources have no external side effects, so normal
// references to them in the config will suffice for proper
// ordering.
continue
}

refs = append(refs, r)
}
}
return refs
}

// parentModuleDependsOn returns the set of vertices that a data sources parent
// module references through the module call's depends_on. The bool return
// value indicates if depends_on was found in a parent module configuration.
func (n ReferenceMap) parentModuleDependsOn(g *Graph, depender graphNodeDependsOn) ([]dag.Vertex, bool) {
func (m ReferenceMap) parentModuleDependsOn(g *Graph, depender graphNodeDependsOn) ([]dag.Vertex, bool) {
var res []dag.Vertex
fromModule := false

Expand All @@ -408,7 +420,7 @@ func (n ReferenceMap) parentModuleDependsOn(g *Graph, depender graphNodeDependsO
continue
}

deps, fromParentModule := n.dependsOn(g, mod)
deps, fromParentModule := m.dependsOn(g, mod)
for _, dep := range deps {
// add the dependency
res = append(res, dep)
Expand Down