Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data source for packer registry #169

Merged
merged 40 commits into from
Aug 6, 2021
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
5d91f37
add file for packer registry
SwampDragons Jul 12, 2021
e594afe
add packe registry to provider.go
SwampDragons Jul 12, 2021
d381e02
packer client
SwampDragons Jul 13, 2021
9092ab2
WIP
azr Jul 14, 2021
d1867aa
Merge branch 'data_source_packer_registry' of ssh://github.com/hashic…
azr Jul 14, 2021
3be9aca
flattenPackerBuildList
azr Jul 14, 2021
7500dac
fix pointer type errors after dependency update
azr Jul 14, 2021
76bb967
Update data_source_packer_image.go
azr Jul 14, 2021
bfe5099
add & run tests
azr Jul 15, 2021
1d3ed90
Update data_source_packer_image.go
azr Jul 15, 2021
ff12e1e
it works !
azr Jul 16, 2021
66254ef
go mod tidy
azr Jul 21, 2021
7afcdc7
fix description for bucket
azr Jul 27, 2021
8063413
nest iteration stuff under iteration
azr Jul 27, 2021
be602f5
start showing builds & hcp_packer_image -> hcp_packer_image_iteration
azr Jul 27, 2021
4dbb8bd
Revert "fix pointer type errors after dependency update"
azr Jul 27, 2021
5cb2338
Update internal/provider/data_source_packer_image_iteration.go
azr Jul 28, 2021
2db3cba
Update internal/provider/data_source_packer_image_iteration.go
azr Jul 28, 2021
aa1be36
up go mods
azr Jul 28, 2021
9874c8b
Update data_source_packer_image_iteration.go
azr Jul 28, 2021
9e5728f
Create packer_image_iteration.md
azr Jul 28, 2021
63d18cf
make channel mandatory for now
azr Jul 28, 2021
630880f
go get github.com/hashicorp/[email protected] && go mod tidy
azr Aug 3, 2021
280ff02
Update internal/provider/data_source_packer_image_iteration.go
azr Aug 3, 2021
9d00c67
Create data-source.tf
azr Aug 3, 2021
ca6d036
Create packer_image_iteration.md.tmpl
azr Aug 3, 2021
7397f83
Update packer_image_iteration.md
azr Aug 3, 2021
73d79d8
Merge branch 'data_source_packer_registry' of ssh://github.com/hashic…
azr Aug 3, 2021
5512bac
Delete data_source_packer_image_iteration_test.go
azr Aug 3, 2021
31760fc
Update data_source_packer_image_iteration.go
azr Aug 3, 2021
6707024
Update packer_image_iteration.md
azr Aug 3, 2021
f4bcb91
Revert "Delete data_source_packer_image_iteration_test.go"
azr Aug 3, 2021
3c2cb91
auto-upsert acc tests data
azr Aug 3, 2021
34b5f2c
go get github.com/hashicorp/[email protected] && tidy
azr Aug 4, 2021
7ae2cd9
Update data_source_packer_image_iteration.go
azr Aug 4, 2021
14ad128
Update data_source_packer_image_iteration_test.go
azr Aug 4, 2021
b835c06
go get -u github.com/hashicorp/terraform-json
azr Aug 4, 2021
d9eacad
Merge remote-tracking branch 'origin/main' into data_source_packer_re…
azr Aug 4, 2021
6343175
manually fix docs
azr Aug 5, 2021
cdc0c02
Add CheckDestroy step for Packer Data Source Test (#180)
Aug 6, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ require (
github.com/hashicorp/go-cty v1.4.1-0.20200414143053-d3edf31b6320
github.com/hashicorp/go-version v1.3.0
github.com/hashicorp/hcl/v2 v2.8.2 // indirect
github.com/hashicorp/hcp-sdk-go v0.10.0
github.com/hashicorp/hcp-sdk-go v0.10.1-0.20210709190444-22372a371cc8
github.com/hashicorp/terraform-exec v0.13.3 // indirect
github.com/hashicorp/terraform-plugin-docs v0.4.0
github.com/hashicorp/terraform-plugin-sdk/v2 v2.5.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -351,8 +351,8 @@ github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ
github.com/hashicorp/hcl/v2 v2.3.0/go.mod h1:d+FwDBbOLvpAM3Z6J7gPj/VoAGkNe/gm352ZhjJ/Zv8=
github.com/hashicorp/hcl/v2 v2.8.2 h1:wmFle3D1vu0okesm8BTLVDyJ6/OL9DCLUwn0b2OptiY=
github.com/hashicorp/hcl/v2 v2.8.2/go.mod h1:bQTN5mpo+jewjJgh8jr0JUguIi7qPHUF6yIfAEN3jqY=
github.com/hashicorp/hcp-sdk-go v0.10.0 h1:RB2GD9orNtINInggRmsw1iC4ODj5X7JagXK5CY+7Db4=
github.com/hashicorp/hcp-sdk-go v0.10.0/go.mod h1:Tm9BAlTkp6jknZ0YNxF/556JBC/meCN1LUmWFN38HsU=
github.com/hashicorp/hcp-sdk-go v0.10.1-0.20210709190444-22372a371cc8 h1:OuweqkFd5uNrEbSEs5tThGjODjYNEm/w5SjY8VmwVUU=
github.com/hashicorp/hcp-sdk-go v0.10.1-0.20210709190444-22372a371cc8/go.mod h1:Tm9BAlTkp6jknZ0YNxF/556JBC/meCN1LUmWFN38HsU=
github.com/hashicorp/logutils v1.0.0 h1:dLEQVugN8vlakKOUE3ihGLTZJRB4j+M2cdTm/ORI65Y=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
github.com/hashicorp/terraform-exec v0.12.0/go.mod h1:SGhto91bVRlgXQWcJ5znSz+29UZIa8kpBbkGwQ+g9E8=
Expand Down
12 changes: 10 additions & 2 deletions internal/clients/client.go
Original file line number Diff line number Diff line change
@@ -1,17 +1,23 @@
package clients

import (
cloud_consul "github.com/hashicorp/hcp-sdk-go/clients/cloud-consul-service/preview/2021-02-04/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-consul-service/preview/2021-02-04/client/consul_service"
cloud_network "github.com/hashicorp/hcp-sdk-go/clients/cloud-network/preview/2020-09-07/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-network/preview/2020-09-07/client/network_service"
cloud_operation "github.com/hashicorp/hcp-sdk-go/clients/cloud-operation/preview/2020-05-05/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-operation/preview/2020-05-05/client/operation_service"
cloud_resource_manager "github.com/hashicorp/hcp-sdk-go/clients/cloud-resource-manager/preview/2019-12-10/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-resource-manager/preview/2019-12-10/client/organization_service"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-resource-manager/preview/2019-12-10/client/project_service"

cloud_consul "github.com/hashicorp/hcp-sdk-go/clients/cloud-consul-service/preview/2021-02-04/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-consul-service/preview/2021-02-04/client/consul_service"

cloud_vault "github.com/hashicorp/hcp-sdk-go/clients/cloud-vault-service/preview/2020-11-25/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-vault-service/preview/2020-11-25/client/vault_service"

cloud_packer "github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/client/packer_service"

sdk "github.com/hashicorp/hcp-sdk-go/httpclient"
)

Expand All @@ -25,6 +31,7 @@ type Client struct {
Organization organization_service.ClientService
Consul consul_service.ClientService
Vault vault_service.ClientService
Packer packer_service.ClientService
}

// ClientConfig specifies configuration for the client that interacts with HCP
Expand Down Expand Up @@ -69,6 +76,7 @@ func NewClient(config ClientConfig) (*Client, error) {
Organization: cloud_resource_manager.New(httpClient, nil).OrganizationService,
Consul: cloud_consul.New(httpClient, nil).ConsulService,
Vault: cloud_vault.New(httpClient, nil).VaultService,
Packer: cloud_packer.New(httpClient, nil).PackerService,
}

return client, nil
Expand Down
26 changes: 26 additions & 0 deletions internal/clients/packer.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package clients

import (
"context"

"github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/client/packer_service"
packermodels "github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/models"
sharedmodels "github.com/hashicorp/hcp-sdk-go/clients/cloud-shared/v1/models"
)

func GetPackerChannelBySlug(ctx context.Context, client *Client, loc *sharedmodels.HashicorpCloudLocationLocation,
bucketName string, channel string) (*packermodels.HashicorpCloudPackerChannel, error) {

getParams := packer_service.NewGetChannelParams()
getParams.BucketSlug = bucketName
getParams.Slug = channel
getParams.LocationOrganizationID = loc.OrganizationID
getParams.LocationProjectID = loc.ProjectID

getResp, err := client.Packer.GetChannel(getParams, nil)
if err != nil {
return nil, err
}

return getResp.Payload.Channel, nil
}
210 changes: 210 additions & 0 deletions internal/provider/data_source_packer_image_iteration.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
package provider

import (
"context"
"log"
"time"

packermodels "github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/models"
sharedmodels "github.com/hashicorp/hcp-sdk-go/clients/cloud-shared/v1/models"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-provider-hcp/internal/clients"
)

// a minute sounds like a lot already since this would be mainly pulling data.
azr marked this conversation as resolved.
Show resolved Hide resolved
var defaultPackerTimeout = time.Minute

func dataSourcePackerImageIteration() *schema.Resource {
return &schema.Resource{
Description: "The Packer Image data source iteration gets the most recent iteration (or build) of an image given an channel.",
azr marked this conversation as resolved.
Show resolved Hide resolved
ReadContext: dataSourcePackerImageRead,
Timeouts: &schema.ResourceTimeout{
Default: &defaultPackerTimeout,
},
Schema: map[string]*schema.Schema{
// Required inputs
"bucket": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any plans to make bucket and channel resources so this can all be managed by TF?

Description: "The slug of the HCP Packer Registry image bucket to pull from.",
Type: schema.TypeString,
Required: true,
ValidateDiagFunc: validateSlugID,
},
// Optional inputs
"channel": {
Description: "The channel that points to the version of the image you want.",
Type: schema.TypeString,
Optional: true,
ValidateDiagFunc: validateSlugID,
},
// computed outputs
"organization_id": {
Description: "The ID of the organization this HCP Packer registry is located in.",
Type: schema.TypeString,
Computed: true,
},
"project_id": {
Description: "The ID of the project this HCP Packer registry is located in.",
Type: schema.TypeString,
Computed: true,
},

// Actual iteration:

"id": {
azr marked this conversation as resolved.
Show resolved Hide resolved
Description: "ID of this iteration",
Type: schema.TypeString,
Computed: true,
},
"incremental_version": {
Description: "Incremental version of this iteration",
Type: schema.TypeInt,
Computed: true,
},
"created_at": {
Description: "Creation time of this iteration",
Type: schema.TypeString,
Computed: true,
},
"builds": {
Description: "Builds for this iteration",
Type: schema.TypeList,
Computed: true,
Elem: &schema.Schema{
Type: schema.TypeSet,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"cloud_provider": {
Type: schema.TypeString,
},
"component_type": {
Type: schema.TypeString,
},
"created_at": {
Type: schema.TypeString,
},
// "id": {
// Type: schema.TypeString,
// },
"images": {
Type: schema.TypeList,
Elem: &schema.Schema{
Type: schema.TypeSet,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"created_at": {
Type: schema.TypeString,
},
"id": {
Type: schema.TypeString,
},
"image_id": {
Type: schema.TypeString,
},
"region": {
Type: schema.TypeString,
},
},
},
},
},
"iteration_id": {
Type: schema.TypeString,
},
"labels": {
Type: schema.TypeMap,
Elem: &schema.Schema{
Type: schema.TypeString,
},
},
"packer_run_uuid": {
Type: schema.TypeString,
},
"status": {
Type: schema.TypeString,
},
"updated_at": {
Type: schema.TypeString,
},
},
},
},
},
},
}
}

func dataSourcePackerImageRead(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
bucketName := d.Get("bucket").(string)
channelSlug := d.Get("channel").(string)
client := meta.(*clients.Client)

loc := &sharedmodels.HashicorpCloudLocationLocation{
OrganizationID: client.Config.OrganizationID,
ProjectID: client.Config.ProjectID,
}

if err := setLocationData(d, loc); err != nil {
return diag.FromErr(err)
}

log.Printf("[INFO] Reading HCP Packer registry (%s) [project_id=%s, organization_id=%s, channel=%s]", bucketName, loc.ProjectID, loc.OrganizationID, channelSlug)

channel, err := clients.GetPackerChannelBySlug(ctx, client, loc, bucketName, channelSlug)
if err != nil {
return diag.FromErr(err)
}

iteration := channel.Pointer.Iteration

d.SetId(channel.Pointer.Iteration.ID)

d.Set("incremental_version", iteration.IncrementalVersion)
d.Set("created_at", iteration.CreatedAt.String())
d.Set("builds", flattenPackerBuildList(iteration.Builds))

return nil
}

func setLocationData(d *schema.ResourceData, loc *sharedmodels.HashicorpCloudLocationLocation) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooo, could make this a shared helper 👀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll maybe do that in another PR, if time !

if err := d.Set("organization_id", loc.OrganizationID); err != nil {
return err
}

if err := d.Set("project_id", loc.ProjectID); err != nil {
return err
}
return nil
}

func flattenPackerBuildList(builds []*packermodels.HashicorpCloudPackerBuild) (flattened []map[string]interface{}) {
for _, build := range builds {
out := map[string]interface{}{
"cloud_provider": build.CloudProvider,
"component_type": build.ComponentType,
"created_at": build.CreatedAt.String(),
"id": build.ID,
"images": flattenPackerBuildImagesList(build.Images),
"iteration_id": build.IterationID,
"labels": build.Labels,
"packer_run_uuid": build.PackerRunUUID,
"status": build.Status,
"updated_at": build.UpdatedAt.String(),
}
flattened = append(flattened, out)
}
return
}

func flattenPackerBuildImagesList(images []*packermodels.HashicorpCloudPackerImage) (flattened []map[string]interface{}) {
for _, image := range images {
out := map[string]interface{}{
"created_at": image.CreatedAt.String(),
"id": image.ID,
"image_id": image.ImageID,
"region": image.Region,
}
flattened = append(flattened, out)
}
return
}
38 changes: 38 additions & 0 deletions internal/provider/data_source_packer_image_iteration_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package provider

import (
"os"
"testing"

"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
)

var (
testAccPackerAlpineProductionImage = `
data "hcp_packer_image_iteration" "alpine" {
bucket = "alpine"
channel = "production"
}`
)

func TestAcc_dataSourcePacker(t *testing.T) {
resourceName := "data.hcp_packer_image_iteration.alpine"

resource.Test(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t, false) },
ProviderFactories: providerFactories,

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this test upserts some test data that TF doesn't manage directly, we'll need an extra step to clean them up. You can add CheckDestroy: testAccPackerCheckDestroy here, then implement the CheckDestroy function to remove all the upserted data and verify the data source is deleted (peering example)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@azr this has been implemented in #180 which @bcmdarroch has reviewed and approved. The change contain a few additional error checks, and a potential bug fix when requesting a channel that has not been assigned a complete iteration.

I confirmed the panic using the provider on an exiting channel with no assigned iteration, and confirmed that the fix errors as expected. Merging the changes as they are for now so that we can get this in. But please follow up on the changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Wilken !

Steps: []resource.TestStep{
// testing that getting the production channel of the alpine image
// works.

{
Config: testConfig(testAccPackerAlpineProductionImage),
Check: resource.ComposeTestCheckFunc(
resource.TestCheckResourceAttrSet(resourceName, "organization_id"),
resource.TestCheckResourceAttrSet(resourceName, "project_id"),
),
},
},
})
}
1 change: 1 addition & 0 deletions internal/provider/provider.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ func New() func() *schema.Provider {
"hcp_hvn": dataSourceHvn(),
"hcp_hvn_route": dataSourceHVNRoute(),
"hcp_vault_cluster": dataSourceVaultCluster(),
"hcp_packer_image_iteration": dataSourcePackerImageIteration(),
},
ResourcesMap: map[string]*schema.Resource{
"hcp_aws_network_peering": resourceAwsNetworkPeering(),
Expand Down