Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data source for packer registry #169

Merged
merged 40 commits into from
Aug 6, 2021
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
5d91f37
add file for packer registry
SwampDragons Jul 12, 2021
e594afe
add packe registry to provider.go
SwampDragons Jul 12, 2021
d381e02
packer client
SwampDragons Jul 13, 2021
9092ab2
WIP
azr Jul 14, 2021
d1867aa
Merge branch 'data_source_packer_registry' of ssh://github.com/hashic…
azr Jul 14, 2021
3be9aca
flattenPackerBuildList
azr Jul 14, 2021
7500dac
fix pointer type errors after dependency update
azr Jul 14, 2021
76bb967
Update data_source_packer_image.go
azr Jul 14, 2021
bfe5099
add & run tests
azr Jul 15, 2021
1d3ed90
Update data_source_packer_image.go
azr Jul 15, 2021
ff12e1e
it works !
azr Jul 16, 2021
66254ef
go mod tidy
azr Jul 21, 2021
7afcdc7
fix description for bucket
azr Jul 27, 2021
8063413
nest iteration stuff under iteration
azr Jul 27, 2021
be602f5
start showing builds & hcp_packer_image -> hcp_packer_image_iteration
azr Jul 27, 2021
4dbb8bd
Revert "fix pointer type errors after dependency update"
azr Jul 27, 2021
5cb2338
Update internal/provider/data_source_packer_image_iteration.go
azr Jul 28, 2021
2db3cba
Update internal/provider/data_source_packer_image_iteration.go
azr Jul 28, 2021
aa1be36
up go mods
azr Jul 28, 2021
9874c8b
Update data_source_packer_image_iteration.go
azr Jul 28, 2021
9e5728f
Create packer_image_iteration.md
azr Jul 28, 2021
63d18cf
make channel mandatory for now
azr Jul 28, 2021
630880f
go get github.com/hashicorp/[email protected] && go mod tidy
azr Aug 3, 2021
280ff02
Update internal/provider/data_source_packer_image_iteration.go
azr Aug 3, 2021
9d00c67
Create data-source.tf
azr Aug 3, 2021
ca6d036
Create packer_image_iteration.md.tmpl
azr Aug 3, 2021
7397f83
Update packer_image_iteration.md
azr Aug 3, 2021
73d79d8
Merge branch 'data_source_packer_registry' of ssh://github.com/hashic…
azr Aug 3, 2021
5512bac
Delete data_source_packer_image_iteration_test.go
azr Aug 3, 2021
31760fc
Update data_source_packer_image_iteration.go
azr Aug 3, 2021
6707024
Update packer_image_iteration.md
azr Aug 3, 2021
f4bcb91
Revert "Delete data_source_packer_image_iteration_test.go"
azr Aug 3, 2021
3c2cb91
auto-upsert acc tests data
azr Aug 3, 2021
34b5f2c
go get github.com/hashicorp/[email protected] && tidy
azr Aug 4, 2021
7ae2cd9
Update data_source_packer_image_iteration.go
azr Aug 4, 2021
14ad128
Update data_source_packer_image_iteration_test.go
azr Aug 4, 2021
b835c06
go get -u github.com/hashicorp/terraform-json
azr Aug 4, 2021
d9eacad
Merge remote-tracking branch 'origin/main' into data_source_packer_re…
azr Aug 4, 2021
6343175
manually fix docs
azr Aug 5, 2021
cdc0c02
Add CheckDestroy step for Packer Data Source Test (#180)
Aug 6, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions docs/data-sources/packer_image_iteration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
# generated by https://github.com/hashicorp/terraform-plugin-docs
page_title: "hcp_packer_image_iteration Data Source - terraform-provider-hcp"
subcategory: ""
description: |-
The Packer Image data source iteration gets the most recent iteration (or build) of an image, given a channel.
---

# hcp_packer_image_iteration (Data Source)

The Packer Image data source iteration gets the most recent iteration (or build) of an image, given a channel.


Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is auto generated but, can I edit things here ?

Copy link
Contributor

@bcmdarroch bcmdarroch Jul 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To edit the generated doc, you'll want to add a packer_image_iteration template to https://github.com/hashicorp/terraform-provider-hcp/tree/main/templates/data-sources. You can follow the example of HVN route, and add whatever you need.

Follow-up question, is this feature generally available? If not, you'll want to add a banner in the template:
-> **Note:** This feature is currently in private/public beta.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's private beta ! I added that, and tweaked things accordingly 👍🏼 thanks !

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a way to preview docs ?

Copy link
Contributor Author

@azr azr Aug 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, some nested attributes — like the builds one — are not documented, even though I did document the fields in the provider code; is there something I can do to fix that ?

Copy link
Contributor

@bcmdarroch bcmdarroch Aug 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use this tool to preview docs, just have to copy-paste the markdown into the form: https://registry.terraform.io/tools/doc-preview

The nested object fields get displayed below in a Nested Schema section: https://github.com/hashicorp/terraform-provider-hcp/blob/3c2cb9185ca3fc616922556fa13d17231337779b/docs/data-sources/packer_image_iteration.md#nested-schema-for-builds

Copy link
Contributor Author

@azr azr Aug 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use this tool to preview docs, just have to copy-paste the markdown into the form: https://registry.terraform.io/tools/doc-preview

Dope, thanks ! Sorry I missed that info.

The nested object fields get displayed below in a Nested Schema section:

Nice, also, I documented them in here:

"builds": {
Description: "Builds for this iteration. An iteration can have more than one build if it took more than one go to build all images.",
Type: schema.TypeList,
Computed: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"cloud_provider": {
Description: "Name of the cloud provider this image is stored-in, if any.",
Type: schema.TypeString,
Computed: true,
},
"component_type": {
Description: "Name of the builder that built this. Ex: 'amazon-ebs.example'",
Type: schema.TypeString,
Computed: true,
},

But they are not fully populated here:

<a id="nestedatt--builds"></a>
### Nested Schema for `builds`
Read-Only:
- **cloud_provider** (String)
- **component_type** (String)
- **created_at** (String)

Is it normal ? Is there something I could do for this ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I looked and long story short, because of the way TF schema works, this is not possible.


<!-- schema generated by tfplugindocs -->
## Schema

### Required

- **bucket** (String) The slug of the HCP Packer Registry image bucket to pull from.
- **channel** (String) The channel that points to the version of the image you want.

### Optional

- **timeouts** (Block, Optional) (see [below for nested schema](#nestedblock--timeouts))

### Read-Only

- **builds** (List of Object) Builds for this iteration. An iteration can have more than one build if it took more than one go to build all images. (see [below for nested schema](#nestedatt--builds))
- **created_at** (String) Creation time of this iteration
- **id** (String) HCP ID of this iteration
- **incremental_version** (Number) Incremental version of this iteration
- **organization_id** (String) The ID of the organization this HCP Packer registry is located in.
- **project_id** (String) The ID of the project this HCP Packer registry is located in.

<a id="nestedblock--timeouts"></a>
### Nested Schema for `timeouts`

Optional:

- **default** (String)


<a id="nestedatt--builds"></a>
### Nested Schema for `builds`

Read-Only:

- **cloud_provider** (String)
- **component_type** (String)
- **created_at** (String)
- **id** (String)
- **images** (List of Object) (see [below for nested schema](#nestedobjatt--builds--images))
- **labels** (Map of String)
- **packer_run_uuid** (String)
- **status** (String)
- **updated_at** (String)

<a id="nestedobjatt--builds--images"></a>
### Nested Schema for `builds.images`

Read-Only:

- **created_at** (String)
- **id** (String)
- **image_id** (String)
- **region** (String)


2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ require (
github.com/hashicorp/go-cty v1.4.1-0.20200414143053-d3edf31b6320
github.com/hashicorp/go-version v1.3.0
github.com/hashicorp/hcl/v2 v2.8.2 // indirect
github.com/hashicorp/hcp-sdk-go v0.10.0
github.com/hashicorp/hcp-sdk-go v0.10.1-0.20210727200019-239ce8d80646
azr marked this conversation as resolved.
Show resolved Hide resolved
github.com/hashicorp/terraform-exec v0.13.3 // indirect
github.com/hashicorp/terraform-plugin-docs v0.4.0
github.com/hashicorp/terraform-plugin-sdk/v2 v2.5.0
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -351,8 +351,8 @@ github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ
github.com/hashicorp/hcl/v2 v2.3.0/go.mod h1:d+FwDBbOLvpAM3Z6J7gPj/VoAGkNe/gm352ZhjJ/Zv8=
github.com/hashicorp/hcl/v2 v2.8.2 h1:wmFle3D1vu0okesm8BTLVDyJ6/OL9DCLUwn0b2OptiY=
github.com/hashicorp/hcl/v2 v2.8.2/go.mod h1:bQTN5mpo+jewjJgh8jr0JUguIi7qPHUF6yIfAEN3jqY=
github.com/hashicorp/hcp-sdk-go v0.10.0 h1:RB2GD9orNtINInggRmsw1iC4ODj5X7JagXK5CY+7Db4=
github.com/hashicorp/hcp-sdk-go v0.10.0/go.mod h1:Tm9BAlTkp6jknZ0YNxF/556JBC/meCN1LUmWFN38HsU=
github.com/hashicorp/hcp-sdk-go v0.10.1-0.20210727200019-239ce8d80646 h1:4WbtYgGIxeRrGC0hGBtOi6XuJ23024H7r724rCL+eXI=
github.com/hashicorp/hcp-sdk-go v0.10.1-0.20210727200019-239ce8d80646/go.mod h1:Tm9BAlTkp6jknZ0YNxF/556JBC/meCN1LUmWFN38HsU=
github.com/hashicorp/logutils v1.0.0 h1:dLEQVugN8vlakKOUE3ihGLTZJRB4j+M2cdTm/ORI65Y=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
github.com/hashicorp/terraform-exec v0.12.0/go.mod h1:SGhto91bVRlgXQWcJ5znSz+29UZIa8kpBbkGwQ+g9E8=
Expand Down
12 changes: 10 additions & 2 deletions internal/clients/client.go
Original file line number Diff line number Diff line change
@@ -1,17 +1,23 @@
package clients

import (
cloud_consul "github.com/hashicorp/hcp-sdk-go/clients/cloud-consul-service/preview/2021-02-04/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-consul-service/preview/2021-02-04/client/consul_service"
cloud_network "github.com/hashicorp/hcp-sdk-go/clients/cloud-network/preview/2020-09-07/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-network/preview/2020-09-07/client/network_service"
cloud_operation "github.com/hashicorp/hcp-sdk-go/clients/cloud-operation/preview/2020-05-05/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-operation/preview/2020-05-05/client/operation_service"
cloud_resource_manager "github.com/hashicorp/hcp-sdk-go/clients/cloud-resource-manager/preview/2019-12-10/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-resource-manager/preview/2019-12-10/client/organization_service"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-resource-manager/preview/2019-12-10/client/project_service"

cloud_consul "github.com/hashicorp/hcp-sdk-go/clients/cloud-consul-service/preview/2021-02-04/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-consul-service/preview/2021-02-04/client/consul_service"

cloud_vault "github.com/hashicorp/hcp-sdk-go/clients/cloud-vault-service/preview/2020-11-25/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-vault-service/preview/2020-11-25/client/vault_service"

cloud_packer "github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/client"
"github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/client/packer_service"

sdk "github.com/hashicorp/hcp-sdk-go/httpclient"
)

Expand All @@ -25,6 +31,7 @@ type Client struct {
Organization organization_service.ClientService
Consul consul_service.ClientService
Vault vault_service.ClientService
Packer packer_service.ClientService
}

// ClientConfig specifies configuration for the client that interacts with HCP
Expand Down Expand Up @@ -69,6 +76,7 @@ func NewClient(config ClientConfig) (*Client, error) {
Organization: cloud_resource_manager.New(httpClient, nil).OrganizationService,
Consul: cloud_consul.New(httpClient, nil).ConsulService,
Vault: cloud_vault.New(httpClient, nil).VaultService,
Packer: cloud_packer.New(httpClient, nil).PackerService,
}

return client, nil
Expand Down
26 changes: 26 additions & 0 deletions internal/clients/packer.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
package clients

import (
"context"

"github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/client/packer_service"
packermodels "github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/models"
sharedmodels "github.com/hashicorp/hcp-sdk-go/clients/cloud-shared/v1/models"
)

func GetPackerChannelBySlug(ctx context.Context, client *Client, loc *sharedmodels.HashicorpCloudLocationLocation,
bucketName string, channel string) (*packermodels.HashicorpCloudPackerChannel, error) {

getParams := packer_service.NewGetChannelParams()
getParams.BucketSlug = bucketName
getParams.Slug = channel
getParams.LocationOrganizationID = loc.OrganizationID
getParams.LocationProjectID = loc.ProjectID

getResp, err := client.Packer.GetChannel(getParams, nil)
if err != nil {
return nil, err
}

return getResp.Payload.Channel, nil
}
229 changes: 229 additions & 0 deletions internal/provider/data_source_packer_image_iteration.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
package provider

import (
"context"
"log"
"time"

packermodels "github.com/hashicorp/hcp-sdk-go/clients/cloud-packer-service/preview/2021-04-30/models"
sharedmodels "github.com/hashicorp/hcp-sdk-go/clients/cloud-shared/v1/models"
"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
"github.com/hashicorp/terraform-provider-hcp/internal/clients"
)

var defaultPackerTimeout = time.Minute

func dataSourcePackerImageIteration() *schema.Resource {
return &schema.Resource{
Description: "The Packer Image data source iteration gets the most recent iteration (or build) of an image, given a channel.",
ReadContext: dataSourcePackerImageRead,
Timeouts: &schema.ResourceTimeout{
Default: &defaultPackerTimeout,
},
Schema: map[string]*schema.Schema{
// Required inputs
"bucket": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any plans to make bucket and channel resources so this can all be managed by TF?

Description: "The slug of the HCP Packer Registry image bucket to pull from.",
Type: schema.TypeString,
Required: true,
ValidateDiagFunc: validateSlugID,
},
"channel": {
Description: "The channel that points to the version of the image you want.",
Type: schema.TypeString,
Required: true,
ValidateDiagFunc: validateSlugID,
},
// computed outputs
"organization_id": {
Description: "The ID of the organization this HCP Packer registry is located in.",
Type: schema.TypeString,
Computed: true,
},
"project_id": {
Description: "The ID of the project this HCP Packer registry is located in.",
Type: schema.TypeString,
Computed: true,
},

// Actual iteration:

"id": {
azr marked this conversation as resolved.
Show resolved Hide resolved
Description: "HCP ID of this iteration",
Type: schema.TypeString,
Computed: true,
},
"incremental_version": {
Description: "Incremental version of this iteration",
Type: schema.TypeInt,
Computed: true,
},
"created_at": {
Description: "Creation time of this iteration",
Type: schema.TypeString,
Computed: true,
},
"builds": {
Description: "Builds for this iteration. An iteration can have more than one build if it took more than one go to build all images.",
Type: schema.TypeList,
Computed: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"cloud_provider": {
Type: schema.TypeString,
Computed: true,
},
"component_type": {
Description: "Name of the builder that built this. Ex: 'amazon-ebs.example'",
Type: schema.TypeString,
Computed: true,
},
"created_at": {
Description: "Creation time of this build.",
Type: schema.TypeString,
Computed: true,
},
"id": {
Description: "HCP ID of this build.",
Type: schema.TypeString,
Computed: true,
},
"images": {
Description: "Output of the build. This will contain the location or cloud identifier for your images.",
Type: schema.TypeList,
Computed: true,
Elem: &schema.Resource{
Schema: map[string]*schema.Schema{
"created_at": {
Description: "Creation time of this image.",
Type: schema.TypeString,
Computed: true,
},
"id": {
Description: "HCP ID of this image.",
Type: schema.TypeString,
Computed: true,
},
"image_id": {
Description: "Cloud Image ID, URL string identifying this image for the builder that built it.",
Type: schema.TypeString,
Computed: true,
},
"region": {
Description: "Region this image was built from. If any.",
Type: schema.TypeString,
Computed: true,
},
},
},
},
"labels": {
Description: "Labels for this build.",
Type: schema.TypeMap,
Computed: true,
Elem: &schema.Schema{
Type: schema.TypeString,
},
},
"packer_run_uuid": {
Description: "UUID of this build.",
Type: schema.TypeString,
Computed: true,
},
"status": {
Description: "Status this build. DONE means that all images tied to this build were successfully built.",
azr marked this conversation as resolved.
Show resolved Hide resolved
Type: schema.TypeString,
Computed: true,
},
"updated_at": {
Description: "Time this build was last updated.",
Type: schema.TypeString,
Computed: true,
},
},
},
},
},
}
}

func dataSourcePackerImageRead(ctx context.Context, d *schema.ResourceData, meta interface{}) diag.Diagnostics {
bucketName := d.Get("bucket").(string)
channelSlug := d.Get("channel").(string)
client := meta.(*clients.Client)

loc := &sharedmodels.HashicorpCloudLocationLocation{
OrganizationID: client.Config.OrganizationID,
ProjectID: client.Config.ProjectID,
}

if err := setLocationData(d, loc); err != nil {
return diag.FromErr(err)
}

log.Printf("[INFO] Reading HCP Packer registry (%s) [project_id=%s, organization_id=%s, channel=%s]", bucketName, loc.ProjectID, loc.OrganizationID, channelSlug)

channel, err := clients.GetPackerChannelBySlug(ctx, client, loc, bucketName, channelSlug)
if err != nil {
return diag.FromErr(err)
}

iteration := channel.Pointer.Iteration

d.SetId(channel.Pointer.Iteration.ID)

if err := d.Set("incremental_version", iteration.IncrementalVersion); err != nil {
return diag.FromErr(err)
}
if err := d.Set("created_at", iteration.CreatedAt.String()); err != nil {
return diag.FromErr(err)
}
if err := d.Set("builds", flattenPackerBuildList(iteration.Builds)); err != nil {
return diag.FromErr(err)
}

return nil
}

func setLocationData(d *schema.ResourceData, loc *sharedmodels.HashicorpCloudLocationLocation) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooo, could make this a shared helper 👀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll maybe do that in another PR, if time !

if err := d.Set("organization_id", loc.OrganizationID); err != nil {
return err
}

if err := d.Set("project_id", loc.ProjectID); err != nil {
return err
}
return nil
}

func flattenPackerBuildList(builds []*packermodels.HashicorpCloudPackerBuild) (flattened []map[string]interface{}) {
for _, build := range builds {
out := map[string]interface{}{
"cloud_provider": build.CloudProvider,
"component_type": build.ComponentType,
"created_at": build.CreatedAt.String(),
"id": build.ID,
"images": flattenPackerBuildImagesList(build.Images),
"labels": build.Labels,
"packer_run_uuid": build.PackerRunUUID,
"status": build.Status,
"updated_at": build.UpdatedAt.String(),
}
flattened = append(flattened, out)
}
return
}

func flattenPackerBuildImagesList(images []*packermodels.HashicorpCloudPackerImage) (flattened []map[string]interface{}) {
for _, image := range images {
out := map[string]interface{}{
"created_at": image.CreatedAt.String(),
"id": image.ID,
"image_id": image.ImageID,
"region": image.Region,
}
flattened = append(flattened, out)
}
return
}
Loading