diff --git a/docs/resources/google_bigquery_table.md b/docs/resources/google_bigquery_table.md new file mode 100644 index 000000000..b4531f742 --- /dev/null +++ b/docs/resources/google_bigquery_table.md @@ -0,0 +1,111 @@ +--- +title: About the google_bigquery_table resource +platform: gcp +--- + +## Syntax +A `google_bigquery_table` is used to test a Google Table resource + +## Examples +``` +describe google_bigquery_table(project: 'chef-gcp-inspec', dataset: 'inspec_gcp_dataset', name: 'inspec_gcp_bigquery_table') do + it { should exist } + + its('expiration_time') { should cmp '1738882264000' } + its('time_partitioning.type') { should eq 'DAY' } + its('description') { should eq 'A BigQuery table' } +end + +describe google_bigquery_table(project: 'chef-gcp-inspec', dataset: 'inspec_gcp_dataset', name: 'nonexistent') do + it { should_not exist } +end +``` + +## Properties +Properties that can be accessed from the `google_bigquery_table` resource: + + * `table_reference`: Reference describing the ID of this table + + * `datasetId`: The ID of the dataset containing this table + + * `projectId`: The ID of the project containing this table + + * `tableId`: The ID of the the table + + * `creation_time`: The time when this dataset was created, in milliseconds since the epoch. + + * `description`: A user-friendly description of the dataset + + * `friendly_name`: A descriptive name for this table + + * `id`: An opaque ID uniquely identifying the table. + + * `labels`: The labels associated with this dataset. You can use these to organize and group your datasets + + * `last_modified_time`: The time when this table was last modified, in milliseconds since the epoch. + + * `location`: The geographic location where the table resides. This value is inherited from the dataset. + + * `name`: Name of the table + + * `num_bytes`: The size of this table in bytes, excluding any data in the streaming buffer. + + * `num_long_term_bytes`: The number of bytes in the table that are considered "long-term storage". + + * `num_rows`: The number of rows of data in this table, excluding any data in the streaming buffer. + + * `type`: Describes the table type + + * `view`: The view definition. + + * `useLegacySql`: Specifies whether to use BigQuery's legacy SQL for this view + + * `userDefinedFunctionResources`: Describes user-defined function resources used in the query. + + * `time_partitioning`: If specified, configures time-based partitioning for this table. + + * `expirationMs`: Number of milliseconds for which to keep the storage for a partition. + + * `type`: The only type supported is DAY, which will generate one partition per day. + + * `streaming_buffer`: Contains information regarding this table's streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer. + + * `estimatedBytes`: A lower-bound estimate of the number of bytes currently in the streaming buffer. + + * `estimatedRows`: A lower-bound estimate of the number of rows currently in the streaming buffer. + + * `oldestEntryTime`: Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available. + + * `schema`: Describes the schema of this table + + * `fields`: Describes the fields in a table. + + * `encryption_configuration`: Custom encryption configuration + + * `kmsKeyName`: Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key. + + * `expiration_time`: The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. + + * `external_data_configuration`: Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table. + + * `autodetect`: Try to detect schema and format options automatically. Any option specified explicitly will be honored. + + * `compression`: The compression type of the data source + + * `ignoreUnknownValues`: Indicates if BigQuery should allow extra values that are not represented in the table schema + + * `maxBadRecords`: The maximum number of bad records that BigQuery can ignore when reading data + + * `sourceFormat`: The data format + + * `sourceUris`: The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one '*' wildcard character and it must come after the 'bucket' name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the '*' wildcard character is not allowed. + + * `schema`: The schema for the data. Schema is required for CSV and JSON formats + + * `googleSheetsOptions`: Additional options if sourceFormat is set to GOOGLE_SHEETS. + + * `csvOptions`: Additional properties to set if sourceFormat is set to CSV. + + * `bigtableOptions`: Additional options if sourceFormat is set to BIGTABLE. + + * `dataset`: Name of the dataset diff --git a/docs/resources/google_bigquery_tables.md b/docs/resources/google_bigquery_tables.md new file mode 100644 index 000000000..8e5164df0 --- /dev/null +++ b/docs/resources/google_bigquery_tables.md @@ -0,0 +1,47 @@ +--- +title: About the google_bigquery_tables resource +platform: gcp +--- + +## Syntax +A `google_bigquery_tables` is used to test a Google Table resource + +## Examples +``` +describe.one do + google_bigquery_tables(project: 'chef-gcp-inspec', dataset: 'inspec_gcp_dataset').table_references.each do |table_reference| + describe google_bigquery_table(project: 'chef-gcp-inspec', dataset: 'inspec_gcp_dataset', name: table_reference.table_id) do + its('expiration_time') { should cmp '1738882264000' } + its('description') { should eq 'A BigQuery table' } + end + end +end +``` + +## Properties +Properties that can be accessed from the `google_bigquery_tables` resource: + +See [google_bigquery_table.md](google_bigquery_table.md) for more detailed information + * `table_references`: an array of `google_bigquery_table` table_reference + * `creation_times`: an array of `google_bigquery_table` creation_time + * `friendly_names`: an array of `google_bigquery_table` friendly_name + * `ids`: an array of `google_bigquery_table` id + * `labels`: an array of `google_bigquery_table` labels + * `last_modified_times`: an array of `google_bigquery_table` last_modified_time + * `locations`: an array of `google_bigquery_table` location + * `num_bytes`: an array of `google_bigquery_table` num_bytes + * `num_long_term_bytes`: an array of `google_bigquery_table` num_long_term_bytes + * `num_rows`: an array of `google_bigquery_table` num_rows + * `types`: an array of `google_bigquery_table` type + * `views`: an array of `google_bigquery_table` view + * `time_partitionings`: an array of `google_bigquery_table` time_partitioning + * `streaming_buffers`: an array of `google_bigquery_table` streaming_buffer + * `schemas`: an array of `google_bigquery_table` schema + * `encryption_configurations`: an array of `google_bigquery_table` encryption_configuration + * `expiration_times`: an array of `google_bigquery_table` expiration_time + * `external_data_configurations`: an array of `google_bigquery_table` external_data_configuration + * `datasets`: an array of `google_bigquery_table` dataset + +## Filter Criteria +This resource supports all of the above properties as filter criteria, which can be used +with `where` as a block or a method. diff --git a/libraries/google/bigquery/property/table_encryption_configuration.rb b/libraries/google/bigquery/property/table_encryption_configuration.rb new file mode 100644 index 000000000..f7e13ae71 --- /dev/null +++ b/libraries/google/bigquery/property/table_encryption_configuration.rb @@ -0,0 +1,29 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableEncryptionConfiguration + attr_reader :kms_key_name + + def initialize(args = nil) + return if args.nil? + @kms_key_name = args['kmsKeyName'] + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_external_data_configuration.rb b/libraries/google/bigquery/property/table_external_data_configuration.rb new file mode 100644 index 000000000..493dc7d03 --- /dev/null +++ b/libraries/google/bigquery/property/table_external_data_configuration.rb @@ -0,0 +1,62 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +require 'google/bigquery/property/table_external_data_configuration_bigtable_options' +require 'google/bigquery/property/table_external_data_configuration_bigtable_options_column_families' +require 'google/bigquery/property/table_external_data_configuration_csv_options' +require 'google/bigquery/property/table_external_data_configuration_google_sheets_options' +require 'google/bigquery/property/table_external_data_configuration_schema' +require 'google/bigquery/property/table_external_data_configuration_schema_fields' +module GoogleInSpec + module BigQuery + module Property + class TableExternalDataConfiguration + attr_reader :autodetect + + attr_reader :compression + + attr_reader :ignore_unknown_values + + attr_reader :max_bad_records + + attr_reader :source_format + + attr_reader :source_uris + + attr_reader :schema + + attr_reader :google_sheets_options + + attr_reader :csv_options + + attr_reader :bigtable_options + + def initialize(args = nil) + return if args.nil? + @autodetect = args['autodetect'] + @compression = args['compression'] + @ignore_unknown_values = args['ignoreUnknownValues'] + @max_bad_records = args['maxBadRecords'] + @source_format = args['sourceFormat'] + @source_uris = args['sourceUris'] + @schema = GoogleInSpec::BigQuery::Property::TableExternalDataConfigurationSchema.new(args['schema']) + @google_sheets_options = GoogleInSpec::BigQuery::Property::TableExternalDataConfigurationGoogleSheetsOptions.new(args['googleSheetsOptions']) + @csv_options = GoogleInSpec::BigQuery::Property::TableExternalDataConfigurationCsvOptions.new(args['csvOptions']) + @bigtable_options = GoogleInSpec::BigQuery::Property::TableExternalDataConfigurationBigtableOptions.new(args['bigtableOptions']) + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options.rb b/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options.rb new file mode 100644 index 000000000..f5da50ab2 --- /dev/null +++ b/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options.rb @@ -0,0 +1,36 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +require 'google/bigquery/property/table_external_data_configuration_bigtable_options_column_families' +module GoogleInSpec + module BigQuery + module Property + class TableExternalDataConfigurationBigtableOptions + attr_reader :ignore_unspecified_column_families + + attr_reader :read_rowkey_as_string + + attr_reader :column_families + + def initialize(args = nil) + return if args.nil? + @ignore_unspecified_column_families = args['ignoreUnspecifiedColumnFamilies'] + @read_rowkey_as_string = args['readRowkeyAsString'] + @column_families = GoogleInSpec::BigQuery::Property::TableExternalDataConfigurationBigtableOptionsColumnFamiliesArray.parse(args['columnFamilies']) + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options_column_families.rb b/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options_column_families.rb new file mode 100644 index 000000000..713f4f835 --- /dev/null +++ b/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options_column_families.rb @@ -0,0 +1,50 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +require 'google/bigquery/property/table_external_data_configuration_bigtable_options_column_families_columns' +module GoogleInSpec + module BigQuery + module Property + class TableExternalDataConfigurationBigtableOptionsColumnFamilies + attr_reader :columns + + attr_reader :encoding + + attr_reader :family_id + + attr_reader :only_read_latest + + attr_reader :type + + def initialize(args = nil) + return if args.nil? + @columns = GoogleInSpec::BigQuery::Property::TableExternalDataConfigurationBigtableOptionsColumnFamiliesColumnsArray.parse(args['columns']) + @encoding = args['encoding'] + @family_id = args['familyId'] + @only_read_latest = args['onlyReadLatest'] + @type = args['type'] + end + end + + class TableExternalDataConfigurationBigtableOptionsColumnFamiliesArray + def self.parse(value) + return if value.nil? + return TableExternalDataConfigurationBigtableOptionsColumnFamilies.new(value) unless value.is_a?(::Array) + value.map { |v| TableExternalDataConfigurationBigtableOptionsColumnFamilies.new(v) } + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options_column_families_columns.rb b/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options_column_families_columns.rb new file mode 100644 index 000000000..9869d4692 --- /dev/null +++ b/libraries/google/bigquery/property/table_external_data_configuration_bigtable_options_column_families_columns.rb @@ -0,0 +1,49 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableExternalDataConfigurationBigtableOptionsColumnFamiliesColumns + attr_reader :encoding + + attr_reader :field_name + + attr_reader :only_read_latest + + attr_reader :qualifier_string + + attr_reader :type + + def initialize(args = nil) + return if args.nil? + @encoding = args['encoding'] + @field_name = args['fieldName'] + @only_read_latest = args['onlyReadLatest'] + @qualifier_string = args['qualifierString'] + @type = args['type'] + end + end + + class TableExternalDataConfigurationBigtableOptionsColumnFamiliesColumnsArray + def self.parse(value) + return if value.nil? + return TableExternalDataConfigurationBigtableOptionsColumnFamiliesColumns.new(value) unless value.is_a?(::Array) + value.map { |v| TableExternalDataConfigurationBigtableOptionsColumnFamiliesColumns.new(v) } + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_external_data_configuration_csv_options.rb b/libraries/google/bigquery/property/table_external_data_configuration_csv_options.rb new file mode 100644 index 000000000..9620948aa --- /dev/null +++ b/libraries/google/bigquery/property/table_external_data_configuration_csv_options.rb @@ -0,0 +1,44 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableExternalDataConfigurationCsvOptions + attr_reader :allow_jagged_rows + + attr_reader :allow_quoted_newlines + + attr_reader :encoding + + attr_reader :field_delimiter + + attr_reader :quote + + attr_reader :skip_leading_rows + + def initialize(args = nil) + return if args.nil? + @allow_jagged_rows = args['allowJaggedRows'] + @allow_quoted_newlines = args['allowQuotedNewlines'] + @encoding = args['encoding'] + @field_delimiter = args['fieldDelimiter'] + @quote = args['quote'] + @skip_leading_rows = args['skipLeadingRows'] + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_external_data_configuration_google_sheets_options.rb b/libraries/google/bigquery/property/table_external_data_configuration_google_sheets_options.rb new file mode 100644 index 000000000..1833dbba8 --- /dev/null +++ b/libraries/google/bigquery/property/table_external_data_configuration_google_sheets_options.rb @@ -0,0 +1,29 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableExternalDataConfigurationGoogleSheetsOptions + attr_reader :skip_leading_rows + + def initialize(args = nil) + return if args.nil? + @skip_leading_rows = args['skipLeadingRows'] + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_external_data_configuration_schema.rb b/libraries/google/bigquery/property/table_external_data_configuration_schema.rb new file mode 100644 index 000000000..6ef8eef8d --- /dev/null +++ b/libraries/google/bigquery/property/table_external_data_configuration_schema.rb @@ -0,0 +1,30 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +require 'google/bigquery/property/table_external_data_configuration_schema_fields' +module GoogleInSpec + module BigQuery + module Property + class TableExternalDataConfigurationSchema + attr_reader :fields + + def initialize(args = nil) + return if args.nil? + @fields = GoogleInSpec::BigQuery::Property::TableExternalDataConfigurationSchemaFieldsArray.parse(args['fields']) + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_external_data_configuration_schema_fields.rb b/libraries/google/bigquery/property/table_external_data_configuration_schema_fields.rb new file mode 100644 index 000000000..c2990b4ae --- /dev/null +++ b/libraries/google/bigquery/property/table_external_data_configuration_schema_fields.rb @@ -0,0 +1,49 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableExternalDataConfigurationSchemaFields + attr_reader :description + + attr_reader :fields + + attr_reader :mode + + attr_reader :name + + attr_reader :type + + def initialize(args = nil) + return if args.nil? + @description = args['description'] + @fields = args['fields'] + @mode = args['mode'] + @name = args['name'] + @type = args['type'] + end + end + + class TableExternalDataConfigurationSchemaFieldsArray + def self.parse(value) + return if value.nil? + return TableExternalDataConfigurationSchemaFields.new(value) unless value.is_a?(::Array) + value.map { |v| TableExternalDataConfigurationSchemaFields.new(v) } + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_schema.rb b/libraries/google/bigquery/property/table_schema.rb new file mode 100644 index 000000000..79fa4940d --- /dev/null +++ b/libraries/google/bigquery/property/table_schema.rb @@ -0,0 +1,30 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +require 'google/bigquery/property/table_schema_fields' +module GoogleInSpec + module BigQuery + module Property + class TableSchema + attr_reader :fields + + def initialize(args = nil) + return if args.nil? + @fields = GoogleInSpec::BigQuery::Property::TableSchemaFieldsArray.parse(args['fields']) + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_schema_fields.rb b/libraries/google/bigquery/property/table_schema_fields.rb new file mode 100644 index 000000000..62bf7298c --- /dev/null +++ b/libraries/google/bigquery/property/table_schema_fields.rb @@ -0,0 +1,49 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableSchemaFields + attr_reader :description + + attr_reader :fields + + attr_reader :mode + + attr_reader :name + + attr_reader :type + + def initialize(args = nil) + return if args.nil? + @description = args['description'] + @fields = args['fields'] + @mode = args['mode'] + @name = args['name'] + @type = args['type'] + end + end + + class TableSchemaFieldsArray + def self.parse(value) + return if value.nil? + return TableSchemaFields.new(value) unless value.is_a?(::Array) + value.map { |v| TableSchemaFields.new(v) } + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_streaming_buffer.rb b/libraries/google/bigquery/property/table_streaming_buffer.rb new file mode 100644 index 000000000..a38f90ba3 --- /dev/null +++ b/libraries/google/bigquery/property/table_streaming_buffer.rb @@ -0,0 +1,35 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableStreamingBuffer + attr_reader :estimated_bytes + + attr_reader :estimated_rows + + attr_reader :oldest_entry_time + + def initialize(args = nil) + return if args.nil? + @estimated_bytes = args['estimatedBytes'] + @estimated_rows = args['estimatedRows'] + @oldest_entry_time = args['oldestEntryTime'] + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_table_reference.rb b/libraries/google/bigquery/property/table_table_reference.rb new file mode 100644 index 000000000..43cdb0060 --- /dev/null +++ b/libraries/google/bigquery/property/table_table_reference.rb @@ -0,0 +1,35 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableTableReference + attr_reader :dataset_id + + attr_reader :project_id + + attr_reader :table_id + + def initialize(args = nil) + return if args.nil? + @dataset_id = args['datasetId'] + @project_id = args['projectId'] + @table_id = args['tableId'] + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_time_partitioning.rb b/libraries/google/bigquery/property/table_time_partitioning.rb new file mode 100644 index 000000000..24f2e0b65 --- /dev/null +++ b/libraries/google/bigquery/property/table_time_partitioning.rb @@ -0,0 +1,32 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableTimePartitioning + attr_reader :expiration_ms + + attr_reader :type + + def initialize(args = nil) + return if args.nil? + @expiration_ms = args['expirationMs'] + @type = args['type'] + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_view.rb b/libraries/google/bigquery/property/table_view.rb new file mode 100644 index 000000000..c263d4f7d --- /dev/null +++ b/libraries/google/bigquery/property/table_view.rb @@ -0,0 +1,33 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +require 'google/bigquery/property/table_view_user_defined_function_resources' +module GoogleInSpec + module BigQuery + module Property + class TableView + attr_reader :use_legacy_sql + + attr_reader :user_defined_function_resources + + def initialize(args = nil) + return if args.nil? + @use_legacy_sql = args['useLegacySql'] + @user_defined_function_resources = GoogleInSpec::BigQuery::Property::TableViewUserDefinedFunctionResourcesArray.parse(args['userDefinedFunctionResources']) + end + end + end + end +end diff --git a/libraries/google/bigquery/property/table_view_user_defined_function_resources.rb b/libraries/google/bigquery/property/table_view_user_defined_function_resources.rb new file mode 100644 index 000000000..c44403a25 --- /dev/null +++ b/libraries/google/bigquery/property/table_view_user_defined_function_resources.rb @@ -0,0 +1,40 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +module GoogleInSpec + module BigQuery + module Property + class TableViewUserDefinedFunctionResources + attr_reader :inline_code + + attr_reader :resource_uri + + def initialize(args = nil) + return if args.nil? + @inline_code = args['inlineCode'] + @resource_uri = args['resourceUri'] + end + end + + class TableViewUserDefinedFunctionResourcesArray + def self.parse(value) + return if value.nil? + return TableViewUserDefinedFunctionResources.new(value) unless value.is_a?(::Array) + value.map { |v| TableViewUserDefinedFunctionResources.new(v) } + end + end + end + end +end diff --git a/libraries/google_bigquery_table.rb b/libraries/google_bigquery_table.rb new file mode 100644 index 000000000..ec1dca42d --- /dev/null +++ b/libraries/google_bigquery_table.rb @@ -0,0 +1,106 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +require 'gcp_backend' +require 'google/bigquery/property/table_encryption_configuration' +require 'google/bigquery/property/table_external_data_configuration' +require 'google/bigquery/property/table_external_data_configuration_bigtable_options' +require 'google/bigquery/property/table_external_data_configuration_bigtable_options_column_families' +require 'google/bigquery/property/table_external_data_configuration_csv_options' +require 'google/bigquery/property/table_external_data_configuration_google_sheets_options' +require 'google/bigquery/property/table_external_data_configuration_schema' +require 'google/bigquery/property/table_external_data_configuration_schema_fields' +require 'google/bigquery/property/table_schema' +require 'google/bigquery/property/table_schema_fields' +require 'google/bigquery/property/table_streaming_buffer' +require 'google/bigquery/property/table_table_reference' +require 'google/bigquery/property/table_time_partitioning' +require 'google/bigquery/property/table_view' +require 'google/bigquery/property/table_view_user_defined_function_resources' + +# A provider to manage Google Cloud BigQuery resources. +class Table < GcpResourceBase + name 'google_bigquery_table' + desc 'Table' + supports platform: 'gcp' + + attr_reader :table_reference + attr_reader :creation_time + attr_reader :description + attr_reader :friendly_name + attr_reader :id + attr_reader :labels + attr_reader :last_modified_time + attr_reader :location + attr_reader :name + attr_reader :num_bytes + attr_reader :num_long_term_bytes + attr_reader :num_rows + attr_reader :type + attr_reader :view + attr_reader :time_partitioning + attr_reader :streaming_buffer + attr_reader :schema + attr_reader :encryption_configuration + attr_reader :expiration_time + attr_reader :external_data_configuration + attr_reader :dataset + def base + 'https://www.googleapis.com/bigquery/v2/' + end + + def url + 'projects/{{project}}/datasets/{{dataset}}/tables/{{name}}' + end + + def initialize(params) + super(params.merge({ use_http_transport: true })) + @fetched = @connection.fetch(base, url, params) + parse unless @fetched.nil? + end + + def parse + @table_reference = GoogleInSpec::BigQuery::Property::TableTableReference.new(@fetched['tableReference']) + @creation_time = @fetched['creationTime'] + @description = @fetched['description'] + @friendly_name = @fetched['friendlyName'] + @id = @fetched['id'] + @labels = @fetched['labels'] + @last_modified_time = @fetched['lastModifiedTime'] + @location = @fetched['location'] + @name = @fetched['name'] + @num_bytes = @fetched['numBytes'] + @num_long_term_bytes = @fetched['numLongTermBytes'] + @num_rows = @fetched['numRows'] + @type = @fetched['type'] + @view = GoogleInSpec::BigQuery::Property::TableView.new(@fetched['view']) + @time_partitioning = GoogleInSpec::BigQuery::Property::TableTimePartitioning.new(@fetched['timePartitioning']) + @streaming_buffer = GoogleInSpec::BigQuery::Property::TableStreamingBuffer.new(@fetched['streamingBuffer']) + @schema = GoogleInSpec::BigQuery::Property::TableSchema.new(@fetched['schema']) + @encryption_configuration = GoogleInSpec::BigQuery::Property::TableEncryptionConfiguration.new(@fetched['encryptionConfiguration']) + @expiration_time = @fetched['expirationTime'] + @external_data_configuration = GoogleInSpec::BigQuery::Property::TableExternalDataConfiguration.new(@fetched['externalDataConfiguration']) + @dataset = @fetched['dataset'] + end + + # Handles parsing RFC3339 time string + def parse_time_string(time_string) + time_string ? Time.parse(time_string) : nil + end + + def exists? + !@fetched.nil? + end +end diff --git a/libraries/google_bigquery_tables.rb b/libraries/google_bigquery_tables.rb new file mode 100644 index 000000000..adbaac999 --- /dev/null +++ b/libraries/google_bigquery_tables.rb @@ -0,0 +1,118 @@ +# frozen_string_literal: false + +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- +require 'gcp_backend' +class Tables < GcpResourceBase + name 'google_bigquery_tables' + desc 'Table plural resource' + supports platform: 'gcp' + + attr_reader :table + + filter_table_config = FilterTable.create + + filter_table_config.add(:table_references, field: :table_reference) + filter_table_config.add(:creation_times, field: :creation_time) + filter_table_config.add(:friendly_names, field: :friendly_name) + filter_table_config.add(:ids, field: :id) + filter_table_config.add(:labels, field: :labels) + filter_table_config.add(:last_modified_times, field: :last_modified_time) + filter_table_config.add(:locations, field: :location) + filter_table_config.add(:num_bytes, field: :num_bytes) + filter_table_config.add(:num_long_term_bytes, field: :num_long_term_bytes) + filter_table_config.add(:num_rows, field: :num_rows) + filter_table_config.add(:types, field: :type) + filter_table_config.add(:views, field: :view) + filter_table_config.add(:time_partitionings, field: :time_partitioning) + filter_table_config.add(:streaming_buffers, field: :streaming_buffer) + filter_table_config.add(:schemas, field: :schema) + filter_table_config.add(:encryption_configurations, field: :encryption_configuration) + filter_table_config.add(:expiration_times, field: :expiration_time) + filter_table_config.add(:external_data_configurations, field: :external_data_configuration) + filter_table_config.add(:datasets, field: :dataset) + + filter_table_config.connect(self, :table) + + def base + 'https://www.googleapis.com/bigquery/v2/' + end + + def url + 'projects/{{project}}/datasets/{{dataset}}/tables' + end + + def initialize(params = {}) + super(params.merge({ use_http_transport: true })) + @params = params + @table = fetch_wrapped_resource('tables') + end + + def fetch_wrapped_resource(wrap_path) + # fetch_resource returns an array of responses (to handle pagination) + result = @connection.fetch_all(base, url, @params) + return if result.nil? + + # Conversion of string -> object hash to symbol -> object hash that InSpec needs + converted = [] + result.each do |response| + next if response.nil? || !response.key?(wrap_path) + response[wrap_path].each do |hash| + hash_with_symbols = {} + hash.each_key do |key| + name, value = transform(key, hash) + hash_with_symbols[name] = value + end + converted.push(hash_with_symbols) + end + end + + converted + end + + def transform(key, value) + return transformers[key].call(value) if transformers.key?(key) + + [key.to_sym, value] + end + + def transformers + { + 'tableReference' => ->(obj) { return :table_reference, GoogleInSpec::BigQuery::Property::TableTableReference.new(obj['tableReference']) }, + 'creationTime' => ->(obj) { return :creation_time, obj['creationTime'] }, + 'friendlyName' => ->(obj) { return :friendly_name, obj['friendlyName'] }, + 'id' => ->(obj) { return :id, obj['id'] }, + 'labels' => ->(obj) { return :labels, obj['labels'] }, + 'lastModifiedTime' => ->(obj) { return :last_modified_time, obj['lastModifiedTime'] }, + 'location' => ->(obj) { return :location, obj['location'] }, + 'numBytes' => ->(obj) { return :num_bytes, obj['numBytes'] }, + 'numLongTermBytes' => ->(obj) { return :num_long_term_bytes, obj['numLongTermBytes'] }, + 'numRows' => ->(obj) { return :num_rows, obj['numRows'] }, + 'type' => ->(obj) { return :type, obj['type'] }, + 'view' => ->(obj) { return :view, GoogleInSpec::BigQuery::Property::TableView.new(obj['view']) }, + 'timePartitioning' => ->(obj) { return :time_partitioning, GoogleInSpec::BigQuery::Property::TableTimePartitioning.new(obj['timePartitioning']) }, + 'streamingBuffer' => ->(obj) { return :streaming_buffer, GoogleInSpec::BigQuery::Property::TableStreamingBuffer.new(obj['streamingBuffer']) }, + 'schema' => ->(obj) { return :schema, GoogleInSpec::BigQuery::Property::TableSchema.new(obj['schema']) }, + 'encryptionConfiguration' => ->(obj) { return :encryption_configuration, GoogleInSpec::BigQuery::Property::TableEncryptionConfiguration.new(obj['encryptionConfiguration']) }, + 'expirationTime' => ->(obj) { return :expiration_time, obj['expirationTime'] }, + 'externalDataConfiguration' => ->(obj) { return :external_data_configuration, GoogleInSpec::BigQuery::Property::TableExternalDataConfiguration.new(obj['externalDataConfiguration']) }, + 'dataset' => ->(obj) { return :dataset, obj['dataset'] }, + } + end + + # Handles parsing RFC3339 time string + def parse_time_string(time_string) + time_string ? Time.parse(time_string) : nil + end +end diff --git a/test/integration/verify/controls/google_bigquery_table.rb b/test/integration/verify/controls/google_bigquery_table.rb new file mode 100644 index 000000000..b5fdb7f3d --- /dev/null +++ b/test/integration/verify/controls/google_bigquery_table.rb @@ -0,0 +1,48 @@ +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- + +title 'Test GCP google_bigquery_table resource.' + +gcp_project_id = attribute(:gcp_project_id, default: 'gcp_project_id', description: 'The GCP project identifier.') +bigquery_table = attribute('bigquery_table', default: { + "table_id": "inspec_gcp_bigquery_table", + "description": "A BigQuery table", + "expiration_time": 1738882264000, + "time_partitioning_type": "DAY" +}, description: 'BigQuery table definition') +dataset = attribute('dataset', default: { + "dataset_id": "inspec_gcp_dataset", + "friendly_name": "A BigQuery dataset test", + "description": "Test BigQuery dataset description", + "location": "EU", + "default_table_expiration_ms": 3600000, + "access_writer_role": "WRITER", + "access_writer_special_group": "projectWriters" +}, description: 'BigQuery dataset definition') +control 'google_bigquery_table-1.0' do + impact 1.0 + title 'google_bigquery_table resource test' + + describe google_bigquery_table(project: gcp_project_id, dataset: dataset['dataset_id'], name: bigquery_table['table_id']) do + it { should exist } + + its('expiration_time') { should cmp bigquery_table['expiration_time'] } + its('time_partitioning.type') { should eq bigquery_table['time_partitioning_type'] } + its('description') { should eq bigquery_table['description'] } + end + + describe google_bigquery_table(project: gcp_project_id, dataset: dataset['dataset_id'], name: 'nonexistent') do + it { should_not exist } + end +end diff --git a/test/integration/verify/controls/google_bigquery_tables.rb b/test/integration/verify/controls/google_bigquery_tables.rb new file mode 100644 index 000000000..f9624d493 --- /dev/null +++ b/test/integration/verify/controls/google_bigquery_tables.rb @@ -0,0 +1,45 @@ +# ---------------------------------------------------------------------------- +# +# *** AUTO GENERATED CODE *** AUTO GENERATED CODE *** +# +# ---------------------------------------------------------------------------- +# +# This file is automatically generated by Magic Modules and manual +# changes will be clobbered when the file is regenerated. +# +# Please read more about how to change this file in README.md and +# CONTRIBUTING.md located at the root of this package. +# +# ---------------------------------------------------------------------------- + +title 'Test GCP google_bigquery_tables resource.' + +gcp_project_id = attribute(:gcp_project_id, default: 'gcp_project_id', description: 'The GCP project identifier.') +bigquery_table = attribute('bigquery_table', default: { + "table_id": "inspec_gcp_bigquery_table", + "description": "A BigQuery table", + "expiration_time": 1738882264000, + "time_partitioning_type": "DAY" +}, description: 'BigQuery table definition') +dataset = attribute('dataset', default: { + "dataset_id": "inspec_gcp_dataset", + "friendly_name": "A BigQuery dataset test", + "description": "Test BigQuery dataset description", + "location": "EU", + "default_table_expiration_ms": 3600000, + "access_writer_role": "WRITER", + "access_writer_special_group": "projectWriters" +}, description: 'BigQuery dataset definition') +control 'google_bigquery_tables-1.0' do + impact 1.0 + title 'google_bigquery_tables resource test' + + describe.one do + google_bigquery_tables(project: gcp_project_id, dataset: dataset['dataset_id']).table_references.each do |table_reference| + describe google_bigquery_table(project: gcp_project_id, dataset: dataset['dataset_id'], name: table_reference.table_id) do + its('expiration_time') { should cmp bigquery_table['expiration_time'] } + its('description') { should eq bigquery_table['description'] } + end + end + end +end