Skip to content

Commit

Permalink
Add Shaped AI tap-iceberg (#1808)
Browse files Browse the repository at this point in the history
  • Loading branch information
BTheunissen authored Jul 18, 2024
1 parent 18d67ba commit 345f741
Show file tree
Hide file tree
Showing 3 changed files with 98 additions and 0 deletions.
1 change: 1 addition & 0 deletions _data/default_variants.yml
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@ extractors:
tap-hotjar: epoch8
tap-hubplanner: airbyte
tap-hubspot: singer-io
tap-iceberg: shaped-ai
tap-idealo-click-report: horze-international
tap-ilevel: singer-io
tap-immuta: immuta
Expand Down
97 changes: 97 additions & 0 deletions _data/meltano/extractors/tap-iceberg/shaped-ai.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
capabilities:
- about
- batch
- catalog
- discover
- schema-flattening
- state
- stream-maps
description: Meltano Tap for ingesting Apache Iceberg data
domain_url: https://iceberg.apache.org
executable: tap-iceberg
keywords:
- meltano_sdk
label: Iceberg
logo_url: /assets/logos/extractors/iceberg.png
maintenance_status: active
name: tap-iceberg
namespace: tap_iceberg
next_steps: ''
pip_url: git+https://github.com/shaped-ai/tap-iceberg.git
quality: unknown
repo: https://github.com/shaped-ai/tap-iceberg
settings:
- description: Compression format to use for batch files.
kind: options
label: Batch Config Encoding Compression
name: batch_config.encoding.compression
options:
- label: Gzip
value: gzip
- label: None
value: none
- description: Format to use for batch files.
kind: options
label: Batch Config Encoding Format
name: batch_config.encoding.format
options:
- label: Jsonl
value: jsonl
- label: Parquet
value: parquet
- description: Prefix to use when writing batch files.
kind: string
label: Batch Config Storage Prefix
name: batch_config.storage.prefix
- description: Root path to use when writing batch files.
kind: string
label: Batch Config Storage Root
name: batch_config.storage.root
- description: The name of the Iceberg catalog
kind: string
label: Catalog Name
name: catalog_name
- description: Additional properties for the Iceberg catalog
kind: object
label: Catalog Properties
name: catalog_properties
- description: The type of Iceberg catalog (e.g., 'hive', 'rest', 'glue')
kind: string
label: Catalog Type
name: catalog_type
- description: The URI of the Iceberg catalog
kind: string
label: Catalog Uri
name: catalog_uri
- description: 'One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization'
kind: array
label: Faker Config Locale
name: faker_config.locale
- description: 'Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator'
kind: string
label: Faker Config Seed
name: faker_config.seed
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
label: Flattening Enabled
name: flattening_enabled
- description: The max depth to flatten schemas.
kind: integer
label: Flattening Max Depth
name: flattening_max_depth
- description: User-defined config values to be used within map expressions.
kind: object
label: Stream Map Config
name: stream_map_config
- description: Config object for stream maps capability. For more information check
out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html).
kind: object
label: Stream Maps
name: stream_maps
settings_group_validation:
- - catalog_type
- catalog_uri
settings_preamble: ''
usage: ''
variant: shaped-ai
Binary file added static/assets/logos/extractors/iceberg.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 345f741

Please sign in to comment.