Cannot run custom DBT transformation git_repo directory does not exist

[zackliston]

Environment

• Airbyte version: 0.35.12
• OS Version / Instance: Kubernetes via Kustomize
• Deployment: Kubernetes
• Source Connector and version: airbyte/source-mongodb-v2 0.1.11
• Destination Connector and version: airbyte/destination-redshift 0.3.23
• Severity: High
• Step where error happened: Sync Job

Current Behavior

When resetting the data or syncing the connector successfully downloads the git_repo

[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > Running: git clone --depth 5 -b master --single-branch  $GIT_REPO git_repo 
[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > Cloning into 'git_repo'... 
[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > Last 5 commits in git_repo: 
[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > acfec19 feat: initial commit with POC 
[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > f93fbfd Initial commit 
[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > /config 
[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > Running: transform-config --config destination_config.json --integration-type redshift --out /config 
[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > Namespace(config='destination_config.json', integration_type=<DestinationType.redshift: 'redshift'>, out='/config') 
[airbyte-worker-f9b5b9686-jz7vh] 2022-01-31 02:47:05 normalization > transform_redshift 

However, once it tries to actually navigate to the git_repo in the entrypoint.sh file, it gets this error

entrypoint.sh: line 6: cd: git_repo: No such file or directory

Expected Behavior

It should be able to navigate to the downloaded git_repo and run the DBT

Logs

logs-163.txt

Steps to Reproduce

1. Set up a mongodb source + redshift destination
2. Turn off the Data Normalization and set to Raw Data - no normalization (however the behaviour was observed with both settings)
3. Setup a single custom transformation
4. Setup for a private repo and the master branch with all other settings default

Are you willing to submit a PR?

Unable to contribute at this time

[ChristopheDuong]

"duplicates" of #5091

[alafanechere]

Hi @zackliston, the custom transformation is not working on Kube (reasons explains in #5091). I'm closing this as it's a duplicate(please track updates on #5091).

[yuhuishi-convect]

Hit the same wall. #5091 has been around for a while. Wondering what's the current status of this issue.

Is there some detailed description of the workaround solutions? For example, how to package a custom image to work as a custom transformation container?