Bad Request ("invalid mount config for type "bind": bind source path does not exist: /builder/home/.config/gcloud")

[MarkEdmondson1234]

Related to #9

It seems similar to #1 but the fixes for OSX don't seem to apply to Cloud Build containers

This cloudbuild.yaml does not found the gcloud instance, perhaps to do with path?

steps:
- name: google/cloud-sdk:alpine
  id: Clone repo
  entrypoint: git
  args: ['clone', 'https://github.com/GoogleCloudPlatform/composer-local-dev.git']
- name: gcr.io/google.com/cloudsdktool/google-cloud-cli:418.0.0
  id: install pip
  dir: composer-local-dev
  script: ls -la && pip install --upgrade pip && pip install . --user
- name: gcr.io/google.com/cloudsdktool/google-cloud-cli:418.0.0
  entrypoint: /bin/sh
  id: composer-dev
  args: 
    - -c
    - |-
      export PATH=/builder/home/.local/bin:$$PATH
      echo $$PATH
      whereis gcloud
      composer-dev list-available-versions --include-past-releases --limit 10
      composer-dev create testairflow \
        --from-source-environment=${_SOURCE_COMPOSER_ENVIRONMENT} \
        --dags-path=/workspace/dags \
        --location=${_SOURCE_COMPOSER_LOCATION} \
        --project=${PROJECT_ID}
      cat /workspace/composer/testairflow/variables.env
      composer-dev start testairflow --debug
      composer-dev run-airflow-cmd testairflow dags list -o table
substitutions:
  _SOURCE_COMPOSER_ENVIRONMENT: 'composer2-practice'
  _SOURCE_COMPOSER_LOCATION: 'europe-west2'

gives this error:

─ Error ──────────────────────────────────────────────────────────────────────╮
│ Failed to create container with an error: 400 Client Error for               │
│ http+docker://localhost/v1.41/containers/create?name=composer-local-dev-test │
│ airflow: Bad Request ("invalid mount config for type "bind": bind source     │
│ path does not exist: /builder/home/.config/gcloud")                          │
│ Possible reason is that composer-dev was installed in the path that is not   │
│ available to Docker. See                                                     │
│ https://cloud.google.com/composer/docs/composer-2/run-local-airflow-environm │
│ ents#troubleshooting-homebrew for more information.                          │
│                                                                              │
│ To print debug messages please add --debug flag.                             │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Error ──────────────────────────────────────────────────────────────────────╮
│ Failed to run command: environment not running.                              │
│                                                                              │
│ To print debug messages please add --debug flag.                             │
╰──────────────────────────────────────────────────────────────────────────────╯

With relevant log with debug:

Created environment under composer/testairflow. 

...                              
                    DEBUG    urllib3.connectionpool:http:/ connectionpool.py:456
                             /localhost:None "GET                               
                             /v1.41/containers/composer-lo                      
                             cal-dev-testairflow/json                           
                             HTTP/1.1" 404 64                                   
                    DEBUG    root:Container not found.        environment.py:482
                             Traceback (most recent call                        
                             last):                                             
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/cli                   
                             ent.py", line 268, in                              
                             _raise_for_status                                  
                                 response.raise_for_status()                    
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/requests/model                   
                             s.py", line 1021, in                               
                             raise_for_status                                   
                                 raise                                          
                             HTTPError(http_error_msg,                          
                             response=self)                                     
                             requests.exceptions.HTTPError:                     
                             404 Client Error: Not Found for                    
                             url:                                               
                             http+docker://localhost/v1.41/co                   
                             ntainers/composer-local-dev-test                   
                             airflow/json                                       
                                                                                
                             The above exception was the                        
                             direct cause of the following                      
                             exception:                                         
                                                                                
                             Traceback (most recent call                        
                             last):                                             
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/composer_local                   
                             _dev/environment.py", line 474,                    
                             in get_container                                   
                                 container =                                    
                             self.docker_client.containers.ge                   
                             t(self.container_name)                             
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/models/                   
                             containers.py", line 925, in get                   
                                 resp =                                         
                             self.client.api.inspect_containe                   
                             r(container_id)                                    
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/utils/d                   
                             ecorators.py", line 19, in                         
                             wrapped                                            
                                 return f(self, resource_id,                    
                             *args, **kwargs)                                   
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/con                   
                             tainer.py", line 783, in                           
                             inspect_container                                  
                                 return self._result(                           
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/cli                   
                             ent.py", line 274, in _result                      
                                 self._raise_for_status(respo                   
                             nse)                                               
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/cli                   
                             ent.py", line 270, in                              
                             _raise_for_status                                  
                                 raise                                          
                             create_api_error_from_http_excep                   
                             tion(e) from e                                     
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/errors.                   
                             py", line 39, in                                   
                             create_api_error_from_http_excep                   
                             tion                                               
                                 raise cls(e,                                   
                             response=response,                                 
                             explanation=explanation) from e                    
                             docker.errors.NotFound: 404                        
                             Client Error for                                   
                             http+docker://localhost/v1.41/co                   
                             ntainers/composer-local-dev-test                   
                             airflow/json: Not Found ("No                       
                             such container:                                    
                             composer-local-dev-testairflow")                   
                    DEBUG    composer_local_dev.environment:C environment.py:583
                             reating container                                  
                    DEBUG    urllib3.connectionpool:http:/ connectionpool.py:456
                             /localhost:None "POST                              
                             /v1.41/containers/create?name                      
                             =composer-local-dev-testairfl                      
                             ow HTTP/1.1" 400 116                               
                    DEBUG    root:Received docker API error   environment.py:613
                             when creating container.                           
                             Traceback (most recent call                        
                             last):                                             
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/composer_local                   
                             _dev/environment.py", line 718,                    
                             in get_or_create_container                         
                                 return self.get_container()                    
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/composer_local                   
                             _dev/environment.py", line 484,                    
                             in get_container                                   
                                 raise                                          
                             errors.EnvironmentNotFoundError(                   
                             ) from None                                        
                             composer_local_dev.errors.Enviro                   
                             nmentNotFoundError: Failed to                      
                             run command: environment not                       
                             running.                                           
                                                                                
                             To print debug messages please                     
                             add --debug flag.                                  
                                                                                
                             During handling of the above                       
                             exception, another exception                       
                             occurred:                                          
                                                                                
                             Traceback (most recent call                        
                             last):                                             
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/cli                   
                             ent.py", line 268, in                              
                             _raise_for_status                                  
                                 response.raise_for_status()                    
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/requests/model                   
                             s.py", line 1021, in                               
                             raise_for_status                                   
                                 raise                                          
                             HTTPError(http_error_msg,                          
                             response=self)                                     
                             requests.exceptions.HTTPError:                     
                             400 Client Error: Bad Request                      
                             for url:                                           
                             http+docker://localhost/v1.41/co                   
                             ntainers/create?name=composer-lo                   
                             cal-dev-testairflow                                
                                                                                
                             The above exception was the                        
                             direct cause of the following                      
                             exception:                                         
                                                                                
                             Traceback (most recent call                        
                             last):                                             
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/composer_local                   
                             _dev/environment.py", line 602,                    
                             in create_container                                
                                 return                                         
                             self.docker_client.containers.cr                   
                             eate(                                              
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/models/                   
                             containers.py", line 906, in                       
                             create                                             
                                 resp =                                         
                             self.client.api.create_container                   
                             (**create_kwargs)                                  
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/con                   
                             tainer.py", line 431, in                           
                             create_container                                   
                                 return                                         
                             self.create_container_from_confi                   
                             g(config, name, platform)                          
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/con                   
                             tainer.py", line 448, in                           
                             create_container_from_config                       
                                 return self._result(res,                       
                             True)                                              
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/cli                   
                             ent.py", line 274, in _result                      
                                 self._raise_for_status(respo                   
                             nse)                                               
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/api/cli                   
                             ent.py", line 270, in                              
                             _raise_for_status                                  
                                 raise                                          
                             create_api_error_from_http_excep                   
                             tion(e) from e                                     
                               File                                             
                             "/builder/home/.local/lib/python                   
                             3.9/site-packages/docker/errors.                   
                             py", line 39, in                                   
                             create_api_error_from_http_excep                   
                             tion                                               
                                 raise cls(e,                                   
                             response=response,                                 
                             explanation=explanation) from e                    
                             docker.errors.APIError: 400                        
                             Client Error for                                   
                             http+docker://localhost/v1.41/co                   
                             ntainers/create?name=composer-lo                   
                             cal-dev-testairflow: Bad Request                   
                             ("invalid mount config for type                    
                             "bind": bind source path does                      
                             not exist:                                         
                             /builder/home/.config/gcloud")