Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect and copy files in a shared windows folder from under the airflow docker container. #659

Open
coolvs4 opened this issue Jan 14, 2024 · 0 comments

Comments

@coolvs4
Copy link

coolvs4 commented Jan 14, 2024

I need my DAG to check for a file in a shared folder in the Windows operating system.
A suitable solution was to use the FileSensor tool.
I use this tool as follows:

wait_file = FileSensor(
        task_id="wait_file",
        filepath="//192.168.0.100/shared/log.csv",
        fs_conn_id='my_smb_connect',
        poke_interval = 3,
        timeout = 15
    )

There are no problems when using folders in the local file system.
I was setting up a connection via the web interface Admin -> Connections.
I set Connection Type to File (path) and everything worked.

However, to access remote resources in Windows, you must use the SMB protocol. I have installed additional packages in the airflow image apache-airflow-providers-samba because it is not available by default.

My connection settings are following:

Connection Id: my_smb_connect
Connection Type: Samba
Host: 192.168.0.100
Schema: 
Login: user
Password: ****
Port: 445

But this does not give any result. The file is still not visible.
Please help me, is it the wrong connection setup?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant