Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make port as optional argument #24022

Closed
2 tasks done
ankurbajaj9 opened this issue May 30, 2022 · 2 comments · Fixed by #24550
Closed
2 tasks done

Make port as optional argument #24022

ankurbajaj9 opened this issue May 30, 2022 · 2 comments · Fixed by #24550
Assignees

Comments

@ankurbajaj9
Copy link
Contributor

Apache Airflow version

2.3.1 (latest released)

What happened

my webhdfs is running behind a proxy.
I need to give a complicated URL and do not want to add port in front of the URL

So there are two options

  1. use Schema to add stuff before webhdfs/v1
  2. make adding of port optional so that complete url before webhdfs can be provided in the hostname

What you think should happen instead

error while connecting to webhdfs
[2022-05-27, 15:24:43 UTC] {webhdfs.py:89} INFO - Read operation on namenode abc.xyz.com failed with error: b''

How to reproduce

run webhdfs behind a proxy without exposing the actual servers

Operating System

Debian

Versions of Apache Airflow Providers

2.2.3 apache-airflow-providers-apache-hdfs

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@ankurbajaj9 ankurbajaj9 added area:core kind:bug This is a clearly a bug labels May 30, 2022
@boring-cyborg
Copy link

boring-cyborg bot commented May 30, 2022

Thanks for opening your first issue here! Be sure to follow the issue template!

@potiuk
Copy link
Member

potiuk commented May 30, 2022

Feel free to propose a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants