Cloudwatch parsing fails for log groups containing slashes #14667
Labels
affected_version:2.0
Issues Reported for 2.0
area:logging
good first issue
kind:bug
This is a clearly a bug
provider:amazon-aws
AWS/Amazon - related issues
Milestone
Comments
|
@mini1989cloud I had to create a set of AWS credentials which are injected as the standard boto3 environment variables in order for Cloudwatch to work. I don't think it knows about or will use an Airflow connection. |
|
I am also having this problem on Apache Airflow 2.1.4 and Amazon Linux 2 as the OS. Thank you maxcountryman for outlining the problem however a fix would be appreciated. |
vsimon
added a commit
to vsimon/airflow
that referenced
this issue
Dec 3, 2021
) When the log group arn contains slashes, the urlparse function parses the group name as part of the 'path' instead of including it as part of the 'netloc'. This fix concatenates both the 'netloc' and 'path' fields together to use as the group arn. For group names without slashes, the functionality remains the same as the group name is still parsed as part of the 'netloc' field and the 'path' field will be empty.
potiuk
pushed a commit
that referenced
this issue
Dec 19, 2021
…19700) When the log group arn contains slashes, the urlparse function parses the group name as part of the 'path' instead of including it as part of the 'netloc'. This fix concatenates both the 'netloc' and 'path' fields together to use as the group arn. For group names without slashes, the functionality remains the same as the group name is still parsed as part of the 'netloc' field and the 'path' field will be empty.
Indeed. anyone providing fix would be great. |
|
@maxcountryman - can you please verify it? I am closing it provisionally (thanks @Taragolis ). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
and others
Apache Airflow version: 2.0.1
Environment:
uname -a):What happened:
When enabling remote logging via Cloudwatch we have observed that you cannot use log groups with slashes in their name, e.g. "/foo/bar/baz" or a more practical example like "/ecs/your-app". This is in our experience a common convention in AWS and doesn't seem to be a documented limitation so we wonder if this is a bug.
What you expected to happen:
We believe that unless this is explicitly designed in this manner this is likely not behaving as desired.
How to reproduce it:
Simply create a log group with forward slashes and you should observe the same effect.
It's even easier to observe this in the REPL, for instance here is the culprit in our code and in the REPL we can reproduce like so:
The text was updated successfully, but these errors were encountered: