Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the content type retrieval #7013

Merged
merged 9 commits into from
Oct 15, 2021
Merged

Fix the content type retrieval #7013

merged 9 commits into from
Oct 15, 2021

Conversation

benmoriceau
Copy link
Contributor

@benmoriceau benmoriceau commented Oct 13, 2021

What

This is addressing #5900, the content-type retrieval was not working properly and some binary rows were getting log.

How

Use the MIME type describe in https://www.iana.org/assignments/media-types/media-types.xhtml

@CLAassistant
Copy link

CLAassistant commented Oct 13, 2021

CLA assistant check
All committers have signed the CLA.

@marcosmarxm
Copy link
Member

@benmoriceau let me know when you feel this contribution can be reviewed.

@benmoriceau benmoriceau temporarily deployed to more-secrets October 13, 2021 17:15 Inactive
@benmoriceau
Copy link
Contributor Author

@benmoriceau let me know when you feel this contribution can be reviewed.

Hello @marcosmarxm, Thanks for the quick feedback. This review is a draft since I am learning the build system. I will check with @jrhizor if I properly respect the good practices when he reach the office.

@benmoriceau benmoriceau temporarily deployed to more-secrets October 13, 2021 18:11 Inactive
@benmoriceau benmoriceau temporarily deployed to more-secrets October 13, 2021 20:00 Inactive
@benmoriceau benmoriceau temporarily deployed to more-secrets October 13, 2021 21:09 Inactive
@benmoriceau benmoriceau temporarily deployed to more-secrets October 15, 2021 00:36 Inactive
@benmoriceau benmoriceau temporarily deployed to more-secrets October 15, 2021 00:39 Inactive
@benmoriceau benmoriceau marked this pull request as ready for review October 15, 2021 00:44
Copy link
Contributor

@jrhizor jrhizor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

The JSON library is the only real issue.

@@ -116,4 +142,17 @@ private static String redactSensitiveInfo(String requestBody) {
return requestBody;
}

private static boolean isValidJson(String json) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're using Jackson for JSON manipulation almost everywhere and we have a few helpers for it. Can we use Jsons.tryDeserialize instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -57,6 +57,7 @@ dependencies {
implementation 'org.glassfish.jersey.media:jersey-media-json-jackson'
implementation 'org.glassfish.jersey.ext:jersey-bean-validation'
implementation "org.flywaydb:flyway-core:7.14.0"
implementation 'org.json:json:20210307'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other comment, we can remove this dependency if we use already imported JSON libs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

build.gradle Show resolved Hide resolved
Comment on lines 120 to 123
final List<String> splitLogs = Lists.newArrayList(logs.split(System.lineSeparator()));
Assertions.assertThat(splitLogs)
.extracting((line) -> line.endsWith(expectedLog) && line.contains(expectedLogLevel))
.containsOnlyOnce(true);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
final List<String> splitLogs = Lists.newArrayList(logs.split(System.lineSeparator()));
Assertions.assertThat(splitLogs)
.extracting((line) -> line.endsWith(expectedLog) && line.contains(expectedLogLevel))
.containsOnlyOnce(true);
final Stream<String> matchingLines = logs.lines()
.filter(line -> line.endsWith(expectedLog))
.filter(line -> line.contains(expectedLogLevel));
assertEquals(1, matchingLines.count());

Nitpick: personally easier to read with a count of matches instead of a stream of booleans and checking for true. I'm not against adding assertj in general though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I switch it to the filter version but I kept assetJ for the comparison.

Comment on lines 39 to 41
private static final String method = "POST";
private static final String remoteAddr = "123.456.789.101";
private static final String url = "/api/v1/test";
Copy link
Contributor

@jrhizor jrhizor Oct 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private static final String method = "POST";
private static final String remoteAddr = "123.456.789.101";
private static final String url = "/api/v1/test";
private static final String METHOD = "POST";
private static final String REMOTE_ADDR = "123.456.789.101";
private static final String URL = "/api/v1/test";

and usages ofc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@benmoriceau benmoriceau temporarily deployed to more-secrets October 15, 2021 15:04 Inactive
@jrhizor jrhizor merged commit ec240ed into master Oct 15, 2021
@jrhizor jrhizor deleted the fix-is-gzip branch October 15, 2021 15:59
schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022
* Fix the content type retrieval

* Re-add the original MIME type

* Fix CS

* Update logic and add a test

* Auto format

* PR comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants