Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TACKLE-268]-Request failed with status 500 on importing invalid csv file #107

Merged
merged 11 commits into from
Aug 23, 2021

Conversation

carlosthe19916
Copy link
Contributor

@carlosthe19916 carlosthe19916 commented Aug 9, 2021

https://issues.redhat.com/browse/TACKLE-268

About creating SQL scripts to change the column sizes

The current definition of the table application_import is something like:

create table application_import (
    id int8 not null,
    businessService varchar(255),
    description varchar(255),
    applicationname varchar(255),
    comments varchar(255),
    recordType1 varchar(255),
    tag1 varchar(255),
    tagType1 varchar(255),
    tag2 varchar(255),
    tagType2 varchar(255),
    tag3 varchar(255),
    tagType3 varchar(255),
    tag4 varchar(255),
    tagType4 varchar(255),
    errorMessage varchar(255),
    isValid boolean,
    primary key (id)
)

In our last meeting, it was suggested to create a new SQL script to alter the application_import table and align the column sizes to the UI; it means, to reduce some column's size. For instance, applicationname varchar(255) will become applicationname varchar(120) . This change will work in an empty database but it will cause problems if for any reason the DB is not empty and the table application_import contains applicationname column values longer than 120.

@codecov
Copy link

codecov bot commented Aug 9, 2021

Codecov Report

Merging #107 (696ae60) into main (181b593) will increase coverage by 0.04%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main     #107      +/-   ##
============================================
+ Coverage     97.27%   97.32%   +0.04%     
- Complexity      219      220       +1     
============================================
  Files            25       25              
  Lines           441      448       +7     
  Branches         22       23       +1     
============================================
+ Hits            429      436       +7     
  Misses            9        9              
  Partials          3        3              
Impacted Files Coverage Δ
...plicationinventory/entities/ApplicationImport.java 100.00% <ø> (ø)
...e/applicationinventory/services/ImportService.java 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 181b593...696ae60. Read the comment docs.

mrizzi and others added 3 commits August 23, 2021 12:58
* TACKLE-271: case insensitive headers now allowed in import csv file (migtools#82)

* TACKLE-290 importer native image tests (migtools#88)

* TACKLE-290 importer native image tests

* TACKLE-290: removed userTransactions in tests

* TACKLE-290: convert to use endpoints

* TACKLE-290: convert to use endpoints so NativeImageTests don't fail

* TACKLE-290:inititialise mocks

* TACKLE-290: initialise mocks

* TACKLE-290: add wiremock http server to enable native image tests to call out to remote rest api

* TACKLE-290: tidy up importer test code

* TACKLE-290: increase test coverage for importer

* TACKLE-290: added extra test

* TACKLE-290: test coverage

* TACKLE-290: fix problem with controls services query parameters not being picked up

* TACKLE-290: remove status field from ApplicationImport

* Tackle 290 - Fix native tests (migtools#6)

* Comment tests

* Restart changes

* comment test

* comment test

* comment test

* Working tests

* build tests inside own gh account

* fix ImportServiceTest

* Remove comments

* restore gh workflow

* restore gh workflow

* TACKLE-290: fix merged test to be native-compliant

* TACKLE-290 Removed 'baseUri' and changed 'mp-rest/uri' props (migtools#7)

* TACKLE-290: alter order of ImportServiceTest methods

* TACKLE-290: alter test method ordering

* TACKLE-290 Refactored some tests and WireMock (migtools#8)

Co-authored-by: Carlos E. Feria Vila <[email protected]>
Co-authored-by: Marco Rizzi <[email protected]>

* Maven pom.xml(deps-dev): Bump mockito-core from 3.8.0 to 3.12.0 (migtools#117)

Bumps [mockito-core](https://github.com/mockito/mockito) from 3.8.0 to 3.12.0.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](mockito/mockito@v3.8.0...v3.12.0)

---
updated-dependencies:
- dependency-name: org.mockito:mockito-core
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Maven pom.xml(deps-dev): Bump wiremock-jre8 from 2.29.0 to 2.30.1 (migtools#120)

Bumps [wiremock-jre8](https://github.com/tomakehurst/wiremock) from 2.29.0 to 2.30.1.
- [Release notes](https://github.com/tomakehurst/wiremock/releases)
- [Commits](wiremock/wiremock@2.29.0...2.30.1)

---
updated-dependencies:
- dependency-name: com.github.tomakehurst:wiremock-jre8
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Maven pom.xml(deps-dev): Bump assertj-core from 2.6.0 to 3.20.2 (migtools#76)

Bumps [assertj-core](https://github.com/assertj/assertj-core) from 2.6.0 to 3.20.2.
- [Release notes](https://github.com/assertj/assertj-core/releases)
- [Commits](assertj/assertj@assertj-core-2.6.0...assertj-core-3.20.2)

---
updated-dependencies:
- dependency-name: org.assertj:assertj-core
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Maven pom.xml(deps-dev): Bump postgresql from 1.15.2 to 1.16.0 (migtools#96)

Bumps [postgresql](https://github.com/testcontainers/testcontainers-java) from 1.15.2 to 1.16.0.
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/master/CHANGELOG.md)
- [Commits](testcontainers/testcontainers-java@1.15.2...1.16.0)

---
updated-dependencies:
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Maven pom.xml(deps-dev): Bump mockito-core from 3.12.0 to 3.12.1 (migtools#121)

Bumps [mockito-core](https://github.com/mockito/mockito) from 3.12.0 to 3.12.1.
- [Release notes](https://github.com/mockito/mockito/releases)
- [Commits](mockito/mockito@v3.12.0...v3.12.1)

---
updated-dependencies:
- dependency-name: org.mockito:mockito-core
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* TACKLE-268 Properly clean data inserted during test execution ('Import-app-8')

Co-authored-by: Mark Brophy <[email protected]>
Co-authored-by: Carlos E. Feria Vila <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# Conflicts:
#	src/test/java/io/tackle/applicationimporter/ImportServiceTest.java
Copy link
Contributor

@mrizzi mrizzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validation approach has been widely discussed on some meetings and the implementation has been done accordingly so no need to further discuss the validation approach in this PR.
I've reviewed the test class in order to enhance where possible.
I'm aware of the need for quickly get this PR merged so I'll provide a PR to this PR to fix all the comments I raised and also for adding the native test once only HTTP endpoints are used in the test.

@ResourceArg(name = KeycloakTestResource.REALM_NAME, value = "quarkus")
}
)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's just one test method so no need to use @Order(X) annotation.

Suggested change
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)

}
)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class Issue268Test extends SecuredResourceTest {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better keep the naming consistent with the other issue-related test class IssueTACKLE282Test and add the link to the issue for a quick reference in case of need.

Suggested change
public class Issue268Test extends SecuredResourceTest {
// https://issues.redhat.com/browse/TACKLE-268
public class IssueTACKLE268Test extends SecuredResourceTest {

Comment on lines 62 to 68
@InjectMock
@RestClient
TagService mockTagService;

@InjectMock
@RestClient
BusinessServiceService mockBusinessServiceService;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we can leverage and be consistent with the usage of the WireMockControlsServices recently added that allows for removing this code.

Comment on lines 59 to 60
@Inject
UserTransaction userTransaction;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relying only on REST endpoints makes it easier for using the test also in native mode.

Comment on lines 77 to 78
Mockito.when(mockTagService.getListOfTags(0, 1000)).thenReturn(Collections.emptySet());
Mockito.when(mockBusinessServiceService.getListOfBusinessServices(0, 1000)).thenReturn(Collections.emptySet());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From above, we can leverage and be consistent with the usage of the WireMockControlsServices recently added that allows for removing this code.

given()
.accept("application/json")
.when()
.get("/import-summary")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a call to the same /import-summary tested in the call straight above so all the tests can be done with just one call.

"[0].'invalidCount'", is(2)
);

ImportSummary summary = ImportSummary.findAll().firstResult();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The summary ID can be retrieve from the above test call to the /import-summary endpoint avoiding the need for executing this query.

given()
.accept("text/csv")
.when()
.get("/csv-export?importSummaryId=" + summary.id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

importSummaryId should be managed properly as a query parameter

Comment on lines 157 to 160
userTransaction.begin();
ApplicationImport.deleteAll();
ImportSummary.deleteAll();
userTransaction.commit();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using the summary ID retrieve in a call above, this transaction can be replaced with a REST delete call.

.log().all()
.statusCode(200).extract().response();

assertEquals(200, response.getStatusCode());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is tested in the above RESTAssured call.

Suggested change
assertEquals(200, response.getStatusCode());

@mrizzi mrizzi mentioned this pull request Aug 23, 2021
Copy link
Contributor

@mrizzi mrizzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As already reported, validation process already discussed outside of this PR.
The test part of the PR has been reviewed so I think we're fine to merge this.
The PR has been also manually tested in a live Tackle instance (thanks @PhilipCattanach)

I approve even if I've been a contributor but we need this to let #92 build successfully.

Thanks @carlosthe19916 for having done this enhancement 👍

@mrizzi mrizzi merged commit ef41e1f into migtools:main Aug 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants