Add more realistic test data

[oliverroick]

Proposed changes in this pull request

This PR fixes a minor annoyance. The test project London 2 has its project extent in London but its test data is in Germany. The test data itself is just a grid of rectangles.

The changes include:

• I used OSM's land use dataset for Greater London, to provide test data in the area of London and to provide more realistic test data (i.e. polygons of varying shape and size). The test data is read from a text file containing 15,000 GeoJSON geometries. By default only the first 4,000 of these geometries are added as locations to the test project.
• To add more geometries to the test project, I extended the loadfixtures command with an option --records that can be used to specify the maximum number of records loaded into the project. It's usage is ./manage.py loadfixtures --records=10000.
• Moved the fixtures code together with the tests data file into the directory core/fixtures

When should this PR be merged

Anytime

Risks

None

Follow up actions

None

Checklist (for reviewing)

General

• Is this PR explained thoroughly? All code changes must be accounted for in the PR description.
• Is the PR labeled correctly? It should have the migration label if a new migration is added.
• Is the risk level assessment sufficient? The risks section should contain all risks that might be introduced with the PR and which actions we need to take to mitigate these risks. Possible risks are database migrations, new libraries that need to be installed or changes to deployment scripts.

Functionality

• Are all requirements met? Compare implemented functionality with the requirements specification.
• Does the UI work as expected? There should be no Javascript errors in the console; all resources should load. There should be no unexpected errors. Deliberately try to break the feature to find out if there are corner cases that are not handled.

Code

• Do you fully understand the introduced changes to the code? If not ask for clarification, it might uncover ways to solve a problem in a more elegant and efficient way.
• Does the PR introduce any inefficient database requests? Use the debug server to check for duplicate requests.
• Are all necessary strings marked for translation? All strings that are exposed to users via the UI must be marked for translation.

Tests

• Are there sufficient test cases? Ensure that all components are tested individually; models, forms, and serializers should be tested in isolation even if a test for a view covers these components.
• If this is a bug fix, are tests for the issue in place There must be a test case for the bug to ensure the issue won’t regress. Make sure that the tests break without the new code to fix the issue.
• If this is a new feature or a significant change to an existing feature has the manual testing spreadsheet been updated with instructions for manual testing?

Documentation

• Are changes to the UI documented in the platform docs? If this PR introduces new platform site functionality or changes existing ones, the changes must be documented in the Cadasta Platform Documentation.
• Are changes to the API documented in the API docs? If this PR introduces new API functionality or changes existing ones, the changes must be documented in the API docs.
• Are reusable components documented? If this PR introduces components that are relevant to other developers (for instance a mixin for a view or a generic form) they should be documented in the Wiki.

[seav]

I have one minor code suggestion. Aside from that, I have some comments on the test data:

1. Since the test data is from OSM and I think it is a substantial extract, we should honor the OSM license by indicating somewhere that the data is from OSM and that it is licensed under ODbL.
2. Why not convert londondata.txt into a full-fledged GeoJSON file so that it can be displayed on a map using GitHub's GeoJSON display feature. The GeoJSON file can also include the attribution/license mentioned above as an attribute.

[seav]

Why not create the choices list as a variable outside the loop so that we don't recreate the list on every loop iteration?

[oliverroick]

On it...

[oliverroick]

I deliberately decided against using GeoJSON. We only need the geometries, so I wanted to have a simple format that only provides the geometries so I can read the file on line at a time. GeoJSON has some overhead that I would need to remove; it adds complexity to the code that is not necessary. I also don't see what we get from being able to view the geometries on a map on Github.

I agree on adding the licensing information, I will add it to the top of the file.

[bjohare]

Looks good.