Questionnaire API #938

oliverroick · 2016-11-16T16:11:36Z

Proposed changes in this pull request

Block questionnaires from uploading if the project has data
Render XForm from questionnaire instance
Add XForm rendering to API view
Extend API to update questioanaires

When should this PR be merged

For sprint 11

Risks

The migration to populate index fields for Question and Questiongroup should be tested on staging against the production database to make sure it works with all XLSForms we currently have in production.

This PR changes the way XForms are created and provided to GeoODK. I have tested with most questionnaires that we use in production but there is a chance that I haven't though of all possible edge cases. Loading forms into GeoODK, collecting and submitting data should be carefully tested on staging.

Follow up actions

Missing permissions were added to policies, when deploying ./manage.py loadpolicies --update needs to be run. \cc @amplifi

bjohare

Spotted a couple of issues that need to be investigated before merging. Other than those, it's looking pretty good..

bjohare · 2016-11-17T14:48:41Z

cadasta/questionnaires/renderer/xform.py

+QUESTION_TYPES = dict(Question.TYPE_CHOICES)
+
+
+class XFormRenderer(BaseRenderer):


The XFormRenderer doesn't populate the meta/instanceID element of the rendered xform. This is required to detect duplicate xform submissions. The meta/instanceID element could be populated either by adding a uuid directly to the meta/instanceID element in the same way as the version is currently populated or by adding something like <bind calculate="concat('uuid:', uuid())" nodeset="/amwkg2a86v48hcaykrc7myng/meta/instanceID" readonly="true()" type="string"/> to the xform model.

bjohare · 2016-11-17T15:10:02Z

cadasta/questionnaires/renderer/xform.py

+QUESTION_TYPES = dict(Question.TYPE_CHOICES)
+
+
+class XFormRenderer(BaseRenderer):


Questions in the rendered xform are not in the the order defined in the xlsform. This matters I think as the form designer probably wants questions answered in order. Also elements like title which provide an introduction to the form in ODK Collect are appearing in the middle of the question sequence.

I addressed this. The order of questions and groups is now determined through the field index, which I added to both Question and QuestionGroup`. This also adds a migration to populate the index field for existing questionnaires.

bjohare · 2016-11-17T16:08:01Z

cadasta/questionnaires/managers.py

-                )
-                instance.md5_hash = self.get_hash(
-                    instance.filename, instance.id_string, instance.version
-                )



Lines 209 - 211 are not required here, they've been moved to the render method in the XFormRenderer.

Right, I left it in there on purpose because this is the part that throws exceptions that are shown to the user as error messages. If we remove it here, we will have to rewrite error handling for XLSForms. I'm not sure if it is worth doing it, since we are thinking about replacing XLSForms.

Ah ok.. sounds good.

bjohare · 2016-11-17T16:34:27Z

cadasta/xforms/views/api.py

    def create(self, request, *args, **kwargs):
        if request.method.upper() == 'HEAD':
            return Response(headers=self.get_openrosa_headers(request),
                            status=status.HTTP_204_NO_CONTENT,)
+        print('REQUEST')


Can this be removed?

bjohare · 2016-11-17T16:34:53Z

cadasta/xforms/views/api.py

@@ -41,10 +41,15 @@ class XFormSubmissionViewSet(OpenRosaHeadersMixin,
    parser_classes = (FormParser, MultiPartParser,)
    serializer_class = XFormSubmissionSerializer

+    def dispatch(self, *args, **kwargs):
+        print(args[0].__dict__)


Can this be removed?

oliverroick · 2016-11-23T11:28:00Z

@bjohare I addressed the requested changes. Before merging we should test the migrations on staging against the production database to make sure the index fields are populated correctly with the XLSForms that are currently in use (/cc @amplifi)

bjohare · 2016-11-23T11:35:55Z

@oliverroick sounds like a plan!

- Block questionnaires from uploading - Render XForm from questionnaire instance - Add XForm rendering to API view - Extend API to update questioanaires

oliverroick added PR: needs review PR: migration labels Nov 16, 2016

oliverroick force-pushed the feature/686-questionnaire-api branch from 52d67c1 to ad59de8 Compare November 16, 2016 17:36

oliverroick closed this Nov 16, 2016

oliverroick reopened this Nov 16, 2016

bjohare suggested changes Nov 17, 2016

View reviewed changes

oliverroick force-pushed the feature/686-questionnaire-api branch 2 times, most recently from 0aaad8d to 61c4389 Compare November 23, 2016 11:22

oliverroick force-pushed the feature/686-questionnaire-api branch 4 times, most recently from bcfe91a to ca4986a Compare November 29, 2016 18:32

bjohare self-assigned this Nov 30, 2016

bjohare added PR: approved and removed PR: needs review labels Nov 30, 2016

oliverroick force-pushed the feature/686-questionnaire-api branch from ca4986a to 66bc9b5 Compare November 30, 2016 13:07

oliverroick changed the title ~~Questioannire API~~ Questionnaire API Nov 30, 2016

oliverroick force-pushed the feature/686-questionnaire-api branch 2 times, most recently from d2d3571 to 46c0f86 Compare December 5, 2016 10:03

oliverroick added 6 commits December 6, 2016 17:12

Questioannire API

042b70e

- Block questionnaires from uploading - Render XForm from questionnaire instance - Add XForm rendering to API view - Extend API to update questioanaires

Add permissions

26efb22

Fix ordering, add UUID

1bba6e2

Remove permission for Xform API view

8bd5670

Refactoring and permissions fix

16df8ae

Fix migrations

db8bf8e

amplifi force-pushed the feature/686-questionnaire-api branch from 46c0f86 to db8bf8e Compare December 6, 2016 07:12

amplifi merged commit d7a6e88 into master Dec 6, 2016

amplifi deleted the feature/686-questionnaire-api branch December 6, 2016 07:23

seav mentioned this pull request Dec 22, 2016

Editing project details does not succeed #1018

Closed

seav mentioned this pull request Jan 3, 2017

Warning Message on XLSForm Reupload #555

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Questionnaire API #938

Questionnaire API #938

oliverroick commented Nov 16, 2016 •

edited

Loading

bjohare left a comment

bjohare Nov 17, 2016

bjohare Nov 17, 2016

oliverroick Nov 23, 2016

bjohare Nov 17, 2016

oliverroick Nov 22, 2016

bjohare Nov 22, 2016

bjohare Nov 17, 2016

oliverroick Nov 22, 2016

bjohare Nov 17, 2016

oliverroick Nov 22, 2016

oliverroick commented Nov 23, 2016

bjohare commented Nov 23, 2016

		QUESTION_TYPES = dict(Question.TYPE_CHOICES)


		class XFormRenderer(BaseRenderer):

Questionnaire API #938

Questionnaire API #938

Conversation

oliverroick commented Nov 16, 2016 • edited Loading

Proposed changes in this pull request

When should this PR be merged

Risks

Follow up actions

bjohare left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oliverroick commented Nov 23, 2016

bjohare commented Nov 23, 2016

oliverroick commented Nov 16, 2016 •

edited

Loading