Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.4 LPO performance update - Fix learner-metrics endpoint #306

Merged
merged 3 commits into from
Dec 14, 2020

Conversation

johnbaldwin
Copy link
Contributor

@johnbaldwin johnbaldwin commented Dec 12, 2020

Improved performance and fixed filter for the 'learner-metrics' API endpoint

  • Updated figures.admin

    • Added 'UserRelatedMixin' to add a search bar on user fields ( username, email, profile__name) and abstracted the 'user_link' method. Added this mixin to the LearnerCourseGradeMetrics and EnrollmentData model views
    • Added more filter terms to both LearnerCourseGradeMetrics and
      EnrollmentData model views
  • Added new module, figures.queries

    • This abstracts the queryset contruction from the viewset and makes testing the queryset easier in the Django shell without needing to invoke the viewset. Probably a pattern I'll continue
    • See the 'site_users_enrollment_data' function docstring for details
  • Cleaned up figures.serializers

    • Minor stuff: Removed list_serializer_class from LearnerMetricsSerializerV2 and removed commented out code
  • Reworked figures.views.LearnerMetricsViewSetV2

    • Removed the query to get course ids. Now just relies on the
      'course' query params for course ids. The query will filter out
      invalid 'course' values. See notes above on 'figures.query'. This
      helps reduce execution time
    • Added new pattern to figures.views by calling an external function
      to construct the queryset to help with testing and debugging

Fix tox.ini issue and minor changes

  • The Hawthorn tests were not running due to a recent typo I added in the tox.ini 'commands' section. Fixed that.
  • Also calling out Hawthorn in the same way as Ginko and Juniper. Now all have Open edX release specific pytest-.ini files and Hawthorn now specifies the 'OPENEDX_RELEASE' also
  • This means that regardless of what the default environment that is set in the settings, the Hawthorn tests will specify the proper environment

Minor figures.sites performance improvement refactoring

  • Reworked functions to reduce queries
  • Updated tests that broke from the refactoring
  • Add .pytest_cache cleanup to Makefile

* Updated figures.admin
    * Added 'UserRelatedMixin' to add a search bar on user fields (
      username, email, profile__name) and abstracted the 'user_link'
      method. Added this mixin to the LearnerCourseGradeMetrics and
      EnrollmentData model views
    * Added more filter terms to both LearnerCourseGradeMetrics and
      EnrollmentData model views

* Added new module, figures.queries
    * This abstracts the queryset contruction from the viewset and makes
      testing the queryset easier in the Django shell without needing to
      invoke the viewset. Probably a pattern I'll continue
    * See the 'site_users_enrollment_data' function docstring for details

* Cleaned up figures.serializers
    * Minor stuff: Removed list_serializer_class from
      LearnerMetricsSerializerV2 and removed commented out code

* Reworked figures.views.LearnerMetricsViewSetV2
    * Removed the query to get course ids. Now just relies on the
      'course' query params for course ids. The query will filter out
      invalid 'course' values. See notes above on 'figures.query'. This
      helps reduce execution time
    * Added new pattern to figures.views by calling an external function
      to construct the queryset to help with testing and debugging
The Hawthorn tests were not running due to a recent typo I added in the
tox.ini 'commands' section. Fixed that.

Also calling out Hawthorn in the same way as Ginko and Juniper. Now all
have Open edX release specific pytest-<named release>.ini files and
Hawthorn now specifies the 'OPENEDX_RELEASE' also

This means that regardless of what the default environment that is set
in the settings, the Hawthorn tests will specify the proper environment
@johnbaldwin johnbaldwin requested a review from thraxil December 13, 2020 21:05
* Reworked functions to reduce queries
* Updated tests that broke from the refactoring
* Add .pytest_cache cleanup to Makefile
@johnbaldwin johnbaldwin force-pushed the john/0.4-fix-learner-metrics-api branch from 0de0dfa to 0cba5fd Compare December 13, 2020 21:08
ginkgo: OPENEDX_RELEASE = GINKGO
hawthorn: OPENEDX_RELEASE = HAWTHORN
hawthorn_multisite: OPENEDX_RELEASE = HAWTHORN
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice! I may reuse it!

@johnbaldwin johnbaldwin merged commit ff12e4f into master Dec 14, 2020
@johnbaldwin johnbaldwin deleted the john/0.4-fix-learner-metrics-api branch December 14, 2020 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants