feat: Snippet CMS 4.0 and djangocms-versioning Data migration #74

adam-murray · 2021-08-12T13:44:28Z

Migration to support CMS4 transition from existing CMS3 installs

…gement command for cms3-cms4

… enabled

…igration

…ned compatibility.

Co-authored-by: Aiky30 <[email protected]>

Aiky30

Sorry Adam, more changes required.

Different issues:

Migrations look to be out of sync to the code
Changes to test code that wasn't needed, only change what you have to on a previous project, especially when it's still maintained in a v3 branch.
Unused factories that can be cleaned

Aiky30 · 2021-09-15T07:21:07Z

djangocms_snippet/models.py

@@ -20,13 +20,13 @@ class Snippet(models.Model):
    """
    name = models.CharField(
        verbose_name=_('Name'),
-        unique=True,
        max_length=255,
    )
    snippet_grouper = models.ForeignKey(
        SnippetGrouper,
        on_delete=models.PROTECT,
        null=True,


You shouldn't need the null or the blank here.

It's added initially for the migration here: https://github.com/django-cms/djangocms-snippet/pull/74/files#diff-7975630682285642c8665843c1405c2016fe129a1bc04adc016a3600bbbc3b49R38
It's then removed here: https://github.com/django-cms/djangocms-snippet/pull/74/files#diff-4ca21242eb21cfa24bd2e3f9e4848d967856c53f97e1cd1838f5dc194efcd14fR21

I think if you were to run the migrations again you would create a new migration file.

Aiky30 · 2021-09-15T07:22:33Z

djangocms_snippet/models.py

@@ -20,13 +20,13 @@ class Snippet(models.Model):
    """
    name = models.CharField(
        verbose_name=_('Name'),
-        unique=True,
        max_length=255,
    )
    snippet_grouper = models.ForeignKey(
        SnippetGrouper,
        on_delete=models.PROTECT,


This doesn't match the migrations that you ran, did you change the protection bit later i.e. after creating the migrations?

https://github.com/django-cms/djangocms-snippet/pull/74/files#diff-4ca21242eb21cfa24bd2e3f9e4848d967856c53f97e1cd1838f5dc194efcd14fR21

If you ran make migrations this will create a new migration file. The migrations files will need to be updated from Cascade.

djangocms_snippet/models.py

Aiky30 · 2021-09-15T07:30:09Z

tests/test_plugins.py

-        self.page.delete()
-        self.home.delete()
-        self.superuser.delete()
+        page_data = {


We should try and change as little as possible of the existing code, this makes maintenance easier. The master branch will receive updates for v3 and we will need to port those updates back in, if there is more noise than needed this can make the job harder.
Can you please move this back to match the original and only change what you need to change, Also the api published param is deprcated and will throw a warning:

self.home = create_page( title="home", template="page.html", language=self.language, "created_by": self.superuser )

Aiky30 · 2021-09-15T07:31:20Z

tests/utils/factories.py

+        abstract = True
+
+
+class TreeNodeFactory(factory.django.DjangoModelFactory):


TreeNodeFactory is not used, please delete it

Aiky30 · 2021-09-15T07:31:31Z

tests/utils/factories.py

+        model = TreeNode
+
+
+class PageFactory(factory.django.DjangoModelFactory):


PageFactory is not used, please delete it

Aiky30 · 2021-09-15T07:31:42Z

tests/utils/factories.py

+        model = Page
+
+
+class PageContentFactory(factory.django.DjangoModelFactory):


PageContentFactory is not used, please delete it

Aiky30 · 2021-09-15T07:31:56Z

tests/utils/factories.py

+        model = PageContent
+
+
+class PageVersionFactory(AbstractVersionFactory):


PageVersionFactory is not used, please delete it

Aiky30 · 2021-09-15T07:32:09Z

tests/utils/factories.py

+        model = Version
+
+
+class PageContentWithVersionFactory(PageContentFactory):


PageContentWithVersionFactory is not used, please delete it

Co-authored-by: Aiky30 <[email protected]>

codecov-commenter · 2021-09-15T07:48:25Z

Codecov Report

Merging #74 (e958a03) into support/django-cms-4.0.x (eeb8072) will decrease coverage by 5.25%.
The diff coverage is 71.15%.

@@                     Coverage Diff                      @@
##           support/django-cms-4.0.x      #74      +/-   ##
============================================================
- Coverage                     98.33%   93.07%   -5.26%     
============================================================
  Files                            16       20       +4     
  Lines                           180      231      +51     
  Branches                          8       12       +4     
============================================================
+ Hits                            177      215      +38     
- Misses                            2       12      +10     
- Partials                          1        4       +3

Impacted Files	Coverage Δ
...ocms_snippet/migrations/0009_auto_20210915_0445.py	`100.00% <ø> (ø)`
...ppet/migrations/0011_cms4_plugin_data_migration.py	`60.00% <60.00%> (ø)`
...ations/0010_cms4_grouper_version_data_migration.py	`65.62% <65.62%> (ø)`
djangocms_snippet/conf.py	`100.00% <100.00%> (ø)`
...ocms_snippet/migrations/0012_auto_20210915_0721.py	`100.00% <100.00%> (ø)`
djangocms_snippet/models.py	`100.00% <100.00%> (ø)`
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eeb8072...e958a03. Read the comment docs.

Aiky30

All good.

Ran this locally with a setup of snippets already created. They create fine and work as expected. :-)

* djangocms-versioning compatibility - Initial Model and cms_config implementation (#75) * feat: Snippet CMS 4.0 and djangocms-versioning Data migration (#74) * feat: Moderation compatibility and version copy method implementation (#77) * feat: Versioning Compatibility Changes (#79) * fix: Fixed bug that causes ChangeView not to use SnippetForm (#80) * feat: Reworked form fields and added the ability to show draft snippets (#81) * fix: Adding valid HTML to the add snippet form throws an error (#82) * fix: form initialisation error in read-only mode (#83) * fix: History URL button broken link on the Snippets change form (#84) * fix: Draft snippet render error due to draft snippets (#86) * feat: Default versioning import user id setting can be set via environment variable (#89) * Release 4.0.0.dev1 (#91) * fix: django-cms 4.0.x - Remove breaking django-treebeard pinning issue (#93) * Release 4.0.0.dev2 (#95) * feat: Removing slug field and hyperlink from list display when versioning is enabled (#98) * fix: Snippet plugin showing ID instead of name(#100) * Fix: Compare view removed unnecessary template logic (#101) * Release 4.0.0.dev3 (#103) * feat: Preview icon renders content in read only (#102) * Release 4.0.0.dev4 (#109) * fix: Added test coverage to admin preview view (#96) * port-feat: pre-commit config added from the v3 workstream (#117) Changes partially taken from this change: a56091c Requires additional work when the django-cms 4.0 workstream is official, the version bump code has been ignored because it would generate official releases only: #116 * feat: django-cms 4.0.x - django 3.2 and Python 3.9 compatibility (#92) * Release 4.0.1.dev1 (#119) * feat: Enable add button on the SnippetPluginForm (#127) * Release 4.0.1.dev2 (#128) * init support for django4.2 (#150) * init support for django4.2 * add change log for add compatiable for django 4.2 and python 3.10 * Update CHANGELOG.rst * Update .pre-commit-config.yaml * fix lint config issue * remove test code * add blackline for flake8 * add pyproject.toml file * add requirement files * fix ci failed issue * format setup.py import section * upgrade isort version to fix pre commit ci issue * upgrade the django-upgrade package to pass through the pre commit ci test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * upgrade packages in pre commit * resort init file changes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * comment the django-upgrade from ci * fix: add missing migration file * fix: update previous file instead of creating new file --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Josh Peng Yu <[email protected]> * fix pyproject.toml config error (#151) * Added site field in models n site filter in admin (#159) * Added site field in models n site filter in admin * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated versioning branch * Add test cases and updated the implementation * Update CHANGELOG.rst Co-authored-by: Mark Walker <[email protected]> * corrected formatting * fixed flake8 issues * removed list_filter --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Mark Walker <[email protected]> * Release/4.1.0 (#155) * bump version to 4.1.0 * bump version to 4.1.0 pyproject.toml * replace test requirement url * update changelog date * update release date. * Update README.rst * Update README.rst * Create PULL_REQUEST_TEMPLATE.md (#85) Add PR template * Only include badges at the top to avoid confusion & removed old badges like travis. (#106) * Added pypi github actions (#108) * FEATURE: adjust setup metadata (#110) * correct doc about default value of DJANGOCMS_SNIPPET_CACHE (#120) * Update to ace 1.9.6, load ace editor from static files if djangocms_static_ace is installed, add dark mode (#123) Fix: Load ace editor from static files if `djangocms-static-ace` is installed Fix: Respect if user has set dark mode Add: Weak dependency on djangocms-static-ace Doc: optional static-ace dependency * ci: Update pypi action branch names (#125) * Bump version: 3.0.0 → 3.1.0 (#126) * fix: django-cms 3.x - Remove breaking django-treebeard pinning issue (#94) * Removed pinning. * Update setup.py --------- Co-authored-by: Fabian Braun <[email protected]> * fix: add noop django 4 migration (#134) * fix: add noop django 4 migration * fix: isort * ci: Prepare release of version 3.1.1 (#139) * Update CHANGELOG.rst * Bump version * Pin treebeard for Django CMS 3.7 tests * Update dj30_cms37.txt * chore: Package overhaul (#140) * ci: Update pre-commit * ci: Run ruff formatter * build: Move package in to src dir * test: Update requirements building * chore: Keeping git happy after moving to src * ci: Add dependabot config * ci: Update tests to run tox * build: Project config * test: Update tox setup * build: Update setup to build_meta * ci: Update pypi workflows * ci: Update linter to ruff * chore: Fix RUF012 * ci: Update linter trigger * chore: Drop python 3.8 * chore: Drop python 3.8 * chore: Adjust coverage config * fix: Test pypi workflow * Bump actions/setup-python from 4 to 5 (#148) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: Test only for migrations within package * Update tests * run tests on all branches * fix tests * Update readme, changelog * Fix common test suite for django CMS 3.11, 4.0, and 4.1 * Update CHANGELOG * Merge branches together * Update precommit-config * fix: ruff precommit * ci: auto fixes from pre-commit hooks for more information, see https://pre-commit.ci * fix: ruff issues * More ruff fixes * Update pyproject.toml * ci: auto fixes from pre-commit hooks for more information, see https://pre-commit.ci * fix: Remove isort and flake8 from setup.cfg * ci: auto fixes from pre-commit hooks for more information, see https://pre-commit.ci * Update setup.cfg * Fix: universal branch (#167) * fix: manifest.in * fix: SnippetAdmin use overrdden change_form_template * ci: auto fixes from pre-commit hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update lint workflow --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Adam Murray <[email protected]> Co-authored-by: Aiky30 <[email protected]> Co-authored-by: Bernard Van Der Vyver <[email protected]> Co-authored-by: Michael Collins <[email protected]> Co-authored-by: 傅瑞曼 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Josh Peng Yu <[email protected]> Co-authored-by: Vipul Narang <[email protected]> Co-authored-by: Mark Walker <[email protected]> Co-authored-by: Nicolai <[email protected]> Co-authored-by: Kaushal Dhungel <[email protected]> Co-authored-by: Simon Krull <[email protected]> Co-authored-by: Liumeo <[email protected]> Co-authored-by: Mark Walker <[email protected]> Co-authored-by: Patrik <[email protected]> Co-authored-by: Mark Walker <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Josh Yu <[email protected]>

adam-murray and others added 30 commits August 9, 2021 13:12

Added cms_config versionable item and grouper

6368d7e

Added admin, requirements for testing and requirements for running

75449d6

Update

9bc367c

Fixed issue in setup.py and added basic first migration

7c9455c

Added migration to cms4

4954ee4

Added migration to cms4

8defe7f

Began to implement versioning switch, added migration as part of mana…

b9f822f

…gement command for cms3-cms4

Moved data migration to management command, added util for versioning…

49e4bb3

… enabled

Added tests for config as well as util for versioning check

dceee9f

Merge branch 'feature/versioned-snippets' into feature/snippet-data-m…

7dfd721

…igration

Updated cms_config

56adbdb

Updated management command

48e125f

Added form, updated model and forms to improve versioned and unversio…

5e12a63

…ned compatibility.

Updated migration, removed unecessary plugin

fdcde12

Merged

13547ae

Removed additional copies of files

9b73d59

Split migration command into migration and command

680ce70

Updates to migration

8dc7f91

Updates to migration

6e937e8

Update migration for review

2df658b

Remove pdb

11a7d3e

Migration upgrade

d7c5dbb

Remove plugin from migration 10, added version creation

8898232

Implemented proper plugin migration

225b9f9

Removed management command

500cf72

Migration 10 now handles lack of versioning gracefully.

2017d90

Add snippet property for backwards compatibility

25e552e

Co-authored-by: Aiky30 <[email protected]>

Cleaned up migration and models

25048ff

Removed unecessary logging

649c4af

Remove redundent comment

bc8d60e

adam-murray added 13 commits September 10, 2021 15:33

Plugin tests now use factories

aa62ced

Linting

9fba572

Linting

aeaa52a

Fixed old issue with migrations

bdf4eaa

Fixed old issue with migrations

aa5a54c

Fixed spelling mistake

2d36a3b

Updated test to no longer use factoris

a8f56f3

Fixed rendering

b87a0d6

Fixed migration error

7375e5b

Updated comments

03df8f7

Migration linting

f895bf1

Remove unique=True from snipept slug field

79eebf4

Removed unique=True on name

38c211d

Aiky30 requested changes Sep 15, 2021

View reviewed changes

Update djangocms_snippet/models.py

ee07ad5

Co-authored-by: Aiky30 <[email protected]>

adam-murray added 10 commits September 15, 2021 11:08

Refactrogin

9b23fa8

Merged

7cbaaa3

Linting

7cb6cc8

Refactored test_plugin to closer to original

b59a31d

Refactoring tests

3153940

Updated tests and migration to make grouper mandatory

3d97dc8

Updated models to even with migrations

366739b

Refactor

3853d19

Linting

e958a03

Removed unused util

c5c236b

Aiky30 approved these changes Sep 15, 2021

View reviewed changes

Aiky30 changed the title ~~Snippet CMS4 Data migration~~ feat: Snippet CMS 4.0 and djangocms-versioning Data migration Sep 15, 2021

Aiky30 merged commit a441d5a into django-cms:support/django-cms-4.0.x Sep 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Snippet CMS 4.0 and djangocms-versioning Data migration #74

feat: Snippet CMS 4.0 and djangocms-versioning Data migration #74

adam-murray commented Aug 12, 2021

Aiky30 left a comment

Aiky30 Sep 15, 2021

Aiky30 Sep 15, 2021

Aiky30 Sep 15, 2021

Aiky30 Sep 15, 2021

Aiky30 Sep 15, 2021

Aiky30 Sep 15, 2021

Aiky30 Sep 15, 2021

Aiky30 Sep 15, 2021

codecov-commenter commented Sep 15, 2021 •

edited

Loading

Aiky30 left a comment

		abstract = True


		class TreeNodeFactory(factory.django.DjangoModelFactory):

		model = TreeNode


		class PageFactory(factory.django.DjangoModelFactory):

		model = Page


		class PageContentFactory(factory.django.DjangoModelFactory):

		model = PageContent


		class PageVersionFactory(AbstractVersionFactory):

		model = Version


		class PageContentWithVersionFactory(PageContentFactory):

feat: Snippet CMS 4.0 and djangocms-versioning Data migration #74

feat: Snippet CMS 4.0 and djangocms-versioning Data migration #74

Conversation

adam-murray commented Aug 12, 2021

Aiky30 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Sep 15, 2021 • edited Loading

Codecov Report

Aiky30 left a comment

Choose a reason for hiding this comment

codecov-commenter commented Sep 15, 2021 •

edited

Loading