style, js, contenttype, import scripts, email templates (#525)

* Squashed commit of the following from BL/contenttypes:
    history
    upgrade to add descrptions to content types
    deleted non-castle icon types
    added new icons for contenttypes

* castle types updated and upgrades to match

* handle max function in css instead of less

* add resuming (quick skipping already imported paths) to import script

* fix folder_contents paging bugs

* allow floats or stringified floats in lat and lng

* update HISTORY

* fix some resume bugs, add delete-resumeable-file flag

* fix too-modern js

* style toggle-advanced like other buttons

* compile resources

* upgrade step

* update HISTORY

* fix max function in less compiler

* compile resources

* typo

* compilation that seems to have worked

* Squashed commit of the following from BL/contenttypes:
    history
    upgrade to add descrptions to content types
    deleted non-castle icon types
    added new icons for contenttypes

* castle types updated and upgrades to match

* update .gitignore

* remove unneeded init-dev and templates folder

* update max statement to compile with newer less/css

* permanently add makefile and package.json

* update package-lock, compiled resources

* update docs and HISTORY

* fix typo

* move from 3004 (old pr) to 3005

* append upgrade description

* add parallax icons

* compile resources, note on svg

* try aligning with flex

* adjust dexterity type descriptions

* icons change and profile/ compile stuff commit

* update/ add svg files

* fix tinymce vertical spacing issue

* fix typo

* add a version for collective.easyform

* swap profile number

* update compilation date

* remove unused inline style

* give recycle bin its own icon

* update package-lock

* add back in compile resource/grunt weirdness

* update compiled files

* update gitignore

* restore init-dev.py

* restore templates makefile

* remove incorrect contents for upgrade 3005

* update descriptions and titles

* update CHANGES

* restore deleted audio.xml contents

* remove easyform after testing icons

* fix error messages for unneeded tiles in dexterity control panel

* add rolemap to upgrade step

* add EmailTemplate to upgrade step

* update CHANGES

* j3cbot templates to add/ move button

* override documentviewer template

* add dexterity types back button

* add missing back button

* update/ simplify back buttons

* update CHANGES

* further arrange svg icons

* recompile resources

* some more icon tweaks

* try to fix initial run of plone-compile-resources

* update README

* tweaks

---------

Co-authored-by: Brian Duncan <[email protected]>

.gitignore

@@ -22,7 +22,6 @@ castle/cms/locales/en/LC_MESSAGES/plone.mo
 pip-selfcheck.json
 *.pyc
 .vscode/
-Gruntfile.js
 log.html
 node_modules/
 report.html
@@ -37,7 +36,5 @@ env
 env2
 dump.rdb
 export/
-/Makefile
-/package.json
-/watch-run.py
-/watchable-grunt.js
+npm-debug.log
+/Gruntfile.js

CHANGES.md

@@ -4,8 +4,13 @@ Changelog
 3.0.0b127 (unreleased)
 ----------------------
 
-- Nothing changed yet.
-
+- Update control panel icons, providing one for every castle configlet
+- Update dexterity types so every one has an icon and a short desription
+- Add EmailTemplate dexterity type for repeated emails
+- Fix data-tiles that don't exist from rendering and spamming output
+  (for dexterity type definitions in control panel in /@@dexterity-types/<DexterityTypeHere>)
+- Allow resuming importjson script
+- fix missing/ inconsistently localed back to site setup button in controlpanels
 
 3.0.0b126 (2023-08-01)
 ----------------------

Makefile

@@ -0,0 +1,17 @@
+.PHONY: less-plone less-plone-logged-in compile-plone compile-plone-logged-in verify-gruntfile-exists
+ERROR_MESSAGE = You must run ./bin/plone-compile-resources --site-id <YOUR_SITE_ID> before these targets will work
+
+verify-gruntfile-exists:
+	test -f Gruntfile.js || ( echo "\n\n$(ERROR_MESSAGE)\n\n"; exit 1 )
+
+less-plone:
+	make verify-gruntfile-exists && grunt less:plone
+
+less-plone-logged-in:
+	make verify-gruntfile-exists && grunt less:plone-logged-in
+
+compile-plone:
+	make verify-gruntfile-exists && grunt compile-plone
+
+compile-plone-logged-in:
+	make verify-gruntfile-exists && grunt compile-plone-logged-in

README.rst

@@ -94,7 +94,12 @@ Development setup on macOS
 7. ``bin/buildout``
 8. Run (in separate terminal windows) ``elasticsearch``, ``redis-server``, ``bin/instance fg``
 9. Browse to http://localhost:8080/
-10. Once you have created a site in the development instance, run init-dev script to populate templates, ``bin/instance run castle/cms/_scripts/init-dev.py``
+
+Once you have created a site in the development instance:
+
+10. Run compile-resources script to allow `Makefile targets <./Makefile>`_ to work, ``bin/plone-compile-resources --site-id <Castle, or whatever you named your site>``
+11. Run init-dev script to populate templates, ``bin/instance run castle/cms/_scripts/init-dev.py``
+12. See the /docs/ folder for further development information
 
 
 Optional Dependencies

castle/cms/_scripts/importjson.py

@@ -46,6 +46,9 @@
     '--stop-if-exception', dest='stop_if_exception', default=False)
 parser.add_argument(
     '--pdb-if-exception', dest='pdb_if_exception', default=False)
+parser.add_argument('--resumable', dest='resumable', default=False, action='store_true')
+parser.add_argument(
+    '--delete-resumable-file', dest='delete_resumable_file', default=False, action='store_true')
 parser.add_argument('--skip-existing', dest='skip_existing', default=True)
 parser.add_argument(
     '--skip-transitioning', dest='skip_transitioning', default=False)
@@ -73,6 +76,10 @@
 stop_if_exception = args.stop_if_exception
 pdb_if_exception = args.pdb_if_exception
 retain_paths = args.retain_paths
+resumable = args.resumable
+delete_resumable_file = args.delete_resumable_file
+successfully_imported_paths = []
+imported_but_not_committed = []
 
 if args.import_paths:
     import_paths = args.import_paths.split(',')
@@ -174,15 +181,23 @@ class CastleImporter(object):
         ('expiration_date', 'setExpirationDate'),
     ]
 
+    @property
+    def successfully_imported_paths(self):
+        return [
+            args.export_directory + successfully_imported_path
+            for successfully_imported_path in successfully_imported_paths
+        ]
+
     def do_import(self):
         self.import_folder(args.export_directory, container=site)
 
     def import_object(self, filepath, container=None):
-        fi = open(filepath)
-        file_read = fi.read()
-        fi.close()
+        if resumable and filepath in self.successfully_imported_paths:
+            print("Skipping {}; Already successfully imported and resuming is enabled".format(filepath))
+            return
         try:
-            data = mjson.loads(file_read)
+            with open(filepath, 'r') as import_file:
+                data = mjson.loads(import_file.read())
         except Exception:
             print("Skipping {}; Unable to read JSON data".format(filepath))
             return
@@ -275,19 +290,24 @@ def import_object(self, filepath, container=None):
                 try:
                     obj = api.content.create(safe_id=True, **creation_data)
                     print('Created {path}'.format(path=path))
+                    imported_but_not_committed.append(path)
                     self.imported_count += 1
                     if self.imported_count % 50 == 0:
                         print('%i processed, committing' % self.imported_count)
                         transaction.commit()
+                        successfully_imported_paths.extend(imported_but_not_committed)
+                        with open('./.successfullyimportedpaths', 'a') as fout:
+                            fout.writelines('\n'.join(imported_but_not_committed) + '\n')
+                            fout.flush()
+                        del imported_but_not_committed[0:]
                 except api.exc.InvalidParameterError:
                     if stop_if_exception:
                         logger.error('Error creating content {}'.format(filepath), exc_info=True)
                         if pdb_if_exception:
                             import pdb
                             pdb.set_trace()
                         raise
-                    logger.error('Error creating content {}'
-                                                .format(filepath), exc_info=True)
+                    logger.error('Error creating content {}'.format(filepath), exc_info=True)
                     return
     # TODO check default folder pages came over as folder with rich text tile
     # TODO any folder pages without default page should have content listing tile
@@ -417,10 +437,20 @@ def create_plain_folder(self, id, container):
 
 
 if __name__ == '__main__':
-
     print('------------------------------')
     print('Start importing')
     print('------------------------------')
+    if resumable:
+        print('------------------------------')
+        print('Resuming enabled, checking for ./.successfullyimportedpaths and loading')
+        # create if it doesn't exist:
+        with open('.successfullyimportedpaths', 'a+'):
+            pass
+        with open('.successfullyimportedpaths', 'r') as fin:
+            for line in fin.readlines():
+                successfully_imported_paths.append(line.replace('\n', ''))
+        print('{} paths loaded'.format(len(successfully_imported_paths)))
+        print('------------------------------')
     if args.overwrite:
         print('------------------------------')
         print('Importing with overwrite enabled')
@@ -429,3 +459,7 @@ def create_plain_folder(self, id, container):
     importer.do_import()
     print('Created {count} Content Items'.format(count=importer.imported_count))
     transaction.commit()
+    print('Import completed')
+    if delete_resumable_file:
+        print('Deleting .successfullyimportedpaths')
+        os.remove('.successfullyimportedpaths')

castle/cms/browser/content/fc.py

@@ -247,7 +247,7 @@ def get_options(self):
         return {
             'title': 'Recycle bin',
             'id': 'recycle',
-            'icon': 'trash',
+            'icon': 'recycle',
             'context': 'default',
             'url': self.context.absolute_url() + '/@@fc-trash',
             'form': {

castle/cms/browser/controlpanel/documentviewer.py

@@ -0,0 +1,12 @@
+from collective.documentviewer.browser.controlpanel import GlobalSettingsForm
+from plone.app.z3cform.layout import wrap_form
+from zope.browserpage.viewpagetemplatefile import ViewPageTemplateFile
+
+
+class CastleDocumentViewerSettingsForm(GlobalSettingsForm):
+    template = ViewPageTemplateFile('templates/documentviewer.pt')
+    label = None
+    description=None
+
+
+GlobalSettingsFormView = wrap_form(CastleDocumentViewerSettingsForm)

castle/cms/browser/controlpanel/templates/announcements.pt

@@ -14,10 +14,8 @@
   <metal:main metal:fill-slot="prefs_configlet_main">
 
     <a
-      href=""
+      href="${context/portal_url}/@@overview-controlpanel"
       id="setup-link"
-      tal:attributes="href string:$portal_url/plone_control_panel"
-      i18n:translate=""
     >
       Site Setup
     </a>

castle/cms/browser/controlpanel/templates/archives-manage.pt

@@ -11,12 +11,13 @@
 
 <metal:main metal:fill-slot="prefs_configlet_main" i18n:domain="plone">
 
-    <a href=""
-       id="setup-link"
-       tal:attributes="href string:$portal_url/@@overview-controlpanel"
-       i18n:translate="">
-       Site Setup
+    <a
+      href="${context/portal_url}/@@overview-controlpanel"
+      id="setup-link"
+    >
+      Site Setup
     </a>
+
     <h1 class="documentFirstHeading"
         i18n:translate="">Archive Management</h1>
 

castle/cms/browser/controlpanel/templates/archives-review.pt

@@ -18,12 +18,13 @@
 
 <metal:main metal:fill-slot="prefs_configlet_main" i18n:domain="plone">
 
-    <a href=""
-       id="setup-link"
-       tal:attributes="href string:$portal_url/@@overview-controlpanel"
-       i18n:translate="">
-       Site Setup
+    <a
+      href="${context/portal_url}/@@overview-controlpanel"
+      id="setup-link"
+    >
+      Site Setup
     </a>
+
     <h1 class="documentFirstHeading"
         i18n:translate="">Archive Management</h1>
 

castle/cms/browser/controlpanel/templates/audit.pt

@@ -12,15 +12,14 @@
             tal:define="connected view/can_connect_to_index;
                         site_path python: '/'.join(context.getPhysicalPath())">
 
-    <a href=""
-       id="setup-link"
-       tal:attributes="href string:$portal_url/plone_control_panel"
-       i18n:translate=""
-       tal:condition="not: view/user">
-        Site Setup
+    <a
+      href="${context/portal_url}/@@overview-controlpanel"
+      id="setup-link"
+      tal:condition="not: view/user">
+    >
+      Site Setup
     </a>
 
-
     <h1 class="documentFirstHeading" tal:content="view/label">View Title</h1>
 
     <div metal:use-macro="context/global_statusmessage/macros/portal_message">

castle/cms/browser/controlpanel/templates/celery.pt

@@ -11,12 +11,13 @@
 
 <metal:main metal:fill-slot="prefs_configlet_main" i18n:domain="plone">
 
-    <a href=""
-       id="setup-link"
-       tal:attributes="href string:$portal_url/@@overview-controlpanel"
-       i18n:translate="">
-       Site Setup
+    <a
+      href="${context/portal_url}/@@overview-controlpanel"
+      id="setup-link"
+    >
+      Site Setup
     </a>
+
     <h1 class="documentFirstHeading"
         i18n:translate="">Celery Tasks</h1>
 

castle/cms/browser/controlpanel/templates/crawler.pt

@@ -10,14 +10,13 @@
 
 <div metal:fill-slot="prefs_configlet_main">
 
-    <a href=""
-       id="setup-link"
-       tal:attributes="href string:$portal_url/plone_control_panel"
-       i18n:translate="">
-        Site Setup
+    <a
+      href="${context/portal_url}/@@overview-controlpanel"
+      id="setup-link"
+    >
+      Site Setup
     </a>
 
-
     <h1 class="documentFirstHeading" tal:content="view/label">View Title</h1>
 
     <div metal:use-macro="context/global_statusmessage/macros/portal_message">