Install javascript packages with npm instead of putting the files directly in the source code

.gitignore

@@ -172,9 +172,14 @@ CACHE/
 # VSCode
 .vscode
 
-# Docker data
 # Docker data
 postgres-data/
 
 src/media/
+src/static/
 *.log
+
+node_modules/
+
+# Ignore all generated static files from the npm module
+src/npm/static/

README.md

@@ -24,22 +24,25 @@ To get started with Project Moore, follow these instructions to set up a
 **development** environment:
 
 1. Install Python 3, at least version 3.6 or up.
-2. [Install postgresql](INSTALLING_POSTGRES.md)
-3. Install the following python packages:
+2. [Install the LTS version of NodeJS](https://nodejs.org). If you are using Ubuntu or Ubuntu for windows, use [NVM](https://github.com/nvm-sh/nvm) to install the LTS version node instead.
+3. [Install postgresql](INSTALLING_POSTGRES.md)
+4. Install the following python packages:
     - python3-venv
     - python3-dev
     - build-essential
     - libpq-dev
-4. Clone the repository.
-5. Copy the file `.env-template` and name the copy `.env`
-6. Fill in the necessary variables in `.env`. `MELOS_URL` and `MELOS_ADMIN` are required. You might have to fill in some database credidentils. Check `src/moore/settings/base.py` for which default values are used if you don't specify and credidentials.
-7. Run `source ./source_me.sh` to create a virtual environment.
-8. Run `pip install --upgrade pip` to make sure that pip is running the latest version
-9. Run `pip install -r dev-requirements.txt`
-10. Use `cd src` to enter the website directory.
-11. Run `./manage.py migrate` to initialize the database.
-12. Run `./manage.py compilemessages` to create all the translations.
-13. Run `./manage.py createsuperuser` to create an admin user. (if the ssn is not passed, the most likely fault lies with the db-credentials)
+5. Clone the repository.
+6. Copy the file `.env-template` and name the copy `.env`
+7. Fill in the necessary variables in `.env`. `MELOS_URL` and `MELOS_ADMIN` are required. You might have to fill in some database credidentils. Check `src/moore/settings/base.py` for which default values are used if you don't specify and credidentials.
+8. Run `source ./source_me.sh` to create a virtual environment.
+9. Run `pip install --upgrade pip` to make sure that pip is running the latest version
+10. Run `pip install -r dev-requirements.txt`
+11. Use `cd src` to enter the website directory.
+12. Run `./manage.py migrate` to initialize the database.
+13. Run `./manage.py compilemessages` to create all the translations.
+14. Run `./manage.py createsuperuser` to create an admin user. (if the ssn is not passed, the most likely fault lies with the db-credentials)
+15. Use `cd npm` and run the command `npx vite build` to compile all javascript libraries.
+16. Go back to `src` by running `cd ..`
 
 During development, you can run a test web server using `./manage.py runserver`.
 
@@ -61,6 +64,7 @@ environment all set up:
 1. Run `docker exec -it moore python src/manage.py compilemessages` to create all the translations
 1. Run `docker exec -it moore python src/manage.py createsuperuser` to create an admin
    user.
+1. Run `docker exec -w src/npm -it npx vite build` to compile all javascript libraries.
 
 The Moore application is now available on `http://localhost:8000` and can be started using `docker-compose up -d` (the `-d` flag starts the instance in the background) and stopped `docker-compose stop`.
 

requirements.txt

@@ -38,3 +38,6 @@ django-jsonschema-form==1.0.3
 
 # Improved environment variable handling
 python-decouple==3.7
+
+# Add customabilitiy for including JS scripts in Forms media
+django-js-asset==2.0.0

src/blocks/templates/blocks/html_editor.html

@@ -0,0 +1,42 @@
+<textarea name="{{ widget.name }}" id="{{widget.attrs.id}}">
+    {% if widget.value %}{{ widget.value }}{% endif %}
+</textarea>
+
+<div id="{{widget.attrs.id}}-target"></div>
+<script type="module">
+    import {
+        EditorState,
+        EditorView,
+        html,
+        basicSetup,
+        theme,
+    } from "/static/codemirror.js";
+
+    const editorSource = document.querySelector("#{{widget.attrs.id}}");
+    editorSource.setAttribute("hidden", "true");
+
+    let startState = EditorState.create({
+        doc: editorSource.value,
+        extensions: [
+            basicSetup,
+            html(),
+            theme,
+            EditorView.updateListener.of((v) => {
+                if (v.docChanged) {
+                    // Each time codemirror is updated the new value is
+                    // set on the actual form field
+                    const doc = v.state.doc;
+                    editorSource.value = doc.toString();
+                }
+            }),
+        ],
+    });
+
+    // Find the editor targtet
+    const editorTarget = document.querySelector("#{{widget.attrs.id}}-target");
+
+    let view1 = new EditorView({
+        state: startState,
+        parent: editorTarget,
+    });
+</script>

src/blocks/widgets.py

@@ -1,35 +1,19 @@
 from django import forms
+from js_asset import JS
 from wagtail.utils.widgets import WidgetWithScript
 
 
 class CodeMirrorWidget(WidgetWithScript, forms.Textarea):
 
-    def render_js_init(self, id, name, value):
-        js = """
-        document.addEventListener('DOMContentLoaded', function(){{
-            CodeMirror.fromTextArea(
-                document.getElementById("{id}"),
-                {{
-                lineWrapping: true,
-                indentUnit: 4,
-                mode: "htmlmixed",
-                autoRefresh: true
-                }}
-            )
-        }});
-        """
-        return js.format(id=id)
+    template_name = 'blocks/html_editor.html'
 
     @property
     def media(self):
-        return forms.Media(
-            css={'all': ('libraries/codemirror/codemirror.css',)},
-            js=(
-                'libraries/codemirror/codemirror.js',
-                'libraries/codemirror/autorefresh.js',
-                'libraries/codemirror/xml.js',
-                'libraries/codemirror/css.js',
-                'libraries/codemirror/javascript.js',
-                'libraries/codemirror/htmlmixed.js',
-            )
-        )
+        # The JS module allows us to add custom attributes to script tags.
+        # In this case we need to add type=module since codemirror.js is
+        # made as an ES module.
+        media = forms.Media(js=[
+            JS("codemirror.js", {"type": "module"})
+        ])
+
+        return media

src/moore/settings/base.py

@@ -36,6 +36,7 @@
     'materialize',
     'members',
     'news',
+    'npm',
     'search',
     'moore',  # include for templatetags
 

src/npm/README.md

@@ -0,0 +1,42 @@
+# NPM modules
+
+This app allows the possiblity to install and use NPM modules within project moore.
+Previously, JavaScript files had to be downloaded and added directly to the source code.
+This makes it difficult to remember and actually update the library and is not a practice used anymore (2023).
+
+This module allows packages to be installed via npm which is the recommended way to install javascript packages.
+
+## How to use
+
+The packages from npm and any other custom code that we add on top of them, do not work directly in browsers. In order to get the code to be usable by browsers, Vite is used to compile them.
+
+1. All javascript files that should be compiled to work in browsers must be added in `vite.config.js` under `build --> rollupOptions --> input`,
+2. Run `npx vite build` to compile the files. This creates a folder called `static` with the files compiled for the browser.
+3. In the part of django where you want to use the javascript code, import the javascript file. How exactly you will do this depends on how you intend to use it. See examples below
+4. Start the django server with `./manage.py runserver` which automatically collects the compiled javascript files from the generated static folder.
+
+## Example
+
+In this example we will install a javascript library called codemirror which replaces an textarea with an code editor (in this case it will be used to write HTML code) on a webpage.
+
+1. First we install codemirror, its languages support for html and its state handler (all three of them were required for the purposes the code editor was going to be used for) `npm i codemirror @codemirror/lang-html @codemirror/state`
+
+2. Next we create a new file called `codemirror.js` in the same directory as `vite.config.js`. In this file we import the parts of the library we want to use and create some extra code that will fit our purpose. Finally they are exported, which allows them to be used by django. **NOTE:** Only those parts that have been exported can be accessed by django!.
+
+![Example code for codemirror.js](images/codemirror-example.png "Example code for codemirror.js")
+
+3. In `vite.config.js` under `build --> rollupOptions --> input`, we add codemirror so that vite will compile it.
+
+![Example of how a javascript file should be added to vite](images/vite-example.png "Example of how a javascript file should be added to vite")
+
+4. We run `npx vite build` in the terminal to compile the files which creates a folder called `static` containing a file called `codemirror.js` which contains the compiled version of our original `codemirror.js` file.
+
+5. In the CodemirrorWidget in django we first have to import the javascript file so that it can be used by django. In this case since the javascript will be used by a form widget, the javascript file can be imported in the forms media object. **Important:** The javascript has to imported with `type=module`
+
+![Example of how the code is imported](images/codemirror-widget.png "Example of how the code is imported")
+
+6. In the widgets template code, the javascript code can now be imported and used as normal javascript.
+
+![Example of how to use the imported javascript file](images/widget-html.png "Example of how to use the imported javascript file")
+
+7. We run `./manage.py runserver` which automatically gets the compile javascript file since it is in a folder called static and makes it available to our code written in django.

src/npm/codemirror.js

@@ -0,0 +1,11 @@
+import { basicSetup, EditorView } from "codemirror";
+import { EditorState } from "@codemirror/state";
+import { html } from "@codemirror/lang-html";
+
+const theme = EditorView.baseTheme({
+    "&.cm-editor": {
+        fontSize: "1rem",
+    },
+});
+
+export { EditorView, EditorState, html, basicSetup, theme };

src/npm/index.js

@@ -0,0 +1,2 @@
+// This file does not serve any purpose but is required by vite
+// to generate each javascript file as separate files

src/npm/materialize.js

@@ -0,0 +1 @@
+export default () => console.log("materilze");