Use this guide to work on the Cree Intelligent Dictionary repo.
Clone the repo and cd
into it, as usual.
git clone https://github.com/UAlbertaALTLab/cree-intelligent-dictionary.git
cd cree-intelligent-dictionary
Get Python 3.9 and Pipenv.
Within the directory, install all Python dependencies:
pipenv install --dev
Install NodeJS version 10 or greater.
With Node installed, install all of the JavaScript dependencies using npm
:
npm install
Create a file named .env
in the project root using the following
template:
# .env
DEBUG=true
USE_TEST_DB=true
These are environment variables that affect whether Django is in debug mode and whether Django should use a smaller test database.
The environment variables go into effect when using pipenv shell
, or by
running a program with pipenv run
. However, pytest
is configured to
always USE_TEST_DB
, regardless of the .env
file contents.
Run pipenv shell
so that all of the Python dependencies work:
pipenv shell
Running
./scripts/dev-bootstrap
creates the test databases for all supported languages.
Note: if you have
USE_TEST_DB=true
, you can skip this
Download crkeng.xml
and place it under CreeDictionary/res/dictionaries/
These files are copyright protected and not allowed on GitHub. Ask coworkers about the ‘altlab’ repo.
If you do not have access to the full crkeng.xml
, set USE_TEST_DB=false
to automatically use the excerpt in this repository at
test_dictionaries/crkeng.xml
. That excerpt can be automatically updated
from the full database file and test_db_words.txt
by the buildtestdbxml
management command.
As is the custom with Django app, migrations are used to initialize the schema. Apply them with
./crkeng-manage migrate
Now import the dictionaries into the database:
./crkeng-manage xmlimport CreeDictionary/res/dictionaries/crkeng.xml
This typically takes 10-15 minutes on the full dictionary.
Our JavaScript and CSS must be built before being used. Run this command:
npm run build
(note: using foreman automatically runs the underlying command in watch mode)
Install foreman with gem install --user foreman
.
Then run foreman start
to run the Django servers for all languages, as
well as the Rollup watch process.
Then you can access the dictionary applications at various port numbers:
- itwêwina: http://127.0.0.1:8000/
- arpeng: http://127.0.0.1:8007/
- cwdeng: http://127.0.0.1:8005/
- srseng: http://127.0.0.1:8009/
Because cookies are not port-specific for historical insecurity
reasons,
you can only be logged in to one 127.0.0.1
development site at a time. If
that becomes problematic, give each development site a unique hostname by
adding the following to /etc/hosts
:
127.0.0.1 arpeng-local
127.0.0.1 cwdeng-local
127.0.0.1 crkeng-local
127.0.0.1 srseng-local
Then you can access the sites with cookie isolation at http://crkeng-local:8000/, http://cwdeng-local:8005/, http://arpeng-local:8007/, and so on.
If you only want to run one dictionary, you can locally comment out lines in the Procfile.
They're located in frontend/
. They're compiled by Rollup to the
appropriate static directory. Note that Rollup allows you to import
or
require()
npm modules in to the frontend JavaScript code; use this
power wisely!
Rollup also minifies the JavaScript when DEBUG=False
.
They're located in frontend/css
. They're compiled by Rollup to the
appropriate static directory. We're using PostCSS to inline
any @import
'd CSS, and to provide a fallback for
CSS custom properties (a.k.a., CSS Variables).
Rollup/PostCSS also minifies the CSS when DEBUG=False
.
pipenv run test
It recognizes the following:
- The Django settings module in
setup.cfg
(forpytest-django
to work) --doctest-modules
--mypy
inPipfile [script]
(to enable doctest and Mypy tests)DEBUG=False
USE_TEST_DB=True
in.env
npm test
Or, for interactive use:
npx cypress open
Eddie “sez”: My workflow is to have three terminals open:
- Vim (editing files)
npm start
— start the development servernpx cypress open
— start Cypress interactively
We use django-toolbar-toolbar. It provides great UI for stack tracing and SQL query profiling.
We format all Python code with Black
To run it on all of the files:
pipenv run format
Protip! Make this a part of your git pre-commit hook!
You don’t need this to run the dictionary, but having these tools installed can be useful if you are building, modifying, or directly interacting with FSTs.
On Mac:
brew install UAlbertaALTLab/hfst/hfst
On Windows:
Download hfst-latest.zip and unpack it. Add the
hfst/bin
folder to your system path.
On Linux:
sudo apt-get install -y hfst
For help, see HFSTOL installation guide.