Skip to content

Developer Guide

Jump to bottom
Kirk Swenson edited this page Nov 6, 2023 · 22 revisions

Table of Contents generated with DocToc

CODAP Developer Guide

These are instructions for setting up a full CODAP development environment. You will need to do this if you wish to modify or debug CODAP itself. For example, you may wish to create an alternate version or contribute bug fixes to the CODAP project. You do not need to do this if you wish to deploy an instance of CODAP to an alternate web server or if you wish to create a plugin (Data Interactive) that will run in CODAP. In these circumstances, you can download a zip file from http://codap.concord.org/releases/zips/ and then unzip it in the file space of your web server. If you wish to develop a CODAP plugin please see: https://github.com/concord-consortium/codap/wiki/CODAP-Data-Interactive-API.

Installing the CODAP development environment

The following are instructions that will install CODAP and its dependencies. There are a variety of ways of installing the dependencies and it is likely that any developer is set up for most dependencies already.

Development Dependencies

The development environment for CODAP requires the following software:

  • Git source code management,
  • Node.js the current version, and
  • Ruby version 1.9.3 or later with the Bundler gem.

The following notes may assist you in getting these packages installed and configured. If you already have these packages installed, whether by the means identified below or by other means, these notes can safely be ignored.

Notes for installing Git

  • If you do not have git installed already, go here.
  • Mac OS does not have git installed natively, but it comes with the Mac Developer Tools(XCode). Mac developers may wish to also install command line tools.
  • Windows:
    • Follow the instructions in the link referenced in the first bullet, above. When prompted check the option for "use git from Windows Command Prompt", pick "Git in Windows Console".
    • Accept defaults for all other options

Notes for installing node.js and npm

  • Mac OS: Here are useful instructions.
  • Windows: Here are useful instructions.
  • Windows users are encouraged to install cygwin. Cygwin emulates a Unix command line environment, including a bash shell.

Notes for installing Ruby

There are several ways to install Ruby.

  • For Mac OS: Ruby is not installed natively on Mac OS, but, as with git, can be found in the Mac Developer Tools(XCode). The version of ruby currently installed with Developer Tools is 2.3.x, which works well with CODAP. You should make sure the Bundler Gem is installed:

    sudo gem install bundler

  • For Mac OS: for users who have Homebrew installed:

        $ brew install ruby
    $ gem install bundler

  • For Windows: Install Ruby 1.9.3 or later with DevKit

    • Go to RubyInstaller
    • Download latest stable version for your windows instance
    • Download DevKit from the same page. If possible, do so in the same installer.
    • Run installer to install Ruby
    • If you downloaded a separate DevKit, unzip it
    • Run "Command Prompt as Ruby" (Go to windows start icon. Search for "Command Prompt with Ruby".)
    c:\Users\username\Downloads>gem install bundler
    • If you downloaded a separate DevKit, run the following in the terminal window:
    c:\Users\username\Downloads>ruby dk.rb init
c:\Users\username\Downloads>ruby dk.rb review
c:\Users\username\Downloads>ruby dk.rb install
    • Install the eventmachine.
    gem install eventmachine

  • For Any platform: It is also possible to install ruby through the Ruby Version Manager(RVM). This is useful if you need to swap between multiple Ruby versions, but is not required for CODAP's purposes. If you elect this path, please refer to the above site for installation instructions. RVM installs the Bundler Gem automatically.

Installing CODAP

Get the CODAP Source

  • In a terminal window:

    $ git clone https://github.com/concord-consortium/codap.git
$ cd codap # we will refer to this as the "CODAP root directory"
$ git submodule update --init --recursive

  • If you intend to modify the CODAP repository for your own purposes or to contribute back, it may be useful to fork it on Github and then clone the fork. To fork CODAP, go to the Github Repository and click on "Fork".

Install the Sproutcore Server Environment

  • In a terminal window:

        $ bundle install

Install NPM Packages

  • We assume you have the NodeJS and the Node Package Manager (npm) installed on your system.

  • In your terminal window, with the current working directory set to the CODAP root directory, run the following commands:

         % npm install --legacy-peer-deps
     % npm run build:bundle

At present, Windows users will need to run the above command from an environment that supports standard Unix command line programs, such as Git Bash, installed above, cygwin, or bash in windows. The terminal window mentioned above should be one of these.

The contents of the bundle change periodically as dependencies are changed. Whenever a code update requires updating the bundle the above steps can be used to build an updated bundle. To simplify the process, the sync script will update git submodules, install/update npm modules, and rebuild the bundle.

    $ npm run sync

Run the CODAP Application Locally

  • In your terminal window, with the current working directory set to the CODAP root directory, run the following commands:

    $ npm start    # or "sc-server --allow-from-ips='*.*.*.*'"

  • This runs the SproutCore server, which serves the application code locally. To run the application, enter the following URL into your web browser of choice: http://localhost:4020/dg. CODAP should run well on the latest release of modern mainstream browsers, including Chrome, Firefox, Safari, and Edge.

  • To verify your build there are two easy things to do:

    • Open your local CODAP instance in a browser. Open each the example documents. They should appear and behave exactly as they would in the official CODAP release.
    • In the terminal window change to the CODAP directory and execute:

    $ npm run test.

    This should execute without error.

Troubleshooting

  • If your local version of CODAP fails to build, try npm run clean and/or npm run sync and try again.
  • If your local version of CODAP fails to load, check the output in the terminal window where you ran 'sc-server' and check your browser's console.

Working with the CODAP development environment

Most changes to the CODAP application source are automatically reflected by simply reloading the browser page at http://localhost:4020/dg. More extensive changes (e.g. adding/removing source files) may require stopping and restarting sc-server and/or removing the generated tmp folder. For npm users, a set of scripts are defined in the package.json file to assist with many of these development tasks. Some of the more commonly used scripts for local development are listed below. See the package.json file for the full list of available scripts.

  • npm run build:bundle - builds the library bundle required by CODAP. Must be run before running CODAP locally for the first time or after bundle contents change.
  • npm run clean - deletes the tmp directory and other build products
  • npm run clean-and-serve - runs the clean script and then starts sc-server
  • npm run codap - starts sc-server
  • npm run lint - runs ESLint/JSHint over the sources
  • npm run start - starts sc-server
  • npm run sync - runs clean, updates submodules, runs npm install, and builds the bundle
  • npm run test - runs CODAP's unit tests

Note that some of these commands require unix or unix-like command line programs, and may not work in pure Windows environments. Windows users may wish to install cygwin or enable bash in windows.

Travis Builds

Branches of CODAP pushed to the Github repository instigate an automated CI build of CODAP and create a development instance of CODAP. If the build is named "foo", the development instance will be "https://codap-dev.concord.org/branch/foo/". The terminating slash("/") in the URL is required. Pushes to the master branch create an instance at "https://codap-dev.concord.org/". The builds can take some time, and can be monitored on the GitHub Actions page.

Updating the Cloud File Manager

CODAP uses the Cloud File Manager library (CFM) to handle most of its document storage/retrieval. Currently, the built CFM files are simply included in the CODAP git repository. To update the version of CFM included in CODAP, it is necessary to build the CFM library with some CODAP-specific options and put the appropriate CFM output files into the apps/dg/resources/cloud-file-manager directory in CODAP. There is an npm script named build:codap in the CFM repository which takes care of all the details as long as the codap and cloud-file-manager repositories are in sibling directories, i.e. something like ../projects/codap and ../projects/cloud-file-manager. With this configuration, from within the codap repository, typing:

$ npm run build:cfm

or from within the cloud-file-manager repository, typing:

$ npm run build:codap

will build the CFM and place the output files in the appropriate place in the codap repository. Under the hood, npm run build:codap executes the following command: bash $ gulp clean-and-build --dest ../codap/apps/dg/resources/cloud-file-manager \ --codap --nojQuery --noReact --noMap If your codap repository is not a sibling of your cloud-file-manager repository or has a different name, you can execute an analogous command with a different destination path. After updating the CFM, simply reloading the page at http://localhost:4020/dg should make the changes available as long as sc-server is running.

Translation/Localization

The master English strings file is lang/strings/en-US.json, which is a standard JSON file except that JavaScript-style comments are allowed. (Comments are stripped before use where appropriate.) Changes to English strings should be made in the master English strings file. All other language files are output files generated by script and are stored in their respective .lproj directories, following SproutCore localization convention. Translations for other languages are managed via the CODAP project (authentication required) on POEditor, which provides free hosting services for open source projects.

Development

After making changes to the master English strings file (lang/strings/en-US.json), run the strings:build script to strip comments and deploy the apps/dg/english.lproj/strings.js file for building:

npm run strings:build

To push changes to the master English strings file to POEditor, run the strings:push script:

npm run strings:push -- -a <poeditor-api-token>

The API token must be provided as an argument to the strings:push script or it can be set as an environment variable:

export POEDITOR_API_TOKEN=<poeditor-api-token>

To update the strings files within the project, run the strings:pull script:

npm run strings:pull -- -a <poeditor-api-token>

As with the strings:push script, the API token must be provided or be set as an environment variable. The strings:pull script builds the English strings as well so all strings files will be up to date.

After pulling updated strings, the modified files can be committed to git, turned into a Github Pull Request, etc. Note that POEditor supports Github integration which could potentially automate part of this, but that requires further investigation.

Unicode escapes are converted to their UTF-8 equivalents when pushed, i.e. strings are viewed/edited in their "user" form in POEditor, and they remain in their UTF-8 form when pulled. For characters that are better left in their Unicode escape form, such as non-printable characters like ZERO-WIDTH-SPACE ("\u200b") and the RIGHT-TO-LEFT-MARK ("\u200f"), the scripts support a custom Unicode escape sequence such that "[u200b]" and "[u200f]" are converted to "\u200b" and "\u200f" respectively when pulled.

The ZERO-WIDTH-SPACE character can be used to indicate that the empty string is the correct translation for a string in a particular language. If the string were simply left untranslated, then POEditor would 1) show it as untranslated in the POEditor UI and 2) replace it with the English string when pulled. The ZERO-WIDTH-SPACE prevents POEditor from treating the string as untranslated, but it is rendered like an empty string.

Adding a language

To add a new language:

  1. Add the language to the POEditor project
  2. Add the language code to the list of languages in bin/strings-pull-project.sh
  3. Create a new {lang}.lproj folder for the language, e.g. es.lproj for Spanish.
  4. Copy the English function_strings.json (at apps/dg/english.lproj/function_strings.json) i nto the new {lang}.lproj folder. Translation of the function_strings.json is not currently supported.
  5. There may at some point be another step for configuring which builds should contain the new language, but for now all languages are built by default.

Testing with Sage Modeler

This section assumes (a) you have a working CODAP development environment, (b)you know what Sage Modeler is and have a reason to test CODAP with it, and (c) the code you wish to debug is on the CODAP side, not the Sage Modeler side.

  • The following URL will invoke a local CODAP development instance in standalone mode with Sage Modeler occupying the entire workspace. It is functionally equivalent to the invocation on the Sage Modeler website.

http://localhost:4020/dg?embeddedMode=yes&hideSplashScreen=yes&hideUndoRedoInComponent=yes&hideWebViewLoading=yes&standalone=SageModeler&di=http://sage.concord.org/sagemodeler.html%3Fstandalone%3Dtrue&di-override=sage&inbounds=true

Clone this wiki locally