Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move Resources out of Electron build #84

Closed
benloh opened this issue Apr 22, 2023 · 15 comments · Fixed by #135
Closed

Move Resources out of Electron build #84

benloh opened this issue Apr 22, 2023 · 15 comments · Fixed by #135
Assignees

Comments

@benloh
Copy link
Contributor

benloh commented Apr 22, 2023

A number of related issues:

  • The current location for meme.loki was designed to work with macOS (e.g. in ~/Documents/meme/) but that does not work on Ubuntu and other OSes.
  • Where and how should the meme.app be distributed and saved? Do we expect people to drag it to "Applications" folder? Or do we distribute the full MEME macOS (was meme-darwin-x64) folder?
  • Where should the runtime *.loki files be stored? (and backups, logs, and screenshots)?
  • Where should the resources be stored?
  • Where should the starting template database be stored?
  • What impact does this have (if any) when running on Ubuntu?

Principles

We want to bullet proof this for people who are downloading it for the first time.

  • If you run meme.app by itself with no resources, it should still be able to run and...
    • create a new meme.loki database from a default _blank database template burned into the Electron app.
    • if *.db.js files have been defined in the data/templates/meme folder, a new loki db should be generated from those files.
    • if there is no data/templates/meme folder, then a new loki db should be generated from data/templates/_blank files.
  • We want to make it possible for users who inadvertently screw up the meme.loki file to be able to have a fallback -- e.g. just remove the meme.loki file and a new one will be generated, either from the data/templates/meme files if they're defined, or from data/templates/_blank.
  • Resources are managed during the build process
    • Since resources.db.js needs to refer to specific resource URLs, we want to keep a copy of relevant sources with the saved db definitions.
    • original resource files are stored with the *.db.js definitions (under the db directory, e.g. data/templates/meme/resources) and copied over to the /MEME macOS/resources folder during the loki build process. This way the default and meme db.js definitions can refer to specific resources and we can guarantee that the resources will be present.

Ideally, something like this?

/MEME macOS/
  README.txt
  LICENSE
  LICENSE.chromium.html
  meme.app
  version
  resources/              -- Active resources are copied here during first run
    source1.pdf           -- Copied from template
    source2.pdf
  data/                   -- ALL other files and folders are under `data`
    README.txt            -- Explains folder structure, how files are used.
    templates/            -- Sourced from `boilerplate/src/system/datasets`?
      README.txt          -- Explains folder structure, how files are used.
      _blank/             -- Default database definition if `meme` db.json is not defined
        *.db.js           -- loki files are rendered from these json definitions
        resources/
            source1.pdf   -- Copied to /resources during build
            source2.pdf
      meme/               -- Optional database definition if `meme.loki` is not present
        *.db.js           -- loki files are rendered from these json definitions
        resources/
            source1.pdf   -- Copied to /resources during build
            source2.pdf
  data/db/
      README.txt          -- Explains folder structure, how files are used.
      meme.loki           -- Default database to open, if empty, built from ./templates/meme or _blank
    logs/
    screenshots/

The *.db.js files are:

      classroomResources.db.js
      classrooms.db.js
      criteria.db.js
      groups.db.js
      models.db.js
      ratingsDefinitions.db.js
      resources.db.js
      sentenceStarters.db.js
      teachers.db.js

See also Dataset Management for info on loki file paths.

@benloh
Copy link
Contributor Author

benloh commented Apr 26, 2023

In GitLab by @jdanish on Apr 26, 2023, 12:28

Just seeing this. Did you want feedback? I agree with the framing! Though maybe we can change the folder name to not have Darwin since that all confuses folks.

@benloh
Copy link
Contributor Author

benloh commented Apr 26, 2023

@jdanish yeah, please comment. This seemed to make the most sense to me. I'm less enamored of the idea of putting things in ~/Documents, but we might also face restrictions about writing to the current directory.

Maybe call the folder "MEME" or "MEME macOS" or "MEME Electron Application" instead of "meme-darwin-x64"?

@benloh
Copy link
Contributor Author

benloh commented Apr 26, 2023

In GitLab by @jdanish on Apr 26, 2023, 13:17

I agree. I think the ideal for ease and assuming we aren't building an install anytime soon (or ever?) is just being able to zip a folder that folks download, and the app and data are all in the same folder. Calling it MEME macOS works for me.

The structure works for me as well, and makes it easy to edit.

I also like the idea that we provide a default in case the rest is hosed or deleted. I can provide those at some point once I know the format. Though given that we can't delete, I'd say one teacher, with one class, etc., makes the most sense?

We might want a readme in there too, but it can go in the zip and I can handle it.

Did I miss anything?

@benloh
Copy link
Contributor Author

benloh commented Apr 26, 2023

A few more thoughts...

  • We want lots of READMEs in each folder to explain how they're used.
  • We'll have to figure out how best to organize things on the dev/code side as well as the distributed side. It may be that we want to be able to add the distribution files or a spec for distribution to the repo so that it can be easily replicated?

I also like the idea that we provide a default in case the rest is hosed or deleted. I can provide those at some point once I know the format. Though given that we can't delete, I'd say one teacher, with one class, etc., makes the most sense?

The easiest approach is probably to do what we're doing now: Edit the *.db.js files in db-templates/default to define what the default db should be? It seems more straightforward than trying to construct a loki db file.

@benloh
Copy link
Contributor Author

benloh commented Apr 26, 2023

In GitLab by @jdanish on Apr 26, 2023, 14:37

I can certainly do that (edit those files).

Realistically, if time is tight, another option is to just hide the db templates on the assumption that only I am likely to muck with them. If we can download the zip file and it runs, that'll work for 98% of our uses, and being able to send a replacement set of resources and or meme.loki handles the rest. Long-term I'd love to be able to have folks tinker with the rest, but realistically this might be all we need and then we wait until we can have the rest edited by a UI:

/MEME macOS/
  README.md
  LICENSE
  LICENSE.chromium.html
  meme.app
  version
  db/
    meme.loki
  logs/
  screenshots/
  resources/
    README.md

Or, maybe have a data folder with a sub-folder for templates and db

data/
    templates/
    db/
         meme.loki

@benloh
Copy link
Contributor Author

benloh commented Apr 26, 2023

In GitLab by @jdanish on Apr 26, 2023, 14:41

Come to think of it, logs and screenshots can go under data so top level we have:

/MEME macOS/
  README.txt
  LICENSE
  LICENSE.chromium.html
  meme.app
  version
  data/
  resources/

and then inside data:

    templates/
    db/
    logs/
    screenshots/

Note I changed the readme to txt for normal people who aren't us 😄

@benloh
Copy link
Contributor Author

benloh commented May 9, 2023

Archiving the first proposed approach (ca. April 22, 2023) for historical purposes.

THIS IS NOT THE FINAL DESIGN

/MEME macOS/
  LICENSE
  LICENSE.chromium.html
  meme.app
  version
  db/
    README.md    -- Explains folder structure, how files are used.
    meme.loki
  db-templates/  -- Sourced from `boilerplate/src/system/datasets`?
    README.md    -- Explains folder structure, how files are used.
    default/
      classroomResources.db.js
      classrooms.db.js
      criteria.db.js
      groups.db.js
      models.db.js
      ratingsDefinitions.db.js
      resources.db.js
      sentenceStarters.db.js
      teachers.db.js
    meme/
      classroomResources.db.js
      classrooms.db.js
      criteria.db.js
      groups.db.js
      models.db.js
      ratingsDefinitions.db.js
      resources.db.js
      sentenceStarters.db.js
      teachers.db.js
  logs/
  screenshots/
  resources/
    README.md

@benloh
Copy link
Contributor Author

benloh commented May 9, 2023

@jdanish A few more questions about resources.

We were anticipating that with different starting databases you might have completely different sets of resources that you would want to use. So rather than having the resources.db.js file define a huge list of resources that may or may not be relevant to a particular database, it makes more sense to have one set of resources for the default fallback database, and another set for the custom-defined meme database. This way we can guarantee that resources are always available.

For example, if you wanted to do water quality investigations in one classroom and deer population investigations in another classroom, you could define a unique set of resources for each and store them in their respective template/meme/ folders.

(We had also toyed with the idea of auto-generating the resources list rather than having to define the resources.db.js, but while we can auto-generate the labels, we couldn't easily generate the type specification.)

Does that make sense? Or are you using resources in a different way?

@benloh
Copy link
Contributor Author

benloh commented May 9, 2023

In GitLab by @jdanish on May 9, 2023, 09:01

That makes sense and I think the “default” can be empty since we would typically not have something across classes. For duplicating resources for 2 classes I assume we could copy and paste?

@benloh
Copy link
Contributor Author

benloh commented May 9, 2023

This kind of depends on your setup.

e.g. if you have one server computer and are setting up multiple teachers and multiple classrooms per teacher they would simply all share the same resources as defined the templates/meme/resources folder. (Each classroom can have it's own subset of resources turned on and off). I would think you would go this route if you have a server that is, say dedicated to water quality investigations. In this case, you don't need to copy and paste, you can just turn resources on and off as needed for each classroom (period, not per teacher).

e.g. if you have two servers, one for water quality and one for deer population, then you probably want to define separate templates for each. If they share a same resource, then just duplicate the resource in both template folders.

@benloh
Copy link
Contributor Author

benloh commented May 9, 2023

Once we get closer to an implementation, we should probably write up a guide, something to the effect of the slack post:

The current thinking is a series of cascading fallbacks.

  1. You can start your Electron app, muck with it however you want. Then zip it and all associated files and send it. In this case, the user would use the data/db/meme.loki file.
  2. You can remove the data/db/meme.loki file. The system will then try to create a loki db out of the data/templates/meme/*.db.js specifications. In other words, you can spec a default db for multiple classrooms this way.
  3. If there is no data/templates/meme defined, it falls back to data/templates/default which is where you might set up a teacher 1 / classroom 1 definition with no resources. Or you might decide that having at least one resource would be helpful for demonstrating the basic links.

So we can have a generic zip with the app and just the default db.

And you can create a second zip, say for a particular study, that has a meme db.js defined so that everyone starts with the same base, then sets up their classrooms.

Or, you can create a 5 different zips, one for every classroom in the study, where you predefine all the class rosters for everyone. Here all the classrooms would start from the same templates/meme/*.db.js files so they all share criteria, resources, etc. But then you open up the app and add classrroom rosters for one class, zip it and send it. Then delete the meme.loki, open up the app again (and a new fresh meme.loki s created from the raw db.js files with no classrooms defined), define a new classroom, zip it, and send it to the second classs.

Also, a summary:

To share a project, you can do one of three things:

  • Send them meme.loki and have them replace their existing meme.loki
  • Send them templates/meme/*.db.js docs
  • Send them templates/default/*.db.js docs

Most likely you would just send them meme.loki

If you only wanted them to view a project (and not do any writing) you could also send them the *.MZIP file that you export from the Electron app.

@benloh
Copy link
Contributor Author

benloh commented May 10, 2023

One additional note: Dragging MZIP files into the Electron app is a researcher feature for Reviewing a Database Archive -- "The original database has not been changed. You can import another archive. Quit and restart app to restore active database." It is not intended as a method of replacing the current running database.

So to seed a classroom, you should use one of the three methods above (replace meme.loki, replace templates/meme, or fall back to templates/default).

@benloh
Copy link
Contributor Author

benloh commented Jun 9, 2023

@scott.campion Some refinements of this as we work through the materials.

  1. The default project/folder/template is now called _blank -- this actually was already implemented and is a better starting point for new projects.
  2. We will not distribute /data/templates/meme/*.db.js files with the build. These should be provided by researchers to seed projects when they distribute the Electron app. (They could also seed by replacing the meme.loki file.
  3. The meme.loki file provided is a demo file that illustrates both how things are set up on the admin side, and a prototypical model that students might use.
  4. See Default Project Files #90 for the files to use.

The main issue description contains the intended design. The discussion merely provides historical context.

@benloh
Copy link
Contributor Author

benloh commented Aug 31, 2023

@Sakelun One possible clarification/refinement/oversight -- sorry I just realized this may not have been clear since I was focused on the end result. The folder structure in the description was assuming that this is the structure in the packaged Electron build. If I'm not mistaken, the packaged electron app currently sits at

boilerplate/dist/meme-darwin-x64

...so with the changes, we're expecting the the built app would sit at:

boilerplate/dist/MEME macOS/

...or whatever we decide to rename meme-darwin-x64 to (assuming it's possible to do that with the electron build scripts).

That leaves the question of where we should be putting the source data files a little unclear. But if we go with the principle of mirroring the app, then it makes sense to add the data files at:

boilerplate/data/
boilerplate/data/templates/
boilerplate/data/templates/_blank/
boilerplate/data/db/
boilerplate/data/db/logs/
boilerplate/data/db/screenshots/
boilerplate/resources

...but perhaps you already were ahead of me on that...

This was referenced Jun 9, 2023
@benloh
Copy link
Contributor Author

benloh commented Oct 27, 2023

Implemented.

@benloh benloh closed this as completed Oct 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants