Runtime folder structure and default/Resource packaging updates #135

Sakelun · 2023-08-31T19:03:55Z

This pull request implements the changes requested in #84, namely:

Data and Resource artifacts separated from the codebase folder and sub-folders to remove the need to re-build, package, and sign when Data/Resources are changed.
Path decisions largely centralized to support further tweaking that may be necessary for wider cross-platform distribution.
Resource sets are now included with their respective Data files; previously there were just datasets. Now, a template includes both.
Resources are now seeded with their respective Data files into the top-level resources folder
Selection of a Data/Resource set ("template") has been adjusted to choose from a prioritized list of template options (environment-specified, meme, or blank)

Note:

As part of this update, the static/dlc folder is no longer present and has been replaced with a top-level resources folder. It is therefore necessary that the Express server remap the static/dlc URL path to this folder to maintain backward compatibility with older datafiles.

@benloh has indicated that legacy file support may not be necessary. If so, it would be preferred to map / within a datafile to be /resources/ for the purposes of serving content via URL (e.g. https://..../resources/fishtank.pdf) and its location on disk /<meme folder>/resources/.

Test Instructions

Test basic paths

nvm use
npm run clean:all -- note the dist and built folders have been removed
npm ci
npm run dev -- compile to make sure it runs
Check http://localhost:3000 and http://localhost:3000/#/admin to make sure app loads
Make sure you prepped the files using the "Preparing for Electron" instructions above.
npm run package
Check boilerplate/dist/MEME macOS to make sure the electron app has been built correctly:
- data/ -- should be empty
- resources/ -- should be empty
- templates/ -- should have a _blank template folder
- LICENSE
- LICENSE.chromium.html
- meme.app
- version
If you haven't already set up code signing, see [Signing](https://github.com/theRAPTLab/meme/blob/dev-src/file-paths/README-signing.md)
Verify that you can run meme.app locally on your machine -- start the Electron app, and load http://localhost:3000/#/admin
npm run appsign

Test create a blank database with `/templates/_blank`

Try different methods of starting a new project from scratch. You should always be able to create a new blank project without any errors.

1. Test `npm run dev` with blank

Delete <repodir>/data and dist
Run npm run dev
A new test.loki file should be created in <repodir>/data/db/test.loki
Going to http://localhost:3000/#/admin you should see a blank project

2. Test `npm run electron` with blank

Delete <repodir>/data and dist
Run npm run electron
A new meme.loki file should be created in <repodir>/data/db/meme.loki
An electron app should be built and started
Going to http://localhost:3000/#/admin you should see a blank project

3. Test `npm run app` with blank

Delete <repodir>/data and dist

Run npm run app => npm should complain

./dist/meme-darwin-x64/meme.app: No such file or directory
MEME EXEC    - ERROR from codesign check
MEME EXEC    - macos will not run this app until it is signed

Run npm run package

Run npm run app => npm should complain

./dist/meme-darwin-x64/meme.app: No such file or directory
MEME EXEC    - ERROR from codesign check
MEME EXEC    - macos will not run this app until it is signed

Run npm run electron to build the meme.loki file.
Run npm run package to build with the necessary depdencies.
Run npm run appsign (NOTE you need to have set up code signing -- see [Signing](https://github.com/theRAPTLab/meme/blob/dev-src/file-paths/README-signing.md))
A new meme.loki file should be created in <repodir>/data/db/meme.loki
An electron app should be built and started
Going to http://localhost:3000/#/admin you should see a blank project

Test with default `/templates/test` template folder

Make sure that npm run dev will generate test.loki based on a starting template and that it'll fall back to templates/_blank if templates/test is not defined.

Test Blank

Delete <repodir>/data and dist
Delete /<repodir>/templates/test (if it exists)
npm run dev
The app should run with a blank project

Test test

Delete <repodir>/data and dist
Add /<repodir>/templates/test -- use the zip file attached.
npm run dev
The app should run with a test project that has three teachers added.

Test with default `/templates/meme` template folder

Make sure the various ways of setting a starting non-blank template work.

Test Blank

Delete <repodir>/data and dist
Delete /<repodir>/templates/meme (if it exists)
npm run dev
The app should run with a blank project

Test npm run electron with meme

Delete <repodir>/data and dist
Add /<repodir>/templates/meme -- use the zip file attached.
npm run electron
The app should run with a test project that has three teachers added.

Test npm run app with meme

Delete <repodir>/data and dist
Add /<repodir>/templates/meme -- use the zip file attached.
npm run electron
npm run package
npm run appsign
npm run app
The app should run with a test project that has three teachers added.

Test meme.app with meme

Delete <repodir>/data and dist
Add /<repodir>/templates/meme -- use the zip file attached.
npm run electron
npm run package
npm run appsign
Double-click meme.app to start Electron
The app should run with a test project that has three teachers added.

Test copy distribution

Make sure it's possible to copy the distribution files to another computer.

Build the electron app (npm run package) and copy the /dist/MEME macOS folder to your new computer/new location, e.g. to ~/Desktop/MEME macOS.
Open a terminal.
cd ~/Desktop/MEME macOS
./install.sh
The quarantine flag should now be fixed.
Double click the meme.app Electron application to start the server.
If you see javascript error, then the quarantine flag was probably not removed successfully. Try typing xattr meme.app -- you should NOT see the line com.apple.quarantine
If you copy the app to another computer you may have to run ./install.sh again.

…aset dependent packaging and signing.

jdanish · 2023-08-31T19:10:14Z

Confirming we don't need the directory to be backwards compatible - I can make quick fixes in any ported data.

benloh · 2023-08-31T19:19:57Z

@benloh has indicated that legacy file support may not be necessary. If so, it would be preferred to map / within a datafile to be /resources/ for the purposes of serving content via URL (e.g. https://..../resources/fishtank.pdf) and its location on disk //resources/.
Confirming we don't need the directory to be backwards compatible - I can make quick fixes in any ported data.

@Sakelun Please go ahead and make that change (if it was necessary to add that).

jdanish · 2023-08-31T20:59:43Z

Not sure if this is a bug or an FYI, but my install did not have a resources folder inside of templates/_blank/ and so was throwing an error. I created an empty folder there and then things appear to be running fine so far.

jdanish · 2023-08-31T21:04:45Z

What url should I type in for something that is in the resources folder? Thanks!

Sakelun · 2023-08-31T22:04:58Z

What url should I type in for something that is in the resources folder? Thanks!

I will be updating that soon, but for the version you have in your hands this moment it maps the URLs as they were, so https://..../static/dlc/tanksim.pdf is the full URL for the browser and you'd leave '../static/dlc/L1 Pond Chlorophyll.pdf' in your data file. The actual resource file would reside in /resources/tanksim.pdf. If it came from a template, it'd have been copied from /templates/<template name>/resources/tanksim.pdf.

jdanish · 2023-08-31T22:38:09Z

Thanks. Confirmed this appears to work though I'll continue to test, but resources, logs, data, all seem to be working in the new directories as expected.

Sakelun · 2023-08-31T22:44:28Z

Not sure if this is a bug or an FYI, but my install did not have a resources folder inside of templates/_blank/ and so was throwing an error. I created an empty folder there and then things appear to be running fine so far.

Thanks for catching that. This situation is now handled gracefully whether the folder is there or not. We've also added a README file to the folder describing its purpose that will ensure it remains a part of the source repository.

benloh · 2023-09-11T20:38:12Z

Not sure if this is a bug or an FYI, but my install did not have a resources folder inside of templates/_blank/ and so was throwing an error. I created an empty folder there and then things appear to be running fine so far.

Thanks for catching that. This situation is now handled gracefully whether the folder is there or not. We've also added a README file to the folder describing its purpose that will ensure it remains a part of the source repository.

@Sakelun Just checking...are you waiting for my approval here? I was under the impression that you were going to add a few more fixes (e.g. "handled gracefully whether the folder is there or not." and "added a README"). Or is that already done and I missed it?

Sakelun · 2023-09-12T01:33:19Z

@Sakelun Just checking...are you waiting for my approval here? I was under the impression that you were going to add a few more fixes (e.g. "handled gracefully whether the folder is there or not." and "added a README"). Or is that already done and I missed it?

The fixes are on the branch already. The only item pending is reorganizing the output folder per our meeting.

…m builds.

Sakelun · 2023-09-29T22:11:03Z

@benloh The distributable package output has been updated - please test and let me know if you encounter any issues.

benloh · 2023-09-30T05:18:52Z

@Sakelun A few issues came up during testing.

1. 'MEME macOS`

Sorry to be so nitpicky, but as of a few years ago the official name is macOS not MacOS, so it'd be great if we could change dist folder to MEME macOS.

2. npm ci error

During testing, I tried to start with a fresh slate to make sure everything will work. So I:

nvm use -- using node v10.20.1
npm run clean:all -- to remove dist, built, and node_modules
npm ci resulted in a warning with node-pre-gyp

clickr:boilerplate loh$ npm ci

> [email protected] install /Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents
> node install

node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(403): https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.2.9/fse-v1.2.9-node-v64-darwin-x64.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v64 ABI, unknown) (falling back to source compile with node-gyp) 
Traceback (most recent call last):
  File "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py", line 50, in <module>
    sys.exit(gyp.script_main())
             ^^^^^^^^^^^^^^^^^
  File "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 554, in script_main
    return main(sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^
  File "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 547, in main
    return gyp_main(args)
           ^^^^^^^^^^^^^^
  File "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 520, in gyp_main
    [generator, flat_list, targets, data] = Load(
                                            ^^^^^
  File "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 136, in Load
    result = gyp.input.Load(build_files, default_variables, includes[:],
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 2782, in Load
    LoadTargetBuildFile(build_file, data, aux_data,
  File "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 391, in LoadTargetBuildFile
    build_file_data = LoadOneBuildFile(build_file_path, data, aux_data,
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 234, in LoadOneBuildFile
    build_file_contents = open(build_file_path, 'rU').read()
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid mode: 'rU' while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:248:12)
gyp ERR! System Darwin 22.6.0
gyp ERR! command "/Users/loh/.nvm/versions/node/v10.20.1/bin/node" "/Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64/fse.node" "--module_name=fse" "--module_path=/Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64" "--napi_version=6" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v64"
gyp ERR! cwd /Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents
gyp ERR! node -v v10.20.1
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/Users/loh/.nvm/versions/node/v10.20.1/bin/node /Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64/fse.node --module_name=fse --module_path=/Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64 --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v64' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:982:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
node-pre-gyp ERR! System Darwin 22.6.0
node-pre-gyp ERR! command "/Users/loh/.nvm/versions/node/v10.20.1/bin/node" "/Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents/node_modules/node-pre-gyp/bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents
node-pre-gyp ERR! node -v v10.20.1
node-pre-gyp ERR! node-pre-gyp -v v0.12.0
node-pre-gyp ERR! not ok 
Failed to execute '/Users/loh/.nvm/versions/node/v10.20.1/bin/node /Users/loh/.nvm/versions/node/v10.20.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64/fse.node --module_name=fse --module_path=/Users/loh/dev/inq-seeds/boilerplate/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64 --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v64' (1)

3. Test Code Signing

I also wanted to test the whole process. So in spite of the node-pre-gyp error, I ran the package and signing commands:
7. npm run dev -- compile to make sure it runs
8. Check http://localhost:3000 and http://localhost:3000/#/admin to make sure app loads
9. Make sure you prepped the files using the "Preparing for Electron" instructions above.
10. npm run package
(https://github.com/theRAPTLab/meme/blob/dev-src/file-paths/README-signing.md)
11. npm run appsign
12. Zipped up the MEME MacOS folder
13. Airdropped the zip to an M1 (Apple Silicon) computer.
14. Unzipped the folder
15. Ran the meme.app
16. Got the following error:

I'm pretty sure this was working last time

I'm not done testing the path issues though.

Sakelun · 2023-09-30T23:45:56Z

@benloh We've discussed these issues offline, to re-cap:

The friendly name for the MacOS folder has been changed to macOS
This was found to be a pre-existing issue that also exists on the current dev branch and is being addressed in fsevents npm errors #137
This is due to Application Translocation on Apple systems. When an application bundle is downloaded, it acquires an extended filesystem attribute com.apple.quarantine that causes the Gatekeeper security system to run the application within a read-only folder in a randomized private path. This prevents file system write operations (such as those necessary to create the .loki file or supporting directories) from succeeding. To maintain the desired file system layout described in this ticket, we have worked around this by including a shell script (install.sh) in the Mac distribution package that can be run to remove the quarantine flag.

Other alternatives would include trialing placing these resources within the bundle (Contents\Resources) - though this may also be read-only; or using an absolute path to a known writeable location such as the user's Documents folder.

benloh · 2023-10-01T06:41:46Z

@jdanish We have a LOT of documentation (especially wiki rewrites) that we need to do to fully test and document this. But if you need to get started Monday, here's the essentials of what you need to know:

Use node v10.9.0 if you run into any npm problems. See fsevents npm errors #137.
If you haven't set up an Apple Developer account and did all the prep work for code signing, I recommend that you not worry about code signing for now and just do an unsigned Electron build. See README-signing.md for more details. You should still be able to copy and run the Electron app.
If you want to build the Electron app and copy it to another machine, you need to run a new install.sh script.
a. Build the electron app (npm run package) and copy the /dist/MEME macOS folder to your new computer/new location, e.g. to ~/Desktop/MEME macOS.
b. Open a terminal.
c. cd ~/Desktop/MEME macOS
d. ./install.sh
e. The quarantine flag should now be fixed.
f. Double click the meme.app Electron application to start the server.
g. If you see javascript error, then the quarantine flag was probably not removed successfully. Try typing xattr meme.app -- you should NOT see the line com.apple.quarantine
If you copy the app to another computer you may have to run ./install.sh again.
The app will start with a default _blank template project. The easiest way to set up a project is to just use the admin site to add content. Here's a quick overview of what you need to know about the file locations (it actually gets much more complicated given the matrix of options).
A. Development Mode: Using Local Dev Server / Digital Ocean running npm run dev or npm run electron
- \meme\data\db\test.loki -- The development mode project db file will be in test.loki. You can copy this file to other installs to copy the project database.
- \meme\resources\* -- To add resources, add them to the \meme\resources\ folder. The URL for resources when adding a new resource on the admin page is .../static/dlc/helloworld.pdf.
B. Production Mode: Using Local Dev Server / Digital Ocean running npm run app
- \meme\data\db\meme.loki -- The production mode project db file will be in meme.loki. You can copy this file to other installs to copy the project database.
- \meme\resources\* -- To add resources, add them to the \meme\resources\ folder. The URL for resources when adding a new resource on the admin page is .../static/dlc/*.pdf.
C. Production Mode: Using Electron App by double-clicking on Electron
- \meme\dist\MEME macOS\data\db\meme.loki -- The production mode project db file will be in meme.loki. You can copy this file to other installs to copy the project database.
- \meme\dist\MEME macOS\resources\* -- To add resources, add them to the \meme\dist\MEME macOS\resources\ folder. The URL for resources when adding a new resource on the admin page is .../static/dlc/*.pdf.

#136.

…d be `npm run package`

benloh · 2023-10-21T22:55:56Z

@Sakelun Sorry this took so long, but I finally wrote up testing instructions and documented what we did.

See #136 for the draft wiki update and testing instructions (at bottom of file).

A few things to note:

I added a few fixes directly, so please do a pull
In meme.js:329 f_DebugApp() is still referencing meme-darwin-x64 in two places. These break npm run app.
- line 333: const { code, stderr } = shell.exec('codesign -dvv ./dist/meme-darwin-x64/meme.app', {
- line 339: shell.exec('./dist/meme-darwin-x64/meme.app/Contents/MacOS/meme');
  I assume you need to use your appPath fix (line 276) to point to the right folder?

in meme.js:166 f_PackageApp()references meme-darwin-x64 in the three of the output messages. Since things seem to be working I'm assuming this is just a display issue rather than a broken reference? The output lines are:

clickr:boilerplate loh$ npm run package
> [email protected] package /Users/loh/dev/inq-seeds/boilerplate
> node meme package
...
MEME EXEC    - installing node dependencies into ./built
**MEME EXEC    - using electron-packager to write 'meme.app' to /Users/loh/dev/inq-seeds/boilerplate/dist/meme-darwin-x64**
Packaging app for platform darwin x64 using electron v3.1.13
**Wrote new app to ../dist/meme-darwin-x64**
**MEME EXEC    - copying templates to output folder: /Users/loh/dev/inq-seeds/boilerplate/dist/meme-darwin-x64**

Either way we should fix the output messages so they point to the right folder (e.g. dist/"MEME macOS"/dist/...

I think I know what the fixes should be, but I wanted to confirm with you to make sure I was reading things correctly. I'm assuming I need to add this and use appPath as the reference:

  const appPath = path.join(
    __dirname,
    'dist',
    platformConfig.friendlyName != null
      ? platformConfig.friendlyName
      : path.basename(platformConfig.packageOutput),
    'meme.app'
  );

jdanish · 2023-10-22T13:03:05Z

I know this wasn't meant for me, but 3 quick comments / questions:

When I ran this, I received the following error:

I followed the instructions and all was then fine. Not sure if that is an issue or just something for me to be aware of?

the readme file being put into the empty resources folder has no extension. Can you add md, txt, or rtf as appropriate so that novices can open it by double-clicking? I simply dragged to VSC but that's not a typical user reaction.
If I read this correctly, I can sign up for a $99/year apple developer license and then I'll also be able to sign, making life easier on our end, is that correct?

Thanks!

jdanish · 2023-10-22T13:45:21Z

Also two issues with the readme:

The screenshot extension is no-longer being supported so please remove this section from the readme: IV.A. Install Screen Capture Extension
The app can no-longer be copied on its own for distribution. Rather, you'd need to (zip and) copy the entire contents of the dist folder (e.g., MEME macOS) otherwise it'll not have updated content and will choke on a number of errors.

I can also do it, but wasn't sure if readme was being actively worked on.

Sakelun · 2023-10-22T20:26:57Z

@benloh I have reviewed your testing results and made the following updates:

In meme.js:329 f_DebugApp() is still referencing meme-darwin-x64 in two places. These break npm run app.

line 333: const { code, stderr } = shell.exec('codesign -dvv ./dist/meme-darwin-x64/meme.app', {

line 339: shell.exec('./dist/meme-darwin-x64/meme.app/Contents/MacOS/meme');
I assume you need to use your appPath fix (line 276) to point to the right folder?

Additional paths have been added to the output of f_GetPlatformConfig() to address the issue above and further support cross-platform usage:

distOutput: absolute path to the distribution folder created at the end of the package step with the friendly platform name
entrypoint: path within the a distribution folder to the executable entry point (can vary per platform), e.g. meme.app/Contents/MacOS/meme (macOS), meme.app/meme (Linux)
appPath: path to the "app" folder within the distribution, nominally "meme.app"; derived from entrypoint

Example output from f_GetPlatformConfig():

{ 
  platform: 'linux',
  arch: 'x64',
  packageOutput: '/home/scott/Projects/SEEDS/seeds-src/dist/meme-linux-x64',
  distOutput: '/home/scott/Projects/SEEDS/seeds-src/dist/MEME Linux',
  appPath: 'meme.app',
  entrypoint: 'meme.app/meme',
  friendlyName: 'MEME Linux'
}

in meme.js:166 f_PackageApp()references meme-darwin-x64 in the three of the output messages. Since things seem to be working I'm assuming this is just a display issue rather than a broken reference? The output lines are:
clickr:boilerplate loh$ npm run package
> [email protected] package /Users/loh/dev/inq-seeds/boilerplate
> node meme package
...
MEME EXEC    - installing node dependencies into ./built
**MEME EXEC    - using electron-packager to write 'meme.app' to /Users/loh/dev/inq-seeds/boilerplate/dist/meme-darwin-x64**
Packaging app for platform darwin x64 using electron v3.1.13
**Wrote new app to ../dist/meme-darwin-x64**
**MEME EXEC    - copying templates to output folder: /Users/loh/dev/inq-seeds/boilerplate/dist/meme-darwin-x64**
Either way we should fix the output messages so they point to the right folder (e.g. dist/"MEME macOS"/dist/...

It's a bit counter-intuitive but the messages are correct; the Electron Packager only seems to allow specification of the output directory where platform-specific builds are placed. The naming of the platform folder is not configurable. The build sequence was renaming the Electron Packager output folder to the friendlier folder name we'd expect as the final step.

To make this clearer, the build sequence has been changed to do the friendly platform folder rename immediately after the electron packaging step with a log message. Subsequent actions (the template copy) now refer to the friendly output folder:

MEME EXEC    - packaging electron app 'meme.app'
MEME EXEC    - erasing ./built and ./dist directories
MEME EXEC    - compiling console, web, system files into ./built
MEME EXEC    - installing node dependencies into ./built
MEME EXEC    - using electron-packager to write 'meme.app' to /home/scott/Projects/SEEDS/seeds-src/dist/meme-linux-x64
Packaging app for platform linux x64 using electron v3.1.13
Wrote new app to ../dist/meme-linux-x64
**MEME EXEC    - renaming package folder to friendly platform name: /home/scott/Projects/SEEDS/seeds-src/dist/MEME Linux
**MEME EXEC    - copying templates to output folder: /home/scott/Projects/SEEDS/seeds-src/dist/MEME Linux
MEME EXEC    - electron app written to /home/scott/Projects/SEEDS/seeds-src/dist/MEME Linux

I think I know what the fixes should be, but I wanted to confirm with you to make sure I was reading things correctly. I'm assuming I need to add this and use appPath as the reference:
  const appPath = path.join(
    __dirname,
    'dist',
    platformConfig.friendlyName != null
      ? platformConfig.friendlyName
      : path.basename(platformConfig.packageOutput),
    'meme.app'
  );

That would have been correct to identify the application folder (for signing). The executable path varies per platform so to better handle all of these circumstances and eliminate redundant code, it's been isolated within the configuration returned by f_GetPlatformConfig() to simplify this elsewhere:

const signedPath = path.join(distOutput, appPath)
const execPath = path.join(distOutput, entryPoint) // needs to be enclosed in double quotes if passing to shell.exec due to spaces

Other Improvements / Fixes

Additional guards have been added to avoid checking the signature using codesign on non-Mac platforms
Bail the process if the platform string isn't recognized rather than fallback to the default Electron package out

Remaining Tasks
I will review your testing notes and:

Consider the relationship between the runtime and build path calculations to see if consolidation is warranted
Test the various build/application entry cases on macOS
Test the various build/application entry cases on Linux

Sakelun · 2023-10-23T14:24:47Z

Hey @jdanish,

the readme file being put into the empty resources folder has no extension. Can you add md, txt, or rtf as appropriate so that novices can open it by double-clicking? I simply dragged to VSC but that's not a typical user reaction.

Thanks for pointing that out! A .md extension has been added to the README within the _blank template.

benloh · 2023-10-24T04:43:37Z

@Sakelun thanks for the meme.js fixes. Confirmed works.
Re the README fix. I think we've been using .txt for tech novices, so READMEs on end-user- and teacher-facing files use .txt because they don't know what an .md file is.

If I read this correctly, I can sign up for a $99/year apple developer license and then I'll also be able to sign, making life easier on our end, is that correct?

@jdanish Yes. See README-signing.md for details on how to implement that. It can be pretty hairy the first time, so give yourself plenty of time.

The screenshot extension is no-longer being supported so please remove this section from the readme: IV.A. Install Screen Capture Extension
The app can no-longer be copied on its own for distribution. Rather, you'd need to (zip and) copy the entire contents of the dist folder (e.g., MEME macOS) otherwise it'll not have updated content and will choke on a number of errors.

@jdanish noted. Actually I think we need to rewrite the the whole section "II. Building and Running the Electron App" section, incorporating the changes from #136.

TO DO

Re-implement caniuse-lite fix
Rewrite README.md, incorporating pathing fixes from Update package instructions #136

jdanish · 2023-10-24T11:46:39Z

Hi all - I just noticed in the most recent readme that we are still using ../static/dlc instead of /resources ... can we instead use resources? Or just assume it is in resources and so file.pdf is actually resources/file.pdf and sub/file.pdf is resources/sub/file.pdf? And either way, if there is an http or https we look on the web?

As noted above, we do not need the static/dlc for backwards compatibility and so it is a bit confusing. If this is a big change, it can wait since I assume we'll revisit this system with the new admin / resource handling and I believe it'll mostly be our team using this app in the short-term. However, if it is easy, seems worth it?

We should also note that sub-folders are also AOK. I can update any help text as needed, though.

Sakelun · 2023-10-24T20:35:11Z

@jdanish

Hi all - I just noticed in the most recent readme that we are still using ../static/dlc instead of /resources ... can we instead use resources? Or just assume it is in resources and so file.pdf is actually resources/file.pdf and sub/file.pdf is resources/sub/file.pdf? And either way, if there is an http or https we look on the web?

As noted above, we do not need the static/dlc for backwards compatibility and so it is a bit confusing. If this is a big change, it can wait since I assume we'll revisit this system with the new admin / resource handling and I believe it'll mostly be our team using this app in the short-term. However, if it is easy, seems worth it?

We should also note that sub-folders are also AOK. I can update any help text as needed, though.

This can be done. It would seem best to go the route of just eliminating the prefix sub-directory within the resource database entirely, so /fishtank.pdf would map to <meme-webserver>/resources/fishtank.pdf.

Eliminate hosting path folder prefix in resource databases

…tic/dlc/" prefix.

jdanish · 2023-10-24T22:47:44Z

Confirming the fix to remove the need for .../static/dlc. You now need /file.pdf (or similar). Ideally, we'd cut the / (so you put file.pdf or folder/file.pdf) but I presume there is some reason to include the / ? Either way, this works better now, thanks!

Sakelun · 2023-10-25T00:00:35Z

@jdanish

Confirming the fix to remove the need for .../static/dlc. You now need /file.pdf (or similar). Ideally, we'd cut the / (so you put file.pdf or folder/file.pdf) but I presume there is some reason to include the / ? Either way, this works better now, thanks!

I wasn't sure, but I've removed it and done some light testing and it doesn't seem consequential.

jdanish · 2023-10-25T00:04:26Z

Thanks! Some quick testing shows that the new version continues to work without the / for a file or sub-folder, and also works for urls. I think this version is more intuitive. I'll keep testing but it seems like this works now.

benloh · 2023-10-25T17:06:23Z

@jdanish Scott noticed that some links used for resources don't work -- some testing for the links. #1 and #4 work. #2/#3 are unlinkable because the target sites don't let themselves be loaded in iframes

Is this an issue? Do we need to add support for opening resource links in other windows?

jdanish · 2023-10-25T17:11:37Z

"Is this an issue? Do we need to add support for opening resource links in other windows?"

@benloh and @Sakelun good catch! I am using more recent data so didn't notice that, though I've seen it before. For now, no it's not an issue. 95% (or more!) of what we use is local pdfs or html pages (for net logo) and then occasionally google docs. It makes a lot more sense to use those things we can control. If that changes long-term we can put a request in but our focus right now is local materials so no worries. Thanks!

benloh

Confirmed works.

Sakelun added 5 commits August 24, 2023 12:56

Runtime folder structure separated from the codebase to eliminate dat…

cb2ac06

…aset dependent packaging and signing.

Removed datasets that are now distributed as templates.

805d0b0

Update path/template logging.

d861b0d

Match module code style conventions.

584af96

Merge remote-tracking branch 'origin/dev' into dev-src/file-paths

b050f40

Sakelun linked an issue Aug 31, 2023 that may be closed by this pull request

Move Resources out of Electron build #84

Closed

Sakelun requested a review from benloh August 31, 2023 19:04

Sakelun self-assigned this Aug 31, 2023

Sakelun changed the base branch from master to dev August 31, 2023 19:05

fix: do not assume resources folder exists in a template.

df2950c

Force VCS inclusion/document use of template resource folder.

b48e4ac

Sakelun added 2 commits September 29, 2023 15:14

Re-arrange output folder structure and begin supporting cross-platfor…

992a2c6

…m builds.

fix: path update for MacOS

3d6ce96

workaround: include shell script to clear Apple quarantine flag

4514868

Sakelun and others added 2 commits October 1, 2023 11:58

fix: update file mode for script to executable for group/anyone.

3aa2058

file-paths. Force node v10.9.0 to address fsevents/node-pre-gyp errors.

3bfe558

#136.

benloh mentioned this pull request Oct 21, 2023

Default Project Files #90

Closed

benloh mentioned this pull request Oct 21, 2023

Request default project #89

Closed

file-paths: Fix bad reference to npm build package in README. Shoul…

0ec426c

…d be `npm run package`

jdanish mentioned this pull request Oct 22, 2023

Dev jd/updating notes #138

Merged

Sakelun added 2 commits October 22, 2023 16:31

Updated platform paths; extended cross-platform support.

3b64284

Add file extension to README

e116820

Rename README.md to README.txt

5f374c4

Resource datasets referring to local resources no longer need "../sta…

60676b8

…tic/dlc/" prefix.

Remove '/' prefix from resource datasets.

d82b667

benloh mentioned this pull request Oct 25, 2023

CRITICAL: Fix 'Browserlist: caniuse-lite is outdated' error that prevents package building - [merged] #127

Closed

benloh approved these changes Oct 27, 2023

View reviewed changes

benloh merged commit 4e49866 into dev Oct 27, 2023

benloh mentioned this pull request Oct 27, 2023

Update package instructions #136

Closed

benloh mentioned this pull request May 22, 2024

Merge v1.4.0-alpha into master theRAPTLab/meme-2023#2

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Runtime folder structure and default/Resource packaging updates #135

Runtime folder structure and default/Resource packaging updates #135

Sakelun commented Aug 31, 2023 •

edited by benloh

Loading

jdanish commented Aug 31, 2023

benloh commented Aug 31, 2023

jdanish commented Aug 31, 2023

jdanish commented Aug 31, 2023

Sakelun commented Aug 31, 2023

jdanish commented Aug 31, 2023

Sakelun commented Aug 31, 2023

benloh commented Sep 11, 2023

Sakelun commented Sep 12, 2023

Sakelun commented Sep 29, 2023

benloh commented Sep 30, 2023

Sakelun commented Sep 30, 2023 •

edited

Loading

benloh commented Oct 1, 2023

benloh commented Oct 21, 2023 •

edited by Sakelun

Loading

jdanish commented Oct 22, 2023

jdanish commented Oct 22, 2023

Sakelun commented Oct 22, 2023 •

edited

Loading

Sakelun commented Oct 23, 2023

benloh commented Oct 24, 2023

jdanish commented Oct 24, 2023

Sakelun commented Oct 24, 2023 •

edited

Loading

jdanish commented Oct 24, 2023

Sakelun commented Oct 25, 2023

jdanish commented Oct 25, 2023

benloh commented Oct 25, 2023

jdanish commented Oct 25, 2023

benloh left a comment

Runtime folder structure and default/Resource packaging updates #135

Runtime folder structure and default/Resource packaging updates #135

Conversation

Sakelun commented Aug 31, 2023 • edited by benloh Loading

Note:

Test Instructions

Test basic paths

Test create a blank database with /templates/_blank

1. Test npm run dev with blank

2. Test npm run electron with blank

3. Test npm run app with blank

Test with default /templates/test template folder

Test with default /templates/meme template folder

Test copy distribution

jdanish commented Aug 31, 2023

benloh commented Aug 31, 2023

jdanish commented Aug 31, 2023

jdanish commented Aug 31, 2023

Sakelun commented Aug 31, 2023

jdanish commented Aug 31, 2023

Sakelun commented Aug 31, 2023

benloh commented Sep 11, 2023

Sakelun commented Sep 12, 2023

Sakelun commented Sep 29, 2023

benloh commented Sep 30, 2023

1. 'MEME macOS`

2. npm ci error

3. Test Code Signing

Sakelun commented Sep 30, 2023 • edited Loading

benloh commented Oct 1, 2023

benloh commented Oct 21, 2023 • edited by Sakelun Loading

jdanish commented Oct 22, 2023

jdanish commented Oct 22, 2023

Sakelun commented Oct 22, 2023 • edited Loading

Sakelun commented Oct 23, 2023

benloh commented Oct 24, 2023

jdanish commented Oct 24, 2023

Sakelun commented Oct 24, 2023 • edited Loading

jdanish commented Oct 24, 2023

Sakelun commented Oct 25, 2023

jdanish commented Oct 25, 2023

benloh commented Oct 25, 2023

jdanish commented Oct 25, 2023

benloh left a comment

Choose a reason for hiding this comment

Sakelun commented Aug 31, 2023 •

edited by benloh

Loading

Test create a blank database with `/templates/_blank`

1. Test `npm run dev` with blank

2. Test `npm run electron` with blank

3. Test `npm run app` with blank

Test with default `/templates/test` template folder

Test with default `/templates/meme` template folder

Sakelun commented Sep 30, 2023 •

edited

Loading

benloh commented Oct 21, 2023 •

edited by Sakelun

Loading

Sakelun commented Oct 22, 2023 •

edited

Loading

Sakelun commented Oct 24, 2023 •

edited

Loading