Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mobile: Support building for web #10650

Merged
merged 191 commits into from
Aug 2, 2024

Conversation

personalizedrefrigerator
Copy link
Collaborator

@personalizedrefrigerator personalizedrefrigerator commented Jun 22, 2024

Summary

[ Try it | Forum post | Implementation progress ]

This pull request allows running Joplin Mobile in a web browser using react-native-web and @sqlite.org/sqlite-wasm.

Note: The demo URL has been updated to https://joplin.github.io/web-app/.

Possible use-cases

  • Running Joplin on devices that can't easily install software.

  • Getting feedback on proposed mobile app changes — clicking on a link in a web browser is much easier than installing an Android APK.

  • Developing and testing plugins for mobile without access to an Android device.

  • Installing non-recommended plugins on iOS.

  • Could allow users to try plugins without installing them:

    • This would be done in a separate pull request. It could be done by hosting a separate copy of the web client, configured to use an in-memory database and virtual filesystem.
  • May simplify developing and testing the mobile app:

    • This could allow creating end-to-end tests for mobile with Playwright (we currently only have these for desktop).
    • On Linux and Windows, manually testing changes to the mobile app currently requires running an Android emulator or access to a physical Android device. On slower/lower-memory computers, Android emulators are very slow and make development more difficult.

Related projects

Demo

The work/mobile-web branch is deployed to https://personalizedrefrigerator.github.io/joplin/web-client/.

Note that this demo works best in a Chromium-based browser. While it loads in Firefox, the initial setup is very slow.

Why not Joplin Desktop?

It seemed simpler to add support for web to the mobile app than the desktop app. In particular, Joplin Desktop depends on a large number of NodeJS-only libraries and APIs (e.g. fs-extra).

Tasks

  • Database
    • Migrate legacy syntax not supported by @sqlite.org/sqlite-wasm
    • Optional (upstream?): Fix slow database access in Firefox? This might be an upstream issue related to sqlite-wasm.
  • Note editor
  • Note viewer
    • Can view rendered markdown.
    • Mermaid
      • Mermaid scripts/styles aren't loaded.
    • KaTeX
      • Partial — KaTeX fonts aren't loaded.
    • External links
    • Internal links
  • Sync
    • Dropbox
    • File system sync (desktop Chromium-based browsers only).
    • Joplin Server — mostly works — see "Joplin Cloud".
      • Only works if served from the same origin as the web client.
    • Optional: OneDrive?
      • Follow-up PR: Authentication workflow needs to be adjusted. For now, it's disabled.
    • Follow-up PR: Joplin Cloud
      • This can only be done after we decide on a hosting URL for the web client.
  • Drawing
    • Inserting
    • Editing
  • Allow installing plugins using "install from file"
    • Chrome
    • Firefox
  • Allow installing plugins from the plugin repo.
  • Camera.
    • Partial — only works on some mobile devices. On other devices, a file picker is shown.
  • Offline access & installability.
    • Currently, files are cached only when first requested. As such, if a font has never been requested (e.g. an icon font only used in settings), it might not be available offline.
  • Optional: Decrease size of bundle.js — extract plugin assets to separate files.
    • Optional — could be done separately. The bundle is currently about 14.5 MiB.
  • Fix accessibility bugs. Edit: On iOS, VoiceOver accessibility seems to work about as well as it does on mobile (has roughly the same issues). Focus issues seem to be similar.
  • The last change to a note is sometimes not saved.
    • Observed in Google Chrome, may occur in other browsers.
    • Existing reports of this issue in the mobile app suggest that this is not specific to the web client.
    • This might be fixed when 70bfb26 is merged.

@personalizedrefrigerator personalizedrefrigerator changed the title Proof of concept: Add web support to Joplin Mobile Proof of concept: Mobile: Support building for web Jun 22, 2024
Copy link
Collaborator Author

@personalizedrefrigerator personalizedrefrigerator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left a few comments that might help while reviewing this pull request!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DialogManager.tsx was added due to the lack of support for Alert.alert in react-native-web. On web, it manages showing custom dialogs based on react-native-paper, with an Alert.alert compatible API. On other platforms, it continues to show dialogs with Alert.alert.

It also sets shim.showMessageBox, allowing plugins and other related code to use the custom dialogs.

Screenshot:
screenshot: rn-paper-based dialog

@@ -215,6 +217,7 @@ class Dropdown extends Component<DropdownProps, DropdownState> {
<TouchableWithoutFeedback
accessibilityElementsHidden={true}
importantForAccessibility='no-hide-descendants'
aria-hidden={true}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aria-hidden={true} is the web version of the "hide from screen readers and other accessibility tools". Currently, we use accessibilityElementsHidden={true} for this on iOS and importantForAccessibility='no-hide-descendants' on Android.

Also see the new components/accessibility/AccessibleView.tsx file.

Comment on lines +114 to +117
// allow-popups-to-escape-sandbox: Allows PDF previews to work on target="_blank" links.
// allow-popups: Allows links to open in a new tab.
permissions: 'allow-scripts allow-modals allow-popups allow-popups-to-escape-sandbox',
allow: 'clipboard-write=(self) fullscreen=(self) autoplay=(self) local-fonts=* encrypted-media=*',
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +26 to +28
// (web only) On web, touching buttons can cause the on-screen keyboard to be dismissed.
// Setting preventKeyboardDismiss overrides this behavior.
preventKeyboardDismiss?: boolean;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, this is used by the markdown toolbar.

Comment on lines +68 to +71
// Intended for web, where resources can't be linked to normally.
public async setResourceFile(id: string, file: Blob) {
this.resourcePathOverrides[id] = URL.createObjectURL(file);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On web, resources are stored in a virtual file system (OPFS). As such, to show them with <img>/<video> and tags, we need to create URLs for them.

@@ -53,6 +53,7 @@ const configScreenStyles = (themeId: number): ConfigScreenStyles => {
const settingContainerStyle: ViewStyle = {
flex: 1,
flexDirection: 'row',
flexBasis: 'auto',
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flex styling is handled slightly differently in react-native-web when compared to react-native. In particular, flex-basis (the default flex size) seems to default to 0 in some cases.

@@ -225,6 +219,7 @@ class LogScreenComponent extends BaseScreenComponent<Props, State> {
{this.state.filter !== undefined ? filterInput : null}
<FlatList
data={this.state.logEntries}
initialNumToRender={100}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works around an issue where log lines weren't showing on web.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related regression testing:

  • On Android 13, I have verified that:
    1. The "select date time" dialog can be opened from the note actions menu for a to-do (from "set alarm").
    2. Clicking "set date" shows a date/time selection dialog.
    3. Clicking "save alarm" asks for alarm permissions (I hadn't previously granted them to Joplin).
    4. Clicking "clear alarm" clears the alarm.

Comment on lines 55 to 58
/*
When installed on iOS, this ensures that the status bar color matches the
system light/dark theme.
*/
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that this is true...

const { reg } = require('../../registry.js');
const { reg } = require('../../registry');
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using registry.js here was causing two copies of registry to be included — the copy from files that require('registry') and the copy from files that require('registry.js'). As a result, certain global variables/properties in registry were not initialized in one copy, but were in the other.

To work around this issue, this pull request changes all registry.js imports to registry imports. Perhaps using metro instead of webpack would be a different way to address this issue?

@laurent22
Copy link
Owner

@personalizedrefrigerator, please let me know when you think this is ready to review

@personalizedrefrigerator
Copy link
Collaborator Author

@personalizedrefrigerator, please let me know when you think this is ready to review

This should be ready for review!

@laurent22
Copy link
Owner

Well I'm not going to say I reviewed these 156 files in detail, but what I did review looked good to me.

It says there are conflicts though - could you check please? It's a bit strange since it doesn't say on which file.

@compgeniuses
Copy link

this is very interesting

@laurent22 laurent22 merged commit f69dffc into laurent22:dev Aug 2, 2024
10 checks passed
@compgeniuses
Copy link

is there or would there be a docker image with docker compose deployment instructions?

@personalizedrefrigerator
Copy link
Collaborator Author

is there or would there be a docker image with docker compose deployment instructions?

I don't think that a docker image will be created for this — hosting the web app should just involve hosting static HTML.

Hosting requirements:

  • A server that supports serving static HTML/JavaScript/WASM/font files.
  • To be served over HTTPS.

Additional notes:

  • For security reasons, make sure that everything with the same origin (protocol, domain name, and port) as the web client is trusted. Everything with the same origin has access to the same storage data, so this prevents other pages from having unnecessary access. For example, if hosting from app.joplincloud.com, app.joplincloud.com/some-page.html and app.joplincloud.com/some/path/here/another-page.html both have access to the same browser persistent storage.
  • The web app is currently hosted by GitHub Pages. Its deployment script can be found here.

@compgeniuses
Copy link

ok, thank you, that was informative.

We have packaged a Joplinserver app for nethserver 8 here https://github.com/geniusdynamics/ns8-joplinserver

Hopefully, we can figure out best way to attach or implement the web app on the root doain of the hosted application.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants