forked from andresvcc/thymio_2_plus
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs(mobile apps): add documentation for mobile apps
added relevant documentation on mobile applications, in particular on their conception and maintenance, explaining the mechanics of operation in the integrated use of web files for iOS and Android, in each case showing how to work with web resources that are stored on the tablet.
- Loading branch information
Showing
19 changed files
with
985 additions
and
92 deletions.
There are no files selected for viewing
102 changes: 102 additions & 0 deletions
102
apps/thymio-suite-mobile-android/IntroductionAndroid.stories.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| import { Meta, Description } from '@storybook/addon-docs' | ||
|
|
||
| <Meta | ||
| title="Mobile app/Android/Introduction" | ||
| parameters={{ | ||
| viewMode: 'docs', | ||
| previewTabs: { | ||
| canvas: { hidden: true } | ||
| }, | ||
| }} | ||
| /> | ||
|
|
||
| # Welcome to the Thymio Suite Mobile for Android | ||
| --- | ||
|
|
||
| Welcome to the Thymio Suite App for Android, a comprehensive mobile application designed to seamlessly interface with the Thymio2+ robotic system through Android tablets. Utilizing the power of React Native, this app offers a robust, cross-platform mobile environment specifically optimized for Android devices. This introduction provides an overview of key features, setup instructions, and operational guidelines to help you maximize the educational potential of your Thymio2+ robots. | ||
|
|
||
| #### Purpose of the App | ||
|
|
||
| The Thymio Suite App aims to facilitate direct interaction between Android tablets and Thymio2+ robots, enhancing the educational experience in both classroom and home settings. The app enables users to program and control Thymio robots with ease, leveraging the advanced capabilities of Android devices to make robotics education both accessible and engaging. | ||
|
|
||
| #### Features | ||
|
|
||
| - **Direct Connectivity:** Uses Bluetooth or Wi-Fi to connect directly with Thymio2+ robots, ensuring stable communication and real-time control. | ||
| - **Interactive Programming Interface:** Offers a user-friendly programming interface that is suitable for all ages, making it straightforward to create, modify, and execute robot programming tasks. | ||
| - **Customizable Robot Settings:** Users can customize robot settings directly from their Android device, providing a flexible and personalized educational tool. | ||
| - **Real-Time Feedback:** Provides real-time feedback and diagnostics to monitor robot performance and identify any issues during operation. | ||
|
|
||
| #### System Requirements | ||
|
|
||
| - **Compatible Devices:** Optimized for Android tablets. | ||
| - **Android Version:** Requires Android 10.0 or later. | ||
| - **Connectivity:** Wi-Fi capability necessary for connecting with the Thymio2+ dongle. | ||
|
|
||
| ### Getting Started | ||
|
|
||
| #### Installation and Setup | ||
|
|
||
| This guide will walk you through the process of setting up the Thymio Suite app on an Android device. Follow these steps to ensure your development environment is ready and to run the app on an Android simulator or a physical device. | ||
|
|
||
| #### Prerequisites | ||
|
|
||
| 1. **Node.js**: Ensure Node.js (v18 or later) is installed. | ||
| 2. **Java JDK**: Install Java Development Kit (JDK 11 is recommended). | ||
| 3. **Android Studio**: Install Android Studio which will also install the latest Android SDK. During installation, make sure to include the Android Virtual Device (AVD) to emulate Android devices. | ||
| 4. **Environment Variables**: | ||
| - Set `JAVA_HOME` to your JDK path. | ||
| - Set `ANDROID_HOME` to your Android SDK location, typically `~/Library/Android/sdk` on macOS or `C:\Users\<Your-Username>\AppData\Local\Android\Sdk` on Windows. | ||
|
|
||
| #### Step 1: Install Yarn | ||
|
|
||
| Install Yarn globally using npm: | ||
|
|
||
| ```bash | ||
| npm install -g yarn | ||
| ``` | ||
|
|
||
| #### Step 2: Go to the project folder | ||
|
|
||
| ```bash | ||
| cd apps/thymio-suite-mobile-android | ||
| ``` | ||
|
|
||
| Run the following command in the project root directory to install all dependencies: | ||
|
|
||
| ```bash | ||
| yarn install | ||
| ``` | ||
|
|
||
| #### Step 3: Set Up Android Development Environment | ||
|
|
||
| 1. **Open Android Studio** and open your project by selecting the `android` folder within your project directory. | ||
| 2. **Check for SDK Updates**: In Android Studio, open the SDK Manager and ensure the required SDK platforms and tools are installed for the Android version you are targeting. | ||
| 3. **Create a Virtual Device**: If you don’t have a physical device, set up an AVD via Android Studio's AVD Manager. | ||
|
|
||
| #### Step 4: Prepare the Android Device | ||
|
|
||
| - **Physical Device**: Enable Developer options and USB debugging on your Android device. | ||
| - **Emulator**: Launch your AVD from Android Studio’s AVD Manager. | ||
|
|
||
| #### Step 5: Run the App | ||
|
|
||
| With your Android device ready and connected (or an emulator running), execute the following commands: | ||
|
|
||
| 1. Start the Metro bundler in a separate terminal window: | ||
|
|
||
| ```bash | ||
| yarn start | ||
| ``` | ||
|
|
||
| 2. Run the app on Android: | ||
|
|
||
| ```bash | ||
| yarn android | ||
| ``` | ||
|
|
||
| This command should install the app on the connected Android device or running emulator. | ||
|
|
||
| #### Step 6: Troubleshooting Common Issues | ||
|
|
||
| - **Build fails**: Check the build output for any errors. Common issues may relate to version incompatibilities or missing SDK tools. | ||
| - **App doesn’t connect to development server**: Ensure your device is on the same network as your development machine, and that the Metro bundler is running. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
143 changes: 143 additions & 0 deletions
143
apps/thymio-suite-mobile-android/src/scratch/scratch-android.stories.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| import { Meta, Description } from '@storybook/addon-docs' | ||
|
|
||
| <Meta | ||
| title="Mobile app/Android/Scratch" | ||
| parameters={{ | ||
| viewMode: 'docs', | ||
| previewTabs: { | ||
| canvas: { hidden: true } | ||
| }, | ||
| }} | ||
| /> | ||
|
|
||
| ## Scratch Interface Documentation Android | ||
|
|
||
| Scratch is a widely recognized visual programming language developed to simplify the process of learning to code. It's particularly useful for educational purposes due to its intuitive drag-and-drop interface. In relation to the Thymio2+ robot, Scratch provides a friendly and engaging way for users, especially younger learners, to program and control their robots. By using Scratch with Thymio2+, students can visually assemble code blocks to control the robot's actions and responses to environmental inputs, making complex concepts in robotics and programming accessible and fun. This integration empowers educators and students to explore the fundamentals of robotics programming in an interactive and supportive environment. | ||
|
|
||
| ## Integration of Scratch in the Thymio Suite Mobile App | ||
|
|
||
| To integrate the Scratch web interface into the Android mobile application, an underlying Android tool is used that allows for serving web files internally without the need for an external web server. Specifically, the files should be placed in the directory /android/app/src/main/assets. They can then be loaded into the WebView using the path file:///android_asset. This method simplifies the deployment process by utilizing the Android platform's native capabilities to handle local web content directly within the app. | ||
|
|
||
| ### Generating the URL | ||
|
|
||
| To construct this URL, necessary data must first be acquired from the 'scanner' page, which can be accessed using the following URL pattern: | ||
|
|
||
| ```javascript | ||
| http://127.0.0.1:3000/scanner/index.html?data=${ | ||
| JSON.stringify({ | ||
| ...LTServices | ||
| })}&gl=${JSON.stringify( | ||
| { | ||
| interface: 'vpl3', | ||
| }, | ||
| )}&lang=${language}` | ||
| ``` | ||
|
|
||
| Here, `LTServices` is an object containing device information structured as follows: | ||
|
|
||
| ```typescript | ||
| interface DeviceInfo { | ||
| port: number; | ||
| txt: { | ||
| uuid: string; | ||
| 'ws-port': string; | ||
| }; | ||
| addresses: string[]; | ||
| name: string; | ||
| fullName: string; | ||
| host: string; | ||
| } | ||
| interface DevicesMap { | ||
| [hostname: string]: DeviceInfo; | ||
| } | ||
| ``` | ||
|
|
||
| ### Purpose of the Scanner Page | ||
|
|
||
| The scanner page uses the data from `LTServices` to scan each TDM and retrieve a list of connected robots. Users can select the appropriate robot on this page, which then enables the Scratch page to be loaded with all necessary data to initiate the interface correctly. | ||
|
|
||
|
|
||
| #### Example URL Structure | ||
|
|
||
| An example URL to access the Scratch interface might look like this: | ||
|
|
||
| - **device Host**: The uuid of the device, typically a Thymio2+ that hosts the TDM (Thymio Device Manager) with which Scratch communicates. | ||
| - **pass**: The password for accessing the Thymio Device Manager. | ||
| - **ws**: The WebSocket ip and port used by the TDM for real-time communication. | ||
|
|
||
| ``` | ||
| file:///android_asset/scratch/index.html?device=%7Bd768ed1a-5831-425a-89a0-50d10f8c1a4c%7D&ws=ws://127.0.0.1:8597&pass=CX87IR | ||
| ``` | ||
| ## File Manager | ||
| The VPL3 interface uses a WebView to provide a block-based programming environment. File operations within this environment—specifically saving current projects and uploading existing ones—are facilitated through communication between the WebView and the native code via `postMessage`. This ensures a seamless integration of web-based components with mobile functionalities. | ||
| ### Workflow Description | ||
| #### Saving Files | ||
| **1. Trigger Save Action:** | ||
| - The save process is initiated by the user from the VPL3 web interface, typically through a save button. | ||
| **2. Capture Data in WebView:** | ||
| - When the save action is triggered, the web environment captures the current state of the program, which is serialized into JSON format. | ||
| **3. PostMessage to Native App:** | ||
| - The serialized data is sent to the native side using `postMessage`. This message includes an action type, such as `{ action: "saveProgram", content: jsonData }`. | ||
| **4. Native Handling:** | ||
| - The native code listens for messages from the WebView. Upon receiving the data, it invokes file writing operations using React Native's file system libraries. | ||
| **5. Confirm and Alert User:** | ||
| - After the file is successfully saved, the native app can send a confirmation back to the WebView or display a native alert confirming the save. | ||
| ```javascript | ||
| // React Native side | ||
| const handleFileUpload = async () => { | ||
| try { | ||
| const result = await DocumentPicker.pick({ type: [DocumentPicker.types.json] }); | ||
| const fileContent = await ReactNativeBlobUtil.fs.readFile(result.uri, 'utf8'); | ||
| webViewRef.current.postMessage(JSON.stringify({ action: "loadProgram", content: fileContent })); | ||
| } catch (err) { | ||
| console.error('File selection error:', err); | ||
| } | ||
| }; | ||
| ``` | ||
|
|
||
| #### Uploading Files | ||
|
|
||
| **1. Open Document Picker:** | ||
| - The user initiates the upload process, prompting the native side to open a document picker for file selection. | ||
|
|
||
| **2. Read and Send Data:** | ||
| - Once a file is selected, its content is read into a string using file reading utilities. | ||
| - This content is then sent back to the WebView using `postMessage`, similar to `{ action: "loadProgram", content: fileString }`. | ||
|
|
||
| **3. Load Data in WebView:** | ||
| - Inside the WebView, an event listener processes the received message and loads the program content into the VPL3 environment. | ||
|
|
||
| ```javascript | ||
| // React Native side | ||
| const handleSaveToFile = (jsonData) => { | ||
| const path = `${ReactNativeBlobUtil.fs.dirs.DocumentDir}/savedProgram.json`; | ||
| ReactNativeBlobUtil.fs.writeFile(path, jsonData, 'utf8') | ||
| .then(() => console.log('Save successful')) | ||
| .catch(err => console.error('Save failed', err)); | ||
| }; | ||
|
|
||
| // WebView side | ||
| window.ReactNativeWebView.postMessage(JSON.stringify({ action: "saveProgram", content: programData })); | ||
| ``` | ||
|
|
||
| ```javascript | ||
| // WebView side | ||
| window.addEventListener('message', function(event) { | ||
| const data = JSON.parse(event.data); | ||
| if (data.action === 'loadProgram') { | ||
| loadProgramFromJSON(data.content); | ||
| } | ||
| }); | ||
| ``` |
Oops, something went wrong.