-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #3 from VirtCode/next
Version v1.4.0
- Loading branch information
Showing
79 changed files
with
2,374 additions
and
2,567 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,45 @@ | ||
| # <img src="brand/banner.svg"/> | ||
| This app allows one to use their smartphone as a computer mouse. The movement of the smartphone is measured over an accelerometer and is then transmitted over Bluetooth to a computer. | ||
| <div align="center"> | ||
| <picture> | ||
| <source media="(prefers-color-scheme: dark)" srcset="brand/title-dark.svg"> | ||
| <img alt="SmartMouse logo and text" src="brand/title-light.svg"> | ||
| </picture> | ||
|
|
||
| <p>SmartMouse is an app that aims to make a smartphone usable as a normal computer mouse. It does estimate its position by processing data from the integrated accelerometer and gyroscope with a custom sensor-fusion algorithm. This data is then transmitted to a host computer over the Bluetooth HID Profile. The mouse buttons are recreated using the Touchscreen.</p> | ||
| </div> | ||
|
|
||
| --- | ||
|
|
||
| ## State | ||
| Currently, this app is still in early development. The position estimation is currently not always correct, which leads to different artifacts and wrong movements of the cursor. Because of that, this app is not yet ready for practical use. | ||
|
|
||
| ## Features | ||
| This app is defined by the following three core features: | ||
|
|
||
| - **Real Position**: The real position of the device is estimated like a normal mouse using the device's internal sensors. | ||
| - **Serverless Connection**: To connect to a target device, the Bluetooth HID Profile is used. Thus, **no** special software or server is required on the target device. | ||
| - **Powerful Interface**: The UI of this app is designed to make it usable for everyone. However, powerful customization of the algorithm and interface is still possible over the advanced settings. | ||
|
|
||
| ## Installation | ||
| To install this app, you can download an APK from the [release section](https://github.com/VirtCode/SmartMouse/releases). This debug APK can then be installed on an Android device. Sometimes, sideloading apps must be enabled in the device settings. | ||
| To install this app, you can download a built debug APK from the [release section](https://github.com/VirtCode/SmartMouse/releases). This APK can easily be installed on your Android device by just opening it. In some cases, sideloading must be enabled in the device settings in to install it. | ||
|
|
||
| After installation, the app should be self-explanatory. First, add your computer to the devices on the "Connect" page. Then connect to it and open the "Mouse" page. Now you can use your device just like a normal mouse, by moving it on a surface and using the touchscreen to click and scroll. | ||
|
|
||
| ## Processing Algorithm | ||
| This app uses a custom sensor-fusion algorithm to estimate the position of the device that is developed specifically for mouse-like movements. | ||
|
|
||
| *This algorithm will be documented here in the future.* | ||
|
|
||
| ## Debugging | ||
| **The following steps and procedures are only intended for debugging and are NOT required for normal usage.** | ||
|
|
||
| For general debugging or just to navigate the source code conveniently, clone this repository and open the Gradle project in Android Studio or a similar IDE. | ||
|
|
||
| To debug the algorithm, another method should be used. This app includes special debugging features. Sensor and processing data can be transmitted in real-time over the network and stored as CSV files on a computer. These files may be analyzed more precisely with data analysis software. Please note that a major delay in the normal transmission via Bluetooth may occur when the debugging transmission is also active. | ||
|
|
||
| Please note that this app only supports Android version 9 or higher. | ||
| Follow these steps to set up a debugging environment: | ||
|
|
||
| ## Debug Environment | ||
| To debug this app or to navigate its source code conveniently, you may clone this repository and open the Gradle project in Android Studio or another IDE. | ||
| 1. Download [SensorServer](https://github.com/VirtCode/SensorServer) on your computer, move it to a directory you know, and run it in a terminal like described on its GitHub page. | ||
| 2. Configure debugging in your SmartMouse app on your smartphone by enabling advanced settings and enabling "Debug Transmission" in the debugging category. Change the hostname to the local IP of your computer. Restart the app for the settings to take effect. | ||
| 3. Now use the app like normal. When you enter the mouse interface, a transmission is automatically started. This transmission is ended when you leave the mouse interface. The ID of this transmission can be seen in the output of the SensorServer. | ||
|
|
||
| ## Note | ||
| This project is part of a matura paper. | ||
| Now you are ready to analyze transmissions which are stored as CSV in your folder by SensorServer. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
116 changes: 116 additions & 0 deletions
116
app/src/main/java/ch/virt/smartphonemouse/customization/CalibrationHandler.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| package ch.virt.smartphonemouse.customization; | ||
|
|
||
| import android.content.Context; | ||
| import android.hardware.Sensor; | ||
| import android.hardware.SensorEvent; | ||
| import android.hardware.SensorEventListener; | ||
| import android.hardware.SensorManager; | ||
| import androidx.preference.PreferenceManager; | ||
| import ch.virt.smartphonemouse.mouse.Calibration; | ||
| import ch.virt.smartphonemouse.mouse.MovementHandler; | ||
| import ch.virt.smartphonemouse.mouse.Parameters; | ||
| import ch.virt.smartphonemouse.mouse.math.Vec3f; | ||
|
|
||
| /** | ||
| * This class is used to measure and save the sampling rate of the inbuilt accelerometer. | ||
| */ | ||
| public class CalibrationHandler implements SensorEventListener { | ||
|
|
||
| private static final float NANO_FULL_FACTOR = 1e-9f; | ||
|
|
||
| private SensorManager manager; | ||
| private Sensor accelerometer; | ||
| private Sensor gyroscope; | ||
|
|
||
| private boolean begun = false; | ||
| private long firstTime = 0; | ||
| private Vec3f gyroSample = new Vec3f(); | ||
| private boolean registered; | ||
|
|
||
| private final Context context; | ||
| private final Calibration calibration; | ||
|
|
||
| /** | ||
| * Creates the calibrator. | ||
| * | ||
| * @param context context to use | ||
| */ | ||
| public CalibrationHandler(Context context) { | ||
| this.context = context; | ||
|
|
||
| calibration = new Calibration((state) -> {}, new Parameters(PreferenceManager.getDefaultSharedPreferences(context))); | ||
| fetchSensor(); | ||
| } | ||
|
|
||
| /** | ||
| * Fetches the sensor from the system. | ||
| */ | ||
| private void fetchSensor() { | ||
| manager = (SensorManager) context.getSystemService(Context.SENSOR_SERVICE); | ||
| accelerometer = manager.getDefaultSensor(MovementHandler.SENSOR_TYPE_ACCELEROMETER); | ||
| gyroscope = manager.getDefaultSensor(MovementHandler.SENSOR_TYPE_GYROSCOPE); | ||
| } | ||
|
|
||
| /** | ||
| * Registers itself as a listener. | ||
| */ | ||
| private void register() { | ||
| if (registered) return; | ||
| manager.registerListener(this, accelerometer, MovementHandler.SAMPLING_RATE); | ||
| manager.registerListener(this, gyroscope, MovementHandler.SAMPLING_RATE); | ||
|
|
||
| registered = true; | ||
| } | ||
|
|
||
| /** | ||
| * Unregisters itself as a listener. | ||
| */ | ||
| private void unregister() { | ||
| if (!registered) return; | ||
| manager.unregisterListener(this, accelerometer); | ||
| manager.unregisterListener(this, gyroscope); | ||
|
|
||
| registered = false; | ||
| } | ||
|
|
||
| /** | ||
| * Starts the measuring process. | ||
| * | ||
| * @param doneListener listener that is executed once the process has finished | ||
| */ | ||
| public void calibrate(Calibration.StateListener doneListener) { | ||
| calibration.setListener((state) -> { | ||
| if (state == Calibration.STATE_END) unregister(); | ||
|
|
||
| doneListener.update(state); | ||
| }); | ||
|
|
||
| begun = false; | ||
| calibration.startCalibration(); | ||
|
|
||
| register(); | ||
| } | ||
|
|
||
| @Override | ||
| public void onSensorChanged(SensorEvent event) { | ||
| if (event.sensor.getType() == MovementHandler.SENSOR_TYPE_ACCELEROMETER) { | ||
|
|
||
| if (!begun) { | ||
| begun = true; | ||
| firstTime = event.timestamp; | ||
| } | ||
|
|
||
|
|
||
| float time = (event.timestamp - firstTime) * NANO_FULL_FACTOR; | ||
| Vec3f acceleration = new Vec3f(event.values[0], event.values[1], event.values[2]); | ||
|
|
||
| calibration.data(time, acceleration, gyroSample); | ||
|
|
||
| } else if (event.sensor.getType() == MovementHandler.SENSOR_TYPE_GYROSCOPE) { | ||
| this.gyroSample = new Vec3f(event.values[0], event.values[1], event.values[2]); | ||
| } | ||
| } | ||
|
|
||
| @Override | ||
| public void onAccuracyChanged(Sensor sensor, int accuracy) {} | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.