Showing system ui without overlaying the application (GameActivity)

[lucasmerlin]

My application is not a game but rather an app using egui as ui, so I'd like to show the android status bar and navigation buttons while my app is open.
I would expect, that if you don't call hideSystemUI and call setDecorFitsSystemWindows with true (in the agdk-eframe example), that the app fits within the system ui.
While the system ui is shown sucessfully, it now overlays the top and the bottom part of the app (in the eframe example, the top bar where you select the different demos is completely hidden by the status bar on my device).

Seems like the underlying SurfaceView created by the GameActivity doesn't respect the setDecorFitsSystemWindows flag. I also tried setting various flags on the views I was able to access in onCreate but I haven't found a reliable way to make the surface view fit inside the system ui.

I also tried updating all the android libraries to the latest version, hoping it'd be a bug in one of them, but it didn't make a difference.

[torokati44]

I think I got around this problem like this: https://github.com/torokati44/ruffle-android/blob/effa0b926797116b08d14117db1ab6fa0dd2d5c9/app/ruffle/src/main/java/rs/ruffle/FullscreenNativeActivity.java#L87-L105
It's hacky as all heck, but perhaps you'll find it useful nevertheless.

[lucasmerlin]

Thanks @torokati44 that is indeed useful!
I managed to get the surfaceview to fit into the system ui by adding android:fitsSystemWindows="true" to the custom ConstraintLayout.

Unfortunately now every touch event seems to be offset by the status bar height, I guess that is what you use the get_loc_on_screen function for?

I'm able to work around this by modifying the winit android implementation by offsetting the pointer position with the contentRect:

                            let location = PhysicalPosition {
                                x: (pointer.x() - self.android_app.content_rect().left as f32) as _,
                                y: (pointer.y() - self.android_app.content_rect().top as f32) as _,
                            };

This does work but feels really wrong, I'm surprised that this isn't handled by the GameActivity code. I'd expect all events to be relative to the surface area, but I guess they don't expect you to modify the surface area size / position and want you to handle any insets in the application?.

[torokati44]

I guess that is what you use the get_loc_on_screen function for?

Yep!

I'm surprised that this isn't handled by the GameActivity code. I'd expect all events to be relative to the surface area

Same.

but I guess they don't expect you to modify the surface area size / position

For me this was one of the main points of switching from native-activity to game-activity.

[ardocrat]

I fixed this just by adding display insets listener inside MainActivity and pass these values into Rust to add paddings to UI accordingly, don't forget that some screens can have cutouts, so it's important to handle them at UI at some cases.

To support cutouts at values/themes.xml you also need to setup:
<item name="android:windowLayoutInDisplayCutoutMode">shortEdges</item>

Below is my code snippet you can call at onCreate of your main Activity, you also need to specify native void onDisplayInsets(int[] cutouts); at same Activity as example, also I am converting pixels to density pixels with (int) (px / context.getResources().getDisplayMetrics().density):

        // Listener for display insets (cutouts) to pass values into native code.
        View content = getWindow().getDecorView().findViewById(android.R.id.content);
        ViewCompat.setOnApplyWindowInsetsListener(content, (v, insets) -> {
            // Setup cutouts values.
            DisplayCutoutCompat dc = insets.getDisplayCutout();
            int cutoutTop = 0;
            int cutoutRight = 0;
            int cutoutBottom = 0;
            int cutoutLeft = 0;
            if (dc != null) {
                cutoutTop = dc.getSafeInsetTop();
                cutoutRight = dc.getSafeInsetRight();
                cutoutBottom = dc.getSafeInsetBottom();
                cutoutLeft = dc.getSafeInsetLeft();
            }

            // Get display insets.
            Insets systemBars = insets.getInsets(WindowInsetsCompat.Type.systemBars());

            // Setup values to pass into native code.
            int[] values = new int[]{0, 0, 0, 0};
            values[0] = Utils.pxToDp(Integer.max(cutoutTop, systemBars.top), this);
            values[1] = Utils.pxToDp(Integer.max(cutoutRight, systemBars.right), this);
            values[2] = Utils.pxToDp(Integer.max(cutoutBottom, systemBars.bottom), this);
            values[3] = Utils.pxToDp(Integer.max(cutoutLeft, systemBars.left), this);

            // Pass values into native code.
            onDisplayInsets(values);

            return insets;
        });

At Rust code you can have something like this at any file, (array contains your insets values):

lazy_static! {
    static ref TOP_DISPLAY_INSET: AtomicI32 = AtomicI32::new(0);
    static ref RIGHT_DISPLAY_INSET: AtomicI32 = AtomicI32::new(0);
    static ref BOTTOM_DISPLAY_INSET: AtomicI32 = AtomicI32::new(0);
    static ref LEFT_DISPLAY_INSET: AtomicI32 = AtomicI32::new(0);
}

#[allow(dead_code)]
#[cfg(target_os = "android")]
#[allow(non_snake_case)]
#[no_mangle]
/// Callback from Java code to update display insets (cutouts).
pub extern "C" fn Java_com_example_MainActivity_onDisplayInsets(
    _env: jni::JNIEnv,
    _class: jni::objects::JObject,
    cutouts: jni::sys::jarray
) {
    use jni::objects::{JObject, JPrimitiveArray};

    let mut array: [i32; 4] = [0; 4];
    unsafe {
        let j_obj = JObject::from_raw(cutouts);
        let j_arr = JPrimitiveArray::from(j_obj);
        _env.get_int_array_region(j_arr, 0, array.as_mut()).unwrap();
    }
    
    TOP_DISPLAY_INSET.store(array[0], Ordering::Relaxed);
    RIGHT_DISPLAY_INSET.store(array[1], Ordering::Relaxed);
    BOTTOM_DISPLAY_INSET.store(array[2], Ordering::Relaxed);
    LEFT_DISPLAY_INSET.store(array[3], Ordering::Relaxed);
}

Works perfectly at egui :)