Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

React Native - glb/gltf model works in emulator/usb connected device but fails in android apk #2992

Closed
uanama opened this issue Sep 4, 2023 · 30 comments · Fixed by #3042
Closed
Labels
bug Something isn't working react-native to do with react-native

Comments

@uanama
Copy link

uanama commented Sep 4, 2023

So my problem is when I try to run the app on my device using the apk without being connected via USB debugging. I am using android. It works fine on emulators but it doesn't work on apk.

This is the code:

Cattura

I tried this too, but it never runs updateText("asset creato");.

Its like it gets stuck in:
const model = await loader.loadAsync(require('./assets/TempioHera/tempioHeraGLB.glb'));

This also happens with GLTFLoader by three/examples/jsm/loaders/GLTFLoader:

Cattura

this is the uri i get:

WhatsApp Image 2023-09-04 at 16 14 24

@CodyJasonBennett
Copy link
Member

CodyJasonBennett commented Sep 4, 2023

Have you seen useLoader and/or are using the latest version 8.14? expo-asset won't work in release mode, we polyfill behavior for this specifically.

import { useLoader } from '@react-three/fiber'
import { GLTFLoader } from 'three-stdlib'

function Model() {
  const gltf = useLoader(GLTFLoader, require('./path/to/asset.glb'))
  return <primitive object={gltf.scene} />
}

@CodyJasonBennett CodyJasonBennett added needs reproduction needs more info needs more information before we can action and removed needs reproduction labels Sep 4, 2023
@uanama
Copy link
Author

uanama commented Sep 5, 2023

i tried to do this:

Cattura

this is my package.json, I use @react-three/fiber 8.14:

Cattura

but i get these errors, both with @react-three/fiber/native and @react-three/fiber

Cattura

Cattura

Cattura

Cattura

errors from the emulator view:

Nuovo progetto (2)

Nuovo progetto (3)

@lexengineer
Copy link

lexengineer commented Sep 5, 2023

Have you seen useLoader and/or are using the latest version 8.14? expo-asset won't work in release mode, we polyfill behavior for this specifically.

import { useLoader } from '@react-three/fiber'
import { GLTFLoader } from 'three-stdlib'

function Model() {
  const gltf = useLoader(GLTFLoader, require('./path/to/asset.glb'))
  return <primitive object={gltf.scene} />
}

I have similar issue on iOS in the release mode. Could you explain please why expo-asset does not work in the release mode? Or maybe share some resources where we can read about that? Just want to dive into details to make sure I understand.

@CodyJasonBennett
Copy link
Member

To clarify, I meant Android APK where uris are shortened for drawables. It remains unknown whether that includes GLB or other assets additionally configured in Metro. I detailed relevant sources in #2980 (comment), with fixes rolled up into #2982 of 8.14.

@CodyJasonBennett CodyJasonBennett added bug Something isn't working and removed needs more info needs more information before we can action labels Sep 5, 2023
@CodyJasonBennett
Copy link
Member

I've confirmed that only image assets are considered drawables, and the polyfills for asset resolution correctly resolve in all cases between dev/APK. This is a new behavior coming from the networking stack and react-native/Expo, so it will be challenging to pinpoint.

@XantreDev
Copy link

Workaround for loading glb

import assert from 'assert';
import { decode } from 'base64-arraybuffer';
import { resolveAsync } from 'expo-asset-utils';
import * as FileSystem from 'expo-file-system';
import { suspend } from 'suspend-react';
import THREE from 'three';
import { GLTF, GLTFLoader } from 'three-stdlib';

async function loadFileAsync({
  asset,
  funcName,
}: {
  asset: unknown;
  funcName: string;
}) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

type ObjectGraph = {
  nodes: Record<string, THREE.Mesh>;
  materials: Record<string, THREE.Material>;
};

// Collects nodes and materials from a THREE.Object3D
export function buildGraph(object: THREE.Object3D) {
  const data: ObjectGraph = { nodes: {}, materials: {} };
  if (object) {
    object.traverse((obj: any) => {
      if (obj.name) {
        data.nodes[obj.name] = obj;
      }
      if (obj.material && !data.materials[obj.material.name]) {
        data.materials[obj.material.name] = obj.material;
      }
    });
  }
  return data;
}
async function loadGLTFAsync({
  asset,
}: {
  asset: unknown;
}): Promise<GLTF & ObjectGraph> {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });

  assert(uri, 'loadGLTFAsync uri should exist');

  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });

  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();

  const res = await loader.parseAsync(arrayBuffer, 'beb');

  if (res.scene) {
    Object.assign(res, buildGraph(res.scene));
  }

  return res as GLTF & ObjectGraph;
}

export const useGLTFCustom = (asset: unknown) =>
  suspend(async () => loadGLTFAsync({ asset }), ['useGLTFCustom', asset]);

@XantreDev
Copy link

In my case i have this error when trying to use useGLTF. I rebuilded native part but it's not the root

 ERROR  The above error occurred in the <ForwardRef> component:

    at anonymous (http://localhost:8081/index.bundle//&platform=android&dev=true&minify=false&app=dev.world.oone.driverapp&modulesOnly=false&runModule=true:469843:24)
    at Suspense
    at Suspense
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)
    at proxy trap (native)

React will try to recreate this component tree from scratch using the error boundary you provided, ErrorBoundary.
 LOG  createFallbackRerender [Error: Could not load 279: undefined]
 ERROR  Error: Could not load 279: undefined

This error is located at:
    in Unknown
    in FiberProvider
    in CanvasWrapper (created by TransformedSpeedometer)
    in RCTView (created by View)
    in View
    in NativeWind.View
    in Unknown (created by TransformedSpeedometer)
    in RCTView (created by View)
    in View
    in NativeWind.View
    in Unknown (created by TransformedSpeedometer)
    in RCTView (created by View)
    in View (created by AnimatedComponent(View))
    in AnimatedComponent(View)
    in Unknown (created by Moti.View)
    in Moti

@Rakha112
Copy link

Workaround for loading glb

import assert from 'assert';
import { decode } from 'base64-arraybuffer';
import { resolveAsync } from 'expo-asset-utils';
import * as FileSystem from 'expo-file-system';
import { suspend } from 'suspend-react';
import THREE from 'three';
import { GLTF, GLTFLoader } from 'three-stdlib';

async function loadFileAsync({
  asset,
  funcName,
}: {
  asset: unknown;
  funcName: string;
}) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

type ObjectGraph = {
  nodes: Record<string, THREE.Mesh>;
  materials: Record<string, THREE.Material>;
};

// Collects nodes and materials from a THREE.Object3D
export function buildGraph(object: THREE.Object3D) {
  const data: ObjectGraph = { nodes: {}, materials: {} };
  if (object) {
    object.traverse((obj: any) => {
      if (obj.name) {
        data.nodes[obj.name] = obj;
      }
      if (obj.material && !data.materials[obj.material.name]) {
        data.materials[obj.material.name] = obj.material;
      }
    });
  }
  return data;
}
async function loadGLTFAsync({
  asset,
}: {
  asset: unknown;
}): Promise<GLTF & ObjectGraph> {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });

  assert(uri, 'loadGLTFAsync uri should exist');

  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });

  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();

  const res = await loader.parseAsync(arrayBuffer, 'beb');

  if (res.scene) {
    Object.assign(res, buildGraph(res.scene));
  }

  return res as GLTF & ObjectGraph;
}

export const useGLTFCustom = (asset: unknown) =>
  suspend(async () => loadGLTFAsync({ asset }), ['useGLTFCustom', asset]);

hey thanks, your workaround works very well to load glb on both dev and APK, but to load the Duck.glb example I get this error
THREE.GLTFLoader: Couldn't load texture {"_h": 1, "_i": 2, "_j": [Error: Cannot create URL for blob!], "_k": null}
Test2

@CodyJasonBennett CodyJasonBennett added the react-native to do with react-native label Sep 12, 2023
@XantreDev
Copy link

Workaround for loading glb

import assert from 'assert';
import { decode } from 'base64-arraybuffer';
import { resolveAsync } from 'expo-asset-utils';
import * as FileSystem from 'expo-file-system';
import { suspend } from 'suspend-react';
import THREE from 'three';
import { GLTF, GLTFLoader } from 'three-stdlib';

async function loadFileAsync({
  asset,
  funcName,
}: {
  asset: unknown;
  funcName: string;
}) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

type ObjectGraph = {
  nodes: Record<string, THREE.Mesh>;
  materials: Record<string, THREE.Material>;
};

// Collects nodes and materials from a THREE.Object3D
export function buildGraph(object: THREE.Object3D) {
  const data: ObjectGraph = { nodes: {}, materials: {} };
  if (object) {
    object.traverse((obj: any) => {
      if (obj.name) {
        data.nodes[obj.name] = obj;
      }
      if (obj.material && !data.materials[obj.material.name]) {
        data.materials[obj.material.name] = obj.material;
      }
    });
  }
  return data;
}
async function loadGLTFAsync({
  asset,
}: {
  asset: unknown;
}): Promise<GLTF & ObjectGraph> {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });

  assert(uri, 'loadGLTFAsync uri should exist');

  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });

  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();

  const res = await loader.parseAsync(arrayBuffer, 'beb');

  if (res.scene) {
    Object.assign(res, buildGraph(res.scene));
  }

  return res as GLTF & ObjectGraph;
}

export const useGLTFCustom = (asset: unknown) =>
  suspend(async () => loadGLTFAsync({ asset }), ['useGLTFCustom', asset]);

hey thanks, your workaround works very well to load glb on both dev and APK, but to load the Duck.glb example I get this error THREE.GLTFLoader: Couldn't load texture {"_h": 1, "_i": 2, "_j": [Error: Cannot create URL for blob!], "_k": null} Test2

In this case embedded textures cannot load, so you should load it separately

  const { nodes, materials } = useGLTFCustrom(planetGltf) 

  const texture = useTexture(planetTexture as unknown as string, (tex) => {
    if (Array.isArray(tex)) {
      throw new Error('Array of textures is not supported');
    }
    tex.flipY = false;
    tex.unpackAlignment = 4;
  });
  const material003 = materials.Planet_Texture as MeshStandardMaterial;
  const earthMaterial = useMemo(() => {
    const material = material003.clone();

    material.map = texture;
    material.emissiveMap = texture;

    return material;
  }, [texture, material003]);
  return (
    <group dispose={null}>
      <PerspectiveCamera
        makeDefault
        far={1000}
        near={0.1}
        fov={60.931}
        position={[0, 0, 13.847]}
      />
      <mesh
        castShadow
        receiveShadow
        geometry={nodes.Planet.geometry}
        material={earthMaterial}
        // material={materials.Planet_Texture}
      />

@Rakha112
Copy link

In this case embedded textures cannot load, so you should load it separately

  const { nodes, materials } = useGLTFCustrom(planetGltf) 

  const texture = useTexture(planetTexture as unknown as string, (tex) => {
    if (Array.isArray(tex)) {
      throw new Error('Array of textures is not supported');
    }
    tex.flipY = false;
    tex.unpackAlignment = 4;
  });
  const material003 = materials.Planet_Texture as MeshStandardMaterial;
  const earthMaterial = useMemo(() => {
    const material = material003.clone();

    material.map = texture;
    material.emissiveMap = texture;

    return material;
  }, [texture, material003]);
  return (
    <group dispose={null}>
      <PerspectiveCamera
        makeDefault
        far={1000}
        near={0.1}
        fov={60.931}
        position={[0, 0, 13.847]}
      />
      <mesh
        castShadow
        receiveShadow
        geometry={nodes.Planet.geometry}
        material={earthMaterial}
        // material={materials.Planet_Texture}
      />

oh yes, thank you very much, the texture has been successfully loaded
Screenshot 2023-09-12 at 19 30 57

@CodyJasonBennett
Copy link
Member

CodyJasonBennett commented Sep 13, 2023

I'm not sure how "Cannot create URL for blob!" is reachable, looking at https://github.com/facebook/react-native/blob/main/packages/react-native/Libraries/Blob/URL.js#L130. I've just been looking into "Could not load 1: undefined" which is a regression with THREE.FileLoader -- 1 refers to a Metro module reference.

@kdmmanapul
Copy link

In this case embedded textures cannot load, so you should load it separately

  const { nodes, materials } = useGLTFCustrom(planetGltf) 

  const texture = useTexture(planetTexture as unknown as string, (tex) => {
    if (Array.isArray(tex)) {
      throw new Error('Array of textures is not supported');
    }
    tex.flipY = false;
    tex.unpackAlignment = 4;
  });
  const material003 = materials.Planet_Texture as MeshStandardMaterial;
  const earthMaterial = useMemo(() => {
    const material = material003.clone();

    material.map = texture;
    material.emissiveMap = texture;

    return material;
  }, [texture, material003]);
  return (
    <group dispose={null}>
      <PerspectiveCamera
        makeDefault
        far={1000}
        near={0.1}
        fov={60.931}
        position={[0, 0, 13.847]}
      />
      <mesh
        castShadow
        receiveShadow
        geometry={nodes.Planet.geometry}
        material={earthMaterial}
        // material={materials.Planet_Texture}
      />

oh yes, thank you very much, the texture has been successfully loaded Screenshot 2023-09-12 at 19 30 57

Hi Rakha112, I was wondering how did you do it in react-native? care to share your code?

@XantreDev
Copy link

He literally shares it))

@kdmmanapul
Copy link

He literally shares it))

Where? Sorry I can't seem to find it. because the one in his comments the code their was a quote reply, and not actually his code with the duck.glb

@XantreDev
Copy link

XantreDev commented Sep 22, 2023

Transform duck.glb with
https://gltf.pmnd.rs/
you should replace useGLTF with useGLTFCustom from snippets higher.
And after it you grab textures images from glb. And use it with useTexture, after it you should replace texture in material

@Rakha112
Copy link

hey @kdmmanapul you can use the code given by @XantreGodlike, and as he said you have to load the textures separately by bake the textures into a .png file. You can clone my repo to try it react-native-3D-Example

@kdmmanapul
Copy link

Thanks guys @XantreGodlike @Rakha112

@kdmmanapul
Copy link

Hmm @Rakha112 @XantreGodlike texture does not seem to work when exported into an APK and run on mobile.

@XantreDev
Copy link

Yep, we are downgraded version to earlier one and patching three fiber(

@XantreDev
Copy link

@CodyJasonBennett

@CodyJasonBennett
Copy link
Member

@kdmmanapul, is this latest R3F? @XantreGodlike, which version are you patching from? I can take a look.

@Rakha112
Copy link

Hey @kdmmanapul. I tried it on 3 devices, Redmi Note 10, Samsung Galaxy Tab A8 and Samsung Galaxy Tab S8, all of them can load the Duck and its textures well using @XantreGodlike workaround. I haven't tried using the latest version
Duck

Here is the APK file maybe you can try

and this is the version of dependencies that I use

  "dependencies": {
    "@react-three/drei": "^9.82.1",
    "@react-three/fiber": "^8.14.1",
    "assert": "^2.1.0",
    "base64-arraybuffer": "^1.0.2",
    "expo": "^49.0.0",
    "expo-asset-utils": "^3.0.0",
    "expo-file-system": "^15.4.4",
    "expo-gl": "~13.0.1",
    "r3f-native-orbitcontrols": "^1.0.8",
    "react": "18.2.0",
    "react-native": "0.72.4",
    "three": "^0.156.1",
    "three-stdlib": "^2.25.1"
  },

@CodyJasonBennett
Copy link
Member

Note that the ^ symbol will install the latest version at the time of install (including minors). If you have a lockfile, that will narrow down which version was resolved. Also when installing a specific version, be sure to pin it by only specifying the exact version (e.g. "8.14.1").

@XantreDev
Copy link

Sorry, I think, i should to recheck with latest version, because seems to be i've used old one

@kdmmanapul
Copy link

Workaround for loading glb

import assert from 'assert';
import { decode } from 'base64-arraybuffer';
import { resolveAsync } from 'expo-asset-utils';
import * as FileSystem from 'expo-file-system';
import { suspend } from 'suspend-react';
import THREE from 'three';
import { GLTF, GLTFLoader } from 'three-stdlib';

async function loadFileAsync({
  asset,
  funcName,
}: {
  asset: unknown;
  funcName: string;
}) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

type ObjectGraph = {
  nodes: Record<string, THREE.Mesh>;
  materials: Record<string, THREE.Material>;
};

// Collects nodes and materials from a THREE.Object3D
export function buildGraph(object: THREE.Object3D) {
  const data: ObjectGraph = { nodes: {}, materials: {} };
  if (object) {
    object.traverse((obj: any) => {
      if (obj.name) {
        data.nodes[obj.name] = obj;
      }
      if (obj.material && !data.materials[obj.material.name]) {
        data.materials[obj.material.name] = obj.material;
      }
    });
  }
  return data;
}
async function loadGLTFAsync({
  asset,
}: {
  asset: unknown;
}): Promise<GLTF & ObjectGraph> {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });

  assert(uri, 'loadGLTFAsync uri should exist');

  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });

  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();

  const res = await loader.parseAsync(arrayBuffer, 'beb');

  if (res.scene) {
    Object.assign(res, buildGraph(res.scene));
  }

  return res as GLTF & ObjectGraph;
}

export const useGLTFCustom = (asset: unknown) =>
  suspend(async () => loadGLTFAsync({ asset }), ['useGLTFCustom', asset]);

hey thanks, your workaround works very well to load glb on both dev and APK, but to load the Duck.glb example I get this error THREE.GLTFLoader: Couldn't load texture {"_h": 1, "_i": 2, "_j": [Error: Cannot create URL for blob!], "_k": null} Test2

In this case embedded textures cannot load, so you should load it separately

  const { nodes, materials } = useGLTFCustrom(planetGltf) 

  const texture = useTexture(planetTexture as unknown as string, (tex) => {
    if (Array.isArray(tex)) {
      throw new Error('Array of textures is not supported');
    }
    tex.flipY = false;
    tex.unpackAlignment = 4;
  });
  const material003 = materials.Planet_Texture as MeshStandardMaterial;
  const earthMaterial = useMemo(() => {
    const material = material003.clone();

    material.map = texture;
    material.emissiveMap = texture;

    return material;
  }, [texture, material003]);
  return (
    <group dispose={null}>
      <PerspectiveCamera
        makeDefault
        far={1000}
        near={0.1}
        fov={60.931}
        position={[0, 0, 13.847]}
      />
      <mesh
        castShadow
        receiveShadow
        geometry={nodes.Planet.geometry}
        material={earthMaterial}
        // material={materials.Planet_Texture}
      />

Regarding this one @XantreGodlike @Rakha112 since we added a condition for Array not being supported, how can we add rougness texture on the model?

Having normalMap and RoughnessMap and etc? Since useTexture by default can have an array of maps right?

@CodyJasonBennett
Copy link
Member

CodyJasonBennett commented Sep 27, 2023

You can omit that callback. I added unpackAlignment since OpenGL/WebGL defaults have fail cases for data textures -- 4 is the default, 1 is safe for bad data. flipY has no effect but may log a warning if used on older versions of expo-gl.

@uanama
Copy link
Author

uanama commented Oct 4, 2023

When I try to use the three-stdlib library in a component I get this error:

catturaGit

This is my package.json:

catturaGit1

@CodyJasonBennett
Copy link
Member

CodyJasonBennett commented Oct 4, 2023

I'd update three-stdlib or anything before reporting to GitHub. I'll look into it regardless on my end.

@uanama
Copy link
Author

uanama commented Oct 12, 2023

I am trying to load a glb that has been compressed either with meshopt or draco, is it possible to load these kind of glb using r3f on both android or iOS?
here is how i tried to set the decoders to gltfLoader:
(the gltfLoader already loads correctly uncompressed glb models)
WhatsApp Image 2023-10-12 at 09 47 51

but i get these warning messages with this approach:

this is what i get when I try loading a GLB compressed using DRACO:
WhatsApp Image 2023-10-12 at 09 41 50

this is what i get when I try loading a GLB compressed using Meshopt:
eb636570-5d4a-45a2-a923-b498ae65a8e2

@CodyJasonBennett
Copy link
Member

CodyJasonBennett commented Oct 17, 2023

I am trying to load a glb that has been compressed either with meshopt or draco, is it possible to load these kind of glb using r3f on both android or iOS?

No, it's not possible to use meshopt or DRACO without JIT support on iOS to implement WebAssembly. Maybe wait until EU 2024 legislature WRT the browser ban which might incidentally help there.


I've merged #3042 for 8.14.6 which reverts changes from #2982 or 8.14 that produces these cryptic promise rejections. I think we'll have to export a native-specific useLoader which uses the above workarounds instead of relying on correct networking behavior upstream.

This undoes changes needed to load textures from a GLB, but should work in Android release mode and anywhere to begin with. Let me know if anything changes on your end and I'll look into a proper fix for textures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working react-native to do with react-native
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants