Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loadAsync not working with FBX [Android] #151

Open
timnlupo opened this issue Dec 3, 2019 · 20 comments
Open

loadAsync not working with FBX [Android] #151

timnlupo opened this issue Dec 3, 2019 · 20 comments

Comments

@timnlupo
Copy link

timnlupo commented Dec 3, 2019

Hello! I'm trying to render a FBX file on Android but am running into an error. It renders an OBJ file fine on Android and the OBJ and FBX fine on iOS. I think the issue is related to #85 and #93 . Same issue as here. Is there any workaround? I've copied the offending error message below. Thanks.

Edit: I traced the issue to line 85 of loadModelsAsync.js, this promise always gets rejected on Android

return new Promise((res, rej) => loader.load(uri, res, onProgress, rej));

Edit 2: I've spent a few hours on this with no luck. GLView does not work with remote debugging which makes this especially difficult. Any suggestions appreciated.

NOTE: This works fine in IOS only having issue with Android.

Object {
  "bubbles": false,
  "cancelable": false,
  "currentTarget": XMLHttpRequest {
    "DONE": 4,
    "HEADERS_RECEIVED": 2,
    "LOADING": 3,
    "OPENED": 1,
    "UNSENT": 0,
    "_aborted": false,
    "_cachedResponse": undefined,
    "_hasError": true,
    "_headers": Object {},
    "_incrementalEvents": true,
    "_lowerCaseResponseHeaders": Object {},
    "_method": "GET",
    "_requestId": null,
    "_response": "",
    "_responseType": "arraybuffer",
    "_sent": true,
    "_subscriptions": Array [],
    "_timedOut": false,
    "_trackingName": "unknown",
    "_url": "file:///data/user/0/host.exp.exponent/cache/ExperienceData/%2540anonymous%252Fthreedemo-3D-expo-2992d51a-3466-48b9-bd1f-6e46e749520c/ExponentAsset-9a80489e046f0d3e164a6f0955a8df98.fbx",
    "readyState": 4,
    "responseHeaders": undefined,
    "status": 0,
    "timeout": 0,
    "upload": XMLHttpRequestEventTarget {
      Symbol(listeners): Object {},
    },
    "withCredentials": true,
    Symbol(listeners): Object {
      "abort": Object {
        "kind": 2,
        "listener": [Function anonymous],
        "next": null,
      },
      "error": Object {
        "kind": 2,
        "listener": [Function anonymous],
        "next": null,
      },
      "load": Object {
        "kind": 2,
        "listener": [Function anonymous],
        "next": null,
      },
      "progress": Object {
        "kind": 2,
        "listener": [Function anonymous],
        "next": null,
      },
    },
  },
  "eventPhase": 2,
  "isTrusted": false,
  "target": XMLHttpRequest {
    "DONE": 4,
    "HEADERS_RECEIVED": 2,
    "LOADING": 3,
    "OPENED": 1,
    "UNSENT": 0,
    "_aborted": false,
    "_cachedResponse": undefined,
    "_hasError": true,
    "_headers": Object {},
    "_incrementalEvents": true,
    "_lowerCaseResponseHeaders": Object {},
    "_method": "GET",
    "_requestId": null,
    "_response": "",
    "_responseType": "arraybuffer",
    "_sent": true,
    "_subscriptions": Array [],
    "_timedOut": false,
    "_trackingName": "unknown",
    "_url": "file:///data/user/0/host.exp.exponent/cache/ExperienceData/%2540anonymous%252Fthreedemo-3D-expo-2992d51a-3466-48b9-bd1f-6e46e749520c/ExponentAsset-9a80489e046f0d3e164a6f0955a8df98.fbx",
    "readyState": 4,
    "responseHeaders": undefined,
    "status": 0,
    "timeout": 0,
    "upload": XMLHttpRequestEventTarget {
      Symbol(listeners): Object {},
    },
    "withCredentials": true,
    Symbol(listeners): Object {
      "abort": Object {
        "kind": 2,
        "listener": [Function anonymous],
        "next": null,
      },
      "error": Object {
        "kind": 2,
        "listener": [Function anonymous],
        "next": null,
      },
      "load": Object {
        "kind": 2,
        "listener": [Function anonymous],
        "next": null,
      },
      "progress": Object {
        "kind": 2,
        "listener": [Function anonymous],
        "next": null,
      },
    },
  },
  "timeStamp": 1575344131854,
  "type": "error",
  Symbol(stop_immediate_propagation_flag): false,
  Symbol(canceled_flag): false,
  Symbol(original_event): Object {
    "type": "error",
  },
}
@timnlupo timnlupo changed the title Android loadAsync not working with FBX loadAsync not working with FBX [Android] Dec 3, 2019
@enzopoeta
Copy link

enzopoeta commented Jan 5, 2020

Hi !
With expo-three 5.3.0 / expo-sdk 36 and react 16.9 I am only able to successfully load obj files. I tried the following extensions without success gltf, glb and dae. I am receiving the following message en try to load them in my android device:
Event {
"isTrusted": false,
}

Thanks
Enzo

@XHMM
Copy link
Contributor

XHMM commented Feb 25, 2020

@enzopoeta I encourtered the same issue.

After checking source code, I find that expo use localUri which use file:// protocal to fetch file, when I changed source code to use uri which use http:// protocal , the error resolved.

Here is the source code diff, hope this can temporary help you. Also wait for expo give us the final solution.

@timnlupo
Copy link
Author

@XHMM This worked for me as well! Should we create a PR @EvanBacon ? Thanks :)

@timnlupo
Copy link
Author

timnlupo commented Mar 2, 2020

The @XHMM fix above does work for local assets, but remote URLs (OBJ or FBX) are failing with

Event {
  "isTrusted": false,
}

Debugging and will update here if I find a fix.

Edit: The following is what the response from resolveAsset looks like for a static asset.

[{"_downloadCallbacks": [], "downloaded": true, "downloading": false, "hash": "624blahblah", "height": null, "localUri": "file:///data/user/0/com.myapp/cache/ExponentAsset-624blahblah.fbx", "name": "Model", "type": "fbx", "uri": "http://10.0.2.2:8081/assets/src/assets/folder/Model.fbx?platform=android&hash=624blahblah", "width": null}]

Notice the uri is http://10.0.2.2:8081/assets/src/assets/folder/Model.fbx?platform=android&hash=624blahblah, hence the fix kind of works because it circumvents the file:// protocol that is throwing this error for unrecognized types (like fbx, gltf, dae, etc. A file pulled from a remote source looks like the following.

[{"_downloadCallbacks": [], "downloaded": true, "downloading": false, "hash": "42ablahblah", "height": null, "localUri": "file:///data/user/0/com.myapp/cache/aws4_request%26X-blahblah", "name": "aws4_request%26X-blahblah", "type": "aws4_request%26X-blahblah", "uri": "file:///data/user/0/com.myapp/cache/aws4_request%26X-blahblah", "width": null}]

So extracting the uri means that it is still using file:// which does not work with any 3D formats other than OBJ. Will continue to play around to see if this can be gamed at all but there is not any literature about this problem online (except for this, which never got resolved).

@timnlupo
Copy link
Author

timnlupo commented Mar 2, 2020

Finally figured it out. The issue comes from FileLoader.load never being trusted by Android. The reason why OBJ is unaffected is because in build/loaders/loadModelsAsync.js the loadObjAsync function has the following.

if (Platform.OS === 'web') {
        return await new Promise((resolve, reject) => loader.load(uri, resolve, () => { }, reject));
}
return loadFileContentsAsync(loader, uri, 'loadObjAsync');

Hence, the load function is never called (but it is in every other problematic file type). To circumvent this we can use the readAsStringAsync function to get base64 data, convert it to an arraybuffer with base64-arraybuffer, and use that arraybuffer in the loaders. I just modified loadArrayBufferAsync because I only care about FBX and GLTF, but assume the DAE loader could be modified similarly. Below is my code.

else if (type.match(/\.fbx$/i)) {
    // const arrayBuffer = await loadArrayBufferAsync({uri: url, onProgress});
    const base64 = await FileSystem.readAsStringAsync(url, {
      encoding: FileSystem.EncodingType.Base64,
    });
    const arrayBuffer = decode(base64);
    const FBXLoader = loaderClassForExtension('fbx');
    const loader = new FBXLoader();
    return loader.parse(arrayBuffer, onAssetRequested);
}

To my testing this works on iOS and Android, remote and local files. It allows you to use file:// so you don't need to make the changes mentioned above. Still would like to hear about an official solution but this workaround works for now.

@giorgioma
Copy link

Same for glTF file loader, I made a fix, maybe it can help
See #165

@ulvido
Copy link

ulvido commented Jul 13, 2020

for android 9 i got this error for loading gltf files. is this related to this topic?

Error:
console.error: { isTrusted" : false }

Screenshot:
Ekran Alıntısı

@Topsyaka077
Copy link

for android 9 i got this error for loading gltf files. is this related to this topic?

Error:
console.error: { isTrusted" : false }

Screenshot:
Ekran Alıntısı

Hey I found another workaround not using expo-three loader and I started to use three.js GTLFLoader

const asset = Asset.fromModule(require('../../../assets/models/toon/scene.gltf')); await asset.downloadAsync(); const loader = new GLTFLoader(); loader.load(asset.uri, (obj) => { scene.add(obj.scene) })

@janweigel
Copy link

Hi @giorgioma
do you have any news on this?
I still can't find a working option to load gltf files into a react native app.
Unfortunately your pull request never got merged.
Is there meanwhile some other way to achieve this?
Any help appreciated!

@haydenlinder
Copy link

@XHMM the diff you provided is not valid anymore. Could you show where you changed the code?

@seand88
Copy link

seand88 commented Dec 2, 2020

stuck on this issue as well, hopefully can patch this somehow...

@krjk333
Copy link

krjk333 commented Dec 7, 2020

I am facing this issue for .obj file too in android. my code is given below. Please help me to solve this

import Expo from "expo";
import React from "react";
import { View, Animated, PanResponder, ActivityIndicator } from "react-native";

import * as THREE from "three";
import ExpoTHREE from "expo-three";
import { GLView } from "expo-gl";
import { Renderer } from "expo-three";
import { OBJLoader } from "three/examples/jsm/loaders/OBJLoader";
import { Asset } from "expo-asset";

export default class App extends React.Component {
  constructor(props) {
    super(props);

    this.state = {
      pan: new Animated.ValueXY(),
    };
  }

  componentWillMount() {
    this._val = { x: 0, y: 0 };
    this.state.pan.addListener((value) => (this._val = value));

    this.panResponder = PanResponder.create({
      onStartShouldSetPanResponder: (e, gesture) => true,
      onPanResponderGrant: (e, gesture) => {
        this.state.pan.setOffset({
          x: this._val.x,
          y: this._val.y,
        });
        this.state.pan.setValue({ x: 0, y: 0 });
      },
      onPanResponderMove: Animated.event([
        null,
        { dx: this.state.pan.x, dy: this.state.pan.y },
      ]),
    });
  }

  render() {
    const panStyle = {
      transform: this.state.pan.getTranslateTransform(),
    };
    return (
      <View
        style={{
          flex: 1,
          backgroundColor: "black",
          justifyContent: "center",
          alignItems: "center",
        }}
      >
        <Animated.View
          {...this.panResponder.panHandlers}
          style={[panStyle, { width: 200, height: 200 }]}
        >
          <GLView
            style={{ flex: 1 }}
            onContextCreate={this._onGLContextCreate}
          />
        </Animated.View>
      </View>
    );
  }

  _onGLContextCreate = async (gl) => {
    const scene = new THREE.Scene();

    const light = new THREE.PointLight(0xff0000, 1, 100);
    light.position.set(50, 50, 50);
    scene.add(light);

    const camera = new THREE.PerspectiveCamera(
      75,
      gl.drawingBufferWidth / gl.drawingBufferHeight,
      0.1,
      1000
    );

    const renderer = new Renderer({ gl });
    renderer.setSize(gl.drawingBufferWidth, gl.drawingBufferHeight);

    const geometry = new THREE.SphereBufferGeometry(1, 36, 36);
    const material = new THREE.MeshBasicMaterial({
      color: 0xafeeee,
    });
    const sphere = new THREE.Mesh(geometry, material);
    const asset = Asset.fromModule(require("./assets/model.obj"));
    await asset.downloadAsync();
    const loader = new OBJLoader();
    loader.load(
      asset.localUri,
      (group) => {
        console.log(group);
        scene.add(group);
      },
      (xhr) => {},
      (error) => {
        scene.add(sphere);
        console.log("error", error);
      }
    );
    //const object
    sphere.castShadow = true;

    camera.position.z = 2;

    const render = () => {
      requestAnimationFrame(render);

      sphere.rotation.x += 0.01;
      sphere.rotation.y += 0.01;

      renderer.render(scene, camera);

      gl.endFrameEXP();
    };
    render();
  };
}

@kyaroru
Copy link

kyaroru commented Aug 24, 2021

After going through so many googling & github issues or even three.js forum (from 0 exp in 3d rendering few weeks back until today), I'm finally able to load these 3 types of models: OBJ, FBX, GLTF - (.glb) in release mode for both android & ios using RN standalone app (barebone RN app with react-native-unimodules installed)

To learn three.js, I created https://github.com/kyaroru/ShibaThree & https://github.com/kyaroru/ReactShibaThree and finally trying out on mobile app using expo-three, but I face so many issues and seems to be unresolvable at first.

Not until today! Argh! So I'm going to share my findings for the newbies out there who are just like me (this might not be an official solution but it just works)

My package.json dependencies:

    "base64-arraybuffer": "^1.0.1", // this is needed at the end! (for FBX and GLTF)
    "expo-gl": "^10.4.2",
    "expo-gl-cpp": "^10.4.1",
    "expo-three": "^5.7.0",
    "react-native-unimodules": "^0.14.6",
    "three": "^0.131.2"

Using expo-three, I tried few different ways to load models:

eg. import OBJLoader or FBXLoader or GLTFLoader directly from three

import { OBJLoader } from 'three/examples/jsm/loaders/OBJLoader';
import { FBXLoader } from 'three/examples/jsm/loaders/FBXLoader';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';

And load using

const loaderFbx = new FBXLoader();
const loaderObj = new OBJLoader();
const loaderGlb = new GLTFLoader();

const model = Asset.fromModule(require('../models/bear/model.obj'));
await model.downloadAsync();
const loader = loaderObj;
loader.load(
      model.uri || '', // .uri / .localUri will not work in release mode on android!
      result => { scene.add(model) },
      onLoad => {},
      onError => {},
);

The above works fine on iOS (even in release mode) but not on android release mode (the APK).
Android will always get this issue #182


Then I tried with loadObjAsync(), it works on Android release mode finally!

import {loadObjAsync} from 'expo-three';

const obj = await loadObjAsync({
  asset: require('../models/bear/model.obj')
});

But .png textures was not loading properly, only .mtl working if I try:

import {loadObjAsync} from 'expo-three';

const obj = await loadObjAsync({
  asset: require('../models/bear/model.obj'),
  assetMtl: require('../models/bear/material.mtl')
});

Then, I tried to look for texture issues & I found this => #185 (comment) (What he explained just strike me in the head.. hmmm)

So I renamed all my textures image file extension to prefix with 'x', and it became : xpng, xjpeg,xjpg - thus it will bundle into raw folder instead of drawable folder when running assembleRelease

Take note also we have to update metro.config.js:

module.exports = {
  ...
  resolver: {
    assetExts: [
      ...,
      'xpng',
      'xjpg',
      'xjpeg',
    ],
  },
};

Then I can finally load OBJ model in android release APK using

import {loadObjAsync, loadTextureAsync} from 'expo-three';
const texture = await loadTextureAsync({
  asset: require('../models/bear/textures.xpng'),
});
const obj = await loadObjAsync({
  asset: require('../models/bear/model.obj')
});
// to map texture to model (in case some newbie like me doesn't know how ><)
obj.traverse(function(object) {
  if (object instanceof THREE.Mesh) {
    object.material.map = texture;
  }
});
scene.add(obj);

OBJ is down, how about FBX & GLTF? To know how things work, I look into loadObjAsync method inside node_modules\expo-three\build\loaders\loadModelsAsync.js

I found out it was actually calling loader.parse() method instead of loader.load()

So I attempted to do similar approach for FBX and GLTF models after referring to #151 (comment)


First, install base64-arraybuffer using npm/yarn

yarn add base64-arraybuffer

Then. In wherever file that you wanted to load fbx or gltf model, add the following codes: (of course we can also fork expo-three and add these new methods - so the newbie don't get stucked! )

import { loadObjAsync, loadTextureAsync } from 'expo-three';
import { resolveAsync } from 'expo-asset-utils';
import { FBXLoader } from 'three/examples/jsm/loaders/FBXLoader';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';
import { FileSystem } from 'react-native-unimodules';
import { decode } from 'base64-arraybuffer';

// this was actually copied from node_modules\expo-three\build\loaders\loadModelsAsync.js
async function loadFileAsync({ asset, funcName }) {
  if (!asset) {
    throw new Error(`ExpoTHREE.${funcName}: Cannot parse a null asset`);
  }
  return (await resolveAsync(asset)).localUri ?? null;
}

// newly added method 
export async function loadFbxAsync({ asset, onAssetRequested }) {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadFbxAsync',
  });
  if (!uri) return;
  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });
  const arrayBuffer = decode(base64);
  const loader = new FBXLoader();
  return loader.parse(arrayBuffer, onAssetRequested);
}

// newly added method
export async function loadGLTFAsync({ asset, onAssetRequested }) {
  const uri = await loadFileAsync({
    asset,
    funcName: 'loadGLTFAsync',
  });
  if (!uri) return;
  const base64 = await FileSystem.readAsStringAsync(uri, {
    encoding: FileSystem.EncodingType.Base64,
  });
  const arrayBuffer = decode(base64);
  const loader = new GLTFLoader();
  return new Promise((resolve, reject) => {
    loader.parse(
      arrayBuffer,
      onAssetRequested,
      result => {
        resolve(result);
      },
      err => {
        reject(err);
      },
    );
  });
}

And that's all! We can now load OBJ, FBX, GLTF models proudly inside Barebone RN app!

  • For GLTF model, only .glb is working for now because using .gltf file will require loading .bin file as well but I haven't figure out how to load .bin file yet 😢)
  • Stay tune!

For more info, can refer to my gist:

or clone this example app that I created:

Programmer life is tough.. 😉

@punksta
Copy link

punksta commented Oct 22, 2021

@kyaroru thank you for the example project!

@sadevn
Copy link

sadevn commented Mar 28, 2022

@kyaroru Thank you for your example.
I checked your repo https://github.com/kyaroru/RNExpoThree
Btw, could you please update your example using the latest expo sdk?

@Neosoulink
Copy link

Neosoulink commented Feb 19, 2023

Hey @kyaroru, I also didn't know how to load .bin files attached to GTLF files.

To solve it (hack it), I converted my .bin file to base 64 and passed it to the URI of the buffer property in the GLTF file, and I ended up with something like this:

{
  "asset": { "generator": "", "version": "2.0"},
  "scene": 0,
  // other props...
  "buffers": [
    {
      "uri": "data:application/octet-stream;base64,yC9gQ2PFtkKT2BPC..."
    },
    {
      "uri": "data:application/octet-stream;base64,yC9gQ2PFtkKT2BPC..."
    }
    // more uri...
  ]
 }

Thank you very much for your solution, it was very helpful 🚀

@ephron-systems
Copy link

How to consume this functions ?

@marsinearth
Copy link

marsinearth commented Jun 18, 2024

  • For GLTF model, only .glb is working for now because using .gltf file will require loading .bin file as well but I haven't figure out how to load .bin file yet 😢)
  • Stay tune!

For more info, can refer to my gist:

or clone this example app that I created:

Programmer life is tough.. 😉

This is not working anymore

@voxoid0
Copy link

voxoid0 commented Oct 16, 2024

Hey @kyaroru, I also didn't know how to load .bin files attached to GTLF files.

To solve it (hack it), I converted my .bin file to base 64 and passed it to the URI of the buffer property in the GLTF file, and I ended up with something like this:

{
  "asset": { "generator": "", "version": "2.0"},
  "scene": 0,
  // other props...
  "buffers": [
    {
      "uri": "data:application/octet-stream;base64,yC9gQ2PFtkKT2BPC..."
    },
    {
      "uri": "data:application/octet-stream;base64,yC9gQ2PFtkKT2BPC..."
    }
    // more uri...
  ]
 }

Thank you very much for your solution, it was very helpful 🚀

I like this idea so I tried it, but with ~25MB in textures for my moddel I seem to have run into a GLTFLoader bug where it can't load such a large uri:
GLTFLoader.js: RangeError: new TypedArray(buffer, [byteOffset], [length]): byteOffset + length * elementSize must be less than buffer.byteLength

@kyaroru
Copy link

kyaroru commented Oct 17, 2024

@kyaroru Thank you for your example. I checked your repo https://github.com/kyaroru/RNExpoThree Btw, could you please update your example using the latest expo sdk?

Hi @sadevn & anyone who need this, I have updated the example using Expo Latest SDK (v51) at https://github.com/kyaroru/RNExpoThree & renamed the old one to https://github.com/kyaroru/RNExpoThreeOld

Hope it helps those who are struggling a lot!

Btw, it's been 3 years! Life has changed but passion never dies!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests