Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AR autogenerate USDZ #2374

Merged
merged 9 commits into from
May 11, 2021
Merged

AR autogenerate USDZ #2374

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
7d90750
automatic ios-src generation
elalish May 10, 2021
78a9347
added docs
elalish May 11, 2021
b3142dd
AR autogenerate USDZ
kolodi May 11, 2021
d58c6f7
separate field to store generated USDZ url
kolodi May 11, 2021
785e876
remove caching for now
kolodi May 11, 2021
899f90d
Removing property. Clean up previous blob url.
kolodi May 11, 2021
42d77de
genrateUsdz boolean bug fix
kolodi May 11, 2021
321fc17
Merge remote-tracking branch 'google/usdz' into USDZexport
kolodi May 11, 2021
a506b6d
Merge branch 'master' into USDZexport
elalish May 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 68 additions & 7 deletions packages/model-viewer/src/features/ar.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@

import {property} from 'lit-element';
import {Event as ThreeEvent} from 'three';
import {USDZExporter} from 'three/examples/jsm/exporters/USDZExporter';

import {IS_AR_QUICKLOOK_CANDIDATE, IS_SCENEVIEWER_CANDIDATE, IS_WEBXR_AR_CANDIDATE} from '../constants.js';
import ModelViewerElementBase, {$needsRender, $renderer, $scene, $shouldAttemptPreload, $updateSource} from '../model-viewer-base.js';
import ModelViewerElementBase, {$needsRender, $progressTracker, $renderer, $scene, $shouldAttemptPreload, $updateSource} from '../model-viewer-base.js';
import {enumerationDeserializer} from '../styles/deserializers.js';
import {ARStatus, ARTracking} from '../three-components/ARRenderer.js';
import {Constructor, waitForEvent} from '../utilities.js';
Expand All @@ -31,7 +32,7 @@ export type ARMode = 'quick-look'|'scene-viewer'|'webxr'|'none';
const deserializeARModes = enumerationDeserializer<ARMode>(
['quick-look', 'scene-viewer', 'webxr', 'none']);

const DEFAULT_AR_MODES = 'webxr scene-viewer quick-look';
const DEFAULT_AR_MODES = 'webxr scene-viewer';

const ARMode: {[index: string]: ARMode} = {
QUICK_LOOK: 'quick-look',
Expand All @@ -57,6 +58,7 @@ const $arMode = Symbol('arMode');
const $arModes = Symbol('arModes');
const $arAnchor = Symbol('arAnchor');
const $preload = Symbol('preload');
const $generatedIosUrl = Symbol('generatedIosUrl');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is vestigial now.


const $onARButtonContainerClick = Symbol('onARButtonContainerClick');
const $onARStatus = Symbol('onARStatus');
Expand Down Expand Up @@ -105,6 +107,8 @@ export const ARMixin = <T extends Constructor<ModelViewerElementBase>>(
protected[$arMode]: ARMode = ARMode.NONE;
protected[$preload] = false;

private[$generatedIosUrl]: string|null = null;

private[$onARButtonContainerClick] = (event: Event) => {
event.preventDefault();
this.activateAR();
Expand Down Expand Up @@ -226,13 +230,17 @@ configuration or device capabilities');
!isSceneViewerBlocked) {
this[$arMode] = ARMode.SCENE_VIEWER;
break;
} else if (
value === 'quick-look' && !!this.iosSrc &&
IS_AR_QUICKLOOK_CANDIDATE) {
} else if (value === 'quick-look' && IS_AR_QUICKLOOK_CANDIDATE) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an extra logic block we'll need below this from my PR. You can either pull it in, or I can add it in mine with the docs update.

this[$arMode] = ARMode.QUICK_LOOK;
break;
}
}

// The presence of ios-src overrides the absence of quick-look ar-mode.
if (!this.canActivateAR && this.iosSrc != null &&
IS_AR_QUICKLOOK_CANDIDATE) {
this[$arMode] = ARMode.QUICK_LOOK;
}
}

if (this.canActivateAR) {
Expand Down Expand Up @@ -352,23 +360,76 @@ configuration or device capabilities');
* Takes a URL to a USDZ file and sets the appropriate fields so that Safari
* iOS can intent to their AR Quick Look.
*/
[$openIOSARQuickLook]() {
const modelUrl = new URL(this.iosSrc!, self.location.toString());
async[$openIOSARQuickLook]() {
const generateUsdz = !this.iosSrc;

this[$arButtonContainer].classList.remove('enabled');

const modelUrl = new URL(
generateUsdz ? await this.prepareUSDZ() : this.iosSrc!,
self.location.toString());

this[$arButtonContainer].classList.add('enabled');

if (this.arScale === 'fixed') {
if (modelUrl.hash) {
modelUrl.hash += '&';
}
modelUrl.hash += 'allowsContentScaling=0';
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read somewhere that Safari doesn't like hashes on blob URLs. Have you tried arScale="fixed" to see if it actually works in QuickLook? I'm not very concerned if it doesn't, so long as it doesn't crash, but I'd be pretty stoked if I'm wrong.


const anchor = this[$arAnchor];
anchor.setAttribute('rel', 'ar');
const img = document.createElement('img');
anchor.appendChild(img);
anchor.setAttribute('href', modelUrl.toString());
if (generateUsdz) {
anchor.setAttribute('download', 'model.usdz');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this do? Is it necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I prepare the Blob, I use simply "application/octet-stream"for the type. So Safari need a "hint" to treat the link as a valid source to open Quick Look, that's why there is a download attribute. As a side effect you can also test the export in desktop browser as it will download the usdz file for you.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, interesting. I think it worked for me because I used a more specific mime type; you can see in my PR. The button still doesn't appear on desktop Safari though, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you have to manually override IS_AR_QUICKLOOK_CANDIDATE constant :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this tricks Safari to open the Quick Look. the file name has been chosen the same as it is in USDZExporter.js, although it could be anything.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my testing it wasn't necessary on my iPhone SE. But it also probably doesn't hurt anything.

}
console.log('Attempting to present in AR with Quick Look...');
anchor.click();
anchor.removeChild(img);
}

async prepareUSDZ(): Promise<string> {
const updateSourceProgress = this[$progressTracker].beginActivity();

if (this[$generatedIosUrl] != null) {
URL.revokeObjectURL(this[$generatedIosUrl]!);
this[$generatedIosUrl] = null;
}

const scene = this[$scene];

const shadow = scene.shadow;
let visible = false;

// Remove shadow from export
if (shadow != null) {
visible = shadow.visible;
shadow.visible = false;
kolodi marked this conversation as resolved.
Show resolved Hide resolved
}

updateSourceProgress(0.2);
kolodi marked this conversation as resolved.
Show resolved Hide resolved

const exporter = new USDZExporter();
const arraybuffer = await exporter.parse(scene.modelContainer);
const blob = new Blob([arraybuffer], {
type: 'application/octet-stream',
});

const url = URL.createObjectURL(blob);
this[$generatedIosUrl] = url;

updateSourceProgress(1);

if (shadow != null) {
shadow.visible = visible;
}

return url;
}
}

return ARModelViewerElement;
Expand Down
8 changes: 4 additions & 4 deletions packages/modelviewer.dev/data/docs.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,13 +243,13 @@
{
"name": "ar-modes",
"htmlName": "arModes",
"description": "A prioritized list of the types of AR experiences to enable. Allowed values are \"webxr\", to launch the AR experience in the browser, \"scene-viewer\", to launch the <a href=\"https://developers.google.com/ar/develop/java/scene-viewer\">Scene Viewer</a> app, \"quick-look\", to launch the iOS Quick Look app. You can specify any number of modes separated by whitespace.",
"description": "A prioritized list of the types of AR experiences to enable. Allowed values are \"webxr\", to launch the AR experience in the browser, \"scene-viewer\", to launch the <a href=\"https://developers.google.com/ar/develop/java/scene-viewer\">Scene Viewer</a> app, \"quick-look\", to launch the iOS Quick Look app. You can specify any number of modes separated by whitespace. Note that the presence of an ios-src will enable quick-look by itself; specifying quick-look here allows us to generate a USDZ on the fly rather than downloading a separate ios-src file.",
"links": [
"<a href=\"../examples/augmentedreality/\">Related examples</a>"
],
"default": {
"default": "webxr scene-viewer quick-look",
"options": "prioritized list of allowed modes"
"default": "webxr scene-viewer",
"options": "prioritized list possible AR modes: webxr, scene-viewer, and quick-look"
}
},
{
Expand Down Expand Up @@ -279,7 +279,7 @@
{
"name": "ios-src",
"htmlName": "iosSrc",
"description": "The url to a <a href=\"https://graphics.pixar.com/usd/docs/Usdz-File-Format-Specification.html\">USDZ</a> model which will be used on <a href=\"https://www.apple.com/ios/augmented-reality/\">supported iOS 12+ devices</a> via <a href=\"https://developer.apple.com/videos/play/wwdc2018/603/\">AR Quick Look</a> on Safari.",
"description": "The url to a <a href=\"https://graphics.pixar.com/usd/docs/Usdz-File-Format-Specification.html\">USDZ</a> model which will be used on <a href=\"https://www.apple.com/ios/augmented-reality/\">supported iOS 12+ devices</a> via <a href=\"https://developer.apple.com/videos/play/wwdc2018/603/\">AR Quick Look</a> on Safari. The presence of this attribute will automatically enable the quick-look ar-mode, however it is no longer necessary. If instead the quick-look ar-mode is specified and ios-src is not specified, then we will generate a USDZ on the fly when the AR button is pressed. This means modifications via the scene-graph API will now be reflected in Quick Look. Hoowever, USDZ generation is not perfect, for instance animations are not yet supported, so in some cases supplying ios-src may give better results.",
"links": [
"<a href=\"../examples/augmentedreality/#ar\"><span class='attribute'>ios-src</span> example</a>"
]
Expand Down
6 changes: 3 additions & 3 deletions packages/modelviewer.dev/examples/augmentedreality/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ <h4>Customize a WebXR Augmented Reality session with HTML, CSS, and JS in Chrome
</div>
<example-snippet stamp-to="webXR" highlight-as="html">
<template>
<model-viewer src="../../assets/ShopifyModels/Chair.glb" poster="../../assets/ShopifyModels/Chair.png" shadow-intensity="1" ar camera-controls alt="A 3D model carousel">
<model-viewer src="../../assets/ShopifyModels/Chair.glb" poster="../../assets/ShopifyModels/Chair.png" shadow-intensity="1" ar ar-modes="webxr scene-viewer quick-look" camera-controls alt="A 3D model carousel">

<button slot="ar-button" id="ar-button">
View in your space
Expand Down Expand Up @@ -345,7 +345,7 @@ <h4>This demonstrates the <code>ar-placement</code> attribute, which defaults to
</div>
<example-snippet stamp-to="wall" highlight-as="html">
<template>
<model-viewer src="../../assets/boom_2_.glb" ar ar-placement="wall" camera-controls alt="A 3D model of some wall art"></model-viewer>
<model-viewer src="../../assets/boom_2_.glb" ar ar-placement="wall" ar-modes="webxr scene-viewer quick-look" camera-controls alt="A 3D model of some wall art"></model-viewer>
</template>
</example-snippet>

Expand Down Expand Up @@ -394,7 +394,7 @@ <h4></h4>
<template>
<div class="demo" style="background: linear-gradient(#ffffff, #ada996); overflow-x: hidden;">
<span style="position: absolute; text-align: center; font-size: 100px; top:50%;">Background</span>
<model-viewer camera-controls src="../../shared-assets/models/glTF-Sample-Models/2.0/AlphaBlendModeTest/glTF-Binary/AlphaBlendModeTest.glb" alt="A 3D transparency test" style="background-color: unset;"></model-viewer>
<model-viewer camera-controls src="../../shared-assets/models/glTF-Sample-Models/2.0/AlphaBlendModeTest/glTF-Binary/AlphaBlendModeTest.glb" ar ar-modes="webxr scene-viewer quick-look" alt="A 3D transparency test" style="background-color: unset;"></model-viewer>
</div>
</template>
</example-snippet>
Expand Down
12 changes: 6 additions & 6 deletions packages/modelviewer.dev/examples/scenegraph/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ <h2 class="demo-title">Swap Model Variants</h2>
</div>
<example-snippet stamp-to="variants" highlight-as="html">
<template>
<model-viewer id="chair" camera-controls src="../../shared-assets/models/glTF-Sample-Models/2.0/SheenChair/glTF-Binary/SheenChair.glb" alt="A 3D model of a chair">
<model-viewer id="chair" camera-controls src="../../shared-assets/models/glTF-Sample-Models/2.0/SheenChair/glTF-Binary/SheenChair.glb" ar ar-modes="webxr scene-viewer quick-look" alt="A 3D model of a chair">
<div class="controls">
<div>Variant: <select id="variant"></select></div>
</div>
Expand Down Expand Up @@ -124,7 +124,7 @@ <h2 class="demo-title">Model Transformations</h2>
</div>
<example-snippet stamp-to="transforms" highlight-as="html">
<template>
<model-viewer id="transform" orientation="20deg 0 0" bounds="tight" shadow-intensity="1" camera-controls ar src="../../shared-assets/models/Astronaut.glb" alt="A 3D model of a chair">
<model-viewer id="transform" orientation="20deg 0 0" bounds="tight" shadow-intensity="1" camera-controls ar ar-modes="webxr scene-viewer quick-look" src="../../shared-assets/models/Astronaut.glb" alt="A 3D model of a chair">
<div class="controls">
<div>Roll: <input id="roll" value="20" size="3" class="number"> degrees</div>
<div>Pitch: <input id="pitch" value="0" size="3" class="number"> degrees</div>
Expand Down Expand Up @@ -195,7 +195,7 @@ <h2 class="demo-title">Change Material Base Color</h2>
</div>
<example-snippet stamp-to="changeColor" highlight-as="html">
<template>
<model-viewer id="color" camera-controls interaction-prompt="none" src="../../shared-assets/models/Astronaut.glb" alt="A 3D model of an astronaut">
<model-viewer id="color" camera-controls interaction-prompt="none" src="../../shared-assets/models/Astronaut.glb" ar ar-modes="webxr scene-viewer quick-look" alt="A 3D model of an astronaut">
<div class="controls", id="color-controls">
<button data-color="1,0,0,1">Red</button>
<button data-color="0,1,0,1">Green</button>
Expand Down Expand Up @@ -236,7 +236,7 @@ <h2 class="demo-title">Change Material Metalness and Roughness Factors</h2>
</div>
<example-snippet stamp-to="changeMaterial" highlight-as="html">
<template>
<model-viewer id="sphere" camera-controls interaction-prompt="none" src="../../shared-assets/models/reflective-sphere.gltf" alt="A 3D model of a sphere">
<model-viewer id="sphere" camera-controls interaction-prompt="none" src="../../shared-assets/models/reflective-sphere.gltf" ar ar-modes="webxr scene-viewer quick-look" alt="A 3D model of a sphere">
<div class="controls">
<div>
<p>Metalness: <span id="metalness-value"></span></p>
Expand Down Expand Up @@ -290,7 +290,7 @@ <h2 class="demo-title">Swap textures - Diffuse/MetallicRoughness</h2>
</div>
<example-snippet stamp-to="swapTextures" highlight-as="html">
<template>
<model-viewer id="lantern" camera-controls src="../../shared-assets/models/glTF-Sample-Models/2.0/DamagedHelmet/glTF-Binary/DamagedHelmet.glb" alt="A 3D model of a helmet">
<model-viewer id="lantern" camera-controls src="../../shared-assets/models/glTF-Sample-Models/2.0/DamagedHelmet/glTF-Binary/DamagedHelmet.glb" ar ar-modes="webxr scene-viewer quick-look" alt="A 3D model of a helmet">
<div class="controls">
<div>
<p>Diffuse</p>
Expand Down Expand Up @@ -346,7 +346,7 @@ <h2 class="demo-title">Swap textures - Normals/Occlusion/Emission</h2>
</div>
<example-snippet stamp-to="swapTextures2" highlight-as="html">
<template>
<model-viewer id="helmet" camera-controls src="../../shared-assets/models/glTF-Sample-Models/2.0/DamagedHelmet/glTF-Binary/DamagedHelmet.glb" alt="A 3D model of a helmet">
<model-viewer id="helmet" camera-controls src="../../shared-assets/models/glTF-Sample-Models/2.0/DamagedHelmet/glTF-Binary/DamagedHelmet.glb" ar ar-modes="webxr scene-viewer quick-look" alt="A 3D model of a helmet">
<div class="controls">
<div>
<p>Normals</p>
Expand Down