Texture Mapping: Missing Rotation

[Ekki]

Hi,

in the meta model, i.e. the data model from which I create my ThreeJS scenes, I'm using the X3D texture mapping manipulation as defined here: http://doc.x3dom.org/author/Texturing/TextureTransform.html

while scaling and offset seem to be there, rotation is missing. Is there any chance to get the rotation in the future?

Thx and BR.

Ekki

[mrdoob]

We have repeat and offset. But I think scale and translate kind of make more sense. So considering that we may have to change that in the future, adding rotation could be nice.

I wonder what's the common formula.

[bhouston]

I can add rotation. we could use it as well. I can do a PR with scale, translate and rotation next week. From reading X3D it seems it is scale then rotation then translation.

In my private Three.JS branch I added these parameters to each texture individually, rather than sharing the same settings across all textures loaded. I could add that as well.

[Ekki]

For the models I create it would be perfect to set these parameters for each texture individually.

Pls let me know if you need my support for testing, reviewing, whatever

[mrdoob]

I can add rotation. we could use it as well. I can do a PR with scale, translate and rotation next week. From reading X3D it seems it is scale then rotation then translation.

That'd be great!

[WestLangley]

Questions to be resolved before coding:

1. Will offset / repeat be replaced with translate / scale ?
2. Won't you also have to add center in addition to rotation? Currently, there is an implicit center, which is in the lower-left ( 0, 0 )?
3. What would the center be if you wanted to scale from, or rotate around, the center of the texture? ( 0.5, 0.5 )?
4. Is scale equivalent to the inverse of repeat? I would think if you want to scale a texture, a scale of 2 would make the texture appear bigger (so you would see less of it).

[bhouston]

If one wants a general solution, one wants to use a 3x3 matrix to do the transform from input UVs to arbitrary UVs prior to texture access. Now I believe that one only needs the 3x2 matrix elements because there is no projection -- thus two vec3s (6 floats in total) would represent arbitrary transforms, which is one element more costly than a vec2 for scale, a vec2 for translation and a single float for rotation (which is 5 floats in total.) This simplifies the specification of the parameters. Thus the specification of the parameters in any form you want (repeat/scale, or the order of operations) can be done via the existing setters on Matrix3 or one could create specialized Texture transform helpers someplace.

I'd recommend this approach, it is straightforward and likely will not need to be updated as we go along - as it is the general and powerful approach.

BTW in classic fixed function OpenGL, they used a 4x4 texture matrix and this generalized well to 3D textures (which can be 3 vec4s , 12 floats, if you do not need projection.) I am not sure we have to go there for the time being, because that is a big price to pay for the rarely used 3D texture capability. Going to 12 floats versus 6 floats is a lot of cost for a rarely used function.

[mrdoob]

Is scale equivalent to the inverse of repeat? I would think if you want to scale a texture, a scale of 2 would make the texture appear bigger (so you would see less of it).

I think so, yeah.

If one wants a general solution, one wants to use a 3x3 matrix to do the transform from input UVs to arbitrary UVs prior to texture access.

Oh! Hadn't thought of that. That sounds great to me.

[WestLangley]

I believe a 3 x 3 matrix would be required. But it would be a bit tricky because ( 0, 0 ) is in the lower left, not in the center of the texture.

Suppose a user wanted to rotate the texture by pi / 4 radians counterclockwise about its center, and zoom in on the texture by a factor of 2 (zooming also from its center).

Question 5. What would the matrix transform be in that case?

I think some sort of helpers would be required so the user could specify the transformation in natural language. I like the idea of using matrices in the shaders, but they will have to be correctly populated.

[bhouston]

@WestLangley wrote:

I believe a 3 x 3 matrix would be required.

Yes, a 3x3 matrix is required but the last line is always going to [0,0,1] for non-projective transformations, so we can skip using a shader parameter for it, and just reconstruct the 3x3 shader with the two vec3s and [0,0,1] on the gpu.

But it would be a bit tricky because ( 0, 0 ) is in the lower left, not in the center of the texture.

I do not understand the question.

Suppose a user wanted to rotate the texture by pi / 4 radians counterclockwise about its center, and zoom in on the texture by a factor of 2 (zooming also from its center).

Question 5. What would the matrix transform be in that case?

We can create it using THREE.Matrix3 this way if these methods existed:

var m = new THREE.Matrix3();  // identity
m.rotate( Math.PI / 4 ); // rotate around 0,0
m.scale( new THREE.Vector2( 2, 2 ) ); // zoom around 0,0
m.translate( new THREE.Vector2( 0.5, 0.5 ) ); // move 0,0 to center of [0,1]x[0,1] texture space

var v1 = new THREE.Vector3()
v1.x = m.elements[0];
v1.y = m.elements[3];
v1.z = m.elements[6];

var v2 = new THREE.Vector3()
v2.x = m.elements[1];
v2.y = m.elements[4];
v2.z = m.elements[7];

// note that in a non-projective matrix, the vector represented by
// ( m.elements[1], m.elements[5], m.elements[8] ), should be (0,0,1).

Upload v1 and v2 to the GPU. Then reconstruct the mat3 in the shader by using v1, v2 and [0,0,1].

[bhouston]

I may have misunderstood the question, you may need to use this set of operations instead:

var m = new THREE.Matrix3();  // identity
m.translate( new THREE.Vector2( -0.5, -0.5 ) ); // move center to 0,0
m.rotate( Math.PI / 4 ); // rotate around 0,0
m.scale( new THREE.Vector2( 2, 2 ) ); // zoom around 0,0
m.translate( new THREE.Vector2( 0.5, 0.5 ) ); // move 0,0 to center of [0,1]x[0,1] texture space

Note I shift now towards 0,0 before the scale/rotation.

[mrdoob]

I think some sort of helpers would be required so the user could specify the transformation in natural language. I like the idea of using matrices in the shaders, but they will have to be correctly populated.

I think we should just do like THREE.Object3D. THREE.Texture would now have translation, rotation, scale and matrix. Matrix will be computed whenever needsUpdate is set to true. For this we should move texture.image to a new THREE.Image type with its own needsUpdate.

[mrdoob]

Upload v1 and v2 to the GPU. Then reconstruct the mat3 in the shader by using v1, v2 and [0,0,1].

Why not just upload a Matrix3fv directly?

[WestLangley]

@mrdoob You are leaving out center -- apparently it is the point around which the rotation and scaling occurs. In three.js, that was always the origin. It is not the same as offset, which you want to rename translate.

Why not just upload a Matrix3fv directly?

Yes. Then the shader math is easy.

Also, the user should not be constructing the matrix. The user should express in natural language.

[WestLangley]

I think the order is:

1. specify the center
2. scale from the center
3. rotate around the center
4. translate

[bhouston]

@mrdoob wrote:

Why not just upload a Matrix3fv directly?

Sure. I've seen the partial matrices used in the past because of the premium placed on shader parameters (Max Fragment Uniform Vectors), but if that isn't an issue, it simplifies things to use the full matrix. I do not know what is the distribution of supported Max Fragment Uniform Vectors in the wild, but it worries me as I added tons of parameters to the shader to support Physically-Based Shading.

From this website I see that I support 1024, which is a fair number: https://www.browserleaks.com/webgl

[bhouston]

It is probably premature optimization to worry about a few floats. So let's go full matrix. :)

[WestLangley]

Let's first agree on the API.

It seems to me there are 7 parameters -- unless you want to hardwire the center to be ( 0.5, 0.5 ) -- in which case, the concept of scaling has a different meaning than we are used to, as it would scale from the mid-point, instead of from the lower-left corner.

[bhouston]

3DS Max, a popular 3D content creation tool offers two sets of parameters:

Real World Mode:

• Offset, Scale, U Angle, V Angle.

Otherwise:

• Offset, Tiling (inverse of Scale), U Angle, V Angle.

The difference in UIs for the two modes doesn't change anything underlying. I guess it allows for specification of U and V angles separately to allow for shear in a user understandable fashion. I do not know how common different U and V angles are in the real world. There is no "center" parameter.

[bhouston]

I think that defaulting center for scale and rotation to (0,0) would be easiest. That is what 3DS Max appears to do and I think that is generally the standard in the industry.

One can decompose any arbitrary 3x3 transform that doesn't include projection or shear into a rotation, scale and translation, so a center parameter is not needed for completeness, just for convenience.

[mrdoob]

Do you know what Blender and Maya does?

[WestLangley]

I think that defaulting center for scale and rotation to (0,0) would be easiest.

@bhouston It would be, but I have never seen a corner rotation requested here or on SO at least. It has always been a request to rotate the image around the image center, which is what X3D appears to support (see link above).

[bhouston]

Unreal Engine 4 doesn't have any fixed way of doing this. You can just modify the UVs via arbitrary nodes in their graph-based shader designer.

Unity 5 doesn't seem to have any fixed way of doing this either.

Can not find an option in Maya for this, they just basically want you to rely on their UV editor. Which is much more general than a uniform transform across all UVs. You can transform subsets of UVs in different ways, etc.

I couldn't find anything in Softimage. Again they seemed to push one to use the UV editor.

Blender again goes the UV editor approach. Their transform tool supports rotation, scale, translation with an optional pivot location: http://wiki.blender.org/index.php/Doc:2.6/Manual/Textures/Mapping/UV/Layout_Editing

[bhouston]

The trend is not to do fixed form UV transforms in the modern game engines. Rather people write custom shaders for the specific transforms they need. I guess for a lot of content, it is annoying to have to manage yet another set of parameters. They just try to get the UVs on the incoming objects fully correct.

One only really needs to do this UV transforms on textures that are mapped dynamically at run-time. This isn't a very common case. Thus maybe one could just add to Geometry, an applyMatrixToUVs( matrix3 ) or something like that. And remove all transform-related parameters (offset, repeat.) You either write your own or you pre-apply it to the geometry. That will actually make things simplier both in the code and in the shaders.

[bhouston]

I wrote a UVTransform operator for Clara.io today and got this result of a test -- just hit play:

https://clara.io/view/94f492f6-9336-4a4c-91b1-26e3a43e4a57/webgl

It runs at 60 fps., but only with 1600 vertices.

[Ekki]

just great! thx so much!

[Ekki]

Happy New Year guys, BTW...

Again, Ben, this looks great! Any time schedule when this could be available to the public? Is there anything I can do to support you? BR, Ekki

[bhouston]

@Ekki I think we need a decision from @mrdoob and @WestLangley as to whether they want to add this to the shaders or as utility to transform UVs on meshes prior to uploading to the card. It is their choice.

[bhouston]

@Ekki This can be implemented as a loader functionality outside of the core ThreeJS as long as the UVs are not animated. In that case it is really simple to do.

[mrdoob]

I think the Matrix3 approach is the way to go. Messing with UVs is... a mess.

[mrdoob]

We could also, potentially, have a different Matrix3 per texture.

[bhouston]

If we go the Matrix3 route, we should have a different Matrix3 per texture. :) I have some plans to redo the texture system so that you can map any texture to any UV, to any input (map, emissiveMap, alphaMap, etc.) But that will come later.

[Ekki]

I would really like that Matrix3-per-Texture way! So I could share the geometries globally and assign 'mapped' textures to each instance. Sounds like a deal! ;-)

[bhouston]

Myself, I am going to be a little gunshy about writing new PRs until I know the fate of the existing PRs as I am worried about conflicts and whether or not I can make use of common.glsl, and diffuseColor/outgoingLight in new PRs.

[WestLangley]

I do not think that users should be expected to specify a matrix. You can implement it that way, but there should be a more natural API -- something like translate, rotate, scale, center.

Rotation is naturally around the center, but scaling can either be from the corner or the center, so I am not sure how that should work.

I am going to be a little gun-shy about writing new PRs

That is completely understandable. I also would like the PRs merged one-at-a-time so I have an opportunity to review them as they are applied.

[bhouston]

@WestLangley I agree we shouldn't use a Matrix3 as the interface. BTW here is the code I wrote for Clara.io to transform the UVs (influenced by our conversations on this issue):

  getUVTransform: function( rotation, tiling, useScale, scale, translation, pivotCenter ) {

    var result = new THREE.Matrix4();

    result.multiply( new THREE.Matrix4().makeTranslation( -pivotCenter.x, -pivotCenter.y, 0 ) );
    result.multiply( new THREE.Matrix4().makeRotationZ( THREE.Math.degToRad( rotation ) ) );
    if( useScale ) {
      result.multiply( new THREE.Matrix4().makeScale( scale.x, scale.y, 0 ) );
    }
    else {
      var safeInverse = function( v ) {
        if( v == 0 ) {
          return 1;
        }
        return 1 / v;
      }
      result.multiply( new THREE.Matrix4().makeScale( safeInverse( tiling.x ), safeInverse( tiling.y ), 0 ) );      
    }
    result.multiply( new THREE.Matrix4().makeTranslation( pivotCenter.x, pivotCenter.y, 0 ) );
    result.multiply( new THREE.Matrix4().makeTranslation( translation.x, translation.y, 0 ) );

    return result;

  },

var applyUVTransform = function() {
  var tempVec3 = new THREE.Vector3();
  return function( uv, matrix ) {
    tempVec3.set( uv.x, uv.y, 0 );
    tempVec3.applyMatrix4( matrix );
    uv.set( tempVec3.x, tempVec3.y );      
  }
}();

I used a Matrix4 because Matrix3 didn't have the necessary tools (makeRotation, makeTranslation, makeScale.)

[WestLangley]

@bhouston I was under the assumption that we would construct the matrix on the javascript side from the user's natural parameters. Something like

matrix3 = new THREE.Matrix3().makeUVTransform( rotate, scale, translate, optionalCenter );

Then, instead of passing offsetRepeat to the shader, we would pass matrix3 instead.

Is that your understanding?

[bhouston]

@WestLangley I agree. The code above can be used, to convert from the Matrix4 I construct to Matrix3 for shader usage just drop column 3 and row 3, the z row/column.

[mrdoob]

Myself, I am going to be a little gunshy about writing new PRs until I know the fate of the existing PRs as I am worried about conflicts and whether or not I can make use of common.glsl, and diffuseColor/outgoingLight in new PRs.

Sorry for the delay. I'm holding these until the 15th. There was a release blocker with morphtargets for the last weeks. Finally figured it out this week. Now I just want to have scripting in the editor in a useful state and will be ready for release on the 15th. Until then I'll be merging only PRs that are simple.

[kkananen]

@mrdoob any update on this? We at vizor.io are implementing this and don't want to diverge too far from your plans. Any new info much appreciated.

[mrdoob]

I'm happy to merge whatever @bhouston and @WestLangley decide.

[Kesshi]

@kkananen Did you finish your implementation of this feature?
Maybe you could make a PR?
For my current project the missing rotation is also big issue.

[nxgraphics]

any status on this ? is it implemented ? is there an example somewhere ? thx

[Mugen87]

Solved with #11863

https://threejs.org/examples/webgl_materials_texture_rotation.html