Skip to content
This repository has been archived by the owner on Dec 22, 2023. It is now read-only.

Update CI #245

Merged
merged 8 commits into from
Jan 28, 2020
Merged

Update CI #245

merged 8 commits into from
Jan 28, 2020

Conversation

lexaknyazev
Copy link
Member

@lexaknyazev lexaknyazev commented Jan 10, 2020

This PR changes the validation CI to use the recently published precompiled validator release. All other pending PRs must be rebased to ensure updated CI status.

A few highlights from existing models:

  • CesiumMan, RiggedFigure, Monster
    • non-normalized weights sum & ignored transforms
  • RiggedSimple, BrainStem
    • ignored transforms
  • InterpolationTest
    • npot texture
  • BoxTextured
    • custom ICC profile (Generic ICC RGB with gamma 1.8) in the texture
  • AlphaBlendModeTest
    • unused textures
  • EnvironmentTest
    • extra fractional part for integer values
  • BoxVertexColors
    • various unused objects

@lexaknyazev lexaknyazev requested a review from emackey January 10, 2020 13:55
@emackey
Copy link
Member

emackey commented Jan 10, 2020

So Travis CI fails this of course, due to stricter checks. Sounds like Cesium Man and a few others have actual errors that need correction. I suppose we should try to get those fixes onto this branch prior to merge.

There are some warnings too, like unused objects and npot textures. Do those prevent Travis from passing? I agree our samples should be more carefully crafted, but there's also something to be said for providing clients with edge cases like these, else engines might get released without testing them.

Also, is there any way Travis could print out a summary at the end, enumerating a simple list of which models need attention in order for tests to pass?

And finally, it looks like Travis has two actions, and the validation happens in both actions. Is this duplicate work, or why does this happen?

Thanks for getting the ball rolling on this. I'll see if I can help fix some of the models, particularly the non-skinning ones.

@emackey
Copy link
Member

emackey commented Jan 10, 2020

@lexaknyazev
Copy link
Member Author

providing clients with edge cases like these

Such cases should not happen unintentionally. E.g., we have a dedicated test for npot textures. Also, there are lots of negative tests in the validator repo and in the Asset-Generator output.

Also, is there any way Travis could print out a summary at the end, enumerating a simple list of which models need attention in order for tests to pass?

The validator sets exit code to 1 if there was a single error among validated assets. Issues of lower severity do not affect CI status.

And finally, it looks like Travis has two actions, and the validation happens in both actions. Is this duplicate work, or why does this happen?

From Travis docs:

If you see two build status icons on your GitHub pull request, it means there is one build for the branch, and one build for the pull request itself (actually the build for the merge of the head branch with the base branch specified in the pull request).

We can disable one of them if needed.

@emackey
Copy link
Member

emackey commented Jan 10, 2020

Such cases should not happen unintentionally.

OK.

Can we allow individual models to specify individual validation settings? The warning for npot textures should be suppressed for the model that intentionally tests them, for example. This would allow us to produce a clean report of the whole repo.

Also, is there any way Travis could print out a summary at the end, enumerating a simple list of which models need attention in order for tests to pass?

The validator sets exit code to 1 if there was a single error among validated assets. Issues of lower severity do not affect CI status.

Sure, but if there are lots of infos and warnings in the mix, and only one model has real errors that turn the CI red, it would be nice to have the offending model(s) called out in an easily discoverable fashion. If one searches the log for the word "Error", there are lots of false positives on Errors: 0 that hide the problem in a sea of noise. I want to scroll to the very bottom and see a message that says something like:

Models with errors:
./2.0/CesiumMan/glTF/CesiumMan.gltf
./2.0/CesiumMan/glTF-Binary/CesiumMan.glb

Otherwise it can take a lot of digging to understand which models are blockers.

This is much less important if we can suppress expected warnings with per-model validation configuration, as presumably any messages in the output should be dealt with.

If you see two build status icons on your GitHub pull request, it means there is one build for the branch, and one build for the pull request itself (actually the build for the merge of the head branch with the base branch specified in the pull request).

We can disable one of them if needed.

Interesting. I suppose it doesn't hurt much to have both, it's just noisy.

@lexaknyazev
Copy link
Member Author

Can we allow individual models to specify individual validation settings?

For now, it's possible to provide a single validation config (with muted or overridden issue codes) for the whole run. To specify individual settings, we need to either:

  • Design a "validation run config" - a JSON with a list of models and their settings (instead of current recursive directory walkthrough).
  • Agree on a "convention" for an asset config located next to each asset (based on a filename).
  • Design a new glTF extension that would contain validation settings.

@lexaknyazev
Copy link
Member Author

As of now

Assets with errors:
	./2.0/CesiumMan/glTF/CesiumMan.gltf
	./2.0/CesiumMan/glTF-Binary/CesiumMan.glb
	./2.0/CesiumMan/glTF-Embedded/CesiumMan.gltf
	./2.0/Monster/glTF/Monster.gltf
	./2.0/Monster/glTF-Binary/Monster.glb
	./2.0/Monster/glTF-Embedded/Monster.gltf
	./2.0/RiggedFigure/glTF/RiggedFigure.gltf
	./2.0/RiggedFigure/glTF-Binary/RiggedFigure.glb
	./2.0/RiggedFigure/glTF-Embedded/RiggedFigure.gltf

@emackey
Copy link
Member

emackey commented Jan 15, 2020

A new CesiumMan model is under development in CesiumGS/cesium#8539. Currently he's still not facing +Z, hopefully will be fixed soon.

@emackey
Copy link
Member

emackey commented Jan 17, 2020

To fix the models here, perhaps the most historically-official way is to get the latest COLLADA2GLTF and re-convert from the sourceModels folder. Questions:

  • Does the latest COLLADA2GLTF version include fixes for these validation failures? Yes for normalization. But there's an issue with the animated version being 90 degrees rotated from the rest pose. Very strange.
  • Where is the latest build of COLLADA2GLTF hosted? release 2.1.5 appears to be the latest.

If that turns out not to be in good shape, could we use another tool such as gltf-pack to correct the errors (without quantizing)? No, it seems gltf-pack always quantizes.

@emackey
Copy link
Member

emackey commented Jan 17, 2020

@lexaknyazev When you get a chance can you update this branch to 2.0.0-dev.3.1 ?

@lexaknyazev
Copy link
Member Author

@emackey
Done.

@emackey
Copy link
Member

emackey commented Jan 24, 2020

CesiumMan is fixed. The process for fixing the remaining two models is now documented in CesiumGS/cesium#8539 (comment). I will need a bit more time to make that all happen.

This was a simple re-run of COLLADA2GLTF 2.1.5 on sourceModels.
Not all models could be fixed by a simple re-run, for example
Monster has a strange 90-degree rotation and broken animation loop.
@emackey
Copy link
Member

emackey commented Jan 27, 2020

🎺 📢 CI has PASSED 🔊 🚀

@emackey
Copy link
Member

emackey commented Jan 27, 2020

Monster was pulled out of sampleModels on this branch. Its animation is too strange. When the animation starts, it jumps 90 degrees to its left, and the animation loop point isn't clean at all. Importing the source COLLADA into Blender really makes a mess there. So I've pulled this asset out entirely, and we should populate this repo in later PRs with more modern, better examples.

Copy link
Member

@emackey emackey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lexaknyazev I think this is ready for merge. Any objections?

@emackey
Copy link
Member

emackey commented Jan 28, 2020

We'll continue to work some of the remaining issues in future PRs. Merging this now to get the new validator implemented in a CI-passable state.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants