Investigate using math types in fj crate

[hannobraun]

Right now, the fj crate doesn't use a math library, instead just using arrays to represent points and vectors. This works well enough for simple models, but might become inconvenient for more complex ones. I assume that any non-trivial model would benefit from a math library, and having to convert data types into arrays would be an inconvenience.

Using the math module in fj seems like the obvious choice. It would need to be extracted into a separate library, but that is desirable anyway. We need to be careful though: To keep modeling productive, compile times are critical. I don't know, if adding math as a dependency in fj could cause models to compile more slowly.

[hannobraun]

Due to #193, a lot has changed since I opened this issue. Any investigation should include whether using the math module from the host application instead of nalgebra makes sense. This would require the math module to be extracted into a separate fj-math library (which would be desirable anyway, see #141).

[therealprof]

Shouldn't the intrinsic types from math be exposed for the modeling as well? At the moment a lot of functions are taking raw f64 or Vec<[f64; 2]> which is problematic for a number of reasons including the lack of input validation, pretty printing, handling convenience but also the ability to use make use of other primitive types -- I would certainly enjoy not having to wildly cast around between whatever type I want (or need to) use and f64 all the time.

Trying to figure out where #246 is actually coming from has not been too much fun, yet. The values are going in as f64, then get bounced around and converted a few times internally and come out wrong at the other end. Constructing them early in the model would help quite a lot following along where the incorrect handling is happening.

[hannobraun]

Shouldn't the intrinsic types from math be exposed for the modeling as well? At the moment a lot of functions are taking raw f64 or Vec<[f64; 2]> which is problematic for a number of reasons including the lack of input validation, pretty printing, handling convenience but also the ability to use make use of other primitive types -- I would certainly enjoy not having to wildly cast around between whatever type I want (or need to) use and f64 all the time.

Yes. See my previous comment in this issue.

Trying to figure out where #246 is actually coming from has not been too much fun, yet. The values are going in as f64, then get bounced around and converted a few times internally and come out wrong at the other end. Constructing them early in the model would help quite a lot following along where the incorrect handling is happening.

Yeah, you're right about that. I'm always careful about including stuff in fj, because I'm worried about compile-times. But I agree that the current situation doesn't work too well. We should be using the same math types everywhere.

[hannobraun]

Updated issue title and description to reflect the current situation.

[hannobraun]

I've experimented with this. The compile times of the star model go up from ~0.5s to ~1.5-1.6s when integrating the math types into fj. This is not acceptable.

I consider this issue blocked on #14, as we'd need some radical progress there, before we can attempt this again.

[LoganDark]

nalgebra is notorious for ridiculous compile times. Have you looked into other crates, such as ultraviolet? I also think the latter would be more suitable to borrow primitives from. See the documentation:

There are a couple relatively unique/novel features in this library, the most important being the use of the Geometric Algebra.

Instead of implementing complex number algebra (for 2d rotations) and Quaternion algebra (for 3d rotations), we use Rotors, a concept taken from Geometric Algebra, to represent 2d and 3d rotations.

What this means for the programmer is that you will be using the Rotor3 type in place of a Quaternion, though you can expect it to do basically all the same things that a Quaternion does. In fact, Quaternions are directly isomorphic to Rotors (meaning they are in essense the same thing, just formulated differently). The reason this decision was made was twofold: first, the derivation of the math is actually quite simple to understand. All the derivations for the code implemented in the Rotor structs in this library are written out in the derivations folder of the GitHub repo; I derived them manually as part of the implementation.

On the other hand, Quaternions are often basically just seen as black boxes that we programmers use to do rotations because they have some nice properties, but that we don’t really understand. You can use Rotors this same way, but you can also easily understand them. Second is that in some sense they can be seen as ‘more correct’ than Quaternions. Specifically, they facilitate a more proper understanding of rotation as being something that occurs within a plane rather than something that occurs around an axis, as it is generally thought. Finally, Rotors also generalize to 4 and even higher dimensions, and if someone wants to they could implement a Rotor4 which retains all the properties of a Rotor3/Quaternion but does rotation in 4 dimensions instead, something which simply is not possible to do with Quaternions.

[hannobraun]

Thank you for your comment, @LoganDark. I didn't know about ultraviolet. It looks really interesting! Although it's probably not the best basis to implement our current fj-math API, seeing how our types are generic over dimension, and their aren't. But maybe that genericness is part of the (compile-time) problem.

Definitely worth experimenting with.

[LoganDark]

our types are generic over dimension, and their aren't

Is there any particular reason why you couldn't use a sealed trait for this? AFAIK, it's definitely possible to smooth over the different structs using a trait.

P.S. (slightly offtopic) is there some switch I can flip to make the newsletter send non-tracking links?

[hannobraun]

Is there any particular reason why you couldn't use a sealed trait for this? AFAIK, it's definitely possible to smooth over the different structs using a trait.

That should work. More trait infrastructure could negatively affect compile times though. No idea if it would be an improvement in the end.

Maybe there's a fundamental conflict here: Being generic over dimensions is valuable in the kernel, but probably less so in model code. Maybe it's a mistake to want to use the same library for both use cases, since the constraints are so different. Would be nice to have just one library, of course, but maybe that's not the best solution.

P.S. (slightly offtopic) is there some switch I can flip to make the newsletter send non-tracking links?

No, sorry. I would like to disable link tracking for the whole list, but my mailing list provider (MailerLite) doesn't support that. I contacted support about that a while ago. They acknowledged the feature request, but made no promises on how much of a priority that is going to be.

Didn't seem worth switching providers over this issues, but I'd definitely prefer having no tracking at all in my emails.

[LoganDark]

That should work. More trait infrastructure could negatively affect compile times though. No idea if it would be an improvement in the end.

Would certainly be less of an impact than nalgebra. The Rust compiler is designed to handle hundreds of different traits per compilation, but nalgebra incurs deep monomorphization costs (tons of codegen).

Maybe there's a fundamental conflict here: Being generic over dimensions is valuable in the kernel, but probably less so in model code. Maybe it's a mistake to want to use the same library for both use cases, since the constraints are so different. Would be nice to have just one library, of course, but maybe that's not the best solution.

Perhaps you could, drumroll please, have separate structs and then... use a trait in the kernel to smooth over both of them. :^)

[hannobraun]

Would certainly be less of an impact than nalgebra. The Rust compiler is designed to handle hundreds of different traits per compilation, but nalgebra incurs deep monomorphization costs (tons of codegen).

Good to know! I have to admit, I don't have great intuition about what kind of feature is going to incur how much of a compile-time cost.

Perhaps you could, drumroll please, have separate structs and then... use a trait in the kernel to smooth over both of them. :^)

I think that would be roughly the same as using two different libraries. And if we decided that having those smoothing-over-traits only available in the kernel would not be enough, it would turn into exactly the same as using two libraries 😄

---

Your suggestion about using ultraviolet gave me a thought: Why not use mint in the fj crate? That could be much more lightweight, and would allow users to choose whatever math library they want to, as long as that library supports mint, which most (all?) of them seem to do.

[LoganDark]

I think that would be roughly the same as using two different libraries. And if we decided that having those smoothing-over-traits only available in the kernel would not be enough, it would turn into exactly the same as using two libraries 😄

Well, if you put the traits in the main math library with the rest of the stuff, it would turn into having one library.

Your suggestion about using ultraviolet gave me a thought: Why not use mint in the fj crate? That could be much more lightweight, and would allow users to choose whatever math library they want to, as long as that library supports mint, which most (all?) of them seem to do.

I don't have any experience with mint, but that sounds like it could work.

[LoganDark]

P.S. did you actually notice cint mint is made by the same dev who made ultraviolet? I didn't actually realize until now but it's actually kind of ironic...

[hannobraun]

I hadn't heard of cint, so thanks for letting me know! Might be a good dependency to use to represent colors, although it doesn't seem to be as accepted in the ecosystem as mint.

[hannobraun]

I don't think that's right? According to crates.io, ultraviolet and cint are by Gray Olson, while mint is by Dzmitry Malyshau.

[LoganDark]

I don't think that's right? According to crates.io, ultraviolet and cint are by Gray Olson, while mint is by Dzmitry Malyshau.

...Oh, I stand corrected. I guess I misread it as mint in the very first place... then wrote cint and still didn't notice... and then corrected myself to mint and now I'm even more confused. Nevermind xD

[hannobraun]

I've had an idea: We could move the math types into a new crate, say fj-core, while leaving the math operations where they are in fj-math. So just the type definitions (and maybe some simple methods like basic constructors and accessors) would move to the new library. Anything non-trivial would stay in fj-math, which would implement the operations via extension traits.

As a result, the kernel could keep using the math types with minimal change, but the fj crate could now depend on the new fj-core (or whatever it ends up being called) and get the math types, without incurring too much of a compile-time cost.

By itself, this would be an incremental improvement. The math types are not any worse than using arrays for vectors and such, while reducing friction between models and kernel. Even increasing reliability, by using Scalar. It wouldn't be a long-term solution though, as having vector operations and other math stuff in models is still desirable.

However, it may be possible to come up with some trait-based proxy approach, so models could call the math operations through the FFI boundary, without having to compile them. I'm not sure about that, as it's just a vague idea at this point. But if that's possible to do in a relatively low-friction way, there's the potential to use this technique for many other APIs beyond just math.

[hannobraun]

This issue is no longer applicable. The affected code has been removed. See A New Direction for context.