Project goals

[jimmycuadra]

I've been trying to think of ways to work on the project while we wait for async/await and the async library ecosystem to settle, and it occurred to me that there's never been any explicit statement of the goals of the project. It often comes up in discussions over Matrix and Twitter, but it'd be useful to have written down, both as a way to guide us during development and as a place to refer to people asking about the project.

---

Here are my initial thoughts about the project's goals:

What Ruma is

• A fully functional, interoperable Matrix homeserver with federation.
• A Rust library for consuming the Matrix APIs.
• A suite of libraries for building Rust software that integrates with Matrix.

Product focus

• Scale: the homeserver should support both small and large scale deployments.
• Performance: the homeserver should run with a minimum of memory and CPU load.
• Stability: Ruma software should minimize the possibility of unrecoverable runtime errors.

Improving the ecosystem

• Spec compliant: Ruma should be a forcing function for completeness and correctness of the Matrix spec. All information needed to implement Ruma should be unambiguously explained in the spec. Whenever implementation would require looking at the details of an existing implementation, the spec should be improved to clarify instead.
• Rust maturity: Ruma should be a proving ground for the suitability of Rust. The needs of the project should provide feedback upstream to the developers of Rust and Rust libraries to improve and stabilize.

---

I'd love to hear other people's thoughts about what they'd like Ruma to focus on.
Feedback about the organization and wording of the above goals would be appreciated, too!

[alexjg]

I think this is a great issue to raise. It's already helped me as I was slightly confused on the status of the project vis a vis spec standardisation and async/await stuff so it may be worth clarifying that directly in the 'status' section on the project readme?

One of the reasons I'm interested in this project is that I'm interested in running many independent matrix servers from within a Kubernetes cluster. Multi-tenant basically. I think that's covered in the goals you've mentioned above but I wonder if other people are also interested in this and whether there's any need to explicitly call it out?

[exul]

Ruma homeserver and Ruma libraries
"What Ruma is" mentions the libraries, whereas "Product focus" describes that developing the homeserver is the focus of the project. Which on one hand I understand, because the libraries were developed as part of the homeserver, on the other hand, I think way more projects would benefit from the libraries. I use the libraries myself and the clean code and the nice APIs are really helpful.
I also think that the libraries (especially the client-server ones) can adapt spec changes faster than the homeserver as a whole.
My main question here is: Will the library development be driven by the homeserver development or will the be developt more independently from the homeserver.
From what I've seen in the past, they were developed pretty independently and I like that.

small and large scale deployments
Supporting large scale deployments usually comes with the trade-off of complexity. Is the idea to support two architectures (the way dendrite does it) or have one that fits all deployments at the cost of setup complexity. Which boils down to complexity in the code vs. complexity in the setup.

developers
Not sure if this belongs here, but I've seen quite a few great open source projects getting abandoned. I think the only way to mitigate this is to have more developers on board. Are there specific plans to address this?

[jimmycuadra]

@alexjg Thanks for the feedback. My plan is to create dedicated pages on the website that present these goals and a status page to give a more high level overview of the status of the project.

In terms of your intended use case, that is absolutely a project goal. It's one of my personal use cases as well. I think you're right that a goals document should include a bit more information about what the supported use cases are.

[jimmycuadra]

@exul Great points. It's probably best to have "product focus" split into multiple sections for the applications and the libraries. It's confusing as written that Ruma "is" three different things but "focuses" on homeserver-specific things. For your main question, I think the answer is "separate," in the sense that we want the libraries to be driven by the use cases of the Rust/Matrix community in addition to the needs of the homeserver. In other words, we'll never close a pull request on a library because the change is not specifically needed by the homeserver. That doesn't mean that integration in the homeserver won't be taken into account. But overall the libraries should have a broader raison d'être than "implementation details of the homeserver that happen to be separate crates."

Re: scale of deployment, my current thinking is that we follow the same approach as Dendrite where there is an all-in-one binary for small deployments and multiple binaries for different parts of the system for large deployments. Some of this is already explained on the website's project pages (specifically the "homeserver comes in two flavors" part) but it makes sense for it to be explained in the goals document too.

Re: sustainability of development, this is a problem with open source software in general. I don't have a magic solution, but some things that may help:

1. Make the project itself useful to lots of people. The bigger the the pool of users, the bigger the pool of potential contributors.
2. Provide guides aimed at developers explaining the architecture of the project, as well as good contribution guidelines.
3. Make major contributors to the project more visible to both show our appreciation for their work, and to let people just discovering the project that it's more than the work of just one dude. This may inspire confidence that the project is more sustainable.

[exul]

Regarding sustainability: You already did a great job by adding frequent contributes to the Ruma organization in a very early stage. Mentioning in the README that contributions from less experienced developers are welcome might encourage them to contribute. Maybe we can even offer a mentor program for some issue (kind of the same way that rust does it).
Another possibility would be to have "office hours" (I shamelessly stole the idea from kops: https://github.com/kubernetes/kops/#office-hours) where new contributors can get help not only via chat but on a call.
All this requires additional time that is "non-development" time, so I think it makes sense to discuss up-front how much the existing community can and wants to invest.

[erlend-sh]

Being a Rust equivalent to the other Matrix homeservers is already of great value, but in order to see adoption outside of the Rust community it'd be beneficial for Ruma to target some specific use cases that aren't currently being addressed by the Matrix ecosystem at large.

My main gripe with Matrix is the lack of a straight forward registration flow. Matrix as-is could easily be providing a better service than the barely-maintained Gitter.im, but there seems to be no interest in supporting "social logins" through the likes of GitHub and GitLab.

The ability to use other applications as a parent userbase is also lacking. users.rust-lang.org could easily be set up so that anyone who signs up for an account there also gets an account on the official Rust Rocket.Chat or Mattermost chat hub (if such a thing existed), but such a setup is far from easy with Matrix.

Thus my wishlist for special homeserver features in Ruma would be as follows:

• Some lightweight equivalent of OmniAuth built into Ruma
• Ruma as the child of a SSO parent - Example in Discourse
• Ruma as the parent of a SSO child - Example in Discourse

[msrd0]

I completely agree with @erlend-sh that it must be far easier to use matrix for users outside the it section. Usually, when I ask them to install matrix, they download the app and go like Hey, why do I need to choose a user id and all that kind of stuff, I just want to enter my phone number and have people find me using that. That said, I know that there is support for 3rd party logins like email or phone numbers, but it is far from useful by now.

If ruma could support signing up using nothing other than a phone number to get started and allowed to set a username later ruma could add some great value and possible users.

[nikhiljha]

• Developer Onboarding

I really like the idea of having a set time each week/month where newer contributors can get help in a call/chat. Starting with a new codebase often leads to a lot of "stupid" questions relating to how things work, and having those resolved quickly would definitely help people who want to contribute but don't know where to start. Yes, it is "non-development" time, but I think it would be just as productive as actually programming something.

• Alternate Signon Methods

The good news is that even with the regular identity server, first you verify your email/phone number and then you bind it to an address. This means that you can make the signup flow go almost exactly like Telegram/WhatsApp (enter phone number, enter verification code, enter name, optional fields to enter username and password).

What Ruma would have to add to make it actually work is a way to authenticate users based on a one time code sent to another session (Telegram) or an SMS to the phone number (WhatsApp). If this is done, I would really like it to be a plugin of some sort. Synapse is super slow and resource-hungry, and I really don't want Ruma to also end up like that.

An easier thing to implement would be oAuth (as erlend-sh mentioned). Having a configuration option that lets you list OAuth providers would be really nice and would allow Ruma clients to be just as easy to use as most popular chat applications.

[jonbayer]

sooo, async/await is settled. or did this project kinda lose steam while waiting.

[jimmycuadra]

async/await is not the only thing we're waiting for. It's a foundational language feature that is necessary for the development of the library ecosystem around it. There's still a lot that needs to happen. Even async/await itself was released as an "MVP" of the feature, with noticeable omissions, like the ability to use async fn in trait methods.