Migrate from Slack to an alternative platform

[cbeams]

This is a Bisq Network proposal. Please familiarize yourself with the submission and review process.

This is a WIP proposal in the sense that it's incomplete and needs a proper owner to take it forward. I wanted to put this placeholder here, though, to make it clear that a number of us would be interested to see a proper proposal for migrating from Slack to Riot (or possibly another platform).

This has been coming for a while, mainly because Slack's business model is incompatible with our needs. It is too expensive to justify paying for it, given how many members we have in our Slack workspace, and while we remain on the free product, our content and conversations regularly disappear, getting 'held hostage' by Slack until we do pay (which again will be never at the current price point).

The issue came to a head this month, however, with the impersonation of @ManfredKarrer by a scammer. Manfred wrote about this in more detail at bisq-network/roles#23 (comment). Whatever platform we move to next will need to have a better security model for user identities than this!

A number of things will need to be addressed in this proposal, including but not limited to:

• Where the instance should be hosted. On the main matrix.org home server, or on a home server run by a contributor? (I imagine the answer will be the latter, but I don't know enough about it to weigh in)
• Is it possible to start on matrix.org and then migrate to a dedicated home server, just to try it out in a lightweight way?
• Feature gap analysis. What do we use in Slack today that Matrix+Riot can't do / don't support?
• Mobile app support. Is it any good?
• Is there a way to bulk-invite the users of our existing Slack workspace to the new Riot workspace (not sure if we'd actually want to do that, but worth knowing the answer)
• ...

Ideally, this will be put together by someone with experience using Matrix, Riot and our Slack instance, but anyone who can catch up on all that is welcome too.

As the current primary Slack Admin, I can help flesh out this proposal by answering questions, but it should be driven by someone else. I do not want to be the owner of any new infrastructure that we move to—this is a good opportunity to improve decentralization.

So if you'd like to own this, please say so in the comments, and follow up with a normal proposal write-up as you see fit.

[cbeams]

I just updated the title to make it more general. I was focusing on Matrix+Riot when writing this, but there are other alternatives, such as Rocket.Chat, MatterMost, etc. Perhaps we can quickly narrow down the field here, but shouldn't assume we'll move to any particular one without some discussion.

See: https://www.reddit.com/r/selfhosted/comments/989768/matrixriot_vs_rocketchat/

[ManfredKarrer]

@cbeams Thanks for adding that proposal!

@chirhonul just told me today that he might be able to operate Rocket.Chat for us. Seems Rocket is used by Blockstream as well, so it is probably a good platform.

[chirhonul]

Yeah, I would be willing to own the role of operating a rocket.chat or other Slack replacement with some reasonable best-effort SLO on an on-going basis. With "best-effort SLO", I mean setting up monitoring / alerting and being responsive in case of service issues, doing backups and other basic technical operation tasks, with an expectation that after initial setup, ongoing maintenance would be less than 5 - 10 hrs / month.

The main complication for me setting up and operating a chat service is that the options for privacy-preserving platform providers is limited, and the price / performance ratio is not that good for the options that do exist. I've used https://www.bacloud.com/ for dev VMs and it's been acceptable for those purposes, but I don't know how well it'd work for more resource-intensive services. I can explore what the resource needs of a rocket.chat instance would be and try setting it up on bacloud.com or other privacy-preserving platforms, but another option would be if someone else can deploy a server and give me access — for practical reasons of redundancy in case of issues, having two or three people have SSH access probably would be good regardless.

[sqrrm]

@chirhonul I agree that bacloud.com is probably not a good enough. I had problems running a price node I think it was. They just shut the VPS down without warning, saying crypto mining was not allowed. After speaking with them explaining that I wasn't mining they had no issue with what I was doing. The main problem being that they're small and not as reliable as a bigger provider.

[ManfredKarrer]

@chirhonul Would it be an option for you if the hosting at a provider where one cannot be anonymous (e.g. AWS,...) is done by someone else and you get the credentials and do all the setup/maintainance. I mean to just separate the hosting billing out?
We had bad experiences (as @sqrrm mentioned) with anonymous hosters as it seems they get more consumed by abusive clients and act therefore also often less reliable. For a infrastructure like that it would be pretty bad if we are out of service of days because the provider has some issues. I also assueme that their security is less professional as big providers and therefor it could add security risks for us as well (prob. not a big issue though). I am aware of the downsides and that it reduces censorship resistence but I think at current stage problems with a unreliable provider are more realistic than others... We could change hoster in future as well...

Ah sorry was not reading your comment fully ;-) Yes would be good to separate those roles and use a high quality hoster. If no one volunteers to be in that role I can take it. But for sake of decentralization it would be good if we find someone else.

[ghost]

@ManfredKarrer ,
A possible solution would be that you rent a domain name specifically for this service only, eg bisq-chat.org, plus the associated hosting, and then give the keys to the person in charge and let him do the job ?
So you won't have to care about the installation etc, but being the domain + hosting owner, you'll still get hands on it.

Possibly you could even create a redirection like chat.bisq.network -> bisq-chat.org , to make things more transparent.

[vycoreb]

why not use something like tox? https://tox.chat/ no need to waste effort on maintenance, better operational security (censorship resistance)

[cbeams]

Tox has a more Skype-like chat model for chatting directly with other users or groups of users, but we're for a Slack-like model, which is channel-based and works well for us.

[vycoreb]

@cbeams there are groups which could be used in exactly the same way or am i missing something?

[cbeams]

Are you familiar with Slack and similar channel-based chat platforms? I doubt that a group-based model would compare well (I used Tox when it first came out, but that was several years ago). I certainly would not want a Skype/WhatsApp-style group model for what we're doing with Bisq.

Groups are usually invite-only and not easily discoverable (if at all). Channels are public to all members of the chat workspace, easily discoverable, anyone can browse, join, etc. This is the model we want.

If Tox's groups are in fact very similar to Slack's channel's, I suppose it would be a candidate for migration, but I doubt this is so.

[vycoreb]

by default they are invite-only but there is already at least one system to automatically invite users https://github.com/JFreegman/ToxBot

[cbeams]

To propose a new chat platform for us to migrate to, it's important to understand how we're using Slack today, what features we use, etc. I'm not sure you have that familiarity, and I believe using something like Tox would not be about a migration to a similar platform, but about using a fundamentally different chat modality. I don't want to reinvent what we're doing. Slack's model works well for us and many others, that's why so many use it. Rocket.Chat, Riot, and I presume other platforms are clones of the Slack approach for a reason. Tox is a different model. If you're really inclined to propose this, go ahead and write it up as I suggested in the description above, but I strongly doubt it's the direction I'd personally want to take, and I have a feeling that other longtime contributors won't either. Happy to be proven wrong, but that won't happen in back-and-forth discussion like this; it'll need to be demonstrated with a proper proposal that shows how Tox will address our needs and make for a smooth migration.

[vycoreb]

everything depends on use cases and there is no magic bullet for everything. i have used your slack a bit, some other slacks in the past and also discords and irc which are similar in some ways.

for the use case of arbitration/customer support tox might be slightly higher friction than slack because you have to download and run a binary. but then the account creation is easier (only username and password for tox). but regarding that use case it might be worth considering that even looking up and joining a slack is too much for many people. i talked to another user of bisq about his experience with it and he told me that one reason that he cannot recommend bisq to less technical people is because sometimes there are issues that require support but he thinks that the process of joining slack etc. to get support is too complicated for a non-technical user. but he would recommend it if we could simplify it and have a feature like on some websites they have a "click here to start a live chat with customer service" thing in the lower right corner.

another use case to consider is that for some users and contributors it is important to maximize the amount of privacy for some discussions that do not have to be public and therefore should not be public. there is no reason for the entire world to know that alice bought some monero and had an issue, bob made a decent amount of money trading and chris is an important contributor (= attack vector).

so the best choice really depends on the use cases. sometimes convenience is more important, sometimes security is important. its good to at least have both options. i wanted to suggest this as an additional option. if you find something else more useful that is perfectly fine.

[chirhonul]

In order to move forward here I suggested to @ManfredKarrer that I'll go ahead and set up a rocket.chat instance on one of the semi-flaky but privacy-preserving platforms I've used in the past. I found that they do have dedicted servers at a reasonable price, so performance should presumably be acceptable, although the risk of being misinterpreted as "crypto mining" or similarly against ToS and getting disabled / shut off could be a risk, as @sqrrm says. I'll give updates here on my progress. It seems that even finding a privacy-preserving way to pay the platform provider is becoming increasingly cumbersome; many payment processors and coin swapping platforms are no longer allowing Monero, and/or enforcing KYC..

[ghost]

@alexej996 was able to put the forum behind an onion address : https://bisq.community/t/onion-address-for-the-forum/6607/5 (thanks to him).
Maybe it's also possible to do the same with a rocket.chat instance ?
Of course, this will also require a strong local internet connexion + UPS.

[ManfredKarrer]

@chirhonul is working on it.

[chirhonul]

I have acquired the bisq.space domain and performed a basic rocket.chat installation:

• https://bisq.space/

The service runs on a single dedicated server currently:

• CPU: Intel® Celeron® Processor G3900T 2C/2T (2M Cache, 2.60 GHz)
• RAM: 16GB DDR4 ECC
• 1 HDD: 3.5" 1TB 6GB/S SATA3
• 2 HDD: 3.5" 1TB 6GB/S SATA3
• RAID Array: RAID 1 (Mirroring)

I would not recommend migrating over all users yet, since there's no monitoring in place and I've only done very basic testing. There also could be performance issues or need for downtime in the future.

I also have not set up an SMTP email server, which seems to be required to deliver "account activation" emails. However in my testing, new users can join and chat even without clicking the activation link from the emails. I tried signing up for a free sendgrid.com account for this purpose, but it was flagged as "high-risk". I've filed a support ticket with them and can update on any progress here.

There seems to be tools available to export Slack data and import it into rocket.chat:

• https://rocket.chat/docs/administrator-guides/import/slack/

There is also a tool to act as a bridge from Slack <-> rocket.chat:

• https://rocket.chat/docs/administrator-guides/import/slack/slackbridge/

I am writing up operational notes for the installation, and will make a plan for how to monitor the basic health of the service and perform backup and other basic operational tasks. The operational planning also will need include a recovery plan in the eventuality of losing the bisq.space domain and/or the server, or suffering prolonged unavailability if I am unavailable in the future. Perhaps having regular backups accessible by a few contributors in addition to clear documentation on how to restore services on a new domain would suffice.

[ManfredKarrer]

Great thanks!
If you need help for applying the new Bisq CI @pedromvpg might be able to help. Here are basics for the new design: https://github.com/bisq-network/bisq-design/blob/master/bisq-refresh.pdf

@pedromvpg Do we have already a place where the new logo is uploaded in diff. formats?

[chirhonul]

I believe that the look-and-feel of the rocket.chat instance can be customized by admin users at:

• https://bisq.space/admin/Assets

I would be happy to give admin privileges to any users you'd like @ManfredKarrer, so others can help configure and moderate the instance. I gave admin privileges to your manfred.karrer to start with, and I believe you should be able to give it to other users.

[ManfredKarrer]

What is the URL for it when connecting from the desktop app? https://bisq.space is not accepted.

[chirhonul]

The URL should be https://bisq.space from the desktop app, and I was able to connect as chi2 from a VM using the desktop app and that URL.

There is a possible issue with the DNS records that could be the cause for the problem connecting in the desktop app. I noticed that there's two A records returned, in addition to the expected IP of the server 185.25.48.173 there's also a second record with 192.64.119.44:

$ dig bisq.space

[..]
;; QUESTION SECTION:
;bisq.space.                    IN      A

;; ANSWER SECTION:
bisq.space.             1237    IN      A       192.64.119.44
bisq.space.             1237    IN      A       185.25.48.173

I'm not sure what the cause of the duplicate A record might be. The domain registrar's DNS settings page is fairly basic but doesn't seem to mention the unexpected record.. perhaps an old record with long TTL?

[ManfredKarrer]

I get a "Timeout trying to connect". Just tried again. Will try again tomorrow. maybe its just an old DNS...

[alexej996]

@chirhonul I only see 185.25.48.173 in the A records, but if the DNS is still giving you trouble you could use cloudflare's DNS nameservers, since they give you good control over the records.

[chirhonul]

Thank you @alexej996. I will consider that if the issues persist, if I can find a privacy-preserving way to use Cloudflare's services.

[alexej996]

I am not aware of any specific privacy issues with cloudflare at the moment.
They only ask you for an email to create an account, you can create it over Tor (probably need JavaScript though) and you don't need any payment info if you are only going to use them as a DNS server.

Although I don't know if I am missing something.

[chirhonul]

I have added a repo containing:

• my notes on installation, maintenance, backups, monitoring and known issues
• all configuration used for the service
• scripts to help with backups and other operational tasks

The parts about monitoring and some other processes describes what I want to set up, and not much of it exists yet!

I hope the repo helps provide info about the setup. I intend for it to be sufficient to revive the service on another domain and/or server in case we cannot use the current one in the future, and will keep improving it:

• https://github.com/chirhonul/bisq.space

[chirhonul]

I get a "Timeout trying to connect". Just tried again. Will try again tomorrow. maybe its just an old DNS...

The DNS resolves to the correct IP only at this time, so the earlier results probably were due to long TTL, as we speculated:

$ dig bisq.space

[...]
;; ANSWER SECTION:
bisq.space.             328     IN      A       185.25.48.173

@ManfredKarrer: Let us know if you are still having issues connecting with the desktop app.

[m52go]

@chirhonul I obtained the Slack archive and began the data import into Rocket. It's only ~60mb unpacked but going at an absolutely glacial pace, could take a day or two.

[chirhonul]

@m52go: Okay, thanks for trying to export data from bisq.slack.com for import into bisq.space.

I think that setting up the bisq.slack.com <-> bisq.space bridge would actually be a better first step. It requires admin rights on bisq.slack.com as well. I included notes on the bridge in an earlier comment:

There is also a tool to act as a bridge from Slack <-> rocket.chat:

    https://rocket.chat/docs/administrator-guides/import/slack/slackbridge/

[chirhonul]

I noticed that the service was very slow and returning 504 Gateway Timeout for me in my browser when visiting https://bisq.space.

All services are running and responding to requests correctly and in a timely manner locally on the server. Resources usage is not very high.

Confusingly, using curl from my laptop I also get a timely response. I'm not sure what would differ in the browser, and since I receive the 504 response I am clearly reaching the nginx frontend on the server..

It is possible that the server has very limited bandwidth, and the upload of admin data, which I can see in the logs, is using up all of it.. the platform claims 1Gbps (50TB traffic inc.), but we need to measure to see if this is the case, and currently it would look unlikely.

[chirhonul]

I still don't know what caused the issues mentioned in my last comment, but there seems to be no problem loading the site over Tor Browser for me now, so perhaps there is not much we can do to investigate at this point. I will continue setting up monitoring and other operational improvements to make it easier to track down the source of such issues in the future.

[m52go]

Update.

The Rocket.Chat instance needs a few external items to be configured before it can be ready for people to use.

1. Email server - password recovery. We will need an email server to send password recovery emails for incoming Slack users to reclaim their identities on Rocket. I'm in touch with @chirhonul to determine the best way to do this.
2. Matterbridge server - mirror Slack messages. The built-in SlackBridge tool from Rocket won't work because of a limitation on the free Bisq Slack account. Matterbridge is an alternative way to accomplish the same thing as SlackBridge, and @mrosseel already has a running Matterbridge instance. I'm working with him in #chat-bridge on Slack to configure it.

The top 2 items will provide us with a minimal but functional chat environment. Less crucial but still important:

3. Integrations. Looks like GitHub is the most critical integration. It needs to be set up manually. There's a script to get started but I'm sure it will need some work.
4. Mobile notifications - push. Also needs to be set up manually...by default, the Rocket mobile apps will work, but there will be no push notifications unless we specify APN and GCM/FCM credentials. Does anyone have accounts set up for these? @joachimneumann? I haven't worked a whole lot with either so I'm not sure what the pricing/burden is like.

Note: until we have an email server, people won't be able to reclaim their usernames. So it might be best to discourage people from joining Rocket until that's taken care of. Already a few users have tried joining and had to make new usernames because their old one was 'taken', which is not ideal.

[ManfredKarrer]

After the DAO launch we will prioritize the transistion to Rocket.Chat.

[KanoczTomas]

I will take the lead on this, will get back to the issue after DAO (making a note for myself).