Joining a channel from Spectrum2 doesn’t work

[linkmauve]

When I join a channel from Spectrum2 I’d expect to see the users who are already present there and interact with them.

Instead, first of all there is no discovery or automated invite sent to the user when they first register to the gateway, as well as no information about the format of the mapped JID, and then both mm_join_chat() and mm_chat_info_defaults() don’t get called.

I did modify the g_strsplit_set() call in mm_chat_info_defaults() to avoid using '/' as a separator, which isn’t something representable in the localpart of a JID, but I’m not even sure it’s the way I should interact with the gateway.

In any case, after I join a room, I see myself in it, without any other participant, and shortly after receive a roster push with my nick in the room, which is definitely not something that should happen.

At some point I did receive messages sent from participants in the channel, but as if they were sent to me in private, and it added them to my roster both on XMPP and on the web UI, without me actually doing anything, just after they spoke.

So this whole thing is a mess, I’m probably doing something wrong, but this prpl too. :)

I can attach logs of me doing anything you want if that may help.

Thanks.

[jaroslawp]

This Spectrum2 I guess ? ... If you run pidgin with --debug we should be able to see how it tries to communicate with the gateway.

[linkmauve]

Yes, this one.

I already have the XML logs from another XMPP client (I don’t use Pidgin), is this what you are interested in?

[jaroslawp]

logs showing how purple-mattermost is communicating with the Spectrum2 - yes.

[linkmauve]

Here is the backend-1.log file generated by this Spectrum backend, anonymised: backend-1.log.txt

Please tell me if you need some parts non-anonymised, or if you need me to do some other action, or to annotate it.

And thanks for looking into it!

[jaroslawp]

OK, I see that when we send 'mm_get_users_by_ids' request:

31715: 2017-07-13 12:51:18,498 INFO  libpurple: mattermost:Fetching url https://mattermost.example.com/api/v3/users/ids
31715: 2017-07-13 12:51:18,498 INFO  libpurple: mattermost:With postdata ["pmirzzru7fdpfmoyxydfx3hmar","p7qjue43fjf1mekzcdpumzhtjc","z7m68jcp37bgb8mjcbnauz5eww","kr46o9w7538h58mnnz3yffm3nh"]}

we do not seem the get the response - we expect something alike:

mattermost: Got response: {"pmirzzru7fdpfmoyxydfx3hmar":{"id":"pmirzzru7fdpfmoyxydfx3hmar","create_at":1463487661344,"update_at":1499087197820,"delete_at":0,"username":"XXXXXXXXXX","auth_data":"", ...

here. but instead it seems we get:

31715: 2017-07-13 12:51:18,497 INFO  libpurple: mattermost:Got response: []

.. as a result we have no user information and the rest of the code fails (yeah .. error checking is far from being complete ..):

31715: 2017-07-13 12:51:18,531 ERROR backend: g_log serv_got_chat_in: assertion 'who != NULL' failed

now: this could be because we actually send a malformed json (my bad ..) : ["el1","el2","el3"]} .. missing opening '{' - so here is what you could try: change in the code:

mm_fetch_url(ma, url, g_strrstr(postdata,"["), mm_get_users_by_ids_response, ids);

to:

mm_fetch_url(ma, url, g_strconcat("{",g_strrstr(postdata,"["),NULL), mm_get_users_by_ids_response, ids);

recompile and retry: please let me know if it helps (we should see in log a response in form {"XXX":{id:"XXX", ...}, "YYY":{id:"YYY",...} ...}.

[linkmauve]

{[…]} isn’t any more valid JSON, according to the API reference you shouldn’t have any {} around it, see https://api.mattermost.com/#tag/users%2Fpaths%2F~1users~1ids%2Fpost

I apparently get a different answer though, but there is another place where [] is the answer.

Here is a fix to send a proper array, you can merge it if you want: 0001-Fix-sending-simple-array-for-user_ids.patch.txt

It doesn’t fix my issue though, I still get [] as an answer, or is it possible that the answer gets displayed before the postdata request?

[jaroslawp]

Thanks for the fix: #61 !
The [] could be a valid answer possibly.. maybe we should get a look at logs again after your fix ?
Your fix makes it work for my MM 3.10 test server: can see buddies now.

[linkmauve]

I was always able to see buddies, but I’m still not able to join MUCs, they only contain myself, no matter the number of users joined.

[jaroslawp]

OK, I think the current master has some problems (for example some channels will appear only on restart .. etc): can you check multi-team branch: https://github.com/EionRobb/purple-mattermost/tree/multi-team
(assuming MUC = Multi User Chat ? = MM Group Channel ?) (note: may be better to move away libpurple buddy list: ~/.purple/blist.xml .. before starting .. not sure if I got everything sorted out correctly while reading data from there...)

[linkmauve]

Spectrum doesn’t use this blist.xml file, I’ve heard. And yes, MUC = Multi User Chat, the way channels should be represented on XMPP.

This multi-team branch doesn’t include your fix, is it up to date?

[jaroslawp]

yes, added it there too.

[p5n]

I've just opened #64, that maybe similar to this.

[linkmauve]

I never got a segfault though, it’s probably a different issue.

[p5n]

Regarding muc support for spectrum2:

• purple_roomlist_room_new must be called with teamid^id^chatname---team as name, so correct string will be passed on join and correct jid will be used
• spectrum2 wants "topic" room property to show room topic (I renamed "name" to "topic")

but there are some more issues, so I still cannot join

[p5n]

Also to show tree of chats for spectrum2 mm_roomlist_expand_category() implementation looks required.
Now team categories and chats displayed as plain list.

[p5n]

I've made quite dirty patch yesterday which makes it usable with spectrum2. I did not make PR because of there is a change to channel_id as primary identifier. I dropped team-id^channel-id^channel-name---team-name format.

https://pastebin.com/tqR1YAaL

[jaroslawp]

Thanks for investigating: we would need to make some more changes as this will break multi-team feature I'm afraid (same channel names across different teams)... let me check what could be done here..

[p5n]

I guess that channel ids are unique between teams. Also I think there may be problems with getting roomlist in spectrum. At least I found that psi+ works well enough, but gajim fails to join into some channels. Also it tries to add all channel people to your jabber roster.

So roomlist and adding buddies need more investigations.

[p5n]

And one more issue: spectrum assumes human readable room names, so channel id as name is slightly painful: you need to rename each room. Maybe there is an ability to provide vcard for rooms, I don't know.

[jaroslawp]

Hmm.. I'm trying to figure out why the team-id^channel-id^channel-name---team-name is not understood (or accepted) by spectrum2 .. is this question of some characters (^? or --- ?) or sthg else ?

[p5n]

It passes "name" to jabber client and then it is used to join channel. So it will work if you pass this string everywhere. But currently as I understand it makes ^ separated string only from serialize which looks not called by spectrum2.

[p5n]

You may try to use it with spectrum. It is quite easy. You need XMPP server and create spectrum2 config (/etc/spectrum2/transports/mattermost.cfg):

[service]
user=nobody
jid = mattermost.XMPP-DOMAIN.com
password = PASSWORD
server = XMPP-SERVER-IP
port = XMPP-SERVER-PORT
backend_host = 127.0.0.1
users_per_backend=10
backend=/usr/bin/spectrum2_libpurple_backend
protocol=prpl-eionrobb-mattermost

[identity]
name=Mattermost Jabber Transport
type=mattermost

[logging]
config = /etc/spectrum2/logging.cfg
backend_config = /etc/spectrum2/backend-logging.cfg

[database]
type = sqlite3

[registration]
enable_public_registration=1

Then run "/usr/bin/spectrum2 -j mattermost.XMPP-DOMAIN.com /etc/spectrum2/transports/mattermost.cfg -n"

[p5n]

You just need to add transport to XMPP-server:
mattermost.XMPP-DOMAIN.com
PASSWORD
XMPP-SERVER-PORT

for example for ejabberd.yml:

  -
    port: SOME-RANDOM-PORT
    access: local
    module: ejabberd_service
    hosts:
      "mattermost.XMPP-DOMAIN.com":
        password: "PASSWORD"

[p5n]

I see that roomlist returned from mm_roomlist_get_list() before it filled with all rooms and spectrum2 looks like copies roomlist to internal structures, so I guess it is race condition.

[jaroslawp]

Thank you for all the information: I will try to have a look at that, but unfortunately for now I'm little bit busy with other projects ... so may take some time ...

[p5n]

You may look into https://github.com/p5n/purple-mattermost
It looks like I was able to make it work with spectrum and human readable names (JIDs).

[jaroslawp]

Cool ! Can you change line termination (I guess that is what makes the diff of whole file ?): master...p5n:master ..

[p5n]

I've done unix2dos and push it but it does not help. it is strange because of
$ git diff v1.2
shows good diff

[p5n]

Ah, it was cached. Now it is better but my emacs also breaks tabs according to coding standard :S

[p5n]

Also I think you should not try to add groupchats into roster. It breaks gajim for example, however psi+ works with it.

[jaroslawp]

@p5n : Please check https://github.com/EionRobb/purple-mattermost/tree/v4api_try_spectrum2

I have ported your changes to v4 api (well, not sure if this is complete since diff/patch did show all sorts of formatting / whitespace etc changes ...).

Unfortunately these changes in current state break the plugin for pidgin:

• adding room from room list does not work anymore.
• room names displayed in buddy list are different than displayed in chat window (and ids are different too so pidgin does not 'realize' that these are the same and misses incoming messages notifications etc ...)
• above also breaks buddy list management: on restart channels (rooms) are not properly
  recognized.

Here is a screenshot of this version:

and here is a screenshot of how it looks like without this patch:

for comparison.

Nevertheless: this version may work with spectrum2: could you please try and review changes ?

Thanks !

[EionRobb]

Sadly the spectrum2 changes shouldn't be necessary as of the latest version of spectrum2. I worked with the spectrum devs a few weeks ago to fix the room list support to work with the correct chat ids that libpurple needs

[jaroslawp]

Why sadly ? even better ! 👍

[p5n]

@jaroslawp thank you, I was able to join channel with v4api_try_spectrum2 branch, but there need more debugging with spectrum2 anyway.

[jaroslawp]

@p4n: thanks: according to what @EionRobb wrote above these changes will not be needed for future spectrum2 versions ... lets see.

[truenicoco]

Any update on this? Using Finch everything seems to be working fine. Using spectrum2 through Gajim I can list the channels I am part of using the "join group chat" dialog but unfortunately, I experience the same issue as @linkmauve , I am alone in the channel and nothing else works.

Examining spectrum2 logs I see:

7987: 2020-04-06 19:04:31,542 INFO  libpurple: mattermost:Got response: {"id":"api.context.404.app_error","message":"Désolé, impossible de trouver cette page.","detailed_error":"There doesn't appear to be an api call for the url='/api/v4/users/username/REDACTEDID^REDACTEDCHANNELNAME---REDACTEDTEAMNAME'.  Typo? are you missing a team_id or user_id as part of the url?","status_code":404}

I am not sure if this comment belongs here or on the spectrum2 repo…