Feature request: import/export API of messages

[helmut72]

Would be cool to import/export (unencrypted) messages from/to Synapse with an API.

An usecase is importing messages from other systems or would be also good to move users from a "testing" Synapse server, which becomes unfortunately "semi-productive", to a new Synapse installation. With a better usernames than "testuser1", "testuser2"...

I read...

https://matrix.org/docs/guides/client-server.html
https://github.com/matrix-org/synapse/tree/master/docs/admin_api

... but couldn't find any API.

Thank you.

[richvdh]

This is closely related to element-hq/element-web#2630, though I couldn't find an issue on the synapse side.

would be also good to move users from a "testing" Synapse server,

It's worth noting that is a lot more complicated than just copying your message history, for reasons noted in the FAQ; however see also https://github.com/matrix-org/matrix-doc/issues/915 and #1209.

[tr3buchet]

Import would be super useful. I suppose a separate tool could be made, considering that as far as i can tell, you'd just need to add the lines to the db and to copy any media files into the correct place. I'm guessing you'd have to translate the format/paths a bit.

[LTangaF]

I've actually been trying to do this with the python-sdk the last few days and am currently blocked on setting historical timestamps. I'm trying to migrate users from a much older chat system, actually, and have figured out a lot of what I need to accomplish, but event timestamps are currently eluding me.

[LTangaF]

I was able to get around the issue by brute force. Basically, I read the original messages and set the system clock before importing each one. If you're running a VM, you have to basically disable host time sync to pull that off. And make sure you don't have any clients pointed at the instance as it'll get confused about presence timestamps jumping around.

[eMPee584]

Import of existing timelines also would be a killer feature for the bridges (i.e. »connect to and import history from Slack«)..
Does the protocol mandate for every event to have the time now?
Importing could be allowed on newly created rooms only so message history could not be fudged around with..

[t3chguy]

Nope it does not, importing is already possible via the application service API's timestamp massaging featureset

[binarymaster]

importing is already possible via the application service API's timestamp massaging featureset

These two pages say that this API has been excluded:

• https://matrix.org/docs/spec/application_service/r0.1.2#timestamp-massaging
• https://spec.matrix.org/unstable/application-service-api/#timestamp-massaging

However there is a recent MSC3316 matrix-org/matrix-spec-proposals#3316 attempting to bring this feature back.

And also there's MSC2716 regarding importing history: matrix-org/matrix-spec-proposals#2716

---

Importing could be allowed on newly created rooms only so message history could not be fudged around with..

I'd like to have this feature too, so I can import message history from my old chat archive using only Synapse Admin APIs, without having to worry about integrating Application Service while I don't actually need it (at least yet).

[GEkuL]

Hi guys, is the feature going to be developed?

I think this is an essential enhancement, so people from other platforms could easily transfer to matrix, gladfully GDPR obliges messengers to export message history, so the only question here, how to import them into the matrix.

It would also be nice to import them in an existing chat selecting who is the other side of the dialog.

I believe that this is a must have feature, even for a simple case of transferring old matrix account messages to newer ones.

Thus, the question is, how it's going?
What's wrong with the feature and when it's going to be implemented?

[christheradioguy]

Just dropping a comment in because this is a feature I'd love to see as well.

My use case is that I've begun using synapse for SMS messaging (via a bridge) and am looking to import all my old SMS message history (currently in a CSV format) so I can have all my historical messages in the same place as any new messages I send or receive.

[helmut72]

Any update on this? Don't want to lost messages. I'm glad that there is now an export feature on Element, but I want to have my chat messages on one place, on the chat server. Like it's possible with a mailserver.

My Synapse server even does have a wrong configuration, I use user:chat.example.com instead user:example.com.

If someone can say at least, that it's possible to import the exported messages from Element, when the import feature is there, this would help a lot to start over a fresh installation.

Thanks.

[squahtx]

An application service using features from MSC2716: Incrementally importing history into existing rooms sounds like it'd do what you want.

It's fairly involved and you'd have to write code to do the import yourself. There's no easy one-click way to import history right now I believe.

[helmut72]

@squahtx Thanks, will follow it.

[helmut72]

Any update on this? It would be enough for now to export chats from my current Synapse in Slack format and after installing my new Synapse, to import them.

Slack import is supported:
https://github.com/Awesome-Technologies/slack-matrix-migration

[HighPriest]

Any info on this?

I have moved server providers and domains a long time ago and had to start all the conversations from the beginning.
I am about to lose access to the old data, so I need a solution to import old conversations ASAP. :(

[GEkuL]

@HighPriest You can google it, seems like there're solutions like db migration. I haven't try it yet, but it seems like a solution.

[HighPriest]

Thank you @GEkuL ! First option from this page seems to be what most people are looking for in my situation, although it is not going to let me merge rooms which have been started again on the new instance.
https://ems.element.io/tools/account-migration

If there is a source to this script somewhere, maybe I would be able to modify it to merge rooms history.