Thoughts about how to improve native XML import/export plugin

[marcbria]

Context

The "portability" of whole journals is a necessity in the OJS community as publicacions jump from one institution to another during their lifetime.

This is demonstrated by the posts in the forum or that at the time of writing this post there are more than 100 issues related to the native OJS xml importer/exporter.

The native import/export plugin has always been a good ally and although journals expect a portability "magic button" for various reasons, that is not possible.

The motivation of this post is to identify and prioritize the needs in relation to this plugin and even speculate on the possibility of defining a new standard for the portability of entire journals between different OJS.

Main problems detected

• Native import/export from different versions does not work because the data models are different.
• The feedback that the user receives from the plugin when the importation fails is not always clear.
• Import/export fails when source and destination settings (eg: sections or multilang) are different.
• Import/export is limited to articles (data from the editorial process are not included).

Possible actions

From least to most important, a possible list of actions that would be done:

1. Add information on the version used to create the export to show warnings when they don't fit.
2. Improve the info offered by the native import-export plugin when an error is found (ie: better display? link to a DIG document? clearer error feedback? suggested solutions to common errors?)
3. Extend the current plugin (or reformulate it as API+plugin) in order to work better between different versions (ie: modify the xml to rename tags)
4. Extend the current plugin (or reformulate it) to export including article history.

Last point would imply, de facto, defining a journal export "standard" to include articles + editorial history + actors involved + section configuration, etc. (in a multi-lang context...) that sounds like a hide task but not impossible.

Related posts

• Extend native import/export plugin to include additional entities #3261
• Forum: OJS 3.3 registering additional filters for XML Export

[asmecher]

@marcbria, we've spoken about XML import/export issues in general internally; the prohibitive challenge is the amount of technical debt involved in mapping everything between PHP objects and XML. I don't consider this approach to be viable without some kind of 3rd party mapping layer to reduce the amount of code required, and I haven't been able to find anything like that.

Instead, we'd like to maintain the XML import/export toolset as-is for existing users, but focus instead on the API for building out future use cases and better coverage of objects. As we build out the UI to be a consumer of the API, we'll gain coverage implicitly.

[marcbria]

I fully understand this.
The intention of the post is just to document and try to define priorities.

To avoid the mapping by working with the API is the line of work for the future that @NateWr and @withanage seem to agree on.

But I feel this in the long term work, and until we get to that point, I think there's a set of "lower development cost" actions that could perhaps be taken before transitioning to the API that would result in a reduction in support requests.

I am referring to:

• Notifying in some way that import/export is not possible between different versions (only identical? only between equal minors?).
• Improve the way OJS reports xml errors.

Could both of these be added to the "to start with" list (or I don't know what you call it) in case some community developer has the time and inclination?

On the larger needs, I have a wild hope that someone in the community is going to take this up before PKP can, so if you don't mind, I'd ask to leave the thread open to get feedback and contrast the priorities I've listed.

[jonasraoni]

Given the amount of users which are using this plugin, I agree with the points above :)

• Compatibility warning: Unfortunately I think we just can get newer versions to emit such warning, we'll need to add a "generatedVersion" attribute (which should be ignorable through a force flag).

• Error reporting: Last time I used the plugin, it was better, but there were still some points to be improved (I think DTD validation errors were being swallowed, not so sure now), so I think it's doable to improve it a little bit more.