Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docbook 5 support #4319

Merged
merged 20 commits into from
Nov 25, 2018
Merged

Add docbook 5 support #4319

merged 20 commits into from
Nov 25, 2018

Conversation

Siedlerchr
Copy link
Member

@Siedlerchr Siedlerchr commented Sep 6, 2018

Fixes #4315
Renamed to docbook 5 export
removed old layout files , replace by the ones from @bernhard-kleine

I added a bib entry and the corresponding xml file for comparision
Please check if all is exported correctly.
I noticed there is also a masterthesis layout file. I think this needs adjustment as well, right?

singleExportDocbookEntry.zip


  • Change in CHANGELOG.md described
  • Tests created for changes
  • Manually tested changed features in running JabRef
  • Screenshots added in PR description (for bigger UI changes)
  • Ensured that the git commit message is a good one
  • Check documentation status (Issue created for outdated help page at help.jabref.org?)

Renamed to docbook 5 export
removed old layout files
@bernhard-kleine
Copy link

bernhard-kleine commented Sep 6, 2018

I have seen three errors and one thing that could be better:

  1. the id="\bibtexkey" should be xml:id="\bibtexkey"

  2. the preamble should contain the item
    xmlns:xlink="http://www.w3.org/1999/xlink", otherwise the validation complains about xlink:href

  3. the numeral 4 in
    xreflabel="\format[Authors(3,Noinitials,EtAl= et al.)]{\author} (\year)"
    should become a 3.
    docbook5export2.zip

  4. the xreflabel might loose the firstname initials. Since this was my first attempt with format the result looks already fine. There is no example with no Initials on the webpage you menthioned, I donot see how to write an author entry with out firstnames. I have now added an option "Noinitials", but I donot know whether that one is valid since it is no mentioned in help.

Furthermore, why not have docbook 4 and docbook 5 export. I really donot know how much that export function has been used. But I would like to keep the old files and rename them to docbook4 respectively and name the new ones docbook5.

I have added the files which need adjustement in the attached zip.
Thank you very much for your help.

* upstream/master:
  update junit to release (#4320)
use lastName only in docbook5
use new docbook template
favor composition over inheritance
add docbook version enum
@Siedlerchr
Copy link
Member Author

@bernhard-kleine I added your changes and also found that the authors formatter has an option LastName which only gives the last name. Is also documented on the page, but overlooked it too and just looked at the code.

I also modified the docbook5.mastersthesis layout to call the docbook 5 authors

singleExportDocbookEntryv2LastName.zip

@bernhard-kleine
Copy link

Thanks a lot: The only item which does not survive validation is the plus in the bibtexkey. But that is another issue. Therefore this is as perfect as it could be. We can finish here. The substitution of the plus from [authshort] should be done by jabref and is not an issue of docbook export.

@Siedlerchr
Copy link
Member Author

Glad it works now as expected.
If you want proper attribution, I need your email address used in github (no reply works as well).
https://help.github.com/articles/creating-a-commit-with-multiple-authors/

It would be nice if you could adjust the documentation of the docbook in the help file. Just click on edit this page at the bottom.

Copy link
Member

@tobiasdiez tobiasdiez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we really need backward support for DocBook 4. Moreover, it would be good if you could add a test(file).

@bernhard-kleine
Copy link

bernhard-kleine commented Sep 7, 2018

When constructing a testcase like Tobias suggested I noticed that the line
<citetitle pubwork="article">\format[XMLChars,RemoveLatexCommands]{\title}</citetitle>
should be changed since it does not take in account other entry types. "article" should become an entrytype macro most probably \entrytype.

This line
\begin{journal} <citetitle pubwork="journal">\format[XMLChars]{\journal}</citetitle>\end{journal}
needs a modification: "journal" should become a macro that accounts for journal, book, thesis etc.

Furthermore it would be relevant to add a line:
\begin{booktitle} \format[XMLChars]{\booktitle}\end{booktitle}
to have the title of the booktitle or something else mentioned.
I tested the following file:
Testdocbookexport.zip

@Siedlerchr
Copy link
Member Author

Siedlerchr commented Sep 7, 2018

The begin and end commands are used to indicate conditional output, e.g. only printed if the field is presented. For thesis there is an extra file. There is a title tag present in the begin.layout
I added the layout files now used. You can use them to create a custom exporter (Options-> Manage custom exporters, point to the .layout file, the other begin and end are automatically found) and then test around with it. The files are loaded at runtime, meaning you don't have to restart JabRef when changing the files content.
layout.zip

@bernhard-kleine
Copy link

That looks comfortable and I will try it in the next week. However, I wonder how you named a macro which retrieves the name of the entrytype?

@Siedlerchr
Copy link
Member Author

\entrytype All fields can be referenced with \fieldname
You can also look at other layout files here: https://github.com/JabRef/layouts.jabref.org

@bernhard-kleine
Copy link

btw my email is [email protected].

@Siedlerchr
Copy link
Member Author

So do we need to add / change the layout or is this current status sufficient?

@bernhard-kleine
Copy link

I did not find the time yet to look into the format for books and incollection, sorry. But it will be different from that of articles.

@bernhard-kleine
Copy link

Hi, I found some time to look after the issue:
I need some help: the conditional output with the attached layout file does not reliably work.
docbook5export.zip

@bernhard-kleine
Copy link

@Siedlerchr I am waiting for your response.

@Siedlerchr
Copy link
Member Author

I will look into that this evening.

* upstream/master: (54 commits)
  Update README.md (#4419)
  Fix generate bibtex key overwrite warning dialog (#4418)
  Fix group hit counter when adding entries (#4413)
  Update README.md
  fix java.nio.file.FileSystemNotFoundException and reorganize code (#4416)
  fixed an issue where corresponding groups are sometimes not highlighted when clicking on entries (#4404)
  Add a few journal abbreviations (#4412)
  Fix display of radio buttons (#4411)
  Update issue templates
  Delete ISSUE_TEMPLATE.md
  Update issue templates
  Integrate mrdlib redesign (#4361)
  fix "save" button in preference dialog not response (#4406)
  show dialog before creating entry (#4405)
  Scrollbar invisible in Preferences -> BibTex Key Pattern (#4287) (#4398)
  Updates (#4402)
  Fixes for all mods exporter tests (#4396)
  Fix EntryType dialog not closed after generate button (#4390)
  Make rank image narrower (#4395)
  Fix showing multiple icons for one menu entry - fixes #4384 (#4392)
  ...
@Siedlerchr
Copy link
Member Author

I found the problem. It seems you really have to use separate files for each entrytype, otherwise it will fill all the fields. The only thing which currently seems not to work is exporting the field school.

The main file the "docbook.layout" is used as fallback if no matching entrytype.layout file is found.
And in the begin and end files no bibtex fields are allowed.

@PhdThesis{JabRef2018,
  author = {{JabRef}},
  title  = {Jabref is awesome!},
  school = {Github},
  year   = {2018},
  type   = {mathesis},
}

layout_separatefiles.zip

@bernhard-kleine
Copy link

What do I have to test it? Which file to select for testing? since one can only load one file for checking.

@Siedlerchr
Copy link
Member Author

It should be sufficient if you put all in one folder and just select the main docbook layout file. All others should be found then

@bernhard-kleine
Copy link

I am just got stuck: I have entered docbook5a in the customize export in Options and now I want to export, The file dialog never opens, I tried with the snapshot from last friday and with the one where you experimentally had my first filter attemps included. I cancelled all windows on my desktop but I could not see the file dialog opening where one can choose the export filter and the place where the file is to go. Strange?

* upstream/master:
  Add submodule pull to circle ci and fix theme loading css (#4443)
  fix groups drag and drop (#4439)
  Convert merge entries dialog to JavaFX (#4410)
  Fix ArrayIndexOutOfBoundsException on second pdf import (#4426)
  Fix radiobuttons in preference menu (#4409)
  Add citation styles as git submodule (#4431)
  Fix highlight color of selected text and progress bar (#4420)
  Fix two new issues
  Fix Violations of Always Use Braces (#4400)
  Delete PreferencesService.java.orig
  Add JabRef icon to installer and change watermark to JabRef (#4421)
  Checkstyle: force braces around code blocks
@Siedlerchr
Copy link
Member Author

There seems to be an issue with the custom export dialog. However, this is currently rewritten in #4394 by @NorwayMaple

Please remove the custom export and then use the inbuilt docbook5 exporter. I now created the following files based on your main file.

layout_fixed.zip

@abepolk
Copy link
Contributor

abepolk commented Nov 1, 2018

@Siedlerchr I'm in the process of rewriting the dialog still. Can I keep going? You're not saying I should stop, are you? I might have time to work on it later today (it is 2pm here).

@Siedlerchr
Copy link
Member Author

Please continue, it would just be nice if you could check that the custom export works fine if you have multiple layout files and select one of them.

@bernhard-kleine
Copy link

@Siedlerchr : There is still an error in the layout file, respectively outside the layout. The macros CreateDocBook5Authors and CreateDocBook5Editors cannot be reached. Therefore the export file shows errors. As I had provided these files already, I wonder what happened.

* upstream/master:
  Bump junit-pioneer from 0.2.2 to 0.3.0 (#4451)
  Add Depandabot badge
  Emphasize that users should try out the newest version before reporting a bug
  Add a preference to add files in entry editor (#4408)
@Siedlerchr
Copy link
Member Author

@bernhard-kleine First of all, you have to explicitly use this branch-version (based on the current master), https://builds.jabref.org/docbookfive/ because the macros for author and editors are referring to the inbuilt java classes which are only available in this branch.

Second, you can try the inbuilt docbook5 exporter, I added the files as resources to the jar.
If the custom-export does not work, you can try to replace the files in the JabRef.jar file. Open it with a zip-file manager, and then under resource\layout\ you see the layout files.

As you probably already saw, it is necessary to add one file per entry type, one generic file is unfortunately not possible. You can take one of the existing as basis for other, and you only need to adjust the fields per entry-type.

@bernhard-kleine
Copy link

I rechecked the export and I found a couple of errors, mostly due to my notsocareful work. However, they errors I found have been changed. Two things remain which I cannot mend: the line
in docbook5.layout
\begin{title} <citetitle pubwork="\entrytype">\format[XMLChars,RemoveLatexCommands]{\title}</citetitle> \end{title}
should habe \format[ToLowerCase]\entrytype, but that didnot work. Also the NoInitials in the line
<biblioentry xreflabel="\format[Authors(3,Noinitials,EtAl= et al.)]{\author} (\year)" xml:id="\bibtexkey">
does not work either. Otherwise the formating seems okay for the entrytypes which are expressively mentioned.
docbook5version20181109.zip

* upstream/master:
  Change location of some fields in the entry editor (#4448)
  Bump applicationinsights-core from 2.2.0 to 2.2.1 (#4488)
  Added feature to add server timezone when connecting to shared database (#4483)
  Bump applicationinsights-logging-log4j2 from 2.2.0 to 2.2.1 (#4487)
  fix save button
  Update archunit from 0.9.1 -> 0.9.2
  Fix move to group always moves first entry (#4457)
  Add eprint cleanup (#4445)
  respect preference state (#4468)
  Set auto-update checkbox enable/disable when reading preferences (#4446)
  Improve MathSciNet fetcher and add ISBN fetcher to entry editor toolbar  (#4444)
  New translations JabRef_en.properties (German) (#4455)
  New Crowdin translations (#4454)
  Fix error in l10n file: Toogle -> Toggle (#4453)
  Remove depandabot
@bernhard-kleine
Copy link

What do I have to write for Noinitials in the macro the formats the authors/editors? This macro does not work.

User lower case formatter for \entrytype
@Siedlerchr
Copy link
Member Author

@bernhard-kleine For the author question: Instead of NoInitials use FullName and for the other one ToLowerCase formatter, it was necessary to include the whole format-command in quotations.
I updated the files accordingly now.

   \begin{title}   <citetitle pubwork="\format[ToLowerCase]{\entrytype}">\format[XMLChars,RemoveLatexCommands]{\title}</citetitle> \end{title}

@bernhard-kleine
Copy link

Fullname does leave the firstnames, that is not intended. I want only lastnames.

@bernhard-kleine
Copy link

I found that "Lastname" does what I want. I am waiting for feedback from the docbook-apps mailing list, since the seriesinfo tag that is mentioned in the guide does not seem to be supported. Otherwise my testfile is properly exported.

@bernhard-kleine
Copy link

bernhard-kleine commented Nov 19, 2018

@Siedlerchr: As far as I am concerned, the conversion is complete. During this work I discovered several flaws in docbook 5.1 which concerns mostly the thesis entries. Obviously these type of papers has been forgotten. There is not a single mentioning in the pubwork attribute. I have brought this to the attention of the people designing docbook 5. Maybe it will be corrected in due time. Once this is done I am willing to adapt the layout files again.
docbook5.1Version20181119.zip

Thanks for the helpful cooperation

@Siedlerchr
Copy link
Member Author

Thanks for the quick feedback. I just changed the layout files according to your updates.
In addtion, I found one little error (NoInitials) and fixed it and I converted all tab charaters to whitespace charaters (was mixed)

When all looks good, I will merge this. The next time you can directly open a new PR by editing the files directly in the browser or you clone the repo and modiy them.
https://github.com/JabRef/jabref/tree/master/src/main/resources/resource/layout

@bernhard-kleine
Copy link

bernhard-kleine commented Nov 19, 2018

@Siedlerchr: I have noticed that the begin/end constructs can not be faithfully nested. Is this intentionally or an error? Nested constructs would have be easier structurally. I solved this only partially.
e.g. (from PHDthesis.layout)

\begin{institution}
	<biblioset relation="phdthesis">\n
		<publisher><publishername>\format[XMLChars]{\institution}</publishername></publisher>
		\begin{year}<pubdate>\year</pubdate>\end{year}
	</biblioset>
	\end{institution}

results in a errorneous

    <pubdate>1958</pubdate>
	</biblioset>

This works but is IMHO not logically correct

<biblioset relation="phdthesis">\n
		\begin{institution}<publisher><publishername>\format[XMLChars]{\institution}</publishername></publisher>\end{institution}
		\begin{year}<pubdate>\year</pubdate>\end{year}
	</biblioset>

@Siedlerchr
Copy link
Member Author

I would not call it an error, but I think the problem is that the layout code generating stuff is not based on a regular grammar, but instead on String Tokenization. And it's already complicated enough

For some export formats like BibtexXML or MODS format we have an xsd schema which is used to generate java classes which are then represent a hardcoded export.

@Siedlerchr
Copy link
Member Author

I am going to merge this now. For future updates we can create a new PR.

@Siedlerchr Siedlerchr merged commit 9213e3c into master Nov 25, 2018
@Siedlerchr Siedlerchr deleted the docbookfive branch November 25, 2018 15:32
Siedlerchr added a commit that referenced this pull request Dec 23, 2018
* upstream/master:
  Add docbook 5 support (#4319)
  Bump java-diff-utils from 2.2.0 to 4.0 (#4507)
  3189 fix group renaming (#4470)
  Create test (#4505)
  Bump archunit-junit5-engine from 0.9.2 to 0.9.3 (#4503)
  Bump archunit-junit5-api from 0.9.2 to 0.9.3 (#4504)
  Bump mockito-core from 2.23.0 to 2.23.4 (#4502)
  Bump guava from 27.0-jre to 27.0.1-jre (#4499)
  Improve page checker (#4498)
  Add color picker to groups dialog (#4492)
  Fix kbd XML closing tag
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants