Readability for citation key patterns #6706

k3KAW8Pnf7mkmdSMPHz27 · 2020-07-23T16:29:05Z

The issue is that AuthorList#fixAuthorForAlphabetization is used to create a String representation of authors' names. This can leave unbalanced brackets if a name piece starts with a latex command, as it is not the intended use of the method.

The suggested fix is to resolve latex in the name of an author and then remove any illegal characters in the post-processing (which is dependent on if it is going to be used as a file path or citation key). As Unicode characters are already allowed in the name of an author, this change should not affect users too much. It will, however,

have slower performance, as all authors are being processed as if they contain latex
in almost all cases that contain latex, it will produce different output than the previous solution

As always, these are only suggestions. Any comments and criticism are encouraged!
The "intuition" behind the suggestion is that latex in Strings will always be hard to deal with, and I think most users expect a pattern to resolve to something as similar as possible to what is shown in the main table.

What remains

verify that the CitationKeyGenerator does not output Unicode keys
~~remove the latex_to_unicode modifier as it should not be used~~
change access modifiers for methods in BracketedPattern, I'd suggest that most should be private or protected
~~change how arguments are parsed for auth.../ed... patterns (readability)~~
add test cases for this issue
test for character classes in regexp
Fix parsing of bracketed patterns
Fix the conversion of unicode to characters allowed in bibtex keys in CitationKeyGenerator

Change in CHANGELOG.md described (if applicable)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (for UI changes)
Checked documentation: Is the information available and up to date? If not created an issue at https://github.com/JabRef/user-documentation/issues or, even better, submitted a pull request to the documentation repository.

Inlining the deprecated method.

Any references in the `String` should already have been resolved by `getResolvedFieldOrAlias`.

On Mac OS X this test case generates an error. The changes should reflect the intent of the test (i.e., are the lists referring to the same files).

src/main/java/org/jabref/logic/citationkeypattern/BracketedPattern.java

String[] changed to List<String> and some refactoring

koppor

Thank you for working on it! To ease reviewing could you add @Test cases for the issue you fixed? By quickly looking at it, I see many lines changed, but no test cases re-activated or added.

For your discussion of the consequences, maybe an ADR could be the proper format.

k3KAW8Pnf7mkmdSMPHz27 · 2020-07-29T20:55:36Z

A test case for this particular issue (where starting brackets can cause a problem) will be added! It is just taking a bit of time since I haven't been able to spend as much time on this as I would have wanted 🙁
I'd argue that it is motivated to make several methods in BracketedPattern private, but then test cases in CitationKeyGeneratorTest depends on them being public -> a lot of refactoring.

For your discussion of the consequences, maybe an ADR could be the proper format.

I should have written most of the pull request in ADR format?

The tests check if authors with names containing Unicode returns the correct number of characters in the citation key.

src/test/java/org/jabref/logic/citationkeypattern/CitationKeyGeneratorTest.java

Co-authored-by: Oliver Kopp <[email protected]>

The result of expanding a bracket must be cleaned before applying modifiers that depends on the number of characters in the result. (truncate)

This fix extends a previous workaround in BracketedPattern.authN. It does not solve the underlying issue.

…-issue-3920

…dSMPHz27/jabref into fix-for-issue-3920

# Conflicts: # src/test/java/org/jabref/logic/citationkeypattern/BracketedPatternTest.java # src/test/java/org/jabref/logic/citationkeypattern/CitationKeyGeneratorTest.java

koppor

Seeing the long list of changes, I would suggest to merge this in - and then work on the two remaining open points. - Otherwise, it will get harder and harder to review.

Nevertheless, a short nitpick on the logger.

(Sorry for review delay)

src/main/java/org/jabref/logic/citationkeypattern/BracketedPattern.java

koppor · 2020-09-20T20:40:08Z

Is it possible to fix checkstyle? This does not look good:

/home/runner/work/jabref/jabref/src/test/java/org/jabref/logic/citationkeypattern/CitationKeyGeneratorTest.java:9: error: package org.jabref.model.bibtexkeypattern does not exist

Co-authored-by: Oliver Kopp <[email protected]>

k3KAW8Pnf7mkmdSMPHz27 · 2020-09-21T17:01:33Z

Seeing the long list of changes, I would suggest to merge this in - and then work on the two remaining open points. - Otherwise, it will get harder and harder to review.

Sorry about that. I didn't really expect to go down this route.

After the merge I'll open up a PR for issue #6892 dealing with parsing.
<edit>
I realized this PR still closes issue #3920. The problem with converting unicode characters is not in the issue tracking system. I can open up a PR for it after #6892 is fixed, if that sounds reasonable?
</edit>

koppor · 2020-09-21T17:42:00Z

Seeing the long list of changes, I would suggest to merge this in - and then work on the two remaining open points. - Otherwise, it will get harder and harder to review.
Sorry about that. I didn't really expect to go down this route.

No need to be sorry. I have to thank you that you keep working on the PR even if much time is spend on the code quality.

After the merge I'll open up a PR for issue #6892 dealing with parsing.

Year, thank you for that!

I realized this PR still closes issue #3920. The problem with converting unicode characters is not in the issue tracking system. I can open up a PR for it after #6892 is fixed, if that sounds reasonable?

Sure, step by step.

…-issue-6707 * upstream/master: (35 commits) Fix a fetcher test for the ShortDOIService (#6945) Fixes Shared Database: Changes filtering in CoarseChangeFilter to attribute property (#6868) Changed default value of "search and store files relative to bibtex file" to true (#6928) Replace comment by just a failure (#6943) Fix: in entry types editor selected field is not removed after first click (#6941) Fix remove actions for entry types in the editor (#6933) Always use Java 15 (#6929) Update DevDocs: workaround for issues with local openjfx maven libraries (#6931) Fixes bugs in the `regex` cite key pattern modifier (#6893) Add missing author Readability for citation key patterns (#6706) Add new author Reset to master and add default case to switch (#6847) Bump mockito-core from 3.5.10 to 3.5.11 (#6924) Bump byte-buddy-parent from 1.10.14 to 1.10.15 (#6923) Bump org.beryx.jlink from 2.21.4 to 2.22.0 (#6925) Bump xmpbox from 2.0.20 to 2.0.21 (#6926) Bump pascalgn/automerge-action from v0.9.0 to v0.10.0 (#6927) Improve parsing of short DOIs (#6920) Bump junit-vintage-engine from 5.6.2 to 5.7.0 (#6910) ...

* upstream/master: (55 commits) Rename menus citation style in preview style (#6899) Fix for some Unicode characters in citation keys (#6938) Add missing authors Fix a fetcher test for the ShortDOIService (#6945) Fixes Shared Database: Changes filtering in CoarseChangeFilter to attribute property (#6868) Changed default value of "search and store files relative to bibtex file" to true (#6928) Replace comment by just a failure (#6943) Fix: in entry types editor selected field is not removed after first click (#6941) Fix remove actions for entry types in the editor (#6933) Always use Java 15 (#6929) Update DevDocs: workaround for issues with local openjfx maven libraries (#6931) Fixes bugs in the `regex` cite key pattern modifier (#6893) Add missing author Readability for citation key patterns (#6706) Add new author Reset to master and add default case to switch (#6847) Bump mockito-core from 3.5.10 to 3.5.11 (#6924) Bump byte-buddy-parent from 1.10.14 to 1.10.15 (#6923) Bump org.beryx.jlink from 2.21.4 to 2.22.0 (#6925) Bump xmpbox from 2.0.20 to 2.0.21 (#6926) ... # Conflicts: # src/main/java/org/jabref/logic/util/DelayTaskThrottler.java

k3KAW8Pnf7mkmdSMPHz27 added 11 commits July 10, 2020 10:47

Drop authEtal dependence on alphabetization

360cfda

Drop use of deprecated method

aa30930

Inlining the deprecated method.

Drop unnecessary method call

4126163

Any references in the `String` should already have been resolved by `getResolvedFieldOrAlias`.

Fix test using nio.file.Path

09f3ca2

On Mac OS X this test case generates an error. The changes should reflect the intent of the test (i.e., are the lists referring to the same files).

Refactor if-else block to switch

facada6

Add parsing of authors as an AuthorList

9e93fe9

Merge branch 'master' into fix-for-issue-3920

ddf3c6f

Fix handling of list of authors

812b9ac

Refactor variable name

15770de

Fix adding institutions twice

d456d1e

Fix attempting to improve readability

53f705c

k3KAW8Pnf7mkmdSMPHz27 changed the title ~~Fix for issue 3920~~ [WIP] Fix for issue 3920 Jul 23, 2020

k3KAW8Pnf7mkmdSMPHz27 marked this pull request as draft July 23, 2020 16:29

Siedlerchr reviewed Jul 23, 2020

View reviewed changes

src/main/java/org/jabref/logic/citationkeypattern/BracketedPattern.java Outdated Show resolved Hide resolved

k3KAW8Pnf7mkmdSMPHz27 added 2 commits July 23, 2020 14:17

Fix types and readability

e62481a

String[] changed to List<String> and some refactoring

Refactor firstAuthorVonAndLast

e0c7507

koppor reviewed Jul 24, 2020

View reviewed changes

k3KAW8Pnf7mkmdSMPHz27 added 5 commits August 7, 2020 10:51

Fix access modifiers

a86ce94

Merge branch 'master' into fix-for-issue-3920

6a836f2

Add test case

8d1774b

Drop unused methods

cf203c9

Add tests for truncated Unicode

715364d

The tests check if authors with names containing Unicode returns the correct number of characters in the citation key.

koppor reviewed Aug 18, 2020

View reviewed changes

src/test/java/org/jabref/logic/citationkeypattern/CitationKeyGeneratorTest.java Outdated Show resolved Hide resolved

k3KAW8Pnf7mkmdSMPHz27 and others added 6 commits August 18, 2020 17:31

Fix missing bracket in test case

ee94869

Co-authored-by: Oliver Kopp <[email protected]>

Add convenience method for cleaning cite keys

ed66bb2

Fix incorrect number of characters in cite key

5b1312f

The result of expanding a bracket must be cleaned before applying modifiers that depends on the number of characters in the result. (truncate)

Fix incorrect number of characters in cite key

864c0ba

This fix extends a previous workaround in BracketedPattern.authN. It does not solve the underlying issue.

Merge remote-tracking branch 'origin/fix-for-issue-3920' into fix-for…

9cac2db

…-issue-3920

Merge branch 'master' into fix-for-issue-3920

25de361

k3KAW8Pnf7mkmdSMPHz27 added 6 commits September 3, 2020 09:53

Add test for parsing of brackets in regex

bde6303

Merge branch 'fix-for-issue-3920' of https://github.com/k3KAW8Pnf7mkm…

4f72ef5

…dSMPHz27/jabref into fix-for-issue-3920

Fix construction of BibEntries

11cbc14

Fix new line in JavaDoc

7c2bcd0

Fix inconsistent bracket use in test patterns

9b68687

Fix more BibEntries constructions

de93751

k3KAW8Pnf7mkmdSMPHz27 mentioned this pull request Sep 8, 2020

Key generator regex() function not handling character classes #6892

Closed

1 task

k3KAW8Pnf7mkmdSMPHz27 added 2 commits September 14, 2020 15:56

Merge branch 'master' into fix-for-issue-3920

609caa7

# Conflicts: # src/test/java/org/jabref/logic/citationkeypattern/BracketedPatternTest.java # src/test/java/org/jabref/logic/citationkeypattern/CitationKeyGeneratorTest.java

Fix mistake in merge

2c73301

koppor requested changes Sep 20, 2020

View reviewed changes

src/main/java/org/jabref/logic/citationkeypattern/BracketedPattern.java Outdated Show resolved Hide resolved

src/main/java/org/jabref/logic/citationkeypattern/BracketedPattern.java Outdated Show resolved Hide resolved

koppor added the status: changes required Pull requests that are not yet complete label Sep 20, 2020

Fix use of withCiteKey

23764b4

k3KAW8Pnf7mkmdSMPHz27 changed the title ~~[WIP] Fix for issue 3920~~ [WIP] Fix parsing and unicode in citation keys Sep 21, 2020

k3KAW8Pnf7mkmdSMPHz27 changed the title ~~[WIP] Fix parsing and unicode in citation keys~~ [WIP] Readability for citation key patterns Sep 21, 2020

k3KAW8Pnf7mkmdSMPHz27 and others added 2 commits September 21, 2020 12:53

Fix logger output

aca6fb1

Co-authored-by: Oliver Kopp <[email protected]>

Fix logger output

7b89653

Co-authored-by: Oliver Kopp <[email protected]>

k3KAW8Pnf7mkmdSMPHz27 added 2 commits September 21, 2020 13:02

Fix logger output

eac7293

Drop test-case outside of PR scope

d598f41

k3KAW8Pnf7mkmdSMPHz27 changed the title ~~[WIP] Readability for citation key patterns~~ Readability for citation key patterns Sep 21, 2020

k3KAW8Pnf7mkmdSMPHz27 marked this pull request as ready for review September 21, 2020 17:26

koppor merged commit efdd24c into JabRef:master Sep 21, 2020

k3KAW8Pnf7mkmdSMPHz27 mentioned this pull request Dec 20, 2020

Updates to institution citation keys #7210

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Readability for citation key patterns #6706

Readability for citation key patterns #6706

k3KAW8Pnf7mkmdSMPHz27 commented Jul 23, 2020 •

edited

Loading

koppor left a comment

k3KAW8Pnf7mkmdSMPHz27 commented Jul 29, 2020

koppor left a comment

koppor commented Sep 20, 2020

k3KAW8Pnf7mkmdSMPHz27 commented Sep 21, 2020 •

edited

Loading

koppor commented Sep 21, 2020

Readability for citation key patterns #6706

Readability for citation key patterns #6706

Conversation

k3KAW8Pnf7mkmdSMPHz27 commented Jul 23, 2020 • edited Loading

koppor left a comment

Choose a reason for hiding this comment

k3KAW8Pnf7mkmdSMPHz27 commented Jul 29, 2020

koppor left a comment

Choose a reason for hiding this comment

koppor commented Sep 20, 2020

k3KAW8Pnf7mkmdSMPHz27 commented Sep 21, 2020 • edited Loading

koppor commented Sep 21, 2020

k3KAW8Pnf7mkmdSMPHz27 commented Jul 23, 2020 •

edited

Loading

k3KAW8Pnf7mkmdSMPHz27 commented Sep 21, 2020 •

edited

Loading