Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Future of Exiv2 #1466

Open
alexvanderberkel opened this issue Feb 3, 2021 · 40 comments
Open

Future of Exiv2 #1466

alexvanderberkel opened this issue Feb 3, 2021 · 40 comments
Assignees
Milestone

Comments

@alexvanderberkel
Copy link
Member

alexvanderberkel commented Feb 3, 2021

The development of v0.27.4 is on track. @clanmills has volunteered to help one last time by shipping v0.27.4 before he retires. I've set up a Zoom meeting to discuss the following two questions:

  1. Should Exiv2 join OIN (see Joining the Open Invention Network (OIN) #1447) .
  2. The future of Exiv2 after v0.27.4

Zoom meeting details:
Topic: Future of Exiv2
Time: Feb 27, 2021 06:00 PM Amsterdam, Berlin, Rome, Stockholm, Vienna
Join Zoom Meeting
https://us05web.zoom.us/j/87388802811?pwd=RStyUUdRak1aNmRENnRKZ3k1Z3U3Zz09
Meeting ID: 873 8880 2811
Passcode: S0qPG1

This is the list of contributors / people who forked and/or worked on Exiv2 over the years on GitHub.
@1480c1 @1div0 @354380o726502 @5l1v3r1 @a17r @adammartak @ahuggel @alexvanderberkel @AmaniBHH
@AndreasMartin72 @andreilabato @Anmol92verma @anmolvermamm @arkiruthis @asp @astippich @Aurora-Community
@awilfox @badola @bellaperez @bentglasstube @BioDataAnalysis @bkuhls @blockspacer @boardman @BohuaCao
@Cadcorp @caomw @carlosbasecodeit @Cethric @cgilles @chan0415 @clanmills @clayne @clthomas
@codingdave @connectthefuture @criadoperez @cryptomilk @csparker247 @CTF-s @cygwin-lem
@czgnp @D4N @danielkaneider @danileo @dbeichl @Deneb99 @derselbst @devil1437

@alexvanderberkel alexvanderberkel added the request feature request or any other kind of wish label Feb 3, 2021
@alexvanderberkel alexvanderberkel added this to the v0.27.4 milestone Feb 3, 2021
@clanmills
Copy link
Collaborator

@boardhead: I hope you will be able to join us.

@clanmills
Copy link
Collaborator

Slides for the meeting.
Future of Exiv2.pdf

@alexvanderberkel
Copy link
Member Author

Meeting notes of 2021-02-27

Participants:

  • Robin Mills
  • Volker Griels-Grabsch
  • Luis Diaz
  • Christoph Hasse
  • Peter Kovar
  • Alexander Esseling
  • Thomas Beutlich

Join OIN (open invention network)?

  • Joining OIN is being suspended in the short-term
  • Decision to evaluate joining the KDE project
  • Benefits: KDE is already part of the open invention network
  • KDE project might bring new contributor and/or maintainer to the project

Exiv2 after v0.27.4?

  • No new maintainer has volunteered.
  • We agreed to simplify the branch structure and "modernise" the code (C++20 support)
  • Current master branch v0.28 will be labeled as old-master and frozen.
  • 0.27-maintenance will be frozen.
  • We'll branch 0.27-maintenance to 'main' and make it the default branch.
  • Luis will port Unique pointer code from old-master to main.
  • Robin will release v0.28 from 'main' in the Fall.

There is no active plant for development of Exiv2 beyond 2021.
We hope to have a week-end retreat in Germany in the Fall to celebrate our achievements.

@clanmills clanmills modified the milestones: v0.27.4, v0.28 Feb 27, 2021
@vog
Copy link
Contributor

vog commented Mar 1, 2021

It was nice to see you again after such a long time!

Just a small nitpick: I'm still in favor of naming the new main branch 'master' rather than 'main', because to any outsider, 'master' is the most obvious name for "the git branch to use" (for final testing, pull requests, etc.).

@phako
Copy link
Contributor

phako commented Mar 1, 2021

That's the new default in git, github and many other installations...

@clanmills
Copy link
Collaborator

Thanks, Volker @vog. Maybe I'll do that. Yesterday I changed the name of 'master' to 'old-master' to discourage anybody from submitting further PRs. I also set '0.27-maintenance' as the default branch.

I should close all the open PRs on 'master' and explain what's going to happen.

By the time we ship 0.27.4 on 2021-05-22, it'll probably be safe and reasonable to branch an new 'master' from 0.27-maintenance. I want to retain the 0.27-maintenance branch in case we need a 0.27.5 security update in future. We haven't had a security issue raised on the code for almost 18 months.

Your comments are appreciated. Thank you for contributing.

@phako. Are you saying 'main' is the new default? I thought somebody said that during the meeting on Saturday.

@phako
Copy link
Contributor

phako commented Mar 1, 2021

yes, at least for github, see https://github.com/github/renaming. I thought git itself had switched as well but I cannot find any reference for it in quick search, only heated discussions why this is "PC gone mad"

@imsodin
Copy link

imsodin commented Mar 1, 2021

It's not (yet?) a default in git, but the default branch has recently(ish) been made configurable: https://github.blog/2020-07-27-highlights-from-git-2-28/#introducing-init-defaultbranch
If you are already restructuring branches, I do suggest using main. I do expect it to be the de-facto norm soon-ish - at least my muscle memory already does m-a-i-TAB (weirdly I didn't drop the TAB and go with m-a-i-n). And the only downside I see is having to switch existing setups from master to main, that's why I think an unrelated branch-restructuration is a good moment to make the switch.

@vog
Copy link
Contributor

vog commented Mar 1, 2021

@phako @imsodin Thanks for clarifying the issue with master! I wasn't aware of that latest developments and forming consensus about switching from master to main.

I fully agree that, given those circumstances, main is the better choice.

By the way, I never liked the wording master that much. Even if it didn't have any connotation to slavery, it was simply a bad naming. You would never guess this default branch name unless you are already familiar with Git. Nevertheless, I followed this naming, because following a suboptimal convention is better than following no conventions at all. It is still a pity that the community bends towards "main" rather than "default" - the latter being the default branch name in Mercurial, and an equally good choice, and then would be automatically compatible with the second-most popular DVCS. But then, perhaps Mercurial will be moving to main in the future as well.

@clanmills
Copy link
Collaborator

Thanks you everybody for your comments. I've had a message from @alexvanderberkel saying 'main' is the new 'master'!

Let's stick to the plan. On 2021-05-22, I'll branch 0.27-maintenance -> main and make 'main' the default branch. I've already renamed 'master' -> 'old-master' to discourage its use.

@boardhead
Copy link
Collaborator

@boardhead: I hope you will be able to join us.

Sorry, I missed this. I haven't been keeping up with all of the Exiv2 posts recently.

@clanmills
Copy link
Collaborator

clanmills commented Apr 1, 2021

Exiv2 v1.00
Updated: 2021-04-1

We are now within sight of Exiv2 v1.00 and could achieve that by the end of 2021. In many ways this is a more desirable goal than KDE. Exiv2 declaring v1.00 says Exiv2 is complete. If KDE wish to take over the project in 2022, that will be fine.

I think it's better to use our limited resources to reach Exiv2 v1.00 on GitHub and not put effort into working with KDE.

Topic Status Comment Owner
To reach v1.0
API Good Unified Metadata Container Milos
C++ Code Good C++ modernisation Luis
Documentation Good Improve README-TESTS.md Robin
Image Handlers Good Add BigTiff
Join OIN and/or KDE Good Alex
Localisation Fair Slovak and Czech
Brazilian Portuguese
Crowdin
German
Peter
Leonardo
Leonardo/Robin
Alex
Project management Good Alex
Release Engineering Good Robin
Samples Good exiv2 -ps enhancements Milos
Security Good Create Security Advisories Kev
Source Code Good Reformat with clang-format Luis
Team Weekend Good September 2021 in Bavaria Alex
Test Framework Good Improve bash_tests Peter
Web Site Good Move to GitHub Robin
XMPsdk Good Consider removing from code base Robin
Deserves Attention
Bugs Good Monitor and respond Robin
Users Good Customer Satisfaction Follow Up Alex
Post v1.0
JXL/codestream Waiting for Standard
WebP2 Waiting for Standard
Deep Test Coverage raw.pixls.us and testimages.pixls.us
Platforms Good Support for Mobile
Preview/Thumbnails Fair The code is weak
Servers Good KDE/GitLab Transition
In Good Order
Back-porting Good
Build Good
Build Server Good
Contributors Good
Enhancements Good
Licensing Good
Scheduling Good
Tools Good

We hope to have a Zoom meeting on the weekend of 24 April or 1 May to discuss the plan and assignments.
It looks the moment as though there will be four groups of 3 contributors in each group:

Group Members Priority Tasks
code Luis, Rosen, Pydera C++11, jp2image.cpp, security
metadata Milos, Christoph, Tom Unified Metadata Container, Canon AFInfo, JPEG>64k
t_l Peter, Leonardo and AlexS. Test and localisation
pm AlexE, Kev, Yuahm Robin Project Management, Security, User support, Release

The schedule looks like:

Topic Date Purpose
Kickoff 2021-05-01 Zoom meeting to discuss the task list
Priority Complete 2021-06-30 Priority features done
Bavaria Week-end 2021-09-18 Fun meet-up. Hiking, talking, boozing
Code Complete 2021-09-30 Finish all shipping features
RC1 2021-10-21 First cut
Code Freeze/RC2 2021-11-15 No more changes
GM 2021-12-15 Shipped

The detailed task/priority list is still below. I hope each group will complete their Priority tasks by 2021-06-30. If we have to drop lower priority tasks to make the schedule, we will. If we hit Priority Complete in late June, success is certain.

The commitment of time per contributor should be of the order of 100 hours. I suspect the metadata group tasks will require more than 100 hours per contributor. I will work on/with any group as requested in addition to dealing with users, release engineering and project management.

Please be aware that the task list is based on open issues on 2021-04-13. New issues will be reported during the life of the project.

  Meaning Priority Description
Required 1 Deliver by Priority Complete
Required 2 Deliver by Code Complete
Optional 3 Will drop if necessary
Objective 4 Not a task
Cannot 5 Not scheduled/no resource
Issue Category Priority Title
#1557 code 1 C++ Modernisation Luis
#1538 code 1 32/64 bit support in Exiv2 Pydera
#1235 code 1 Add ARM job to Travis-CI
#1228 code 1 UBSan violations
#898 code 1 Add Safe::cast
#1355 code 1 Simplify cmake/compilerFlags.cmake
#1062 code 1 Investigate JPG2000 check_boxes
#950 code 1 CVE infinite loop in Exiv2::Jp2Image::encodeJp2Header
#1537 code 2 Fix special case of box size == 1 in jp2image.cpp
#1525 code 2 Replace jp2image with bmffimage
#1501 code 2 make uninstall leaves artefacts in standard locations
#1542 code 2 Function name free() in class EXIV2API DataBuf
#1406 code 2 fcf-protection only exists on x86
#1405 code 2 readMetadata - improve warning & error handling
#1404 code 2 kerAliasesNotSupported: Please sendXMP packet to ...
#1348 code 2 Move utilities out of the Image class
#1217 code 2 CMake does not check for duplicates
#917 code 2 ABI compatibility checker
#1320 code 2 clang-tidy: sort includes LLVM style
#1022 code 2 Add RAII Exiv2::Library to init/free resources
#956 code 2 Add compiler flags to test software security
#1514 code 3 Replace every EXV_ symbol with EXIV2_
#1417 code 3 Improve console output under Windows for UTF-8
#1060 code 3 Replace deprecated SHGetFolderPathA
#565 code 3 Test for alpha bit detection in VP8L chunks
#1552 code 4 Refactoring Tasks for v1.00
#1279 code 5 Unicode Basic Multilingual Plane

#1505 metadata 1 Unified Metadata Container Milos
#1515 metadata 1 Investigate removing xmpsdk from code base
#1343 metadata 1 Comment and EXIF data cannot be read from PNG files
#1549 metadata 1 Implement Adobe Exif >64k in JPEG
#1026 metadata 1 Rewrite sanity checks for CiffComponent size & offset
#459 metadata 1 add namespace to XMP toolkit - exiv2 and libexempi
#1543 metadata 1 Canon AFInfo Rotation makerNote
#1541 metadata 2 Add pretty printer for GPSImgDirection & GPSTrack
#1524 metadata 2 Error: Directory Samsung2 with 21313 entries considered invalid; not read.
#1419 metadata 2 Support for tag PrintImageMatching 0xc4a5
#1416 metadata 2 Iptc.Application2.DateCreated/TimeCreated formats are different for read/write
#1397 metadata 2 Support for DNG 1.5 Milos
#1334 metadata 2 report Roll Angle Tom
#279 metadata 2 Whitebalance values
#1085 metadata 2 Fix Irix lens names in pentaxmn_int.cpp
#280 metadata 2 Reading temperature values form RAW files
#1081 metadata 3 Changing Tag shortens MakerNote on Olympus
#1401 metadata 3 XmpParser::encode() swallowing exceptions
#1547 metadata 3 fix_1416_iptc_DateCreated
#1556 metadata 5 BigTiff Support
#1506 metadata 5 JXL/codestream support
#710 metadata 5 Extract PreviewImage from JPEG
#904 metadata 5 Canon 1Ds thumbnail colour corrupted
metadata 5 WebP2 Support

#1516 t_l 1 Build is using incorrect include path Fixed
#1510 t_l 1 Crowdin localisation Leonardo
#549 t_l 2 Localisation for v1.00
#1421 t_l 2 Fix regression in Canon lens detection Alex S
#1428 t_l 2 Test all Canon lenses Alex S
#930 t_l 2 Add test for issue 755
#896 t_l 2 Enable git LFS testing
#1550 t_l 4 Development of the python test framework Peter

pm 1 Security Kev
#1447 pm 1 Joining OIN Alex
pm 2 Week-end in Bavaria Alex
#1494 pm 2 Joining KDE community Alex
pm 4 User support Robin
#1521 pm 2 Deprecate exiv2.org and binary builds for v1.00 Robin
#1462 pm 3 Exiv2 Project Status Robin
#1018 pm 4 Exiv2 RoadMap Robin

@1div0
Copy link
Collaborator

1div0 commented Apr 1, 2021

I can and will finish localisation to Slovak & Czech, my primary and secondary languages.

@1div0
Copy link
Collaborator

1div0 commented Apr 1, 2021

Also, please correct JXP/codestream to JXL as an abbreviation of JPEG XL. Thanks.

@clanmills
Copy link
Collaborator

Thanks for your input, Peter (@1div0). This is forming a nice project. Nobody has to do a lot of work and nobody is stuck with this forever and ever.

Apparently KDE have localisation folks to whom we can send the files. I've never discovered how to access that channel.

I'm thinking about bullding RC2 next week to include the JXL/bmff code. The code will then be frozen and I will open branch 'main'. I'm meeting with Luis this afternoon to discuss auto_ptr/C++20.

@1div0
Copy link
Collaborator

1div0 commented Apr 1, 2021

I do have a very good experience with https://crowdin.com for ML based localisation. Probably worth to explore.

@lbschenkel
Copy link
Collaborator

If there's interest, I could volunteer to localize to Brazilian Portuguese (my native language). I also have previous experience with Crowdin.

@clanmills
Copy link
Collaborator

@lbschenkel Thanks. I've updated the task table. You've got that task.

Localisation is documented in README.md. It's driven by gettext technology. I believe this is a commonly used approach.

Some companies provide free services and products to open-source projects. Perhaps you could explore that with Crowdin. As Exiv2 has no revenue, so we are unable to pay for a license.

@lbschenkel
Copy link
Collaborator

I'm also familiar with gettext, so that is not a problem to me. However, Crowdin can be easier for non-developers to contribute translations since they can do it directly via the website compared to editing PO files, which is a bit awkward.

Crowdin is free-of-charge for open source projects, and I could apply on behalf of the project, but only the project lead can do so: https://crowdin.com/page/open-source-project-setup-request

@clanmills
Copy link
Collaborator

I've filled in the application form. I think they expect you to have already created something on their platform. I don't want to start learning about Crowdin. Is it a big job to tool up to use this platform? Can you set up the necessary resources?

@lbschenkel
Copy link
Collaborator

It's not hard. I can volunteer to set the project up as an experiment if you want, and then you can judge if it's something that you think that could be useful to the project.

But I think if I do that, it'll be tied to my account at first. I should be able to transfer it to somebody else's account.

@clanmills
Copy link
Collaborator

I have opened a new issue "Investigate Localisation using Crowdin" for further discussion. #1510

Please be aware this topic "The Future of Exiv2" is circulated to about 200 people who have contributed to Exiv2 since 2004.

@clanmills clanmills added TeamExiv2 and removed request feature request or any other kind of wish labels Apr 11, 2021
@alexvanderberkel
Copy link
Member Author

Hi guys,

I warmly welcome you to the kick-off meeting for our work on Exiv2. Please find attached the prepared meeting slides and meeting details.

The meeting is scheduled for:
2021-04-24 10:00 UTC / 11:00 BST / 12:00 CEST / 23:00 NZDT

Zoom details
https://us02web.zoom.us/j/9129667335?pwd=cjFvWHNnMlpwOExYSE44dHFrdWtyZz09
Meeting ID: 912 966 7335
Passcode: 2ngkdc

The full backlog for the project can be found on GitHub:
#1466

Exiv2 v1.0.pdf

@alexvanderberkel
Copy link
Member Author

I just realized that we cannot use the zoom details of Robin and he is not attending. Please use this link

Video call link: https://meet.google.com/ceo-uckx-uof

@postscript-dev
Copy link
Collaborator

For Exiv2 1.0, the 'code complete' is scheduled for 2021-09-30. Is this still going ahead?

@clanmills
Copy link
Collaborator

When I put that in the schedule, I was intending to drive the project and "make it happen". Code-complete is an important milestone to achieve the goal of releasing v1.00 by the end of 2021.

I've now retired and leaving Alex to manage the project. He's likely to call a zoom meeting to ask everybody about goals to which they can commit and deliver.

My impression is that maintenance, security fixes and user support for Exiv2 is going well. Development of v1.00 is sluggish. However Luis has done the C++11 modernisation and that's important enough to release "main" even although other priority one features (such as Unified Metadata Container and removing xmpsdk from the code base) have not been developed.

@kevinbackhouse
Copy link
Collaborator

kevinbackhouse commented Sep 28, 2021

Do all major API changes need to be finished before 1.0 is released? I would quite like to try switching the standard size type from long to size_t, but that's going to a non-trivial project. I also have some smaller changes that I would like to make, like some ongoing changes to DataBuf and adding readOrThrow to BasicIo.

@clanmills
Copy link
Collaborator

It is very desirable to make big API changes before declaring v1.00. I know Alex has been intending to call a team meeting and this kind of thing should be discussed. Developing every priority one features probably cannot be achieved for several years.

Perhaps a good way forward is to aim to release main as v0.9 with C++11 support next year (say June 2022?). The intention with the v0.9 family is to avoid API changes on a series of half-yearly releases (v0.9.1, v0.9.2 and so on). Eventually reaching v1.00 in a few years time.

@alexvanderberkel
Copy link
Member Author

I think we discuss when we think we are able to release 1.0 and who's committed to work on certain topics. Putting new things into the backlog I would say doesn't put us in a bad situation as long as we have people to work on it.

@alexvanderberkel
Copy link
Member Author

I have set up a call in the past for 1.0 but there was only one participant. That is why I have not scheduled another call so far.

@clanmills
Copy link
Collaborator

I think it's kind of difficult to commit to a schedule without full time engineers. I was able to deliver releases to a schedule because I was working full-time on Exiv2. Everybody has full-time and demanding jobs.

With the Paris Climate Treaty, Ms. Figueres asked the countries "what can you do?" and drew the canvas from the bids. And I think that's how you'll have to approach Exiv2. Ask "What can you do by March 2021?" and that's the spec for v0.9.

Always allow 3 months from Code-complete to GM to allow for several RCs. If code is to ship in June, it has to be written, documented and tested by the end of March.

@clanmills
Copy link
Collaborator

Release notes for v0.27.5: #1018 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants