Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

T1 optimisations jpeg2000 #172

Closed
gcode-importer opened this issue Aug 29, 2012 · 16 comments
Closed

T1 optimisations jpeg2000 #172

gcode-importer opened this issue Aug 29, 2012 · 16 comments
Assignees
@gcode-importer
Labels
bug Priority-High

Comments

@gcode-importer
Copy link

Originally reported on Google Code with ID 172

Hi all,

Here are some patches to openjpeg 1.5 which speed up the T1 encoding a bit:

http://carlh.net/software/openjpeg

On the plus side, encoding a large (1998x1080) image takes about 75% of the time with
the patches compared to the release openjpeg (and the result is bitwise identical).

Caveats: no speed-up to decoding, and very limited regression testing.

It's a work in progress, so it's a bit rough round the edges.

Comments welcome.

Reported by malaterre on 2012-08-29 06:59:59
@gcode-importer
Copy link
Author 

Attaching as patch directly

Reported by malaterre on 2012-08-29 07:00:32

- _Attachment: [current.patch](https://storage.googleapis.com/google-code-attachments/openjpeg/issue-172/comment-1/current.patch)_

@gcode-importer
Copy link
Author

Reported by malaterre on 2012-09-10 09:30:43

@gcode-importer
Copy link
Author 

The code is not using C90 convention, which makes it hard to apply as-is:

/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c: In function
't1_enc_updateflags':
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:400:3: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:402:3: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:410:3: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:412:3: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:428:4: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:441:4: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c: In function
't1_enc_sigpass_step':
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:493:2: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c: In function
't1_enc_refpass_step':
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:745:2: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c: In function
't1_enc_clnpass_step':
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:983:2: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:996:3: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c: In function
'allocate_buffers':
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:1330:2: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:1346:11: error:
pointer targets in initialization differ in signedness [-Werror=pointer-sign]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:1346:2: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:1352:4: error:
pointer targets in assignment differ in signedness [-Werror=pointer-sign]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:1359:5: error:
pointer targets in assignment differ in signedness [-Werror=pointer-sign]
/home/mathieu/Projects/openjpeg/branches/openjpeg-1.5/libopenjpeg/t1.c:1360:3: error:
ISO C90 forbids mixed declarations and code [-Werror=edantic]
cc1: all warnings being treated as errors
make[2]: *** [applications/mj2/CMakeFiles/frames_to_mj2.dir/__/__/libopenjpeg/t1.c.o]
Error 1

Reported by malaterre on 2012-09-10 09:31:57

@gcode-importer
Copy link
Author 

Attached a new version of the patch with a few tweaks and also C90 compatibility.

Reported by [email protected] on 2012-09-10 10:56:56

- _Attachment: [patch2](https://storage.googleapis.com/google-code-attachments/openjpeg/issue-172/comment-4/patch2)_

@gcode-importer
Copy link
Author 

Also, I just noticed that I didn't put back support for the J2K_CCP_CBLKSTY_VSC flag.
 Although this doesn't seem to be set anywhere within libopenjpeg; could it be removed?

Reported by [email protected] on 2012-09-10 10:58:15

@gcode-importer
Copy link
Author

Reported by malaterre on 2014-03-14 15:14:23

@gcode-importer
Copy link
Author 

Hi,

I updated the patch for tag 2.1.0, thought I'll share it with everyone.
Still no support J2K_CCP_CBLKSTY_VSC & limited non regression tests (I use a custom
set of images as I do not have access to external svn proxy at work - proxy issues)

Please find some time ratios below (using also updated patch from Issue 220). The whole
encoding time is taken into account, not only t1 enc. Input images are 8bit grayscale
images encoded using 9/7 wavelet. Timings include 8bit->32bit conversion.

0,878843317 (linux x86 gcc4.4)
0,878999168 (linux x64 gcc4.4)
0,906621081 (linux armv7 gcc4.6)
0,924342459 (linux armv5 gcc4.6)
0,878114055 (windows x86 vc7)
0,832016242 (windows x86 vc8)
0,880686491 (windows x64 vc8)
0,829962365 (windows x86 vc10)
0,884205368 (windows x64 vc10)

Regards,
Matthieu

Reported by mayeut on 2014-05-28 07:07:11

- _Attachment: [openjpeg-2.1.0-t1-enc.patch](https://storage.googleapis.com/google-code-attachments/openjpeg/issue-172/comment-7/openjpeg-2.1.0-t1-enc.patch)_

@gcode-importer
Copy link
Author 

Hi Matthieu,

Thanks for the patch. Is there a way to put back support for J2K_CCP_CBLKSTY_VSC in
the patch ? 

J2K_CCP_CBLKSTY_VSC is actually set through the "parameters->mode" field

Antonin

Reported by detonin on 2014-06-02 09:15:45

@gcode-importer
Copy link
Author 

Hi Antonin,

I do not feel very comfortable doing this. For now, this work has not required much
understanding of the algorithm, it was only a port from Carl's patch to 2.1.0 release.
Maybe Carl can lend a hand on this one ?

I'll see what I can do but no promises (& I probably won't have time to start digging
into this before august)

Reported by mayeut on 2014-06-04 17:48:48

@gcode-importer
Copy link
Author 

+1 for getting this patch into the mainline

Reported by boxerab on 2014-07-18 03:44:00

@gcode-importer
Copy link
Author 

Regarding VSC, is this feature used very much? I understand that it leads to poorer
compression ratios than default t1 encoding.

Reported by boxerab on 2014-07-18 03:45:29

@gcode-importer
Copy link
Author

Reported by detonin on 2014-07-18 11:07:41

  • Labels added: Priority-High
  • Labels removed: Priority-Medium

@gcode-importer
Copy link
Author 

For those who may find it useful, I have taken Matthieu's version of the patch and created
a branch for it in my OpenJPEG mirror:

https://github.com/OpenJPEG/openjpeg/tree/carls-t1-patch

Reported by boxerab on 2014-09-15 20:52:22

@gcode-importer gcode-importer self-assigned this Jun 11, 2015
@rouault
Copy link
Collaborator

rouault commented May 18, 2017

openjpeg-2.1.0-t1-enc.patch can be found in
http://storage.googleapis.com/google-code-attachments/openjpeg/issue-172/comment-7/openjpeg-2.1.0-t1-enc.patch (referenced from original Google issue https://code.google.com/archive/p/openjpeg/issues/172)

@rouault
Copy link
Collaborator

rouault commented May 18, 2017

On MAPA.tif 13498x9944 3 bands 8bit depth, opj_compress goes from 68 s (user time) on bare 2.1.0 to 61 s on 2.1.0+openjpeg-2.1.0-t1-enc.patch, so 10% decrease.
On 8c05f00a-ae05-4dd5-bdc7-a1b5eed4ebfb.tif 15595x111128 3 bands 8bit depth, opj_compress goes from 83s to 75s, so 10% decrease too

rouault added a commit to rouault/openjpeg that referenced this issue May 20, 2017
@rouault
T1: use more compact flags to optimize cache usage in encoder passes. (…
4f63db5 
…uclouvain#172)

Ported from Carl Hetherington work (actually through Matthieu Darbois's port
on top of OpenJPEG 2.1.0)

Can reduce total encoding time by 10-15%

WARNING: VSC mode is not implemented, and so is a temporary regression
that must be fixed.
rouault added a commit to rouault/openjpeg that referenced this issue May 20, 2017
@rouault
T1: use more compact flags to optimize cache usage in encoder passes. (…
022cdb6 
…uclouvain#172)

Ported from Carl Hetherington work (actually through Matthieu Darbois's port
on top of OpenJPEG 2.1.0)

Can reduce total encoding time by 10-15%

WARNING: VSC mode is not implemented, and so is a temporary regression
that must be fixed.
rouault added a commit to rouault/openjpeg that referenced this issue May 20, 2017
@rouault
T1: use more compact flags to optimize cache usage in encoder passes. (…
b1c0c37 
…uclouvain#172)

Ported from Carl Hetherington work (actually through Matthieu Darbois's port
on top of OpenJPEG 2.1.0)

Can reduce total encoding time by 10-15%

WARNING: VSC mode is not implemented, and so is a temporary regression
that must be fixed.
rouault added a commit to rouault/openjpeg that referenced this issue May 23, 2017
@rouault
Tests: test opj_compress in VSC mode (related to uclouvain#172)
3beccb2
rouault added a commit to rouault/openjpeg that referenced this issue May 23, 2017
@rouault
Tests: test opj_compress in VSC mode (related to uclouvain#172)
02c2579
rouault added a commit to rouault/openjpeg that referenced this issue May 23, 2017
@rouault
Tests: test opj_compress in VSC mode (related to uclouvain#172)
2d2c368
rouault added a commit that referenced this issue May 23, 2017
@rouault
Merge pull request #935 from rouault/add_compress_vsc_test
6e97d87 
Tests: test opj_compress in VSC mode (related to #172)
rouault added a commit to rouault/openjpeg that referenced this issue May 23, 2017
@rouault
T1: use more compact flags to optimize cache usage in encoder passes. (…
cd12414 
…uclouvain#172)

Ported from Carl Hetherington work (actually through Matthieu Darbois's port
on top of OpenJPEG 2.1.0)

Can reduce total encoding time by 10-15%

WARNING: VSC mode is not implemented, and so is a temporary regression
that must be fixed.
@rouault
Copy link
Collaborator

rouault commented Jun 2, 2017

Implementation in #945

@rouault rouault closed this as completed Jul 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@gcode-importer gcode-importer

Labels
bug Priority-High
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rouault @gcode-importer