New gs 10.01.0 from brew yields many subtle failures #7336

PaulWessel · 2023-03-23T16:05:02Z

After doing brew update and brew upgrade I got gs 10.01.0 which seems to yield numerous tiny failures where lines are just processed and rendered slightly differently. So suddenly I have 85 failures on my hand that hours ago were fine. Hoping @GenericMappingTools/core can check to verify this. I guess 85 DVC updates are no big deal but this is kind of annoying.

seisman · 2023-03-23T16:22:33Z

Please note that our Linux CI is still using old gs versions. Thus, if we update the PS files, the Linux CI may break.

PaulWessel · 2023-03-23T17:49:51Z

As a brew master, is there a command that takes me back to 10.00?

seisman · 2023-03-24T06:12:45Z

There is no easy way to install old versions of a homebrew package, but you can try:

change into the homebrew-core directory. For me, it's /opt/homebrew/Library/Taps/homebrew/homebrew-core/Formula
Download the old gs recipe file ghostscript.rb (https://raw.githubusercontent.com/Homebrew/homebrew-core/7ef0570825ab93c0557c3561e66cfeb6b78686be/Formula/ghostscript.rb), which provides gs 10.0.0
Replace the new ghostscript.rb file with the old one
Reinstall ghostscript

PaulWessel · 2023-03-24T13:21:34Z

Step 4 redownloads 10.01 even though I had placed the previous there. Perhaps I don’t know how to “reinstall”

also, when you say you are using “old” gs, are vi talking 9.x?

joa-quim · 2023-03-24T13:53:12Z

Don't remember how but one can tell brew to run a locally stored formula. So you'd run brew on formula after step 2.

PaulWessel · 2023-03-25T13:31:28Z

Just to show some of the craziness happening once 10.01 got used (presumably also via gm). Look at this failing example 23:

Yes it still fails with what you think is not too much given the picture?

/opt/homebrew/bin/gm compare: image difference exceeds limit (0.0233046 > 0.003).

PaulWessel · 2023-03-25T13:45:54Z

Was able to roll back to 10.00 so failures are back to normal.

seisman · 2023-03-26T11:09:35Z

Step 4 redownloads 10.01 even though I had placed the previous there. Perhaps I don’t know how to “reinstall”

also, when you say you are using “old” gs, are vi talking 9.x?

This looks pretty bad. Attached are ex023 PNG images generated by gs 9.54 and gs 10.01.

A significant change is that in the gs 10.01 version, the character "k" in the contour annotations is almost invisible.

However, the PDF version from gs 10.01 seems OK.

ex23.pdf

PaulWessel · 2023-03-26T11:31:02Z

Yes, the PDF look fine - I wonder if this is a gm/gs issue since it makes no sense.

seisman · 2023-03-26T11:55:16Z

It's a gs issue because gm is not used when converting ps to png.

KristofKoch · 2023-07-13T09:07:55Z

Just a heads-up:

Ghostscript […] has been found vulnerable to a critical-severity remote code execution flaw.

The flaw is tracked as CVE-2023-3664, having a CVSS v3 rating of 9.8, and impacts all versions of Ghostscript before 10.01.2, which is the latest available version released three weeks ago.

Source: Bleeping Computer: Critical RCE found in popular Ghostscript open-source PDF library

PaulWessel · 2023-07-13T09:18:09Z

At summit we should build 10.01.2 and see if they have fixed the bugs affecting us.

joa-quim · 2023-07-13T09:25:29Z

Don’t think they have. In that recent report of malfunction from Dan he was using 10.01.2

remkos · 2023-07-19T15:21:42Z

I tried both ghostscript 10.00.0 and 10.01.2 on ex23.ps
ex23.ps

Results are here for ghostscript 10.00.0 and rendering to png, jpg, and plotting on Xwindows.

Results are here for ghostscript 10.01.2 and rendering to png, jpg, and plotting on Xwindows.

remkos · 2023-07-19T16:05:43Z

This patch to ghostscript appears to fix it: https://git.ghostscript.com/?p=ghostpdl.git;a=blobdiff;f=base/gspaint.c;h=b3500c1d5282972dbb5ae1a27452c27644a4af32;hp=d67ef36bce6cacf2d02c547a9f4337a420667ff6;hb=c52b4a80cdfc6d9b7d04f52c4a872717aeeff73c;hpb=772d6f3f65b9453df52fb3f12058613842e3565c

It is not yet applied to ghostscript 10.01.2 that was released in June. But with this patch applied, things are OK again.

PaulWessel · 2023-07-20T19:00:32Z

Thanks to @remkos, macports now delivers a patched gs 10.02.1 that seems to work fine.

seisman · 2023-07-21T02:38:13Z

Do you think we should has a page that lists the working gs versions?

WalterHFSmith · 2023-07-21T05:57:39Z

Yes, insofar as we know enough to make a useful one for most users.

…

On Jul 20, 2023, at 10:38 PM, Dongdong Tian ***@***.***> wrote: Do you think we should has a page that lists the working gs versions? — Reply to this email directly, view it on GitHub <#7336 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APUT6GLYKEYK74TN7M6POWTXRHTSDANCNFSM6AAAAAAWFNEBDE>. You are receiving this because you are on a team that was mentioned.

PaulWessel · 2023-07-21T07:18:24Z

@joa-quim might have good intel on this. Pre 9.56 had a few buggers, then 10.0.0 right?

remkos · 2023-07-21T10:02:59Z

Thanks to @remkos, macports now delivers a patched gs 10.02.1 that seems to work fine.

In fact, it is gs 10.01.2, which is the latest release.
The patch that is applied is this one: https://git.ghostscript.com/?p=ghostpdl.git;a=commit;h=c52b4a80cdfc6d9b7d04f52c4a872717aeeff73c

seisman · 2023-08-24T01:28:14Z

FYI, Ghostscript has released the first candidate release of 10.02.0.

I just built it from the source and ran the GMT full tests. Now I have 13 failing tests (similar to what we're having using gs 9.56.1):

	 22 - doc/scripts/GMT_App_F_stand+_iso+.sh (Failed)
	320 - test/gmt_core/testsuite.sh (Failed)
	442 - test/grdfill/gridfill.sh (Failed)
	489 - test/grdimage/url_map.sh (Failed)
	552 - test/grdvector/met_uv.sh (Failed)
	556 - test/grdvector/shrink.sh (Failed)
	583 - test/greenspline/gspline_7.sh (Failed)
	603 - test/modern/longbasemap.sh (Failed)
	618 - test/movie/movie_indicator_de.sh (Failed)
	624 - test/movie/trajectory5.sh (Failed)
	728 - test/psevents/event_lines.sh (Failed)
	729 - test/psevents/eventlabelbox.sh (Failed)
	730 - test/psevents/events.sh (Failed)

I didn't check all the tests carefully, but the failures look reasonable.

The example ex23 mentioned in #7336 (comment) also looks good with gs 10.02.0.

PaulWessel · 2023-08-24T10:35:36Z

Hi @seisman I am trying to build on my M1 Mac. Run ./configure then make and get at the end:

	/bin/sh <./obj/ldt.tr
Undefined symbols for architecture arm64:
  "_iconv", referenced from:
      _opvp_to_utf8 in gdevopvp.o
  "_iconv_close", referenced from:
      _opvp_to_utf8 in gdevopvp.o
  "_iconv_open", referenced from:
      _opvp_to_utf8 in gdevopvp.o
ld: symbol(s) not found for architecture arm64

Anything I need to initialise first, args to configure ?

seisman · 2023-08-24T10:44:51Z

./configure works for me on Linux and Intel Mac. I don't have M1 Mac at hand, so can't help here.

PaulWessel · 2023-08-24T10:54:40Z

No worries, I think I did do this before so there must be a way. Obviously the gs team do not use Macs I think.

seisman · 2023-08-24T11:29:40Z

Hi @seisman I am trying to build on my M1 Mac. Run ./configure then make and get at the end:
	/bin/sh <./obj/ldt.tr
Undefined symbols for architecture arm64:
  "_iconv", referenced from:
      _opvp_to_utf8 in gdevopvp.o
  "_iconv_close", referenced from:
      _opvp_to_utf8 in gdevopvp.o
  "_iconv_open", referenced from:
      _opvp_to_utf8 in gdevopvp.o
ld: symbol(s) not found for architecture arm64
Anything I need to initialise first, args to configure ?

This post may be helpful: https://stackoverflow.com/questions/57734434/libiconv-or-iconv-undefined-symbol-on-mac-osx

joa-quim · 2023-08-24T13:44:41Z

Tested 10.02.0 on Windows and seems to work. Same number of failures as before.

PaulWessel · 2023-08-24T15:04:51Z

I have about 80 failures. Either tiny things (mixcompose.sh):

or very bad (imgshade.sh):

Not among @seisman's list and I am guessing no among @joa-quim's either. I have a bunch that is related to different reference grids (like the last one here), and probably round off (top one)? Note sure what I should do. What data server do you use for your tests? Cannot be the same as me (oceania)?

joa-quim · 2023-08-24T15:10:52Z

These latest clearly look originated in different grid versions. We shouldn’t have these type of tests.

PaulWessel · 2023-08-24T15:28:11Z

Well, we cannot avoid having plots that need to be compared. I agree we should avoid such test if an alternative way without needing gs, but I dont see how given these types of modules. What we do need is to run these of a static set of data that does not get updated. Remember that lots of tests results from bug reports and we turned those into tests.

joa-quim · 2023-08-24T19:05:44Z

They could use small synthetic grids.

seisman · 2023-09-14T00:51:46Z

gs 10.02.0 is out.

I built the source code and ran the full tests on Linux. Here are a list of failures I have:

	 22 - doc/scripts/GMT_App_F_stand+_iso+.sh (Failed)
	 52 - doc/scripts/GMT_Defaults_1c.sh (Failed)
	235 - doc/examples/ex38/ex38.sh (Failed)
	264 - test/api/apimodern.sh (Failed)
	321 - test/gmt_core/testsuite.sh (Failed)
	380 - test/grd2cpt/equalarea.sh (Failed)
	399 - test/grdcontour/contours.sh (Failed)
	443 - test/grdfill/gridfill.sh (Failed)
	452 - test/grdfilter/threads.sh (Failed)
	455 - test/grdgradient/aspect.sh (Failed)
	471 - test/grdimage/grdcyclic.sh (Failed)
	474 - test/grdimage/grdread.sh (Failed)
	490 - test/grdimage/url_map.sh (Failed)
	553 - test/grdvector/met_uv.sh (Failed)
	557 - test/grdvector/shrink.sh (Failed)
	584 - test/greenspline/gspline_7.sh (Failed)
	600 - test/modern/imagepluscpt.sh (Failed)
	604 - test/modern/longbasemap.sh (Failed)
	617 - test/modern/viewpluscpt.sh (Failed)
	621 - test/movie/movie_indicator_f_ver.sh (Failed)
	625 - test/movie/trajectory5.sh (Failed)
	730 - test/psevents/eventlabelbox.sh (Failed)
	731 - test/psevents/events.sh (Failed)

I didn't check all these failures carefully, but most of them look good to me.

In summary, gs 10.02.0 looks like a working version for GMT.

remkos · 2023-09-17T08:02:09Z

I created a PR for Ghostscript 10.02.0 on macports. It was accepted and merged yesterday, so macports users can get it easily now.

seisman · 2023-09-18T00:11:49Z

FYI, conda and homebrew also updated their ghostscript to 10.02.0

ghomsi123 · 2023-09-20T10:14:22Z

Thanks @seisman for the updates. Transparency is still not working on my MBpro Ventura.
I'm using the 6.4.0 version installed via brew. I've even linked gs10.2.0 with gmt and it still doesn't work.

joa-quim · 2023-09-20T10:16:55Z

With GMT6.4 you need a gs version < 10 to have transparency.

ghomsi123 · 2023-09-20T10:21:46Z

Thanks @joa-quim. Any ideas or process for downgrading gs version < 10?

joa-quim · 2023-09-20T10:29:31Z

Homebrew can do it but I don't remember how (very occasional Mac user)

ghomsi123 · 2023-09-20T10:34:47Z

Surely @seisman and @PaulWessel have tricks to find out!

seisman · 2023-09-20T10:35:49Z

Any ideas or process for downgrading gs version < 10?

It's not easy to downgrade gs to <10, because many gs dependencies are too new to be compatible with gs 9.

I think it's much easier to install the GMT master branch instead. With homebrew, you just need to run

brew install gmt --HEAD

ghomsi123 · 2023-09-20T10:37:53Z

Thanks @seisman. Is the master branch compatible with the current version of gs?

joa-quim · 2023-09-20T10:39:04Z

Can that --HEAD be limited to recent commits only to avoid having to download the ~1 GB repo?

Is the master branch compatible with the current version of gs?

Yes.

seisman · 2023-09-20T10:44:25Z

Can that --HEAD be limited to recent commits only to avoid having to download the ~1 GB repo?

I believe it only downloads the latest commit using git clone --depth=1.

joa-quim · 2023-09-20T10:50:38Z

Ah, that is good. We should probably advertise that in our installing instructions.

seisman · 2023-09-20T10:54:42Z

We should probably advertise that in our installing instructions.

We already did it https://github.com/GenericMappingTools/gmt/blob/master/INSTALL.md#install-via-homebrew

seisman · 2023-09-24T05:50:11Z

Since gs 10.02.0 seems working well with GMT master branch, I think we can close the issue, although we still need to update the baseline images if necessary.

PaulWessel added the bug Something isn't working label Mar 23, 2023

PaulWessel self-assigned this Mar 23, 2023

remkos mentioned this issue Apr 26, 2023

ghostscript: update to 10.01.0 macports/macports-ports#18139

Closed

12 tasks

remkos added the discussion Topics for longer discussion label Jul 21, 2023

seisman mentioned this issue Aug 19, 2023

Bump to gs 9.56.1 conda-forge/ghostscript-feedstock#27

Merged

5 tasks

seisman mentioned this issue Aug 25, 2023

Figure.text: Support non-ASCII characters in the 'text' parameter GenericMappingTools/pygmt#2638

Merged

7 tasks

seisman closed this as completed Sep 24, 2023

seisman mentioned this issue May 11, 2024

pygmt.show_versions should warn if incompatible packages are installed GenericMappingTools/pygmt#3242

Closed

New gs 10.01.0 from brew yields many subtle failures #7336

New gs 10.01.0 from brew yields many subtle failures #7336

Comments

PaulWessel commented Mar 23, 2023

seisman commented Mar 23, 2023

PaulWessel commented Mar 23, 2023

seisman commented Mar 24, 2023

PaulWessel commented Mar 24, 2023

joa-quim commented Mar 24, 2023

PaulWessel commented Mar 25, 2023

PaulWessel commented Mar 25, 2023

seisman commented Mar 26, 2023

PaulWessel commented Mar 26, 2023

seisman commented Mar 26, 2023

KristofKoch commented Jul 13, 2023

PaulWessel commented Jul 13, 2023

joa-quim commented Jul 13, 2023

remkos commented Jul 19, 2023 • edited Loading

remkos commented Jul 19, 2023

PaulWessel commented Jul 20, 2023

seisman commented Jul 21, 2023

WalterHFSmith commented Jul 21, 2023 via email

PaulWessel commented Jul 21, 2023

remkos commented Jul 21, 2023

seisman commented Aug 24, 2023

PaulWessel commented Aug 24, 2023

seisman commented Aug 24, 2023

PaulWessel commented Aug 24, 2023

seisman commented Aug 24, 2023

joa-quim commented Aug 24, 2023

PaulWessel commented Aug 24, 2023

joa-quim commented Aug 24, 2023

PaulWessel commented Aug 24, 2023

joa-quim commented Aug 24, 2023

seisman commented Sep 14, 2023

remkos commented Sep 17, 2023

seisman commented Sep 18, 2023

ghomsi123 commented Sep 20, 2023

joa-quim commented Sep 20, 2023

ghomsi123 commented Sep 20, 2023

joa-quim commented Sep 20, 2023

ghomsi123 commented Sep 20, 2023

seisman commented Sep 20, 2023 • edited Loading

ghomsi123 commented Sep 20, 2023

joa-quim commented Sep 20, 2023

seisman commented Sep 20, 2023

joa-quim commented Sep 20, 2023

seisman commented Sep 20, 2023

seisman commented Sep 24, 2023

remkos commented Jul 19, 2023 •

edited

Loading

seisman commented Sep 20, 2023 •

edited

Loading