shift() on complex test 2067.4. fails on CRAN

[jangorecki]

Test 2067.4 is failing on (at the moment only there) r-devel-windows-x86_64. Recent change to handling complex NA type seems quite likely to be related.

Test 2067.4 ran without errors but failed check that x equals y:
  > x = shift(list(z, 1:3))
  First 2 of 2 (type 'list'):
  [[1]]
  [1] NA 1+3i 2+2i
  
  [[2]]
  [1] NA 1 2
  
  > y = list(c(NA, z[1:2]), c(NA, 1:2))
  First 2 of 2 (type 'list'):
  [[1]]
  [1] NA 1+3i 2+2i
  
  [[2]]
  [1] NA 1 2

[jangorecki]

based on the info from mailing list behavior is not yet finally decided, therefore probably best to escape test for newer R

Fails on 2023-09-29 r85235 ucrt but not on 2023-09-28 r85227

[MichaelChirico]

best to escape test for newer R

Even better is to write a test using base functionality -- if the base test fails, skip our test.

[mmaechler]

based on the info from mailing list behavior is not yet finally decided, therefore probably best to escape test for newer R

I think you misunderstood what has been in the mailing list.
I think the current R-devel (printing/formatting of complex) will stay as it is; ditto for the coercion of numeric & logical NA to complex. Unfortunately, the behavior of when NAs remain NAs (rather than "other" NaN's from C point of view) in computations is much more platform dependent than some of us (you and I, e.g.) have assumed previously.

[jangorecki]

best to escape test for newer R

Even better is to write a test using base functionality -- if the base test fails, skip our test.

base cannot really be tested here, the only thing there we could test base for is that c() coerces NA to NA_complex_.

[jangorecki]

shift produces complex(real=NA, imaginary=0) but it needs to produce complex(real=NA, imaginary=NA)

[jangorecki]

It looks like current master already fixes this problem. So we just need to find find out which commit fixes that and cherry pick to hotfix branch

[MichaelChirico]

Could someone please link the relevant mailing list thread for future reference? Thanks.

[MichaelChirico]

Thanks Jan for poking me on fixing this. Documenting my process for tracking down 5f9df4d as the issue that fixed things.

1. Updated r-devel to r85472
2. Ensure I can reproduce the CRAN issue on 1.14.8
3. Ensure I can reproduce it's being fixed on current master
4. Simplify the issue to the minimal possible code, I found Im() was 0 on broken versions but NA on fixed versions, so I used is.na(Im(data.table::shift(0+1i)))
5. git bisect as follows:

# (starting from 'master')
# TRUE <--> is.na=TRUE, FALSE <--> is.na=FALSE
git bisect start --term-bad=TRUE --term-good=FALSE
git bisect TRUE
git checkout 1.14.8
git bisect FALSE

# At each bisection commit, run
${R_DEVEL_BIN}/R CMD INSTALL . && ${R_DEVEL_BIN}/Rscript -e "is.na(Im(data.table::shift(0+1i)))"
# Then run 'git bisect TRUE' or 'git bisect FALSE' matching the `Rscript` output

[MichaelChirico]

Found the thread: https://stat.ethz.ch/pipermail/r-devel/2023-September/082864.html

[MichaelChirico]

best to escape test for newer R

Even better is to write a test using base functionality -- if the base test fails, skip our test.

base cannot really be tested here, the only thing there we could test base for is that c() coerces NA to NA_complex_.

Hmm, can't we use as.complex(NA) instead?

I see

# R 4.3.2
Im(as.complex(NA))
# [1] NA

# r-devel r85472
Im(as.complex(NA))
# [1] 0

So just using c(as.complex(NA), z[1:2]) works on "all" versions of R AFAICT.

[jangorecki]

already fixed in devel version, 65edf39 fixes that for hotfix release