read_sas() failure examples on compressed data

[ajdamico]

hi, not sure if files like these should be supported? i didn't find anything in the haven or ReadStat docs saying compressed .sas7bdat files are not allowed. thanks

tf <- tempfile()
download.file( "http://www2.census.gov/programs-surveys/ahs/2002/AHS_2002_Value_Label_Package.zip" , tf , mode = 'wb' )
z <- unzip( tf )
sas_file <- grep( "\\.sas7bdat$" , z , value = TRUE )
haven::read_sas( grep( "\\.sas7bdat$" , z , value = TRUE ) )
# sas7bdat::read.sas7bdat(sas_file)
# Error in sas7bdat::read.sas7bdat(sas_file) : 
  # file contains compressed data


tf <- tempfile()
download.file( "http://www2.census.gov/programs-surveys/ahs/2004/AHS_2004_Value_Labels_Package.zip" , tf , mode = 'wb' )
z <- unzip( tf )
sas_file <- grep( "\\.sas7bdat$" , z , value = TRUE )
haven::read_sas( sas_file )
# sas7bdat::read.sas7bdat(sas_file)
# Error in sas7bdat::read.sas7bdat(sas_file) : 
  # file contains compressed data

[BioStatMatt]

There is no reason why they shouldn't be supported, but they aren't supported by read.sas7bdat. Not sure if ReadStat has figured it out, but Parso library reads all types of compressed files.

…

On Jan 15, 2017, at 5:31 AM, Anthony Damico ***@***.***> wrote: hi, not sure if files like these should be supported? i didn't find anything in the haven or ReadStat docs saying compressed .sas7bdat files are not allowed. thanks tf <- tempfile() download.file( "http://www2.census.gov/programs-surveys/ahs/2002/AHS_2002_Value_Label_Package.zip" , tf , mode = 'wb' ) z <- unzip( tf ) sas_file <- grep( "\\.sas7bdat$" , z , value = TRUE ) haven::read_sas( grep( "\\.sas7bdat$" , z , value = TRUE ) ) # sas7bdat::read.sas7bdat(sas_file) # Error in sas7bdat::read.sas7bdat(sas_file) : # file contains compressed data tf <- tempfile() download.file( "http://www2.census.gov/programs-surveys/ahs/2004/AHS_2004_Value_Labels_Package.zip" , tf , mode = 'wb' ) z <- unzip( tf ) sas_file <- grep( "\\.sas7bdat$" , z , value = TRUE ) haven::read_sas( sas_file ) # sas7bdat::read.sas7bdat(sas_file) # Error in sas7bdat::read.sas7bdat(sas_file) : # file contains compressed data — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[ajdamico]

thanks, but it also fails for these files

devtools::install_github("biostatmatt/sas7bdat.parso")
tf <- tempfile()
download.file( "http://www2.census.gov/programs-surveys/ahs/2002/AHS_2002_Value_Label_Package.zip" , tf , mode = 'wb' )
z <- unzip( tf , exdir = tempdir() )
sas_file <- grep( "\\.sas7bdat$" , z , value = TRUE )
sas7bdat.parso::read.sas7bdat.parso( sas_file )
#Error in .jcall(sps, "S", "s7b2csv", s7bfile, csvfile) : 
#  java.lang.IndexOutOfBoundsException: Index: 0, Size: 0

[BioStatMatt]

Yes, that uses an old version of Parso. I do have the updates Parso code (Java) somewhere. I'll dig it up if you want to give it a spin.

…

On Jan 15, 2017, at 7:39 AM, Anthony Damico ***@***.***> wrote: thanks, but it also fails for these files devtools::install_github("biostatmatt/sas7bdat.parso") tf <- tempfile() download.file( "http://www2.census.gov/programs-surveys/ahs/2002/AHS_2002_Value_Label_Package.zip" , tf , mode = 'wb' ) z <- unzip( tf , exdir = tempdir() ) sas_file <- grep( "\\.sas7bdat$" , z , value = TRUE ) sas7bdat.parso::read.sas7bdat.parso( sas_file ) #Error in .jcall(sps, "S", "s7b2csv", s7bfile, csvfile) : # java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[rogerjdeangelis]

I want to thank Evan and Matt for opening up SAS datasets for R programmers.

Matt your documentation on SAS datasets has opened my eyes and haven has added speed and flexibility.

The code below reads and writes compressed SAS datasets with free
commercial software in combination with open source R(haven) or Python
Unfortunately WPS does not create a SAS data
set within the R script.

It would be wonderful if sas7bdat or haven provided a
write SAS capability, the missing piece.
WPS does not limit the size of the SAS dataset from R in the
free edition.

utl_compressed_r.zip

Converting CSV text numbers to double floats is very expensive so
building a binary SAS dataset has major advantages. I would initially
only support 8 byte float and of course character variables.

[hadley]

Follow progress at WizardMac/ReadStat#21 - this will be added to haven soon after it's added to ReadStat.