Uploading file to dataset with RECFM=FBA shifts lines one char to the right

[ghost]

If RECFM=FBA and LRECL=80 for the dataset and there are any lines in the file that are 80 characters long, the last character gets truncated on upload, resulting in the following error from the z/OSMF API:

{
    "category": 6,
    "rc": 8,
    "reason": 520,
    "message": "fwrite() error",
    "stack": "  1 3B903F38+00000096 DatasetWriter::throwRWException(int,int,const char*)\n  2 3B8F7D20+000004BA DatasetWriter::copyTextFrom(ServletInputStream&)\n  3 3B8F2510+00000A3C PutDatasetServlet::doPut()\n  4 3B90AA98+000000DC TsoServlet::service()\n  5 3B937E40+0000017E TsoServlet::run()\n  6 3B937818+00000386 main\n  7 0DA3AF08+0000127E CEEVROND\n  8 3B0F1390+000000B4 EDCZHINV\n  9 0D8D4C68+000001C6 CEEBBEXT\n",
    "details": [
        "EDC5003I Truncation of a record occurred during an I/O operation. (errno2=0xC0460022)",
        "__amrc: __last_op=151 __code=0x00000000"
    ]
}

[leofang-zOSMF]

Could you please upload your file here so that I can download it and recreate the issue in house?

[uzust01]

Long story short: Either:

• Use a variable block format.
• Check the length of the line before submitting it.
• Use longer LRECL.
• Avoid the control character record format (FB instead of FBA).

Additional explanation and links
FBA means fixed length (blocked, ANSI format). There does not seem to be any conversion of the data you are passing, but an ANSI/ISO control character is added to the first byte of the record for printing purposes (or tapes and punchcards, which are not relevant here). This generates an extra space, sometimes more than one or less, depending on the control character. It can be added by using some JES2 or JES3 defaults or set for each data set.
MVS JCL Reference - Page 467: CONTROL parameter
IBM Doc - ISO/ANSI control characters
The control character can be removed by using additional JCLs with options like REMOVECC (Ask Professor Sort - Page 25), but not while writing to the data set and this can affect the record.

This does not seem like any kind of issue - neither Zowe-CLI nor z/OSMF. Functionality matches what is documented.

[leofang-zOSMF]

OK so as my understanding, you will close use this issue since it works as design. Then I will not spend extra effort on it.

[tucker01]

Hey @leofang-zOSMF I don't believe we have a resolution yet. The problem seems to be that if you download a FBA member using z/OSMF and then upload the member the control character is added. This doesn't seem like desired behavior. I'm hoping we can get the exact scenario from the person who initially posted the issue on slack.

[tucker01]

I was able to reproduce the problem with a dataset with the following attributes:

$ zowe files ls ds "public.fba.test" -a
-
  dsname: PUBLIC.FBA.TEST
  blksz:  6160
  catnm:  ICF.USERID.USERCAT4
  cdate:  2019/10/15
  dev:    3390
  dsntp:  PDS
  dsorg:  PO
  edate:  ***None***
  extx:   1
  lrecl:  80
  migr:   NO
  mvol:   N
  ovf:    NO
  rdate:  2019/10/15
  recfm:  FBA
  sizex:  211
  spacu:  TRACKS
  used:   1
  vol:    TSU008
  vols:   TSU008

I had a line in the dataset:

 BROWSE           PUBLIC.FBA.TEST(TEST) - 01.05    Line 0000000000 Col 001 079
 Command ===>                                                  Scroll ===> CSR 
********************************* Top of Data **********************************
hey this is a line in my FBA dataset that goes all the way to the end 000100011 
******************************** Bottom of Data ********************************

If I download (using the text) header, I see the following:

$ zowe files dl ds "PUBLIC.FBA.TEST(TEST)" -f test.fba
Data set downloaded successfully.
Destination: test.fba
$ zowe files upload ftds test.fba "public.fba.test(test)"
file_to_upload: 1
success:        0
error:          1
skipped:        0


success: false
from:    C:\Users\me\projects\scratch\test.fba
to:      PUBLIC.FBA.TEST(TEST)
error:
  message:
          z/OSMF REST API Error:
      Rest API failure with HTTP(S) status 500
      category: 6
      rc:       8
      reason:   520
      message:  fwrite() error
      details:
        - EDC5003I Truncation of a record occurred during an I/O operation. (errno2=0xC0460022)
        - __amrc: __last_op=151 __code=0x00000000

Dataset member now looks like this:

 BROWSE       PUBLIC.FBA.TEST(TEST) - 01.06    Line 0000000000 Col 001 079
 Command ===>                                                  Scroll ===> CSR 
********************************* Top of Data **********************************
 hey this is a line in my FBA dataset that goes all the way to the end 00010001 
******************************** Bottom of Data ********************************

Notice the missing 1

@tjohnsonBCM or @zFernand0, correct me if I'm wrong, but we have confirmed this through postman (eliminating anything zowe cli might be doing).

[jshewbert]

Good morning. I am the person that originally posted the concern on slack.

@tucker01 is correct. The concern is not that the first character is “reserved”. The concern is that every time a use the z/OSMF API (via Zowe CLI) to upload a file into a FBA dataset an extra character is added and everything is “shifted” one column to the right, truncating records in my case.

[tucker01]

@leofang-zOSMF, were you able to reproduce the problem?

[leofang-zOSMF]

@tucker01, as uzust01 commented 13 days ago, it seems that FBA type has some special rules. Could you please have a try with Windows FTP to upload a file into a FBA dataset to see the results?

[uzust01]

@tucker01 @leofang-zOSMF @jshewbert @tjohnsonBCM
I tried to FTP a file with less than 80 characters record length and download it again using FTP both times - the same thing happens. An empty character that does not exist in the original uploaded files is added to each line of the downloaded file.

When I copy from FBA to FB data set in ISPF it also adds an empty character to each record.

[jshewbert]

@uzust01 I do not have those same results. If I use SFTP to download a file from a FBA data set and then upload the same file back into the same FBA data set, it DOES NOT add a character.

I have tried SFTP from FB to FBA and from FBA to FB data sets. It does not add characters in either scenario.

BTW, we are using Co:Z for our OpenSSH implementation on z/OS.

[ghost]

From my read of this, one should be able to 'round trip' an FBA file (i.e. download and then upload it). That seems a very reasonable requirement.

[leofang-zOSMF]

Hi everyone, could one of you please open an RFA to z/OSMF with below link:
https://www.ibm.com/developerworks/rfe/execute?use_case=submitRfe

Due to the special attribute of ANSI format，the current REST API adds one extra charactor. This is not a defect, but I agree with this should be a reasonable requirement. Maybe we could use a new header statement to control adding that extra charactor or not. Let me know the RFE number after you submit. Thanks a lot.

[jshewbert]

Done
https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=137769

[github-actions]

Thank you for creating a bug report. If you haven't already, please ensure you have provided steps to reproduce it and as much context as possible.

[JTonda]

New link to issue: https://ibm-z-hardware-and-operating-systems.ideas.ibm.com/ideas/ZOS-I-2696