append function not working with CAS

[rajeshkz]

I'm trying to append a SAS dataframe to CAS table with the append() function. However this gives me an error "ERROR: Update access is not supported for file PUBLIC.LSTM_TRAINING_INPUT_2_TMP.DATA."
I understand that in CAS the proc append is not supported and that is the cause of the error from append() function. Is there any alternative that would support an append operation with saspy.
I could create a new dataset with sas.dataframe2sasdata() and do a dataset with append=yes option ( "data lib.SAS-dataset (append=yes) ; set .... " ) however with this I'm getting column size mismatch issues so this is not an option.
Is there any other alternative available or a fix to the append() function to support CAS tables?

[tomweber-sas]

I have to say that I'm not sure, I never use CAS. But I can look into it. Can you show me the code you're running that's accessing CAS, so I can tell that I'm doing the same thing you are. Are you using a CAS libname? I'd like to reproduce what your doing to be sure I'm looking at the right thing.

Thanks,
Tom

[rajeshkz]

Thans for the response. My original code is complex, but below is the code snippet that would reproduce the issue.
I'm assigning the CAS libbame and using PUBLIC CAS lib. Pls note that the CAS do not support proc append and hence the cause of the problem.

import os
import os.path
from os import path
import time
import math
import saspy
from saspy.sasdata import SASdata as SASdata
import tempfile
import retaillib.environmentvariables as ev
import pandas as pd
import pytest
from pandas.testing import assert_frame_equal
from importlib import import_module, invalidate_caches
from pathlib import Path
from pprint import pprint
import urllib3

sas_base_machine = "My-Viya4-BaseServerName"
user = "MyUser"
password = "My-Password"
SAS_COMPUTE_CONTEXT = "Data Mining compute context"
sascfg = (
    f"SAS_config_names=['httpsviya']                           \n"
    f"httpsviya = {{'ip'      :  '{sas_base_machine}',         \n"
    f"              'context' : '{SAS_COMPUTE_CONTEXT}',    \n"
    f"              'user' : '{user}',                         \n"
    f"              'pw' : '{password}',                       \n"
    f"              'options' : ['fullstimer', 'memsize=1G']   \n"
    f"            }}\n"
)
temp = tempfile.NamedTemporaryFile(suffix=".py")
temp.close()
temp = open(temp.name, "w")
temp.write(sascfg)
temp.close()
sas = saspy.SASsession(cfgfile=temp.name, verify=False)
init_code = """
    cas casauto; 
    caslib _all_ assign;
    data public.cars;
        set sashelp.cars;
    run;
"""
results_dict = sas.submit(init_code)
# create a pandas dataframe to append to public.cars
df = sas.sasdata2dataframe("cars", "sashelp")
df = pd.DataFrame(df)
dataset_name_ds = sas.sasdata("cars", "public")
# code here gives an error
dataset_name_ds.append(df)
sas.endsas()

The above code Gives the error below.

SAS server started using Context Data Mining compute context with SESSION_ID=3a9f7529-81b9-45e1-8ced-3fe4cdf54a45-ses0000
.......local/lib/python3.10/site-packages/saspy/sasiohttp.py:822: UserWarning: Noticed 'ERROR:' in LOG, you ought to take a look and see if there was a problem
  warnings.warn("Noticed 'ERROR:' in LOG, you ought to take a look and see if there was a problem")
6546  proc append base=public.'cars'n
6547              data=WORK.'_temp_df'n;
ERROR: Update access is not supported for file PUBLIC.CARS.DATA.
6548  run;

NOTE: Statements not processed because of errors noted above.
NOTE: PROCEDURE APPEND used (Total process time):
      real time           0.01 seconds
      user cpu time       0.00 seconds
      system cpu time     0.01 seconds
      memory              479.34k
      OS Memory           28972.00k
      Timestamp           01/27/2023 06:27:28 PM
      Step Count                        18  Switch Count  0
      Page Faults                       0
      Page Reclaims                     32
      Page Swaps                        0
      Voluntary Context Switches        8
      Involuntary Context Switches      0
      Block Input Operations            0
      Block Output Operations           8
      
NOTE: The SAS System stopped processing this step because of errors.

SAS server terminated for SESSION_ID=3a9f7529-81b9-45e1-8ced-3fe4cdf54a45-ses0000

[tomweber-sas]

ok, So what I posted a wile ago seem like it would be the answer for your case!

[tomweber-sas]

Hey, I just went and tried something out, which worked. Don't know if this is the answer for how you're doing it, but it worked w/out issue. The key here was using the outdsopts={} option of df2sd() to use append=yes on the data step; which is a supported way to append to a CAS table, as it turns out.

tom64-7> python3
Python 3.9.12 (main, Apr  5 2022, 06:56:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import saspy
>>> sas = saspy.SASsession()
SAS server started using Context SAS Job Execution compute context with SESSION_ID=ee4bbf21-d0c6-462e-b21f-bb73c97096d2-ses0000
>>> sas
Access Method         = HTTP
SAS Config name       = mwf
SAS Config file       = /opt/tom/github/saspy/saspy/sascfg_personal.py
WORK Path             = /opt/sas/viya/config/var/tmp/compsrv/default/ee4bbf21-d0c6-462e-b21f-bb73c97096d2/SAS_work68B7000000B3_sas-compute-server-405d3735-8048-4542-b7bc-eed6ede4179a-vlrcg/
SAS Version           = V.04.00M0P01262023
SASPy Version         = 4.5.0
Teach me SAS          = False
Batch                 = False
Results               = Pandas
SAS Session Encoding  = utf-8
Python Encoding value = utf_8
SAS process Pid value = 179


>>> sas.saslib('x', 'cas')
12   libname x cas  ;
NOTE: The session CASAUTO connected successfully to Cloud Analytic Services sas-cas-server-default-client using port 5570. The UUID
      is 5cff25d7-c6bb-d747-ac0d-15b5b6342d0c. The user is sastpw and the active caslib is CASUSER(sastpw).
NOTE: The SAS option SESSREF was updated with the value CASAUTO.
NOTE: The SAS macro _SESSREF_ was updated with the value CASAUTO.
NOTE: The session is using 3 workers.
NOTE: Libref X was successfully assigned as follows:
      Engine:        CAS
      Physical Name: 5cff25d7-c6bb-d747-ac0d-15b5b6342d0c


>>> sas.datasets('x')
13   proc datasets dd=x;
NOTE: No tables are available in caslib CASUSER(sastpw) of Cloud Analytic Services.
                                                             Directory

                                        Libref         X
                                        Engine         CAS
                                        Physical Name  5cff25d7-c6bb-d747-ac0d-15b5b6342d0c
                                        Session UUID   5cff25d7-c6bb-d747-ac0d-15b5b6342d0c
                                        Session Name   CASAUTO
                                        Server Host    sas-cas-server-default-client
                                        Server Port    5570
                                        Caslib         CASUSER(sastpw)
WARNING: No matching members in directory.
13 !                     quit;

NOTE: PROCEDURE DATASETS used (Total process time):
      real time           0.42 seconds
      cpu time            0.26 seconds



>>> df = sas.sd2df('cars','sashelp')
df.head()
>>> df.head()
    Make           Model   Type Origin DriveTrain     MSRP  Invoice  EngineSize  Cylinders  Horsepower  MPG_City  MPG_Highway  Weight  Wheelbase  Length
0  Acura             MDX    SUV   Asia        All  36945.0  33337.0         3.5        6.0       265.0      17.0         23.0  4451.0      106.0   189.0
1  Acura  RSX Type S 2dr  Sedan   Asia      Front  23820.0  21761.0         2.0        4.0       200.0      24.0         31.0  2778.0      101.0   172.0
2  Acura         TSX 4dr  Sedan   Asia      Front  26990.0  24647.0         2.4        4.0       200.0      22.0         29.0  3230.0      105.0   183.0
3  Acura          TL 4dr  Sedan   Asia      Front  33195.0  30299.0         3.2        6.0       270.0      20.0         28.0  3575.0      108.0   186.0
4  Acura      3.5 RL 4dr  Sedan   Asia      Front  43755.0  39014.0         3.5        6.0       225.0      18.0         24.0  3880.0      115.0   197.0
>>> ct = sas.df2sd(df, 'cascars','x')
>>> sas.datasets('x')
6535  proc datasets dd=x;
                                                             Directory

                                        Libref         X
                                        Engine         CAS
                                        Physical Name  5cff25d7-c6bb-d747-ac0d-15b5b6342d0c
                                        Session UUID   5cff25d7-c6bb-d747-ac0d-15b5b6342d0c
                                        Session Name   CASAUTO
                                        Server Host    sas-cas-server-default-client
                                        Server Port    5570
                                        Caslib         CASUSER(sastpw)


                                            Number   Number
                                    Member      of       of                               Data
                        #  Name     Type      Rows  Columns  Last Modified                Encoding  Scope

                        1  CASCARS  DATA       428       15  01/27/2023 15:18:14          utf-8     Session
6535!                     quit;

NOTE: PROCEDURE DATASETS used (Total process time):
      real time           0.40 seconds
      cpu time            0.17 seconds



>>> ct2 = sas.df2sd(df, 'cascars','x', outdsopts={'append':'yes'})
>>> ct2.obs()
856
>>> sas.datasets('x')
13022  proc datasets dd=x;
                                                             Directory

                                        Libref         X
                                        Engine         CAS
                                        Physical Name  5cff25d7-c6bb-d747-ac0d-15b5b6342d0c
                                        Session UUID   5cff25d7-c6bb-d747-ac0d-15b5b6342d0c
                                        Session Name   CASAUTO
                                        Server Host    sas-cas-server-default-client
                                        Server Port    5570
                                        Caslib         CASUSER(sastpw)


                                            Number   Number
                                    Member      of       of                               Data
                        #  Name     Type      Rows  Columns  Last Modified                Encoding  Scope

                        1  CASCARS  DATA       856       15  01/27/2023 15:18:14          utf-8     Session
13022!                     quit;

NOTE: PROCEDURE DATASETS used (Total process time):
      real time           0.22 seconds
      cpu time            0.07 seconds



>>>

[rajeshkz]

Thanks for the input. I'll give this a try.