-
-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
codegen / logic issue for "GIXSQLCursorDeclareParams" #88
Comments
re enhancement: as long as the SQLCA is not back-up'd and restored around this it is an actual error because the values "magically change" between invocations and the programs run as if the last statement was an error. Readjusting the generation to do the preparation only once and only directly before the cursor is used the first time would definitely be both more work an enhancement improving performance and the ability to debug actual errors. |
You are right, the current behaviour is definitely a “logic” bug. I also guess that “selective/lazy” initialization should only apply to cursors declared in WORKING-STORAGE (those are the ones referenced in the “startup declarations”. |
I have implemented what follows, let me know if it might work. It is all activated with a new preprocessor flag (
You can look at the attached (preprocessed) program, it is the same code as in #81, only preprocessed with the |
That seems very reasonable - I'd drop the I further suggest to move the complete block to the end of the processed program - this way you do not have that |
You may want to insert the call twice as follows: GIXSQL* EXEC SQL AT :DBS
GIXSQL* OPEN VM2
GIXSQL* END-EXEC.
GIXSQL IF GIXSQL-CI-F-TSQL027A-VM2 = ' ' THEN
GIXSQL PERFORM GIXSQL-CI-P-TSQL027A-VM2
GIXSQL IF SQLCODE = 0
GIXSQL MOVE 'X' TO GIXSQL-CI-F-TSQL027A-VM2
GIXSQL CALL STATIC "GIXSQLCursorOpen" USING
GIXSQL BY REFERENCE SQLCA
GIXSQL BY REFERENCE "TSQL027A_VM2" & x"00"
GIXSQL END-CALL
GIXSQL END-IF
GIXSQL ELSE
GIXSQL CALL STATIC "GIXSQLCursorOpen" USING
GIXSQL BY REFERENCE SQLCA
GIXSQL BY REFERENCE "TSQL027A_VM2" & x"00"
GIXSQL END-CALL
GIXSQL END-IF ... likely better: less code one minor comparision generated more: GIXSQL* EXEC SQL AT :DBS
GIXSQL* OPEN VM2
GIXSQL* END-EXEC.
GIXSQL IF GIXSQL-CI-F-TSQL027A-VM2 = ' ' THEN
GIXSQL PERFORM GIXSQL-CI-P-TSQL027A-VM2
GIXSQL IF SQLCODE = 0
GIXSQL MOVE 'X' TO GIXSQL-CI-F-TSQL027A-VM2
GIXSQL END-IF
GIXSQL END-IF
GIXSQL IF GIXSQL-CI-F-TSQL027A-VM2 = 'X' THEN
GIXSQL CALL STATIC "GIXSQLCursorOpen" USING
GIXSQL BY REFERENCE SQLCA
GIXSQL BY REFERENCE "TSQL027A_VM2" & x"00"
GIXSQL END-CALL
GIXSQL END-IF --> marker is only set if the initialization worked and the cursor access is only done if it did, too. |
I agree, it is simpler and more readable.
In this case I wanted to avoid a jump, it might be more costly than checking the variable before (if needed) jumping, but I guess a lot depends on compiler optimizations.
I am not sure about this : if there is no |
That's already edited out :-) The check with the sqlcode and then checking both ' ' and 'X' seems better in any case.
only my intent to still leave the |
Done, this is the new preprocessed output. |
Rechecked with 2 PCs - the new preprocessed output file is identical to the old one, maybe GH needs another name for it? |
No, "I" need a check, because I uploaded the wrong file, sorry :-) |
You may want to indent the lines within |
In theory, yes, but this would (probably) lead to a hard crash if the program flow is wrong and it - erroneously - performs a
Ok, done |
There should be a test case ensuring this does not happen, and from a quick glance I see no reason why it should: gixsql/runtime/libgixsql/gixsql.cpp Lines 658 to 663 in 6b8ca51
It sets SQLCA as expected.
Yes, that's the point. Looking forward to 1.0.17 with upcoming fixes and features .I guess that may land somewhere next week? |
Yes, that's the release date I am more or less planning. |
- Added support for "smart" cursor initialization (#88) - Added support for EXECUTE prepared-statement INTO #(87) - Fixed a logging problem (#84) - Fixed "wrong generated COBOL in 1.0.16" (#83) - Fixed "missing "close" for spdlog?" (#82) - Added support for using prepared statements in cursors (#81) - Variable length fields indicators are now 32-bit long by default (#80) - Added support for using variable length fields with prepared statements (#79) - Added upport for using group fields in INSERT and SELECT..INTO statements (#6) - Added support for more connection string formats (including ocesql compatibility) (#16) - Added Support for DISCONNECT ALL (#89) - Performed some refactoring to improve code size - Fixed a few memory leaks
Looks like the very good changes you already had for this did not get into 1.0.17? I see |
Have you run gixpp with |
No, of course not, as it isn't mentioned in https://github.com/mridoni/gixsql/releases/tag/v1.0.17 - is there a reason for making this optional (even more: not default)? I'm just executing the gixsql wrapper script (so as a workaround I'll add it there). |
Note: the flag changes the codegen, but does not work, the compiler complains about a bunch of not existing variables and sections:
|
Ok, I will look into it again (and set "smart cursor init" as default). As you know my test passes, so there is probably something in the code you are testing with that exposes a bug not present in my own test code. I'll try to dig deeper. Thanks |
That has exposed bugs for the VARLENGTH32 issue, too, so maybe you get a reproducer "for free". |
I couldn't reproduce a broken |
Think I've found it: it happens if you have the cursor declarations within the program, not within |
Yes, they should, but they didn't... 😞 I will look into it, thanks for your investigation. |
Another issue seen with cursor declarations in The variable |
- Added new Oracle driver, based on ODPI - Added new SQLite driver - All the drivers have been updated and now implement the complete set of supported features - Solution for "PG: issue with prepared statements" (#99) - Solution for "PCursors cannot be re-opened after close" (#98) - Solution for "libgixpp: setStatus is called for errors without DBI parm passed - sets SQLERRM" (#94) - Solution for "error handling (especially for 07001)" (#92) - Solution for "show-stopper bug in pgsql_prepare" (#91) - Solution for "PREPARE does not work with VARLENGTH groups (ocesql compat)" (#79) - Partial solution for "PREPARE does not work with VARLENGTH groups (ocesql compat)" (#68) - Solution for "The PostgreSQL driver needs START TRANSACTION before using cursors" (#14) - Solution for "FR: support EXEC SQL VAR" (#21) - Fixed a bug in "problems with "codegen / logic issue for "GIXSQLCursorDeclareParams" (#88) - Solution for "FR: allow mapping of "NoRecCode"' (#95) - added --no-rec-code parameter to gixpp - Tokens in the parser have been labeled to improve diagnostics (pulled PR #96 by @GitMensch) - Fixed COMP-3 handling in drivers other than PostgreSQL - Rewrote the test suite (still MSTest-based) to dynamically generate a matrix of test to be run on the various platforms/database drivers - Added options for parameter generation in gixpp (-a was removed) - Added new GIXSQL_FIXUP_PARAMS option for runtime, to automatically convert parameter format in prepared statments - "Native" cursors are now the default for the PostgreSQL driver - "Smart" cursor initialization is now the default for all cursors, including those declared in WORKING-STORAGE (-L was removed from gixpp), should fix #101 - Removed dynamic cursor emulation from the ODBC driver when using PostgreSQL
Something is still amiss here: |
Ok, I will look into it: it passed all the test cases, but it is really possible that this particular program flow eluded my tests. Thanks |
I'm still looking for the error at hand, the parameters seem correct as Minor thing I've seen during debugging: |
This could be solved with a bit of refactoring anyway, if necessary: for each cursor there are two paragraphs emitted in the initialization code at the end of the module:
|
Sounds good. Note that if this is about the field information (type, length, address) only, then this is only necessary for items in |
Yes, but at that point it would be more linear to behave like this in every case: the cost of doing it while opening a cursor, compared with the actual "open" on the DBMS side, is almost non-existent. |
Retested: The codegen error with missing sections for cursor reads when the DECLARE is part of |
Yes, a test case would be useful, since in all evidence the one I am using is not catching this. Thanks |
It always feels a bit strange if you take a big program and reduce it to the max for a minimal reproducer... Here it is: IDENTIFICATION DIVISION.
PROGRAM-ID. TE609.
DATA DIVISION.
WORKING-STORAGE SECTION.
*-----------------------------------------------------------*
EXEC SQL BEGIN DECLARE SECTION END-EXEC.
*
01 DBS PIC X(010).
01 TABID PIC S9(018).
01 MODTIME PIC X(026).
77 KEY01-PART PIC 9(008) VALUE 42.
*
EXEC SQL END DECLARE SECTION END-EXEC.
EXEC SQL INCLUDE SQLCA END-EXEC.
*-----------------------------------------------------------*
*-----------------------------------------------------------*
PROCEDURE DIVISION.
TMAIN SECTION.
P030-00.
*
EXEC SQL AT :DBS
DECLARE CSKEY01N CURSOR FOR
SELECT TABID, MODTIME FROM TAB
WHERE
KEY01 >= (
LPAD(:KEY01-PART , 008, '0')
)
ORDER BY KEY01 ASC
END-EXEC.
*
IF SQLCODE NOT = 0
GO TO P031-99.
*
EXEC SQL
OPEN CSKEY01N
END-EXEC.
*
IF SQLCODE NOT = 0
GO TO P031-99.
*
P031-20.
*
EXEC SQL
FETCH CSKEY01N
INTO
:TABID,
:MODTIME
END-EXEC.
EXEC SQL
CLOSE CSKEY01N
END-EXEC.
*
P031-99.
GOBACK. Producing a preparsed TE609.cob and compiling that leads to:
which is correct as the section was not inserted. |
Ok, thanks, I'll look into it as soon as I can. |
This was actually quite easy: I forgot to remove this test: gixsql/libgixpp/TPESQLProcessing.cpp Lines 648 to 649 in 927cbcf
but fixing this exposed another problem: it seems that this declaration is not properly parsed:
and preprocessing fails. If I change it to (only modifying the level):
it works (even though I obviously cannot run it). I will try to see if I can solve this too and release it with 1.0.18a |
Commenting out this line in the lexer makes it work, but even if it shouldn't be a problem, I will have (at least) to re-run all the tests for regressions. gixsql/libgixpp/gix_esql_scanner.ll Line 1296 in 927cbcf
|
|
Ok, thanks. I will try removing |
... and best include a new test with both. |
It would be nice to intercept the |
The separate issue is created, I suggest to let the scanner pass the level 78 if it is already in the parser and error out there, or just leave it as "skipped" (then the parser will error correctly with "unknown parameter"). It is fine to go with level 77 "as soon as possible", which seems (as no work other than the test case is missing) to be 1.0.18.a; if it is more work you may want to error in the parser for that, too. |
The packages are being built, tomorrow I will run some more tests. |
- Changed logging system initializationto avoid possible crashes - Fixed some spelling problems in the documentation - Fixed some memory leaks in the runtime - Solution for "sqlstate by driver may set 00000 for errors (in GixSQL?)" (#114) - Additional fix for "codegen / logic issue for "GIXSQLCursorDeclareParams" (#88) - Allowing variable with level 77/78 in the parser
During testing with a patched 1.0.18 (including the possibly bad workaround for #119) I got "strange errors". gixsql/gixsql-tests/data/TSQL017E.cbl Lines 24 to 42 in 5ed37df
Currently the codegen is somewhat like GIXSQL* EXEC SQL AT :DBS
GIXSQL* DECLARE CRSR017D CURSOR FOR
GIXSQL* SELECT TABID, MTIME FROM TAB
GIXSQL* WHERE
GIXSQL* KEY01 >= (
GIXSQL* MYFUNC(:KEY-ID-1 , :KEY-ID-2 , 099, '1')
GIXSQL* )
GIXSQL* ORDER BY KEY01 ASC
GIXSQL* END-EXEC.
*
IF SQLCODE NOT = 0
GO TO P003-C.
*
GIXSQL* EXEC SQL
GIXSQL* OPEN CRSR017D
GIXSQL* END-EXEC.
GIXSQL IF GIXSQL-CI-F-TSQL017E-CRSR017D = ' ' THEN
GIXSQL PERFORM GIXSQL-CI-P-TSQL017E-CRSR017D
GIXSQL IF SQLCODE = 0
GIXSQL MOVE 'X' TO GIXSQL-CI-F-TSQL017E-CRSR017D
GIXSQL END-IF
GIXSQL END-IF
GIXSQL IF GIXSQL-CI-F-TSQL017E-CRSR017D = 'X' THEN
GIXSQL CALL "GIXSQLCursorOpen" USING
GIXSQL BY REFERENCE SQLCA
GIXSQL BY REFERENCE "TSQL017E_CRSR017D" & x"00"
GIXSQL END-CALL
GIXSQL END-IF.
*
IF SQLCODE NOT = 0
GO TO P003-C. It could be solved by adjusting the codegen for the "DECLARE in PROCEDURE DIVISION" case as follows (duplicating a the declare part) GIXSQL* EXEC SQL AT :DBS
GIXSQL* DECLARE CRSR017D CURSOR FOR
GIXSQL* SELECT TABID, MTIME FROM TAB
GIXSQL* WHERE
GIXSQL* KEY01 >= (
GIXSQL* MYFUNC(:KEY-ID-1 , :KEY-ID-2 , 099, '1')
GIXSQL* )
GIXSQL* ORDER BY KEY01 ASC
GIXSQL* END-EXEC.
GIXSQL IF GIXSQL-CI-F-TSQL017E-CRSR017D = ' ' THEN
GIXSQL PERFORM GIXSQL-CI-P-TSQL017E-CRSR017D
GIXSQL IF SQLCODE = 0
GIXSQL MOVE 'X' TO GIXSQL-CI-F-TSQL017E-CRSR017D
GIXSQL END-IF
GIXSQL ELSE
GIXSQL MOVE 0 TO SQLCODE
GIXSQL END-IF
*
IF SQLCODE NOT = 0
GO TO P003-C.
*
GIXSQL* EXEC SQL
GIXSQL* OPEN CRSR017D
GIXSQL* END-EXEC.
GIXSQL IF GIXSQL-CI-F-TSQL017E-CRSR017D = ' ' THEN
GIXSQL PERFORM GIXSQL-CI-P-TSQL017E-CRSR017D
GIXSQL IF SQLCODE = 0
GIXSQL MOVE 'X' TO GIXSQL-CI-F-TSQL017E-CRSR017D
GIXSQL END-IF
GIXSQL END-IF
GIXSQL IF GIXSQL-CI-F-TSQL017E-CRSR017D = 'X' THEN
GIXSQL CALL "GIXSQLCursorOpen" USING
GIXSQL BY REFERENCE SQLCA
GIXSQL BY REFERENCE "TSQL017E_CRSR017D" & x"00"
GIXSQL END-CALL
GIXSQL END-IF.
*
IF SQLCODE NOT = 0
GO TO P003-C. This is important for current code that checks the SQLCODE after the DECLARE and has an appropriate error handling there; it is even more important when the program is called the second time and the last time the program was called SQLCODE left with 100 for "no record found" - and now "seem to fail" on the |
Ok, I will look into it, but it might be a week or so until I find the time to work on it again, this is why I released v1.0.18a now. (I am also trying to release v1.0.18a for Gix-IDE, but today uploads on GitHub seem to be really really slow) |
Sounds fair. |
I'm aware that there's a lot ongoing from reading your dev-blog (mostly related to Gix Debugger and to packaging of Gix IDE)... Could you come up with a change for that (as it is still the reason the codebase in question - previously using ocesql - cannot be switched over to GixSQL) and release 1.0.18b, possibly with including #127? Edit: While this seems like an easier issue to solve it would not make the codebase working because of #119, which, as I've understood has a target of GixSQL 1.0.19. |
A program using cursors get the following generated directly before the first section (I hope it isn't before / in
DECLARATIVES
if those exist);The issues here are:
_gixsqlCursorDeclare
always ending inDBERR_CURSOR_EXISTS
and SQLCA being updated with error codes.Suggestion to solve this:
If this takes "too long for the next version" (I personally see it as big drawback in GixSQL and therefore suggest to fix that "soon") then generate a
MOVE SQLCA TO SQLCA-GIXSQL-BACKUP
directly afterESQL STARTUP DECLARATIONS (START)
and aMOVE SQLCA-GIXSQL-BACKUP TO SQLCA-GIXSQL-BACKUP
directly beforeESQL STARTUP DECLARATIONS (END)
.The text was updated successfully, but these errors were encountered: