-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Quartz][Oracle] Unable to store job details when using Quarkus Quartz Scheduler with Oracle #43720
Comments
/cc @machi1990 (quartz), @manovotn (quartz,scheduler), @mkouba (quartz,scheduler) |
I suppose this is an issue on my end but as of now I am unable to run the reproducer - namely the docker image refuses to start 🤔 |
@manovotn just run the test |
I think that it makes sense to introduce the @dcdh Would you care to send a pull request? |
Well, the container is started with the test; I guess it's the devservice timing out, yea. Anyway, I agree with Martin that this looks like we initially overlooked the oracle delegate. If you were to send a PR, we'd be happy to review it :) |
Ok, I will provide a PR tonight. Keep you in touch. |
OracleDelegate is declared inside a subpackage of |
Yes, that's what I meant when I said " |
Sorry Martin, monday is hard today |
No problem. It's the same for me! :D |
I made the changes, but I do not thinks it is the root cause no matter the use of But, the new way Oracle is handling boolean representation between Oracle23 and version below. In my reproducer using the default oracle devservice I am unable to find a documentation related on Boolean new type and breaking changes :/ Could you validate it on your side too, to validate what I am saying. That mind complex ... what should we do ? |
Ok I found it from the release note available here https://www.oracle.com/fr/database/technologies/appdev/jdbc-downloads.html for
The worst part of this sentence is the fact that it will not check the database type used to store the boolean ... breaking change :/ |
Multiple axes of resolution:
On my side I maintain a legacy infrastructure which is not using Oracle 23c and I guess it should be the case for a lot of organizations. What should we do ? |
We should definitely file a new issue in the Quartz repo. Unfortunately, the community was not very active until recently. So we will see what happens next. If I understand it correctly it will work once you change the type of the boolean columns from Also pls send the pull request with the |
@mkouba I confirm that remplacing VARCHAR2(1) to BOOLEAN is working with ok I will raise an issue on Quartz Repo. I was thinking, meanwhile, what do you think, if we add the specific property to deactivate the feature this way And also add a log and update the guide on Oracle and Quarkus Quartz. Because, even if we merge the PR, it is a partial fixed. No one will be able to test or run in dev mode with Quartz and Oracle together without specifying a lower version of Oracle. Or maybe another way: downgrade the version of Oracle used by the dev service. I do not know the legal implication to go from an oracle-free to oracle-xe ? Please let me know. |
+1
+1
Have you verified that I wonder if we shouldn't log a warning and instruct the user to use BOOLEAN instead?
Hm, the guide is postgres-based. So I'm not quite sure where to put this note.
I have no idea. By the way I've just notice that CC @yrodiere |
By changing VARCHAR2(1) to BOOLEAN we will breaks all clients (organizations) which are not using Oracle 23 databases (I am on this case). |
@dcdh You can use the |
It is doable.
|
Waiting for @yrodiere feedbacks |
Let me sum up so that we're sure on the same page ;-)
Workarounds:
Action items
|
@mkouba is it the responsibility to the developer to import by himself the quartz tables ? I guess it is the case, because I did not found any way from Quarkus to create all tables if not present, but I may be wrong - could you confirm please ? So in this case we can do nothings programmatically to check the column format used for Oracle Boolean representation. Thus, we can only update the Quarkus Oracle guide and put a reference to it from the Quarkus Quartz guide. What do you think about it ? |
Yes, it is.
+1 |
Ok so I will just update the guide. Keep you in touch |
-1 to change default behavior of JDBC drivers, regardless of how Hibernate ORM behaves. It would just be confusing as documentation on the Oracle JDBC driver would advertise one behavior, and in Quarkus we'd have another one. +1 to document the breaking change in the Oracle JDBC driver when using Oracle 23+. FWIW before using
See #43462 |
@yrodiere thanks. I will made a PR on the guideline this weekend. |
I need to updated two guides How, where to do it regarding oracle ? |
We don't have a specific guide for Oracle, just one for all datasources. Which does include subsections about Oracle, but well... not sure this is the right place. Since we're talking about a breaking change, maybe an entry in the relevant migration guide would make sense? |
@dcdh another note I am using Quarkus Quartz with Oracle 23c and using Quarkus Liquibase and it works fine. Liquibase properly creates the table for Oracle 23C at least I have not seen any errors. <databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.1.xsd
http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<property name="table_prefix" value="QRTZ_"/>
<property name="blob_type" value="BYTEA" dbms="postgresql"/>
<property name="blob_type" value="BLOB"/>
<changeSet id="quartz-init" author="quartz">
<createTable tableName="${table_prefix}LOCKS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="LOCK_NAME" type="VARCHAR(40)">
<constraints nullable="false"/>
</column>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, LOCK_NAME" tableName="${table_prefix}LOCKS"/>
<createTable tableName="${table_prefix}FIRED_TRIGGERS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="ENTRY_ID" type="VARCHAR(95)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="INSTANCE_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="FIRED_TIME" type="BIGINT">
<constraints nullable="false"/>
</column>
<column name="SCHED_TIME" type="BIGINT">
<constraints nullable="false"/>
</column>
<column name="PRIORITY" type="INTEGER">
<constraints nullable="false"/>
</column>
<column name="STATE" type="VARCHAR(16)">
<constraints nullable="false"/>
</column>
<column name="JOB_NAME" type="VARCHAR(200)"/>
<column name="JOB_GROUP" type="VARCHAR(200)"/>
<column name="IS_NONCONCURRENT" type="BOOLEAN"/>
<column name="REQUESTS_RECOVERY" type="BOOLEAN"/>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, ENTRY_ID" tableName="${table_prefix}FIRED_TRIGGERS"/>
<createIndex tableName="${table_prefix}FIRED_TRIGGERS" indexName="IDX_${table_prefix}FT_INST_JOB_REQ_RCVRY">
<column name="SCHED_NAME"/>
<column name="INSTANCE_NAME"/>
<column name="REQUESTS_RECOVERY"/>
</createIndex>
<createIndex tableName="${table_prefix}FIRED_TRIGGERS" indexName="IDX_${table_prefix}FT_J_G">
<column name="SCHED_NAME"/>
<column name="JOB_NAME"/>
<column name="JOB_GROUP"/>
</createIndex>
<createIndex tableName="${table_prefix}FIRED_TRIGGERS" indexName="IDX_${table_prefix}FT_JG">
<column name="SCHED_NAME"/>
<column name="JOB_GROUP"/>
</createIndex>
<createIndex tableName="${table_prefix}FIRED_TRIGGERS" indexName="IDX_${table_prefix}FT_T_G">
<column name="SCHED_NAME"/>
<column name="TRIGGER_NAME"/>
<column name="TRIGGER_GROUP"/>
</createIndex>
<createIndex tableName="${table_prefix}FIRED_TRIGGERS" indexName="IDX_${table_prefix}FT_TG">
<column name="SCHED_NAME"/>
<column name="TRIGGER_GROUP"/>
</createIndex>
<createIndex tableName="${table_prefix}FIRED_TRIGGERS" indexName="IDX_${table_prefix}FT_TRIG_INST_NAME">
<column name="SCHED_NAME"/>
<column name="INSTANCE_NAME"/>
</createIndex>
<createTable tableName="${table_prefix}CALENDARS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="CALENDAR_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="CALENDAR" type="${blob_type}">
<constraints nullable="false"/>
</column>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, CALENDAR_NAME" tableName="${table_prefix}CALENDARS"/>
<createTable tableName="${table_prefix}PAUSED_TRIGGER_GRPS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, TRIGGER_GROUP" tableName="${table_prefix}PAUSED_TRIGGER_GRPS"/>
<createTable tableName="${table_prefix}SCHEDULER_STATE">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="INSTANCE_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="LAST_CHECKIN_TIME" type="BIGINT">
<constraints nullable="false"/>
</column>
<column name="CHECKIN_INTERVAL" type="BIGINT">
<constraints nullable="false"/>
</column>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, INSTANCE_NAME" tableName="${table_prefix}SCHEDULER_STATE"/>
<createTable tableName="${table_prefix}JOB_DETAILS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="JOB_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="JOB_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="DESCRIPTION" type="VARCHAR(250)"/>
<column name="JOB_CLASS_NAME" type="VARCHAR(250)">
<constraints nullable="false"/>
</column>
<column name="IS_DURABLE" type="BOOLEAN">
<constraints nullable="false"/>
</column>
<column name="IS_NONCONCURRENT" type="BOOLEAN">
<constraints nullable="false"/>
</column>
<column name="IS_UPDATE_DATA" type="BOOLEAN">
<constraints nullable="false"/>
</column>
<column name="REQUESTS_RECOVERY" type="BOOLEAN">
<constraints nullable="false"/>
</column>
<column name="JOB_DATA" type="${blob_type}"/>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, JOB_NAME, JOB_GROUP" tableName="${table_prefix}JOB_DETAILS"/>
<createIndex tableName="${table_prefix}JOB_DETAILS" indexName="IDX_${table_prefix}J_GRP">
<column name="SCHED_NAME"/>
<column name="JOB_GROUP"/>
</createIndex>
<createIndex tableName="${table_prefix}JOB_DETAILS" indexName="IDX_${table_prefix}J_REQ_RECOVERY">
<column name="SCHED_NAME"/>
<column name="REQUESTS_RECOVERY"/>
</createIndex>
<createTable tableName="${table_prefix}TRIGGERS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="JOB_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="JOB_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="DESCRIPTION" type="VARCHAR(250)"/>
<column name="NEXT_FIRE_TIME" type="BIGINT"/>
<column name="PREV_FIRE_TIME" type="BIGINT"/>
<column name="PRIORITY" type="INTEGER"/>
<column name="TRIGGER_STATE" type="VARCHAR(16)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_TYPE" type="VARCHAR(8)">
<constraints nullable="false"/>
</column>
<column name="START_TIME" type="BIGINT">
<constraints nullable="false"/>
</column>
<column name="END_TIME" type="BIGINT"/>
<column name="CALENDAR_NAME" type="VARCHAR(200)"/>
<column name="MISFIRE_INSTR" type="smallint"/>
<column name="JOB_DATA" type="${blob_type}"/>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" tableName="${table_prefix}TRIGGERS"/>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_C">
<column name="SCHED_NAME"/>
<column name="CALENDAR_NAME"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_G">
<column name="SCHED_NAME"/>
<column name="TRIGGER_GROUP"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_JG">
<column name="SCHED_NAME"/>
<column name="JOB_GROUP"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_N_G_STATE">
<column name="SCHED_NAME"/>
<column name="TRIGGER_GROUP"/>
<column name="TRIGGER_STATE"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_N_STATE">
<column name="SCHED_NAME"/>
<column name="TRIGGER_NAME"/>
<column name="TRIGGER_GROUP"/>
<column name="TRIGGER_STATE"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_NEXT_FIRE_TIME">
<column name="SCHED_NAME"/>
<column name="NEXT_FIRE_TIME"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_NFT_MISFIRE">
<column name="SCHED_NAME"/>
<column name="MISFIRE_INSTR"/>
<column name="NEXT_FIRE_TIME"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_NFT_ST">
<column name="SCHED_NAME"/>
<column name="TRIGGER_STATE"/>
<column name="NEXT_FIRE_TIME"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_NFT_ST_MISFIRE">
<column name="SCHED_NAME"/>
<column name="MISFIRE_INSTR"/>
<column name="NEXT_FIRE_TIME"/>
<column name="TRIGGER_STATE"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_NFT_ST_MISFIRE_GRP">
<column name="SCHED_NAME"/>
<column name="MISFIRE_INSTR"/>
<column name="NEXT_FIRE_TIME"/>
<column name="TRIGGER_GROUP"/>
<column name="TRIGGER_STATE"/>
</createIndex>
<createIndex tableName="${table_prefix}TRIGGERS" indexName="IDX_${table_prefix}T_STATE">
<column name="SCHED_NAME"/>
<column name="TRIGGER_STATE"/>
</createIndex>
<createTable tableName="${table_prefix}BLOB_TRIGGERS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="BLOB_DATA" type="${blob_type}"/>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" tableName="${table_prefix}BLOB_TRIGGERS"/>
<createTable tableName="${table_prefix}SIMPROP_TRIGGERS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="STR_PROP_1" type="VARCHAR(512)"/>
<column name="STR_PROP_2" type="VARCHAR(512)"/>
<column name="STR_PROP_3" type="VARCHAR(512)"/>
<column name="INT_PROP_1" type="INTEGER"/>
<column name="INT_PROP_2" type="INTEGER"/>
<column name="LONG_PROP_1" type="BIGINT"/>
<column name="LONG_PROP_2" type="BIGINT"/>
<column name="DEC_PROP_1" type="NUMERIC(13,4)"/>
<column name="DEC_PROP_2" type="NUMERIC(13,4)"/>
<column name="BOOL_PROP_1" type="BOOLEAN"/>
<column name="BOOL_PROP_2" type="BOOLEAN"/>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" tableName="${table_prefix}SIMPROP_TRIGGERS"/>
<createTable tableName="${table_prefix}CRON_TRIGGERS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="CRON_EXPRESSION" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="TIME_ZONE_ID" type="VARCHAR(80)"/>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" tableName="${table_prefix}CRON_TRIGGERS"/>
<createTable tableName="${table_prefix}SIMPLE_TRIGGERS">
<column name="SCHED_NAME" type="VARCHAR(120)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_NAME" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="TRIGGER_GROUP" type="VARCHAR(200)">
<constraints nullable="false"/>
</column>
<column name="REPEAT_COUNT" type="BIGINT">
<constraints nullable="false"/>
</column>
<column name="REPEAT_INTERVAL" type="BIGINT">
<constraints nullable="false"/>
</column>
<column name="TIMES_TRIGGERED" type="BIGINT">
<constraints nullable="false"/>
</column>
</createTable>
<addPrimaryKey columnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" tableName="${table_prefix}SIMPLE_TRIGGERS"/>
<addForeignKeyConstraint baseTableName="${table_prefix}TRIGGERS" constraintName="${table_prefix}TRIGGERS_SCHED_NAME_FKEY" baseColumnNames="SCHED_NAME, JOB_NAME, JOB_GROUP" referencedTableName="${table_prefix}JOB_DETAILS" referencedColumnNames="SCHED_NAME, JOB_NAME, JOB_GROUP"/>
<addForeignKeyConstraint baseTableName="${table_prefix}SIMPLE_TRIGGERS" constraintName="${table_prefix}SIMPLE_TRIGGERS_SCHED_NAME_FKEY" baseColumnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" referencedTableName="${table_prefix}TRIGGERS" referencedColumnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP"/>
<addForeignKeyConstraint baseTableName="${table_prefix}CRON_TRIGGERS" constraintName="${table_prefix}CRON_TRIGGERS_SCHED_NAME_FKEY" baseColumnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" referencedTableName="${table_prefix}TRIGGERS" referencedColumnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP"/>
<addForeignKeyConstraint baseTableName="${table_prefix}SIMPROP_TRIGGERS" constraintName="${table_prefix}SIMPROP_TRIGGERS_SCHED_NAME_FKEY" baseColumnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" referencedTableName="${table_prefix}TRIGGERS" referencedColumnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP"/>
<addForeignKeyConstraint baseTableName="${table_prefix}BLOB_TRIGGERS" constraintName="${table_prefix}BLOB_TRIGGERS_SCHED_NAME_FKEY" baseColumnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP" referencedTableName="${table_prefix}TRIGGERS" referencedColumnNames="SCHED_NAME, TRIGGER_NAME, TRIGGER_GROUP"/>
</changeSet>
</databaseChangeLog>
|
My PR was accepted at Quartz to add an ORacle 23 script. |
Given it's going to be properly documented on the Quartz side in the next version of Quartz, I think we can close this one. As for adding it to the migration guide, I think we cannot really track all the breaking changes of all the components we support. You will have to refer to the appropriate release notes of each project. Now maybe we should have a summary of the component updates in the migration guides when we release. I'll see if it's possible to extract this info without too much hassle. |
Describe the bug
While doing a migration from an old application using quartz 2 (from my memory) to the last version of quarkus, I notice an issue when the quartz job is stored in the database.
It failed because in qrtz_job_details boolean likes IS_DURABLE is stored using one varchar (0 or 1) type and it expected 5 varchar type for TRUE or FALSE.
Expected behavior
Following the reproducer associated with this issue:
Actual behavior
The reproducer fails to start because at startup the quartz job definition is stored and an sql issue is thrown regarding varchar length too small to store boolean value representation.
How to Reproduce?
FYI, the init script is coming from
tables_oracle.sql
provided by quartz dependency. I've just commented thedelete
anddrop table
blocs because it was failing at startup.Output of
uname -a
orver
Linux 2a02-8428-dff8-c601-234b-8c10-a3c4-2308.rev.sfr.net 6.10.10-200.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 18:26:09 UTC 2024 x86_64 GNU/Linux
Output of
java -version
openjdk version "21.0.4" 2024-07-16 OpenJDK Runtime Environment (Red_Hat-21.0.4.0.7-2) (build 21.0.4+7) OpenJDK 64-Bit Server VM (Red_Hat-21.0.4.0.7-2) (build 21.0.4+7, mixed mode, sharing)
Quarkus version or git rev
3.15.1
Build tool (ie. output of
mvnw --version
orgradlew --version
)Apache Maven 3.9.6 (Red Hat 3.9.6-6)
Additional information
It failed to store the job because it expect a boolean column definition using a 5 varchar length to store TRUE or FALSE as string.
The oracle table definition has not changed from many years and it used one varchar to store boolean value.
We should keep this definition.
When using an oracle db-kind, the
QuarkusStdJDBCDelegate
is used. I guess this issue is coming from it. Maybe we should provide a custom implementation for oracle (which is not the case).I guess to fix it, in
QuartzProcressor
the buildstep guessDriver should be updated fromto
particular code
with
QuarkusOracleDelegate
having the same beahvior than other QuarkusdatasourceKind
DelegateThe text was updated successfully, but these errors were encountered: