Use BigInt for integer primary key in sqlite for diesel-cli

.github/workflows/ci.yml

@@ -260,7 +260,7 @@ jobs:
         env:
           BACKEND: ${{ matrix.backend }}
         run: |
-           (cd examples/${{ matrix.backend }} && rustup run ${{ matrix.rust }} bash test_all)
+          (cd examples/${{ matrix.backend }} && rustup run ${{ matrix.rust }} bash test_all)
 
       - name: Test migrations-internals
         shell: bash

CHANGELOG.md

@@ -3,9 +3,9 @@
 All user visible changes to this project will be documented in this file.
 This project adheres to [Semantic Versioning](http://semver.org/), as described
 for Rust libraries in [RFC #1105](https://github.com/rust-lang/rfcs/blob/master/text/1105-api-evolution.md)
-For any named minimal supported Rust version we guarantee that it is possible to build Diesel with the 
-default features enabled using some set of dependencies. Those set of dependencies is not necessarily 
-an up to date version of the specific dependency. We check this by using the unstable `-Z minimal-version` cargo flag. 
+For any named minimal supported Rust version we guarantee that it is possible to build Diesel with the
+default features enabled using some set of dependencies. Those set of dependencies is not necessarily
+an up to date version of the specific dependency. We check this by using the unstable `-Z minimal-version` cargo flag.
 Increasing the minimal supported Rust version will always be coupled at least with a minor release.
 
 ## Unreleased
@@ -19,14 +19,15 @@ Increasing the minimal supported Rust version will always be coupled at least wi
 * Support for connection instrumentation. This allows to inspect any query run by your application
 * Logging in diesel-cli
 * Support for libsqlite3-sys 0.28
+* Add `sqlite-integer-primary-key-is-bigint` configuration option, usable with SQLite 3.37 or above, allowing to use `BigInt` for `INTEGER PRIMARY KEY` columns in SQLite for tables without the `WITHOUT ROWID` attribute ([SQLite doc](https://www.sqlite.org/lang_createtable.html#rowid)).
 
 ### Changed
 
 * The minimal officially supported rustc version is now 1.70.0
 
 ## [2.1.0] 2023-05-26
 
-### Changed 
+### Changed
 
 * The minimal officially supported rustc version is now 1.65.0
 
@@ -48,7 +49,7 @@ Increasing the minimal supported Rust version will always be coupled at least wi
 
 ## [2.0.4] 2023-04-18
 
-## Fixed 
+## Fixed
 
 * Workaround the missing name resolution in rust-analyzer. This should fix type inference for some diesel queries. (It remains broken for queries containing `.filter()`/`.inner_join()`/`.left_join()`. These require fixes in rust-analyzer itself)
 * Fixed a bug that could lead to inserting null values instead of empty values for custom sqlite types
@@ -59,9 +60,9 @@ Increasing the minimal supported Rust version will always be coupled at least wi
 
 * Support for `libsqlite3-sys` 0.26
 
-## [diesel_derives 2.0.2] 2023-03-13 
+## [diesel_derives 2.0.2] 2023-03-13
 
-## Fixed 
+## Fixed
 
 * Fixing the fallout of a breaking change from `quote` by not using their internal API
 
@@ -130,7 +131,7 @@ Increasing the minimal supported Rust version will always be coupled at least wi
   in such a way to support constructing a dynamic value depending on this type.
 
 * Added a `without-deprecated` feature that unconditionally disables deprecated items.
-  Use this feature flag to verify that none of your dependencies is setting 
+  Use this feature flag to verify that none of your dependencies is setting
   the `with-deprecated` flag internally.
 
 * Added support for PostgreSQL's `SIMILAR TO` and `NOT SIMILAR TO`.
@@ -154,7 +155,7 @@ Increasing the minimal supported Rust version will always be coupled at least wi
 
 * Diesel CLI will now generate SQL type definitions for SQL types that are not supported by diesel out of the box. It's possible to disable this behavior via the `generate_missing_sql_type_definitions` config option.
 
-* Added an option to `#[derive(Insertable)]` that let you insert `NULL` values instead of `DEFAULT` values for `Option<T>` 
+* Added an option to `#[derive(Insertable)]` that let you insert `NULL` values instead of `DEFAULT` values for `Option<T>`
 
 * Added support for all the derive attributes being inside `#[diesel(...)]`
 
@@ -259,12 +260,12 @@ Increasing the minimal supported Rust version will always be coupled at least wi
   card implementations for types implementing `Queryable<ST, DB>` or `QueryableByName<DB>`
   so non generic code does not require any change. For generic code you likely need to
   replace a trait bound on `Queryable<ST, DB>` with a trait bound on `FromSqlRow<ST, DB>`
-  and a bound to `QueryableByName<DB>` with `FromSqlRow<Untyped, DB>`. 
+  and a bound to `QueryableByName<DB>` with `FromSqlRow<Untyped, DB>`.
 
 * CLI flags of `only-tables` and `except-tables` are now interpreted as regular expressions.
   Similarly, `only_tables` and `except_tables` in `diesel.toml` are treated as regular expressions.
 
-* Now you can sort column fields by name with the `column-sorting` option. 
+* Now you can sort column fields by name with the `column-sorting` option.
   It can be set to either `ordinal_position` (default) or `name`.
   This ensures stable sorting even if columns are removed and re-added.
 
@@ -277,25 +278,25 @@ Increasing the minimal supported Rust version will always be coupled at least wi
 
 * `TypeMetadata::MetadataLookup` is now `?Sized`.
 
-* Multiple implementations of `Connection<Backend=Pg>` are now possible 
+* Multiple implementations of `Connection<Backend=Pg>` are now possible
   because of the new `PgMetadataLookup` trait.
 
 * For the `Pg` backend, `TypeMetadata::MetadataLookup` has changed to `dyn PgMetadataLookup`.
 
-* Diesel's migration framework was rewritten from the ground. Existing migrations continue to 
+* Diesel's migration framework was rewritten from the ground. Existing migrations continue to
   be compatible with the rewrite, but code calling into `diesel_migrations` requires an update.
   See the [migration guide](2-0-migration) for details.
 
 * `eq_any()` now emits a `= ANY()` expression for the postgresql backend instead of `IN()`
 * `ne_all()` now emits a `!= ALL()` expression for the postgresql backend instead of `NOT IN()`
-* The sqlite backend now uses a single batch insert statement if there are now default values present 
+* The sqlite backend now uses a single batch insert statement if there are now default values present
   in the values clause
 
 * The MySQL connection is using the CLIENT_FOUND_ROWS from now on. This means that updating rows without changing any values will return the number of matched rows (like most other SQL servers do), as opposed to the number of changed rows.
 
-* The definition of `ToSql::to_sql` and `QueryFragment::walk_ast` has changed to allow serializing values without 
+* The definition of `ToSql::to_sql` and `QueryFragment::walk_ast` has changed to allow serializing values without
   copying the value itself. This is useful for database backends like sqlite where you can directly share a buffer
-  with the database. Beside of the changed signature, existing impls of this trait should remain unchanged in almost 
+  with the database. Beside of the changed signature, existing impls of this trait should remain unchanged in almost
   all cases.
 
 * The `PIPES_AS_CONCAT` sql_mode is no longer set
@@ -342,14 +343,14 @@ queries or set `PIPES_AS_CONCAT` manually.
 * We've refactored our type level representation of nullable values. This allowed us to
   fix multiple long standing bugs regarding the correct handling of nullable values in some
   corner cases (#104, #2274)
-  
+
 * Parenthesis are now inserted around all infix operations provided by diesel's `ExpressionMethods` traits
 
 * Queries containing a `distinct on` clause check now on compile time that a compatible order clause was set.
 
 * Implementations of custom SQLite SQL functions now check for panics
 
-* `diesel print-schema` now generates `Array<Nullable<ST>>` rather than `Array<ST>` for Postgres Array types. Existence of 
+* `diesel print-schema` now generates `Array<Nullable<ST>>` rather than `Array<ST>` for Postgres Array types. Existence of
   `NULL` values in database arrays would previously result in deserialization errors. Non-nullable arrays are now opt
   in (by schema patching).
 
@@ -363,8 +364,8 @@ queries or set `PIPES_AS_CONCAT` manually.
 
 * `diesel::pg::upsert` has been deprecated to support upsert queries on more than one backend.
   Please use `diesel::upsert` instead.
-  
-* `diesel::dsl::any` and `diesel::dsl::all` are now deprecated in 
+
+* `diesel::dsl::any` and `diesel::dsl::all` are now deprecated in
    favour of `ExpressionMethods::eq_any()` and `ExpressionMethods::ne_all()`
 
 

diesel_cli/Cargo.toml

@@ -36,7 +36,7 @@ pq-sys = { version = ">=0.4, <0.6.0", optional = true }
 diffy = "0.3.0"
 regex = "1.0.6"
 serde_regex = "1.1"
-diesel_table_macro_syntax = {version = "0.1", path = "../diesel_table_macro_syntax"}
+diesel_table_macro_syntax = { version = "0.1", path = "../diesel_table_macro_syntax" }
 syn = { version = "2", features = ["visit"] }
 tracing = "0.1"
 tracing-subscriber = { version = "0.3.10", features = ["env-filter"] }

diesel_cli/src/cli.rs

@@ -148,6 +148,18 @@ pub fn build_cli() -> Command {
                         .num_args(0..=1)
                         .require_equals(true),
                 )
+                .arg(
+                    Arg::new("sqlite-integer-primary-key-is-bigint")
+                        .long("sqlite-integer-primary-key-is-bigint")
+                        .requires("SCHEMA_RS")
+                        .action(ArgAction::SetTrue)
+                        .help(
+                            "For SQLite 3.37 and above, detect `INTEGER PRIMARY KEY` columns as `BigInt`, \
+                             when the table isn't declared with `WITHOUT ROWID`.\n\
+                             See https://www.sqlite.org/lang_createtable.html#rowid for more information.\n\
+                             Only used with the `--diff-schema` argument."
+                        ),
+                )
                 .arg(
                     Arg::new("table-name")
                         .index(2)
@@ -289,6 +301,16 @@ pub fn build_cli() -> Command {
                 .action(clap::ArgAction::Append)
                 .number_of_values(1)
                 .help("A list of derives to implement for every automatically generated SqlType in the schema, separated by commas."),
+        )
+        .arg(
+            Arg::new("sqlite-integer-primary-key-is-bigint")
+                .long("sqlite-integer-primary-key-is-bigint")
+                .action(ArgAction::SetTrue)
+                .help(
+                    "For SQLite 3.37 and above, detect `INTEGER PRIMARY KEY` columns as `BigInt`, \
+                     when the table isn't declared with `WITHOUT ROWID`.\n\
+                     See https://www.sqlite.org/lang_createtable.html#rowid for more information."
+                ),
         );
 
     let config_arg = Arg::new("CONFIG_FILE")

diesel_cli/src/config.rs

@@ -90,6 +90,8 @@ pub struct PrintSchema {
     pub generate_missing_sql_type_definitions: Option<bool>,
     #[serde(default)]
     pub custom_type_derives: Option<Vec<String>>,
+    #[serde(default)]
+    pub sqlite_integer_primary_key_is_bigint: Option<bool>,
 }
 
 impl PrintSchema {

diesel_cli/src/infer_schema_internals/inference.rs

@@ -3,7 +3,7 @@ use diesel::result::Error::NotFound;
 use super::data_structures::*;
 use super::table_data::*;
 
-use crate::config::Filtering;
+use crate::config::{Filtering, PrintSchema};
 
 use crate::database::InferConnection;
 use crate::print_schema::{ColumnSorting, DocConfig};
@@ -185,10 +185,15 @@ fn get_column_information(
 fn determine_column_type(
     attr: &ColumnInformation,
     conn: &mut InferConnection,
+    #[allow(unused_variables)] table: &TableName,
+    #[allow(unused_variables)] primary_keys: &[String],
+    #[allow(unused_variables)] config: &PrintSchema,
 ) -> Result<ColumnType, crate::errors::Error> {
     match *conn {
         #[cfg(feature = "sqlite")]
-        InferConnection::Sqlite(_) => super::sqlite::determine_column_type(attr),
+        InferConnection::Sqlite(ref mut conn) => {
+            super::sqlite::determine_column_type(conn, attr, table, primary_keys, config)
+        }
         #[cfg(feature = "postgres")]
         InferConnection::Pg(ref mut conn) => {
             use crate::infer_schema_internals::information_schema::DefaultSchema;
@@ -259,11 +264,10 @@ pub fn load_foreign_key_constraints(
 pub fn load_table_data(
     connection: &mut InferConnection,
     name: TableName,
-    column_sorting: &ColumnSorting,
-    with_docs: DocConfig,
+    config: &PrintSchema,
 ) -> Result<TableData, crate::errors::Error> {
     // No point in loading table comments if they are not going to be displayed
-    let table_comment = match with_docs {
+    let table_comment = match config.with_docs {
         DocConfig::NoDocComments => None,
         DocConfig::OnlyDatabaseComments
         | DocConfig::DatabaseCommentsFallbackToAutoGeneratedDocComment => {
@@ -272,15 +276,11 @@ pub fn load_table_data(
     };
 
     let primary_key = get_primary_keys(connection, &name)?;
-    let primary_key = primary_key
-        .iter()
-        .map(|k| rust_name_for_sql_name(k))
-        .collect();
 
-    let column_data = get_column_information(connection, &name, column_sorting)?
+    let column_data = get_column_information(connection, &name, &config.column_sorting)?
         .into_iter()
         .map(|c| {
-            let ty = determine_column_type(&c, connection)?;
+            let ty = determine_column_type(&c, connection, &name, &primary_key, config)?;
 
             let ColumnInformation {
                 column_name,
@@ -298,6 +298,11 @@ pub fn load_table_data(
         })
         .collect::<Result<_, crate::errors::Error>>()?;
 
+    let primary_key = primary_key
+        .iter()
+        .map(|k| rust_name_for_sql_name(k))
+        .collect::<Vec<_>>();
+
     Ok(TableData {
         name,
         primary_key,