Control stack exhausted (no more space for function call frames). This is probably due to heavily nested or infinitely recursive function calls, or a tail call that SBCL cannot or has not optimized away. #810

antonkulaga · 2018-06-16T14:49:06Z

pgloader crashes with control stack exceptions. Here is the command I type:

docker  run --rm --name pgloader --security-opt seccomp=unconfined --network host -v /pipelines/data:/pipelines/data dimitri/pgloader:latest pgloader /pipelines/data/GEOmetadb.sqlite postgresql://postgres:changeme@localhost/sequencing

Here are the errors

WARNING:
Couldn't re-execute SBCL with proper personality flags (/proc isn't mounted? setuid?)
Trying to continue anyway.
2018-06-16T14:43:28.037000Z LOG Migrating from #<SQLITE-CONNECTION sqlite:///pipelines/data/GEOmetadb.sqlite {1005FB27E3}>
2018-06-16T14:43:28.039000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://postgres@localhost:5432/sequencing {10061FB983}>
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

antonkulaga@lab:/pipelines/sources/GEOmetadb-scala/databases/postgres$ ./migrate.sh
2018-06-16T14:45:42.036000Z LOG Migrating from #<SQLITE-CONNECTION sqlite:///pipelines/data/GEOmetadb.sqlite {1005FB27E3}>
2018-06-16T14:45:42.038000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://postgres@localhost:5432/sequencing {10061FBAA3}>
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

The text was updated successfully, but these errors were encountered:

antonkulaga · 2018-06-16T15:22:59Z

Looks like this issue is specific to official docker container :(

diegonc · 2018-07-04T20:32:07Z

I just reproduced this issue without docker. (debian, pgloader 3.5 from sid built on stretch)

$ pgloader --version
pgloader version "3.5.2"
compiled with SBCL 1.3.14.debian

dimitri · 2018-07-04T22:11:15Z

Do you have a --debug output that I could have a look at? Or even better, could you share the SQLite input file so that I can try and reproduce the bug locally, then fix it?

marekr · 2018-07-04T22:26:41Z

I too am trying to use pgloader version 3.5.2 with the same error without docker

I'm attempting to migrate a Mysql Gitlab database to Postgres.

pgloader version "3.5.2" compiled with SBCL 1.3.14.debian

mroszko@mince3:~$ sudo -u postgres pgloader --debug commands.load
sb-impl::*default-external-format* :UTF-8
tmpdir: #P"/tmp/pgloader/"
2018-07-04T22:25:52.025000Z NOTICE Starting pgloader, log system is ready.
2018-07-04T22:25:52.066000Z INFO Starting monitor
2018-07-04T22:25:52.087000Z INFO Parsed command:
LOAD DATABASE
     FROM mysql://git:meep@localhost/gitlabhq_production
     INTO postgresql://postgres@unix://var/run/postgresql:/gitlabhq_production

WITH include no drop, truncate, disable triggers, create no tables,
     create no indexes, preserve index names, no foreign keys,
     data only

ALTER SCHEMA 'gitlabhq_production' RENAME TO 'public'

;




2018-07-04T22:25:52.115000Z DEBUG CONNECTED TO #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://postgres@unix://var/run/postgresql:5432/gitlabhq_production {1006DED9A3}>
2018-07-04T22:25:52.115000Z DEBUG SET client_encoding TO 'utf8'
2018-07-04T22:25:52.115000Z DEBUG SET application_name TO 'pgloader'
2018-07-04T22:25:52.132000Z LOG Migrating from #<MYSQL-CONNECTION mysql://git@localhost:3306/gitlabhq_production {1006DEC423}>
2018-07-04T22:25:52.132000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://postgres@unix://var/run/postgresql:5432/gitlabhq_production {1006DED9A3}>
2018-07-04T22:25:52.188000Z DEBUG CONNECTED TO #<MYSQL-CONNECTION mysql://git@localhost:3306/gitlabhq_production {1006DEC423}>
2018-07-04T22:25:52.192000Z SQL MySQL: sending query: -- params: db-name
--         table-type-name
--         only-tables
--         only-tables
--         including
--         filter-list-to-where-clause incuding
--         excluding
--         filter-list-to-where-clause excluding
  select c.table_name, t.table_comment,
         c.column_name, c.column_comment,
         c.data_type, c.column_type, c.column_default,
         c.is_nullable, c.extra
    from information_schema.columns c
         join information_schema.tables t using(table_schema, table_name)
   where c.table_schema = 'gitlabhq_production' and t.table_type = 'BASE TABLE'



order by table_name, ordinal_position;
2018-07-04T22:25:54.656000Z NOTICE Prepare PostgreSQL database.
2018-07-04T22:25:54.660000Z DEBUG CONNECTED TO #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://postgres@unix://var/run/postgresql:5432/gitlabhq_production {1012AC5A33}>
2018-07-04T22:25:54.660000Z DEBUG SET client_encoding TO 'utf8'
2018-07-04T22:25:54.660000Z DEBUG SET application_name TO 'pgloader'
2018-07-04T22:25:54.665000Z DEBUG BEGIN
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution
2018-07-04T22:25:54.999000Z INFO Stopping monitor

What I am doing here?

marekr · 2018-07-04T22:52:50Z

Since I have two separate gitlab instances and servers that I'm migrating. The one that has

mroszko@btdebian:~$ pgloader --version
pgloader version "3.5.2"
compiled with SBCL 1.2.4.debian
mroszko@btdebian:~$ uname -a
Linux btdebian 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux

worked ok and migrated.

The one with

mroszko@mince3:~$
pgloader version "3.5.2" 
compiled with SBCL 1.3.14.debian
mroszko@mince3:~$ uname -a
Linux mince3 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux

fails with the above debug log

dimitri · 2018-07-04T23:23:16Z

Current version of SBCL is 1.4.9 (see http://www.sbcl.org/platform-table.html for downloading it), is it possible for you to test if the bug is still to be found in that version? If that's the case, we'll need to investigate with SBCL people maybe.

marekr · 2018-07-04T23:44:07Z

Sadly

./build/bin/pgloader --version
pgloader version "3.5.2"
compiled with SBCL 1.4.9

still fails.

I seem to be encountering incredibly strange compiler errors trying to use SBCL 1.2.4 like the server was built with to try it on this server

marekr · 2018-07-04T23:59:20Z

Well SBCL 1.2.4 seems incompatible though somehow the debian builds use it with patches I guess?

Newer than 1.2.5 keeps resulting in this error

Fatal INPUT-ERROR-IN-LOAD:
  READ error during LOAD:

    Package ASDF/SYSTEM-REGISTRY does not exist.

      Line: 17, Column: 94, File-Position: 13863

And all this LISP stuff is way above my head here :/

marekr · 2018-07-05T00:19:41Z

So I stole pgloader from the "good" server and copied it to the "bad" server. It still goes kaboom the same way.

So its probably not SBCL related but something else? Maybe postgres version? The "bad" server is running postgres 10. The good server is running 9.6.

diegonc · 2018-07-05T01:29:00Z

I’m working with Postgres 10, too. I’ll get the details tomorrow.

marekr · 2018-07-06T00:01:45Z

@dimitri i built 3.5.1 from the github releases. It works ok. Appears to be a bug in 3.5.2.

pgloader version "3.5.1~devel"
compiled with SBCL 1.4.9

dimitri · 2018-07-06T09:02:24Z

I still have no way to reproduce the problem here, so I'm waiting on more information (like a source SQLite file or something) that would help me reproduce the bug and then fix it.

marekr · 2018-07-06T13:32:44Z

@dimitri Will mysql (and pg) dumps work? (I have the problem with mysql). I can probably sanitize the db before sending a dump.

I can probably also bisect down to the specific commit between 3.5.1 and 3.5.2.

dimitri · 2018-07-08T18:40:46Z

@marekr Yeah a MySQL dump would work, I don't think I need the pg one unless you're using data only mode and a schema generated by another tool than pgloader, or manually. See my email address on my profile here. Thanks!

AsinRay · 2018-07-12T11:06:46Z

2018-07-12T11:00:46.054000Z LOG Migrating from #<MYSQL-CONNECTION mysql://[email protected]:3306/eth_spider_18 {100605E943}>
2018-07-12T11:00:46.057000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://postgres@UNIX:5432/xx {10062A7D33}>
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

=====================================
OS: debian 9    Src : MySQL 5.7    Dest: PG 10.4
=====================================

Help me!

dimitri · 2018-07-12T15:49:44Z

@AsinRay I will need more information to be able to help you. Version of pgloader you're using, --debug output, and if that's possible for you share the source database with me, or give me access to a box where I can reproduce the bug.

AsinRay · 2018-07-13T01:22:14Z

@dimitri my pgloader version:

pgloader version "3.5.2"
compiled with SBCL 1.3.14.debian

debug info:

postgres@eth-calc:~$ pgloader --debug mysql://root:'siOiZ2/qWLAY'@127.0.0.1/eth_spider_18 pgsql:///xx 
sb-impl::*default-external-format* :UTF-8
tmpdir: #P"/tmp/pgloader/"
2018-07-13T01:22:45.017000Z NOTICE Starting pgloader, log system is ready.
2018-07-13T01:22:45.035000Z INFO Starting monitor
2018-07-13T01:22:45.039000Z INFO     SOURCE: "mysql://root:siOiZ2/[email protected]/eth_spider_18"
2018-07-13T01:22:45.040000Z INFO SOURCE URI: #<PGLOADER.SOURCE.MYSQL:MYSQL-CONNECTION mysql://[email protected]:3306/eth_spider_18 {100605E943}>
2018-07-13T01:22:45.040000Z INFO     TARGET: "pgsql:///xx"
2018-07-13T01:22:45.040000Z INFO TARGET URI: #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://postgres@UNIX:5432/xx {10062A34C3}>
2018-07-13T01:22:45.040000Z DEBUG LOAD DATA FROM #<PGLOADER.SOURCE.MYSQL:MYSQL-CONNECTION mysql://[email protected]:3306/eth_spider_18 {100605E943}>
2018-07-13T01:22:45.044000Z DEBUG CONNECTED TO #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://postgres@UNIX:5432/xx {10062A34C3}>
2018-07-13T01:22:45.044000Z DEBUG SET client_encoding TO 'utf8'
2018-07-13T01:22:45.045000Z DEBUG SET application_name TO 'pgloader'
2018-07-13T01:22:45.056000Z LOG Migrating from #<MYSQL-CONNECTION mysql://[email protected]:3306/eth_spider_18 {100605E943}>
2018-07-13T01:22:45.057000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://postgres@UNIX:5432/xx {10062A34C3}>
2018-07-13T01:22:45.092000Z DEBUG CONNECTED TO #<MYSQL-CONNECTION mysql://[email protected]:3306/eth_spider_18 {100605E943}>
2018-07-13T01:22:45.095000Z SQL MySQL: sending query: -- params: db-name
--         table-type-name
--         only-tables
--         only-tables
--         including
--         filter-list-to-where-clause incuding
--         excluding
--         filter-list-to-where-clause excluding
  select c.table_name, t.table_comment,
         c.column_name, c.column_comment,
         c.data_type, c.column_type, c.column_default,
         c.is_nullable, c.extra
    from information_schema.columns c
         join information_schema.tables t using(table_schema, table_name)
   where c.table_schema = 'eth_spider_18' and t.table_type = 'BASE TABLE'
order by table_name, ordinal_position;
2018-07-13T01:22:45.114000Z SQL MySQL: sending query: -- params: db-name
--         table-type-name
--         only-tables
--         only-tables
--         including
--         filter-list-to-where-clause incuding
--         excluding
--         filter-list-to-where-clause excluding
SELECT s.table_name, s.constraint_name, s.ft, s.cols, s.fcols,
       rc.update_rule, rc.delete_rule
FROM
 (
  SELECT tc.table_schema, tc.table_name,
         tc.constraint_name, k.referenced_table_name ft,
             group_concat(         k.column_name
                          order by k.ordinal_position) as cols,
             group_concat(         k.referenced_column_name
                          order by k.position_in_unique_constraint) as fcols
        FROM information_schema.table_constraints tc
        LEFT JOIN information_schema.key_column_usage k
               ON k.table_schema = tc.table_schema
              AND k.table_name = tc.table_name
              AND k.constraint_name = tc.constraint_name
      WHERE     tc.table_schema = 'eth_spider_18'
            AND k.referenced_table_schema = 'eth_spider_18'
            AND tc.constraint_type = 'FOREIGN KEY'
GROUP BY tc.table_schema, tc.table_name, tc.constraint_name, ft
 ) s
             JOIN information_schema.referential_constraints rc
               ON rc.constraint_schema = s.table_schema
              AND rc.constraint_name = s.constraint_name
              AND rc.table_name = s.table_name;
2018-07-13T01:22:45.119000Z SQL MySQL: sending query: -- params: db-name
--         table-type-name
--         only-tables
--         only-tables
--         including
--         filter-list-to-where-clause incuding
--         excluding
--         filter-list-to-where-clause excluding
  SELECT table_name, index_name, index_type,
         sum(non_unique),
         cast(GROUP_CONCAT(column_name order by seq_in_index) as char)
    FROM information_schema.statistics
   WHERE table_schema = 'eth_spider_18'
GROUP BY table_name, index_name, index_type;
2018-07-13T01:22:45.153000Z NOTICE Prepare PostgreSQL database.
2018-07-13T01:22:45.156000Z DEBUG CONNECTED TO #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://postgres@UNIX:5432/xx {10062A34C3}>
2018-07-13T01:22:45.156000Z DEBUG SET client_encoding TO 'utf8'
2018-07-13T01:22:45.156000Z DEBUG SET application_name TO 'pgloader'
2018-07-13T01:22:45.159000Z DEBUG BEGIN
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution
2018-07-13T01:22:45.519000Z INFO Stopping monitor

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

SarahTournon · 2018-07-13T08:33:48Z

Hi,
Since today, I'm experimenting exactly the same issue .
pgloader version 3.5.2 compiled with SBCL 1.3.1.debian
I'm working on a linux VM

Linux vmaussie2 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

And yesterday everything was working fine ...
The only thing that has changed (except that today is friday 13th...) is that I made an update of my Ubuntu ...
But I also tried in my other VM Linux vmaussie2 4.4.0-127-generic , same issue.

I do not upgrade in SBCL 1.4.9 since it seems not resolving anything .

Thanks in advance for any help .

dimitri · 2018-07-13T08:58:34Z

@AsinRay thanks! can you show your load command (without sensitive information), please?

SarahTournon · 2018-07-13T12:41:28Z

pgloader mysql://root:XXXXX@localhost/tempo_migration postgresql://usermigrate:XXXXX@localhost/tempo_migration Dimitri Fontaine <[email protected]> a écrit :

…

@AsinRay thanks! can you show your load command (without sensitive information), please? -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #810 (comment)

dimitri · 2018-07-13T13:34:32Z

@SarahTournon is it possible for you to share the source database with me, or give me access to an environment where I can reproduce the bug? I'm yet to find a way to reproduce it locally so that I can solve it, and really, I wish I was in a position to fix that one...

SarahTournon · 2018-07-13T14:03:44Z

here it is , the dababase is very small , only one table... Dimitri Fontaine <[email protected]> a écrit :

…

@SarahTournon is it possible for you to share the source database with me, or give me access to an environment where I can reproduce the bug? I'm yet to find a way to reproduce it locally so that I can solve it, and really, I wish I was in a position to fix that one... -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #810 (comment)

dimitri · 2018-07-13T15:08:26Z

@SarahTournon It seems like the email attachement didn't make it to Github's for us to see it. Can you either send an email to my personal address (see my GitHub profile) or upload the file here on Github directly?

dimitri · 2018-07-13T15:35:17Z

Got the database, failed to reproduce the bug locally again. Will try later with a debian/ubuntu environment (docker or otherwise).

ccwang002 · 2018-07-19T22:41:42Z

I encountered the same issue loading a SQLite database to a PosgreSQL database (10.4) after upgrading the pgloader:

pgloader --version
pgloader version "3.5.2"
compiled with SBCL 1.3.14.debian

Do you still need a reproducible example?

dimitri · 2018-07-20T07:11:05Z

@ccwang002 yeah, another try would help testing that bug, thanks!

ccwang002 · 2018-07-20T14:51:04Z

I was able to create a reproducible example on a fresh Debian stretch instance on Google Cloud Engine. I was trying to load this example SQLite database in the documentation (~70MB) into an empty PosgreSQL database.

I installed the PostgreSQL 10.4 (10.4-2.pgdg90+1) and pgloader (3.5.2-2.pgdg90+1) following the instructions at https://www.postgresql.org/download/linux/debian/:

$ sudo apt-get install postgresql-10 pgloader sqlite3
$ createdb mydb
$ wget https://github.com/lerocha/chinook-database/raw/master/ChinookDatabase/DataSources/Chinook_Sqlite_AutoIncrementPKs.sqlite

$ pgloader --version
pgloader version "3.5.2"
compiled with SBCL 1.3.14.debian

Log of pgloader --debug --verbose ./Chinook_Sqlite_AutoIncrementPKs.sqlite postgresql:///mydb

sb-impl::*default-external-format* :UTF-8
tmpdir: #P"/tmp/pgloader/"
2018-07-20T14:47:24.016000Z NOTICE Starting pgloader, log system is ready.
2018-07-20T14:47:24.036000Z INFO Starting monitor
2018-07-20T14:47:24.039000Z INFO     SOURCE: "./Chinook_Sqlite_AutoIncrementPKs.sqlite"
2018-07-20T14:47:24.039000Z INFO SOURCE URI: #<PGLOADER.SOURCE.SQLITE:SQLITE-CONNECTION sqlite:///home/lbwang/./Chinook_Sqlite_AutoIncrementPKs.sqlite {100601AE03}>
2018-07-20T14:47:24.039000Z INFO     TARGET: "postgresql:///mydb"
2018-07-20T14:47:24.040000Z INFO TARGET URI: #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://lbwang@UNIX:5432/mydb {10062634C3}>
2018-07-20T14:47:24.060000Z LOG Migrating from #<SQLITE-CONNECTION sqlite:///home/lbwang/./Chinook_Sqlite_AutoIncrementPKs.sqlite {100601AE03}>
2018-07-20T14:47:24.060000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://lbwang@UNIX:5432/mydb {10062634C3}>
2018-07-20T14:47:24.073000Z SQL SQLite: SELECT tbl_name
  FROM sqlite_master
 WHERE tbl_name = 'sqlite_sequence'
2018-07-20T14:47:24.077000Z SQL SELECT tbl_name
  FROM sqlite_master
 WHERE type='table'
       AND tbl_name <> 'sqlite_sequence'


2018-07-20T14:47:24.081000Z NOTICE SQLite column Album.AlbumId uses a sequence
2018-07-20T14:47:24.084000Z NOTICE SQLite column Artist.ArtistId uses a sequence
2018-07-20T14:47:24.084000Z NOTICE SQLite column Customer.CustomerId uses a sequence
2018-07-20T14:47:24.085000Z NOTICE SQLite column Employee.EmployeeId uses a sequence
2018-07-20T14:47:24.085000Z NOTICE SQLite column Genre.GenreId uses a sequence
2018-07-20T14:47:24.085000Z NOTICE SQLite column Invoice.InvoiceId uses a sequence
2018-07-20T14:47:24.085000Z NOTICE SQLite column InvoiceLine.InvoiceLineId uses a sequence
2018-07-20T14:47:24.086000Z NOTICE SQLite column MediaType.MediaTypeId uses a sequence
2018-07-20T14:47:24.086000Z NOTICE SQLite column Playlist.PlaylistId uses a sequence
2018-07-20T14:47:24.086000Z NOTICE SQLite column Track.TrackId uses a sequence
2018-07-20T14:47:24.118000Z NOTICE Prepare PostgreSQL database.
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution
2018-07-20T14:47:24.430000Z INFO Stopping monitor

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

If I did the same thing on my local Mac machine (OSX 10.13.6) with Homebrew pgloader (3.5.2; SBCL 1.4.9), the command can run successfully.

Thanks for helping this issue. I will also try to self compile the older version as a workaround (will update here later).

ccwang002 · 2018-07-20T15:55:38Z

I compiled the older versions of pgloader using the same SBCL (1.3.14) and here are the results running the same example database with the most recent 3 versions:

3.5.2: Failed
3.5.1: Successful
3.4.1: Successful

I decided to copy the self-compiled pgloader binary (3.4.1) to the production server (the binary seems to be self contained), and it worked again for my case. So I think something went unexpected between version 3.5.1 and 3.5.2.

Again, thanks for making this package and hope you can bisect the culprit commit soon :)

The debian/Ubuntu packaging would defeat the quite simple regexp parsing PostgreSQL version string that we have in pgloader. To make it more robust, make it more open to unforeseen strings. See #800, see #810.

dimitri · 2018-11-30T14:41:58Z

Hi @ahjones, good spot! I think it might be just that, having a NIL in the :variant doesn't strike me as a good thing in our case. I just improved the parsing of the version string to support Debian and ubuntu packaging. Please try again and let me know!

CL-USER> (loop for version-string 
            in '("PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.2058"
                 "PostgreSQL 10.1 on x86_64-apple-darwin14.5.0, compiled by Apple LLVM version 7.0.0 (clang-700.1.76), 64-bit"
                 "PostgreSQL 10.6 (Ubuntu 10.6-1.pgdg14.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 4.8.4-2ubuntu1~14.04.4) 4.8.4, 64-bit")
              collect (multiple-value-list
                       (pgloader.pgsql::parse-postgresql-version-string version-string)))
(("8.0.2" "8.0" :REDSHIFT) ("10.1" "10" :PGDG) ("10.6" "10" :PGDG))

ahjones · 2018-11-30T18:57:20Z

Hi @dmitri, your fix works. Thanks very much for your help.

dimitri · 2018-11-30T19:53:13Z

Closing issue as we might have found the problem. Feel free to reopen if necessary of course.

corentin-cres · 2018-12-05T16:01:14Z

Hello, I'm having the same issue with Docker & pgloader 3.5.2.
I do not understand what to patch based on the conversation on this thread.

Will this issue be resolved in the next version of pgloader ?

dimitri · 2018-12-05T17:24:23Z

@johndoudou please compile from current sources, all the patches highlighted in the conversation have been merged there already. You should not have the bug with a fresh compile from the current master's branch. Please report success or failure!

corentin-cres · 2018-12-06T10:40:33Z

Reporting success :)

I ran into the issue where brew install --HEAD is unable to build from source (https://github.com/dimitri/pgloader/issues/832) but got the building working with a simple make pgloader.

The import was effectively working afterwards, using the built build/bin/pgloader.
Thanks !

ghost · 2018-12-09T20:40:34Z

Hello! I'm pretty new to all of this, but I'm having a similar warning pop up in my command line. It's not an error per se, but I'm wondering if there's something I can/should do to check that everything's alright before I proceed with my Heroku deployment. Thanks so much in advance!

dimitri · 2018-12-10T01:21:40Z

Hi @az326 ; it's actually not a WARNING but a FATAL error here. Which version of pgloader are you using? Can you try the release candidate at
https://github.com/dimitri/pgloader/releases/tag/v3.6.0 and report success/failure?

ghost · 2018-12-10T02:00:14Z

Hi @dimitri . I'm using pgloader 3.5.2. Yes, sure, but is there any way I can install the new pgloader without homebrew? I'm having trouble installing homebrew in the IDE I'm using.

dimitri · 2018-12-10T04:01:41Z

Yes, download the bundle tarball from the GitHub release page, expand the archive and type make in the directory obtained. That should output lots of compile-time logs and then you have ./bin/pgloader.

ghost · 2018-12-10T15:58:18Z

Gotcha, just finished that, although it's still showing "pgloader version '3.5.2'" when I check the version. I've also got an even weirder error message now...

(venv) ShuwendeMBP:website andrewzhang$ pgloader project.db postgres://albgdbybtdqgqj:8101fce883d1fd7d071804875221f5eca6c81df7b57b70eaa1f53756cf5a4fa4@ec2-54-227-249-201.compute-1.amazonaws.com:5432/dbks6mfdlftje0?sslmode=require

2018-12-10T15:28:15.022000Z LOG Data errors in '/private/tmp/pgloader/'
KABOOM!
FATAL error: Failed to connect to pgsql at "ec2-54-227-249-201.compute-1.amazonaws.com" (port 5432) as user "albgdbybtdqgqj": The alien function "SSL_load_error_strings" is undefined.
An unhandled error condition has been signalled:
   Failed to connect to pgsql at "ec2-54-227-249-201.compute-1.amazonaws.com" (port 5432) as user "albgdbybtdqgqj": The alien function "SSL_load_error_strings" is undefined.




What I am doing here?

Failed to connect to pgsql at "ec2-54-227-249-201.compute-1.amazonaws.com" (port 5432) as user "albgdbybtdqgqj": The alien function "SSL_load_error_strings" is undefined.

dimitri · 2018-12-11T01:56:01Z

Are you sure you're using the newly built binary rather than the old one still?

sixsixtrample · 2019-01-30T16:12:06Z

Build from current source, on OSX, migrating MySQL to Postgres(11) I receive the same error:

2019-01-29T17:17:07.779000-07:00 DEBUG SET client_encoding TO 'utf8'
2019-01-29T17:17:07.863000-07:00 DEBUG SET application_name TO 'pgloader'
KABOOM!
2019-01-29T17:17:08.336000-07:00 DEBUG BEGIN
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution
2019-01-29T17:17:08.806000-07:00 INFO Stopping monitor

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

equinton · 2019-02-08T10:37:36Z

I had encountered the same problem with the package provided by Ubuntu (18.04.1 lts).
I have solved it by recompiling from source.

GerBreOwn · 2019-04-11T12:04:01Z

I had the same problem with pgloader 3.5.2 on Linux Mint 19.1. I downloaded and compiled pgloader 3.6.1 and when it was run from ./bin/pgloader it worked as advertised and it was VERY FAST to migrate the database from MariaDB to Postgresql. Thanks for the great product.
Gerald

wyldbill · 2020-05-13T15:06:11Z

was the patch for deb based linux ever merged?

phoe · 2020-05-13T15:07:04Z

Which version is currently available in your debian distribtion?

bessgeor · 2020-06-25T13:35:23Z

I've got a similar issue.
My environment is Ubuntu 18.04 under WSL1 on Windows 10 Pro 1909 (build 18363.900).
I've installed pgloader 3.5.2 (compiled with SBCL 1.4.5.debian) from postgresql apt.
I'm trying to load okato.dbf file from this archive into my PostgreSQL installation (PostgreSQL 12.2, compiled by Visual C++ build 1914, 64-bit) running on Windows system with following code:

LOAD DBF
        FROM /mnt/e/[path]/okato.dbf
    INTO postgresql://[credentials]@localhost/[database]
    WITH truncate, create table;

pgloader's output:

KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

bessgeor · 2020-06-25T14:36:42Z

I've got a similar issue.
My environment is Ubuntu 18.04 under WSL1 on Windows 10 Pro 1909 (build 18363.900).
I've installed pgloader 3.5.2 (compiled with SBCL 1.4.5.debian) from postgresql apt.
I'm trying to load okato.dbf file from this archive into my PostgreSQL installation (PostgreSQL 12.2, compiled by Visual C++ build 1914, 64-bit) running on Windows system with following code:
LOAD DBF
        FROM /mnt/e/[path]/okato.dbf
    INTO postgresql://[credentials]@localhost/[database]
    WITH truncate, create table;
pgloader's output:
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

Seems fixed in 3.6.2

jvtrigueros · 2020-06-28T23:57:47Z

I was having this issue as well with the Docker container migrating a SQLite database to PostgresQL and I solved it by switching to the CCL version:

$ docker run --rm -it dimitri/pgloader:ccl.latest bash
root@bced8654dcf9:/# pgloader --version
pgloader version "3.6.3~devel"
compiled with Clozure Common Lisp Version 1.11.5/v1.11.5  (LinuxX8664)

I hope this helps someone else :D

Ocramius · 2020-09-22T10:47:41Z

Just hit this as well - running dmitri/pgloader:ccl.latest as per #810 (comment) (sha256:79b3ea97a3e08cb728e58790a77ef8b3a86bd014e3f0a7a50034e189c999c40b at the time of this writing) works.

The one distributed with debian (3.6.2) is still borked.

radusuciu · 2021-07-23T18:33:55Z

For others that stumble upon this, I encountered this error today and it was because I had misspelled a table name in a clause like this:

load database
    from sqlite://foo.db
    including only tables names like Foo

Great tool by the way!

jeremybusk · 2021-10-12T17:49:35Z

On Ubuntu 20.04 I had to download latest release and compile

Something like this

apt update && apt install -y build-essential sbcl

curl -LO https://github.com/dimitri/pgloader/releases/download/v3.6.2/pgloader-bundle-3.6.2.tgz
tar xf pgloader-bundle-3.6.2.tgz
cd pgloader-bundle-3.6.2
make

cp bin/pgloader /usr/local/bin/pgloader

/usr/local/bin/pgloader

Kulgar · 2021-10-19T13:40:09Z

@jeremybusk : thanks, this worked for me on Ubuntu 20 too ;-)

sigam-Zq · 2024-01-30T07:14:23Z

系统如下

root@dev1:~/shunhuaRoad/gaussTools/pgloader# uname -a
Linux dev1 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

apt install pgloader 下载的版本如下

root@dev1:~/shunhuaRoad/gaussTools/pgloader# pgloader --version
pgloader version "3.6.1"
compiled with SBCL 1.4.15.debian

使用如下文件

root@dev1:~/shunhuaRoad/gaussTools/pgloader# cat my.load
LOAD DATABASE
FROM mysql://user:passwd@host:port/database
INTO postgresql://user:passwd@host:port/database

WITH include drop, create tables, create indexes, reset no sequences,
workers = 8, concurrency = 1,
multiple readers per thread, rows per range = 50000

CAST

type varchar when(= 1 precision) to "boolean" drop typemod keep default keep not null;

报错如下

root@dev1:~/shunhuaRoad/gaussTools/pgloader# pgloader --debug my.load
pgloader version 3.6.1
sb-impl::default-external-format :UTF-8
tmpdir: #P"/tmp/pgloader/"
2024-01-30T07:09:14.008000Z NOTICE Starting pgloader, log system is ready.
2024-01-30T07:09:14.017000Z INFO Starting monitor
2024-01-30T07:09:14.021000Z LOG pgloader version "3.6.1"
2024-01-30T07:09:14.045000Z INFO Parsed command:
LOAD DATABASE
FROM mysql://root:[email protected]:23306/shunhuaroad
INTO postgresql://ninuo03:Opengauss@@123@127.0.0.1:5432/ninuo03

WITH include drop, create tables, create indexes, reset no sequences,
workers = 8, concurrency = 1,
multiple readers per thread, rows per range = 50000

CAST

type varchar when(= 1 precision) to "boolean" drop typemod keep default keep not null;

2024-01-30T07:09:14.161000Z DEBUG CONNECTED TO #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://[email protected]:5432/ninuo03 {10069DE513}>
2024-01-30T07:09:14.161000Z DEBUG SET client_encoding TO 'utf8'
2024-01-30T07:09:14.163000Z DEBUG SET application_name TO 'pgloader'
2024-01-30T07:09:14.184000Z LOG Migrating from #<MYSQL-CONNECTION mysql://[email protected]:23306/shunhuaroad {10069DCFE3}>
2024-01-30T07:09:14.184000Z LOG Migrating into #<PGSQL-CONNECTION pgsql://[email protected]:5432/ninuo03 {10069DE513}>
2024-01-30T07:09:14.222000Z DEBUG CONNECTED TO #<MYSQL-CONNECTION mysql://[email protected]:23306/shunhuaroad {10069DCFE3}>
2024-01-30T07:09:14.225000Z SQL MySQL: sending query: -- params: db-name
-- table-type-name
-- only-tables
-- only-tables
-- including
-- filter-list-to-where-clause incuding
-- excluding
-- filter-list-to-where-clause excluding
select c.table_name, t.table_comment,
c.column_name, c.column_comment,
c.data_type, c.column_type, c.column_default,
c.is_nullable, c.extra
from information_schema.columns c
join information_schema.tables t using(table_schema, table_name)
where c.table_schema = 'shunhuaroad' and t.table_type = 'BASE TABLE'

order by table_name, ordinal_position;
2024-01-30T07:09:14.384000Z SQL MySQL: sending query: -- params: db-name
-- table-type-name
-- only-tables
-- only-tables
-- including
-- filter-list-to-where-clause incuding
-- excluding
-- filter-list-to-where-clause excluding
SELECT s.table_name, s.constraint_name, s.ft, s.cols, s.fcols,
rc.update_rule, rc.delete_rule

FROM
(
SELECT tc.table_schema, tc.table_name,
tc.constraint_name, k.referenced_table_name ft,

         group_concat(         k.column_name
                      order by k.ordinal_position) as cols,

         group_concat(         k.referenced_column_name
                      order by k.position_in_unique_constraint) as fcols

    FROM information_schema.table_constraints tc

    LEFT JOIN information_schema.key_column_usage k
           ON k.table_schema = tc.table_schema
          AND k.table_name = tc.table_name
          AND k.constraint_name = tc.constraint_name

  WHERE     tc.table_schema = 'shunhuaroad'
        AND k.referenced_table_schema = 'shunhuaroad'
        AND tc.constraint_type = 'FOREIGN KEY'

GROUP BY tc.table_schema, tc.table_name, tc.constraint_name, ft
) s
JOIN information_schema.referential_constraints rc
ON rc.constraint_schema = s.table_schema
AND rc.constraint_name = s.constraint_name
AND rc.table_name = s.table_name;
2024-01-30T07:09:14.949000Z SQL MySQL: sending query: -- params: db-name
-- table-type-name
-- only-tables
-- only-tables
-- including
-- filter-list-to-where-clause incuding
-- excluding
-- filter-list-to-where-clause excluding
SELECT table_name, index_name, index_type,
sum(non_unique),
cast(GROUP_CONCAT(column_name order by seq_in_index) as char)
FROM information_schema.statistics
WHERE table_schema = 'shunhuaroad'

GROUP BY table_name, index_name, index_type;
2024-01-30T07:09:14.981000Z INFO Processing source catalogs
2024-01-30T07:09:15.057000Z NOTICE Prepare PostgreSQL database.
2024-01-30T07:09:15.080000Z DEBUG CONNECTED TO #<PGLOADER.PGSQL:PGSQL-CONNECTION pgsql://[email protected]:5432/ninuo03 {10069DE513}>
2024-01-30T07:09:15.080000Z DEBUG SET client_encoding TO 'utf8'
2024-01-30T07:09:15.082000Z DEBUG SET application_name TO 'pgloader'
2024-01-30T07:09:15.087000Z DEBUG BEGIN
KABOOM!
INFO: Control stack guard page unprotected
Control stack guard page temporarily disabled: proceed with caution
2024-01-30T07:09:15.421000Z INFO Stopping monitor

What I am doing here?

Control stack exhausted (no more space for function call frames).
This is probably due to heavily nested or infinitely recursive function
calls, or a tail call that SBCL cannot or has not optimized away.

PROCEED WITH CAUTION.

拉取 3.6.7 版本docker 使用成功

docker run --rm -v /xxx/:/xxx/ --network xxx_net ghcr.io/dimitri/pgloader pgloader /xxx/my.load

版本查看

root@dev1:/shunhuaRoad/gaussTools/pgloader# docker run --rm ghcr.io/dimitri/pgloader pgloader --version
pgloader version "3.6.7devel"
compiled with SBCL 2.2.9.debian

dimitri added the Needs more testing / information label Jul 6, 2018

dimitri mentioned this issue Jul 13, 2018

Control stack exhausted #800

Closed

dimitri closed this as completed Nov 30, 2018

anhi mentioned this issue Feb 13, 2019

pgloader fails for windows versions of PostgreSQL #908

Closed

TestSubject1498 mentioned this issue Feb 1, 2022

Pgloader - MSSQL to Postgres (YB) throwing TDS_MAX_CONN error #1354

Open

5 tasks

Control stack exhausted (no more space for function call frames). This is probably due to heavily nested or infinitely recursive function calls, or a tail call that SBCL cannot or has not optimized away. #810

Control stack exhausted (no more space for function call frames). This is probably due to heavily nested or infinitely recursive function calls, or a tail call that SBCL cannot or has not optimized away. #810

Comments

antonkulaga commented Jun 16, 2018

antonkulaga commented Jun 16, 2018

diegonc commented Jul 4, 2018 • edited Loading

dimitri commented Jul 4, 2018 • edited Loading

marekr commented Jul 4, 2018 • edited Loading

marekr commented Jul 4, 2018 • edited Loading

dimitri commented Jul 4, 2018

marekr commented Jul 4, 2018

marekr commented Jul 4, 2018 • edited Loading

marekr commented Jul 5, 2018 • edited Loading

diegonc commented Jul 5, 2018

marekr commented Jul 6, 2018 • edited Loading

dimitri commented Jul 6, 2018

marekr commented Jul 6, 2018 • edited Loading

dimitri commented Jul 8, 2018

AsinRay commented Jul 12, 2018 • edited by dimitri Loading

dimitri commented Jul 12, 2018

AsinRay commented Jul 13, 2018 • edited by dimitri Loading

SarahTournon commented Jul 13, 2018 • edited by dimitri Loading

dimitri commented Jul 13, 2018

SarahTournon commented Jul 13, 2018 via email • edited Loading

dimitri commented Jul 13, 2018

SarahTournon commented Jul 13, 2018 via email • edited Loading

dimitri commented Jul 13, 2018

dimitri commented Jul 13, 2018

ccwang002 commented Jul 19, 2018

dimitri commented Jul 20, 2018

ccwang002 commented Jul 20, 2018

ccwang002 commented Jul 20, 2018 • edited Loading

dimitri commented Nov 30, 2018 • edited Loading

ahjones commented Nov 30, 2018

dimitri commented Nov 30, 2018

corentin-cres commented Dec 5, 2018

dimitri commented Dec 5, 2018

corentin-cres commented Dec 6, 2018

ghost commented Dec 9, 2018

dimitri commented Dec 10, 2018

ghost commented Dec 10, 2018 • edited by ghost Loading

dimitri commented Dec 10, 2018

ghost commented Dec 10, 2018 • edited by dimitri Loading

dimitri commented Dec 11, 2018

sixsixtrample commented Jan 30, 2019

equinton commented Feb 8, 2019

GerBreOwn commented Apr 11, 2019

wyldbill commented May 13, 2020

phoe commented May 13, 2020

bessgeor commented Jun 25, 2020 • edited Loading

bessgeor commented Jun 25, 2020

jvtrigueros commented Jun 28, 2020

Ocramius commented Sep 22, 2020

radusuciu commented Jul 23, 2021

jeremybusk commented Oct 12, 2021

Kulgar commented Oct 19, 2021

sigam-Zq commented Jan 30, 2024

diegonc commented Jul 4, 2018 •

edited

Loading

dimitri commented Jul 4, 2018 •

edited

Loading

marekr commented Jul 4, 2018 •

edited

Loading

marekr commented Jul 4, 2018 •

edited

Loading

marekr commented Jul 4, 2018 •

edited

Loading

marekr commented Jul 5, 2018 •

edited

Loading

marekr commented Jul 6, 2018 •

edited

Loading

marekr commented Jul 6, 2018 •

edited

Loading

AsinRay commented Jul 12, 2018 •

edited by dimitri

Loading

AsinRay commented Jul 13, 2018 •

edited by dimitri

Loading

SarahTournon commented Jul 13, 2018 •

edited by dimitri

Loading

SarahTournon commented Jul 13, 2018 via email •

edited

Loading

SarahTournon commented Jul 13, 2018 via email •

edited

Loading

ccwang002 commented Jul 20, 2018 •

edited

Loading

dimitri commented Nov 30, 2018 •

edited

Loading

ghost commented Dec 10, 2018 •

edited by ghost

Loading

ghost commented Dec 10, 2018 •

edited by dimitri

Loading

bessgeor commented Jun 25, 2020 •

edited

Loading