Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MoVR Load not working against 20.2.0 #56821

Closed
vy-ton opened this issue Nov 17, 2020 · 3 comments
Closed

MoVR Load not working against 20.2.0 #56821

vy-ton opened this issue Nov 17, 2020 · 3 comments
Assignees

Comments

@vy-ton
Copy link
Contributor

vy-ton commented Nov 17, 2020

When running the following command using movr-load I'm getting the following error:

roachprod run $CLUSTER:10 <<EOF
    sudo docker run --rm cockroachdb/movr:20.07.1 --app-name "movr-load" --url "postgres://root@$CLUSTER-0010.roachprod.crdb.io:26257/movr?sslmode=disable" load --multi-region --num-users 100 --num-rides 100 --num-vehicles 10 --city "new york" --city "chicago" --city "seattle"
EOF

Error

```Status: Downloaded newer image for cockroachdb/movr:20.07.1
[INFO] (MainThread) connected to movr database @ postgres://[email protected]:26257/movr?sslmode=disable
[INFO] (MainThread) Loading multi region MovR
[INFO] (MainThread) initializing tables
[INFO] (MainThread) applying schema changes to make this database multi-region (this may take up to a minute).
[INFO] (MainThread) altering primary keys...
[INFO] (MainThread) altering foreign keys and dropping old indexes...
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.UndefinedObject: index "vehicles_auto_index_fk_owner_id_ref_users" does not exist


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./loadmovr.py", line 574, in <module>
    run_data_loader(conn_string, cities= get_cities(args.city), num_users= args.num_users, num_rides= args.num_rides, num_vehicles= args.num_vehicles, num_histories= args.num_histories,
  File "./loadmovr.py", line 449, in run_data_loader
    with MovR(conn_string, init_tables=(not skip_reload_tables), multi_region = use_multi_region, echo=echo_sql) as movr:
  File "/usr/src/app/movr.py", line 29, in __init__
    self.run_multi_region_transformations()
  File "/usr/src/app/movr.py", line 249, in run_multi_region_transformations
    self.run_queries_in_separate_transactions(queries_to_run["fk_alters"])
  File "/usr/src/app/movr.py", line 184, in run_queries_in_separate_transactions
    run_transaction(sessionmaker(bind=self.engine),
  File "/usr/local/lib/python3.8/site-packages/cockroachdb/sqlalchemy/transaction.py", line 34, in run_transaction
    return _txn_retry_loop(session, callback, max_retries)
  File "/usr/local/lib/python3.8/site-packages/cockroachdb/sqlalchemy/transaction.py", line 79, in _txn_retry_loop
    ret = callback(conn)
  File "/usr/src/app/movr.py", line 185, in <lambda>
    lambda session: session.execute(query))
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1175, in execute
    return self._connection_for_bind(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1055, in _execute_clauseelement
    ret = self._execute_context(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1195, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1411, in _handle_dbapi_exception
    util.raise_from_cause(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedObject) index "vehicles_auto_index_fk_owner_id_ref_users" does not exist
 [SQL: 'DROP INDEX vehicles_auto_index_fk_owner_id_ref_users;'] (Background on this error at: http://sqlalche.me/e/f405)```
@blathers-crl
Copy link

blathers-crl bot commented Nov 17, 2020

Hi @vy-ton, please add a C-ategory label to your issue. Check out the label system docs.

While you're here, please consider adding an A- label to help keep our repository tidy.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

@rafiss
Copy link
Collaborator

rafiss commented Nov 19, 2020

Looks like the issue is that the altering foreign keys and dropping old indexes... is trying to drop the vehicles_auto_index_fk_owner_id_ref_users index, which doesn't exist. So we could probably fix this by using DROP INDEX IF EXISTS, if possible.

@rafiss
Copy link
Collaborator

rafiss commented Nov 19, 2020

fixed by cockroachdb/movr#104

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants