Remove Flyway and move migrations to Sequelize #97

rmeritz · 2020-05-19T22:06:59Z

Remove the flyway migration and replace with sequelize migrations and sequelize seeds
I tried several methodologies to create the initial migration. I tried using a command-line tool but it was no longer maintained. I tried writing it by hand with the generated models as my starting point, but there were very subtle differences in the index creation that caused errors. I ended up going with using a pgdump of the original state of the database and running the commands from that because it creates a db in the exact same state as before. This means the first migrations is non-traditional but it still allows us to use normal sequelize migrations moving forward which was the main point of this work.
Setup new migrations to run on both Vagrant and locally
Change all json datatypes to a jsonb
Update the local Vagrant configuration to use a separate github oauth app so the sign up flow works correctly when testing with Vagrant
Update documentation of how to work with the db locally
Install the dotfile-cli as a way to easily load the environment variables
Add configuration for the sequelize-cli so it can be used with yarn run sequelize -- <ARGS>
Update documentation for information on how to deploy and how to test deploys with Vagrant
I've tested the app locally and in Vagrant and deleted the db on staging and deployed there

Without passing in additional configuration options.

You can use dotenv to load the enviornment variable and it is much cleaner than exporting than manually.

I looked into a bunch of ways of doing this automatically but none of them worked with the current version of sequelize but it is the most asked for plus-oned feature there is. I just ended up writting it up manually.

https://www.depesz.com/2014/03/25/waiting-for-9-4-introduce-jsonb-a-structured-format-for-storing-json/

Additionally remove flyway migraions and add deploy documentation.

This has the correct redirects.

https://www.npmjs.com/package/dotenv-cli

Now `yarn run sequelize -- db:migrate:undo` works.

resolving merge conflict due to capitalization file name change (Database.md -> database.md)

…ize migration system

evmiguel

@rmeritz Looks good. I added some more to the documentation for changing a flyway managed db to sequelize management. Nonblocking question: what is server/migrations/.gitignore for?

evmiguel · 2020-05-20T18:58:28Z

Vagrantfile

@@ -13,6 +13,8 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  # conflict with that of another project.
  config.vm.network :private_network, ip: '192.168.10.40'

+  config.vm.post_up_message = 'The app is now running at http://192.168.10.40/'


Vagrant changes with config update work beautifully!

evmiguel · 2020-05-20T20:00:54Z

docs/database.md

This is awesome!

evmiguel · 2020-05-20T20:02:04Z

server/migrations/20200518185559-initial-migration.js

+const Promise = require('bluebird');
+const fs = require('fs');
+
+module.exports = {


I like this, and I think it's ok to create our initial state from the flyway dump.

evmiguel · 2020-05-20T20:04:02Z

server/seeders/20200518222159-role.js

@@ -0,0 +1,15 @@
+'use strict';
+
+module.exports = {


evmiguel · 2020-05-20T20:05:21Z

server/seeders/20200518223540-test_status.js

@@ -0,0 +1,15 @@
+'use strict';
+
+module.exports = {


evmiguel · 2020-05-20T20:05:33Z

server/seeders/20200518223022-browser.js

@@ -0,0 +1,15 @@
+'use strict';
+
+module.exports = {


Loving these seeders!

rmeritz · 2020-05-20T20:16:52Z

I added some more to the documentation for changing a flyway managed db to sequelize management.

Your docs look really good. Thanks!

Nonblocking question: what is server/migrations/.gitignore for?

It can be removed. It was added so that the folder would be tracked by git even though there was nothing it in. But then I put stuff in the folder and it became irrelevant. It had made since at some point though.

rmeritz added 15 commits May 19, 2020 13:58

Ensure sequalizer cli can create models

9c85cc4

Without passing in additional configuration options.

Add instructions for connecting to db w/dotenv

46146bb

You can use dotenv to load the enviornment variable and it is much cleaner than exporting than manually.

Write migration in Sequelize

0a326ec

I looked into a bunch of ways of doing this automatically but none of them worked with the current version of sequelize but it is the most asked for plus-oned feature there is. I just ended up writting it up manually.

Switch JSON to JSONB

6998f11

https://www.depesz.com/2014/03/25/waiting-for-9-4-introduce-jsonb-a-structured-format-for-storing-json/

Use database dump to create migration file

8394438

Add initial data with seeds

810a4d9

Deploy without flyway

b5183c7

Fix deploy to work without flyway

d0c681a

Additionally remove flyway migraions and add deploy documentation.

Add a Vagrant up message to help users find app

468e3bd

Remove long hardcoded paths in ansible command

8361ad4

Add a new github oauth app just for Vagrant

035ac5c

This has the correct redirects.

Change datatype in initalial migration to jsonb

c67d722

Use the dotenv-cli to load enviornment vars

cb51871

https://www.npmjs.com/package/dotenv-cli

Fix Addd typo

4f83745

Fix the undoing the original migration

bfb899d

Now `yarn run sequelize -- db:migrate:undo` works.

rmeritz requested review from spectranaut and evmiguel May 19, 2020 22:08

evmiguel added 3 commits May 20, 2020 15:03

Rename Database.md to database.md

7685c6c

resolving merge conflict due to capitalization file name change (Database.md -> database.md)

Merge branch 'main' into migrations-rmeritz

5de4279

update documentation for users who will migrate from flyway to sequel…

b648ffe

…ize migration system

evmiguel approved these changes May 20, 2020

View reviewed changes

Delete .gitignore

c995bbb

evmiguel merged commit bb3aeb0 into main May 20, 2020

evmiguel deleted the migrations-rmeritz branch May 20, 2020 20:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove Flyway and move migrations to Sequelize #97

Remove Flyway and move migrations to Sequelize #97

rmeritz commented May 19, 2020 •

edited

Loading

evmiguel left a comment

evmiguel May 20, 2020

evmiguel May 20, 2020

evmiguel May 20, 2020

evmiguel May 20, 2020

evmiguel May 20, 2020

evmiguel May 20, 2020

rmeritz commented May 20, 2020

Remove Flyway and move migrations to Sequelize #97

Remove Flyway and move migrations to Sequelize #97

Conversation

rmeritz commented May 19, 2020 • edited Loading

evmiguel left a comment

Choose a reason for hiding this comment

evmiguel May 20, 2020

Choose a reason for hiding this comment

evmiguel May 20, 2020

Choose a reason for hiding this comment

evmiguel May 20, 2020

Choose a reason for hiding this comment

evmiguel May 20, 2020

Choose a reason for hiding this comment

evmiguel May 20, 2020

Choose a reason for hiding this comment

evmiguel May 20, 2020

Choose a reason for hiding this comment

rmeritz commented May 20, 2020

rmeritz commented May 19, 2020 •

edited

Loading