Setup test database for running tests for database

[cychu42]

Part of #238
This sets up a test database starchart_test in the container test_database, and it can be used to run tests without interfering with the other database container or any other database.

For convenience, I added a .env.test file, for devs to run cp .env.test .env to change the database connection string to the test database.

Changes

• Add test database in in docker-compose.yml
• Add .env.test
• Ignore the volume at /mysql-data-test for git, Prettier, ESlint, and Docker.

Testing

1. Run cp .env.test .env
2. Run docker compose up -d to make containers like usual
3. Run npm run steup to connect, push schema, and seed the database
4. Run npm run db:studio to check that the database works

[cychu42]

I wonder if there's some issue with the what GitHub action is using to run hadolint. Saw the same test error in Won's PR that has nothing to do with Dockerfile.

[humphd]

I filed hadolint/hadolint-action#79 on the hadolint-action failures.

[cychu42]

Looks like it's fixed.

[cychu42]

Glad you showed me that SQL setup file method. It's pretty interesting.

Both databases now run under the same database container.
I just use root user to access the databases for now, since I'm not sure why we need another account, given the containers are for local development and will be thrown away anyway.
/mysql/data will be where data is persisted, while /mysql/init has the SQL file for creating the two databases and the root user.

[humphd]

Let's not rename mysql-data. We can move the .sql you created into config/ and then mount it where it needs to go in Docker.

For tests vs. running dev, the app uses dev-secrets/DATABASE_URL. How will we deal with differences here?

Ask @dadolhay if there is anything to know about with respect to using the root user like this. I think it's fine, but I'm not an expert.

Also, do you want to blow-away the test-db every time you init in your .sql? It might make it easier to run tests.

[cychu42]

For tests vs. running dev, the app uses dev-secrets/DATABASE_URL. How will we deal with differences here?

I'm not sure I understand. Is updating dev-secrets/DATABASE_URL not enough?

[humphd]

It is not. The DATABASE_URL in the env is only used for calling Prisma cli. At runtime, the app uses a secret.

[cychu42]

Looking at the test and dev scripts, we use SECRETS_OVERRIDE=1 to use what's inside /dev-secrets for testing and development, instead of using Docker secrets.

[cychu42]

How about this? I have applied changes according to your feedbacks, but I keep the original setup for starchart database intact in docker-compose.yml so we don't have to worry about changing much (including the secrets). I just add code relevant to the SQL file to make that test database.
If someone wants to switch to test database, they just run cp .env.test .env, and then run npm run setup or npx prisma db push if the schema isn't there already.

In my mind, I'm not sure it's necessary to drop the test database for testing, since tests should probably clean the test database before running tests, to avoid interference between each tests or any manual testing.

I haven't heard from Denes. Given both startchart user and root user have superuser access, I don't expect there to be a difference, and root user is now only used for the test database anyway.

[humphd]

I like this. The only thing I'd like to see improved is not having a duplicate .env in here. If we have a lot of these files, people have to remember to update them. Can we eliminate the need for that?

Obviously people running the tests (e.g., CI) would need to set this value to override it.

I don't want to make this impossible, but I also don't want more maintenance hassle.

[cychu42]

The .env.test is just an alternative to manually replacing the connection string.
I guess we can tell people to do that in CONTRIBUTION.md with copy/paste, unless we want to start setting the connection string env variable in every test script.

[cychu42]

If we don't want another .env fiIe, I guess adding it into scripts is cleaner than asking people to change connection string back and forth by hand every time? That would definitely need an instruction in CONTRIBUTION.md, so I added it.

[Eakam1007]

This doesn't need to be done in this PR, but would it be worth to add a new setup:test script to do this instead?

[cychu42]

Perhaps. If someone can write a script that has a command to parse the file to change the variable, runs npm run setup, then changes it back.

[Eakam1007]

Ah I think we can simply copy the setup command to setup:test, and override the value for DATABASE_URL

[cychu42]

@Eakam1007 do you want to file an issue?

[Eakam1007]

I think I might just create a PR for this today. Seems simple enough (unless I am missing something)