Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support for using multiple databases in datastore #1090

Merged
merged 75 commits into from
Sep 29, 2023

Commits on Mar 20, 2023

  1. Configuration menu
    Copy the full SHA
    11524c1 View commit details
    Browse the repository at this point in the history
  2. A simple test with multidb support

    Adds databaseId to the request options, client options and code for carrying over the client options to the request options as well as a test to make sure it actually works.
    danieljbruce committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    e06bee0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    1f464a5 View commit details
    Browse the repository at this point in the history
  4. Add additional checks with get

    The test should include additional checks to ensure get is running the way it is supposed to.
    danieljbruce committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    10da9c8 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    ecfce7d View commit details
    Browse the repository at this point in the history
  6. other datastore should delete the entity

    Since we are saving things to another datastore we need to delete the data from that datastore.
    danieljbruce committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    3b6a42c View commit details
    Browse the repository at this point in the history
  7. Add a test for the getDatastore method

    The user needs a way to get the database Id from the client so that they can keep track of their clients more easily.
    danieljbruce committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    e254be5 View commit details
    Browse the repository at this point in the history
  8. Add a unit test for database id

    Ensure that the database id gets passed all the way down to the generated layer and gets included in the request.
    danieljbruce committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    77970c8 View commit details
    Browse the repository at this point in the history
  9. Do an additional check for the sake of tests

    We should check for the existance of datastore.options so that the code doesn’t fail if they are not provided at all.
    danieljbruce committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    3b86ce2 View commit details
    Browse the repository at this point in the history
  10. initialize generated client so it can be mocked

    The test on kokoro is failing because in that environment a project id isn’t defined. The solution is to eliminate the get request so that a call never reaches the backend and initialize the gapic client so it can be mocked out and then catch any call that reaches the mock.
    danieljbruce committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    ea54946 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2023

  1. comments and setDatabaseId

    Add docs comments for newly added functions and add a test that ensures setDatabaseId works correctly.
    danieljbruce committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    38d955c View commit details
    Browse the repository at this point in the history
  2. Refactor of database name

    Don’t use the same string multiple times. When the database name we are using for testing changes then we should only have to make that change in one place.
    danieljbruce committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    57bc2eb View commit details
    Browse the repository at this point in the history
  3. correction to test against old database

    This was a typo. We want to test against the database we were using before
    danieljbruce committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    c9856b2 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2023

  1. getRequestWithDatabaseId function

    Add the getRequestWithDatabaseId function and change it so that it can easily be used in multiple places and the tests still pass.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    543e486 View commit details
    Browse the repository at this point in the history
  2. Change function name to addDatabaseIdToRequest

    The request options should contain databaseId so rename the function as appropriate.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    dde1eba View commit details
    Browse the repository at this point in the history
  3. Comments in all the places to add reqOpts

    This commit contains a comment in all the places we want to add the reqOpts change.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    8ba512a View commit details
    Browse the repository at this point in the history
  4. Add a warning to a certain test for easier debug

    Add a warning because the way the assert checking was done before, the test suite was not receiving any indication of which test was failing which makes debugging really hard.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    de946b2 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    dd7b06c View commit details
    Browse the repository at this point in the history
  6. datastore request id

    Add request ids back in all the places they are required
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    f167ce3 View commit details
    Browse the repository at this point in the history
  7. addDatabaseIdToRequest

    Add addDatabaseIdToRequest to the transaction test framework so that it can be used in the tests.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    364845b View commit details
    Browse the repository at this point in the history
  8. Solve all the test errors for createReadStream

    Mock out add database id for the tests in createReadStream that correspond to createReadStream.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    7b5f902 View commit details
    Browse the repository at this point in the history
  9. addDatabaseIdToRequest mock for runQueryStream

    Add the addDatabaseIdToRequest mock to the tests for every runQueryStream test so that the tests pass.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    a3bdb92 View commit details
    Browse the repository at this point in the history
  10. By default, mock out addDatabaseIdToRequest

    addDatabaseIdToRequest should be a default function in datastore object of request so that we don’t have to mock it out in many different places.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    9d7fab1 View commit details
    Browse the repository at this point in the history
  11. Revert "addDatabaseIdToRequest mock for runQueryStream"

    This reverts commit a3bdb92.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    d9a96bf View commit details
    Browse the repository at this point in the history
  12. Revert "Solve all the test errors for createReadStream"

    This reverts commit 7b5f902.
    danieljbruce committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    7543f57 View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2023

  1. linting

    lint an import
    danieljbruce committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    3f3e965 View commit details
    Browse the repository at this point in the history
  2. Revert "datastore request id"

    This reverts commit f167ce3.
    danieljbruce committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    32bb92b View commit details
    Browse the repository at this point in the history
  3. Revert "This fixed the bug for the save operation"

    This reverts commit dd7b06c.
    danieljbruce committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    92f8778 View commit details
    Browse the repository at this point in the history
  4. Revert "Comments in all the places to add reqOpts"

    This reverts commit 8ba512a.
    danieljbruce committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    6159882 View commit details
    Browse the repository at this point in the history
  5. Introduce addDatabaseIdRequest in request

    Introduce adding the databaseId in request layer again.
    danieljbruce committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    9410652 View commit details
    Browse the repository at this point in the history
  6. Declare a constant so that tests don’t run

    Importing test into the system-test folder causes the system-test command to also run the tests. Eliminate this import.
    danieljbruce committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    b34bf17 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2023

  1. Remove setDatabaseId

    Remove setDatabaseId as it provides a way for the user to get confused about the current state of the client.
    danieljbruce committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    2e1999a View commit details
    Browse the repository at this point in the history
  2. Add comment for addDatabaseIdToRequest function

    The function needs a comment to be consistent with docs.
    danieljbruce committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    bd97d00 View commit details
    Browse the repository at this point in the history
  3. Address header issue

    The new util file needs a header
    danieljbruce committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    d41ca60 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    1519ad3 View commit details
    Browse the repository at this point in the history
  5. Add databaseId for encoding and decoding keys

    The databaseId should be encoded on keys so that the user can write on a record by record basis
    danieljbruce committed Mar 28, 2023
    Configuration menu
    Copy the full SHA
    0ebabd5 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2023

  1. Revert "Add databaseId for encoding and decoding keys"

    This reverts commit 0ebabd5.
    danieljbruce committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    6449d10 View commit details
    Browse the repository at this point in the history
  2. Try a test with the namespace

    Try out a new test that looks at data details
    danieljbruce committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    e9ee522 View commit details
    Browse the repository at this point in the history
  3. Use the other datastore in the test

    Use another datastore with key
    danieljbruce committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    1ba9756 View commit details
    Browse the repository at this point in the history
  4. Create a test that looks at specific details

    The test should look at specific details in the key and should return the data with the right author.
    danieljbruce committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    d35cdee View commit details
    Browse the repository at this point in the history
  5. Add additional checks to existing test

    Add additional checks to existing test to show default database is unaffected.
    danieljbruce committed Mar 29, 2023
    Configuration menu
    Copy the full SHA
    769417d View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e694341 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    f307f4d View commit details
    Browse the repository at this point in the history

Commits on Mar 31, 2023

  1. Merge branch 'main' of https://github.com/googleapis/nodejs-datastore

    …into multi-db
    
    # Conflicts:
    #	system-test/datastore.ts
    danieljbruce committed Mar 31, 2023
    Configuration menu
    Copy the full SHA
    0f27ea4 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2023

  1. Configuration menu
    Copy the full SHA
    76a3dfc View commit details
    Browse the repository at this point in the history

Commits on May 19, 2023

  1. Use the database called multidb-test

    multidb-test is the name of the database set up on the project used in Github automation so we need to change this name in code to use that new database.
    danieljbruce committed May 19, 2023
    Configuration menu
    Copy the full SHA
    3fd19fb View commit details
    Browse the repository at this point in the history

Commits on May 31, 2023

  1. Configuration menu
    Copy the full SHA
    d4e3f91 View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2023

  1. Eliminate function on index that adds the id

    Currently a function is used as a passthrough to the function on the util file. This unnecessarily exposes a function to the user that they shouldn’t be concerned with.
    danieljbruce committed Jun 9, 2023
    Configuration menu
    Copy the full SHA
    771390c View commit details
    Browse the repository at this point in the history
  2. Fix the test so that the assertion error bubbles

    The assertion error should bubble up to the user if we send it through the callback.
    danieljbruce committed Jun 9, 2023
    Configuration menu
    Copy the full SHA
    ae15bc4 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6b5c542 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Add types so that purpose of function is clear

    Types currently set to any should be more specific. This will help clarify the purpose of the function that is used.
    danieljbruce committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    da64dab View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Configuration menu
    Copy the full SHA
    8d170fd View commit details
    Browse the repository at this point in the history
  2. Add types and names so that the function works

    The types should be more specific and the names should be more explicit and clear.
    danieljbruce committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    d49e5fc View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e7082ab View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2023

  1. Eliminate repeated code fragments

    The test case has the same lines for each secondary database key and these lines should not be repeated so we regroup them into better data structures for the job
    danieljbruce committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    b50b065 View commit details
    Browse the repository at this point in the history
  2. Better names and code organization

    The names of some variables were changed to align with others and code was moved so that code pertaining to the same function is grouped together
    danieljbruce committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    f3aadc2 View commit details
    Browse the repository at this point in the history
  3. Change the variable name of the default key

    There is only one default key so we change the variable name so that it doesn’t mention 1
    danieljbruce committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    65605e8 View commit details
    Browse the repository at this point in the history
  4. Refactor the post key hierarchy out

    Refactor the hierarchy out for a cleaner test
    danieljbruce committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    6986d4c View commit details
    Browse the repository at this point in the history

Commits on Aug 18, 2023

  1. Configuration menu
    Copy the full SHA
    0378dea View commit details
    Browse the repository at this point in the history
  2. Revert "Add only to the test we are interested in"

    This reverts commit 0378dea.
    danieljbruce committed Aug 18, 2023
    Configuration menu
    Copy the full SHA
    a190512 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. Configuration menu
    Copy the full SHA
    4b62317 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. Merge branch 'main' of https://github.com/googleapis/nodejs-datastore

    …into multi-db
    
    # Conflicts:
    #	src/index.ts
    #	system-test/datastore.ts
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    88e20ed View commit details
    Browse the repository at this point in the history
  2. Add second database id

    In the transaction tests we add the databaseId parameter. The databaseId parameter will run tests against the secondary database.
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    739796f View commit details
    Browse the repository at this point in the history
  3. Remove unnecessary mocks

    The mocks for addDatabaseIdToRequest are not necessary for these two files since they were only useful for the way the code was written before so should be removed.
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    87450ef View commit details
    Browse the repository at this point in the history
  4. Remove unused imports

    Imports that were used in old mocks no longer exist in the file so should be removed.
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    28b5aaa View commit details
    Browse the repository at this point in the history
  5. Add parameterized testing to the test/index file

    Parameterized testing can be added to the test/index file to test to see how the tests behave with databaseId set to the second database and without this setting too.
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    46a5627 View commit details
    Browse the repository at this point in the history
  6. Add parameterized testing to test/transaction.ts

    Parameterized testing should be used on transaction.ts. This commit adds the testing with async.
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    2cf3d98 View commit details
    Browse the repository at this point in the history
  7. This ensures that the tests still run on the deft

    Tests need to run on the default database as well as the secondary database. This commit ensures that they do.
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    9505083 View commit details
    Browse the repository at this point in the history
  8. Add parameterized testing for the system tests

    Parameterized testing for the system tests will ensure that tests run with the default database as well as with the secondary database.
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    e34735a View commit details
    Browse the repository at this point in the history
  9. Use default datastore variable in new tests

    The intent of the datastore variable in the new tests is to be the default database so that we are able to see if reads/writes affect the default or secondary database in these tests. They should not use the parameterized version of datastore.
    danieljbruce committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    eb55764 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. inline add database id

    Now that the tests don’t need to mock out adding database id function, we can inline the code that adds the database id to the request options.
    danieljbruce committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    992117f View commit details
    Browse the repository at this point in the history
  2. Modify the namespace in parameterized tests

    Parameterized tests should have a different namespace for the second set of tests for sure so we add this prefix.
    danieljbruce committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    7312f49 View commit details
    Browse the repository at this point in the history
  3. Move databaseId to SharedQueryOptions for alignme

    The databaseId should be moved to shared query options as discussed.
    danieljbruce committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    aef80a1 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2023

  1. Run the linter

    Linting was broken so that the PR could be read. Ran the linter again.
    danieljbruce committed Sep 27, 2023
    Configuration menu
    Copy the full SHA
    72255ef View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2023

  1. Configuration menu
    Copy the full SHA
    07dc746 View commit details
    Browse the repository at this point in the history