Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for repoGet request #6273

Merged
merged 56 commits into from
Jun 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
db50631
WIP
maneesht May 13, 2022
46cd378
Simulate add/remove event registration in repoGetValue
jmwski May 13, 2022
c334171
Fixed duplicate onValue calls
maneesht May 17, 2022
f7298df
Updated test file
maneesht May 18, 2022
00a5907
Updated integration test to include get
maneesht May 19, 2022
3ea9604
Reduced timeout for tests
maneesht May 20, 2022
9520427
Removed unnecessary imports
maneesht May 20, 2022
e7dd651
Added back imports
maneesht May 20, 2022
f7ed576
Removed snapshot param
maneesht May 20, 2022
98c6e34
Addressed comments
maneesht May 23, 2022
43f1f8a
Fixed test
maneesht May 23, 2022
9394170
Removed only
maneesht May 23, 2022
2f82052
Removed unnecessary import
maneesht May 23, 2022
c976987
Updated test
maneesht May 23, 2022
fc4cf5c
Added extra test
maneesht May 24, 2022
7c31ece
Merge branch 'master' into fix-get-cache
maneesht May 25, 2022
3959cf3
Fixed formatting
maneesht May 25, 2022
536e103
Create rotten-tables-brush.md
maneesht May 25, 2022
3812a33
Added documentation
maneesht May 25, 2022
a8d8ea6
Passed formatting
maneesht May 25, 2022
7de3c12
Merge remote-tracking branch 'origin/master' into fix-get-cache
maneesht May 26, 2022
766f46d
Addressed comments
maneesht May 26, 2022
04356e4
WIP
maneesht May 31, 2022
a467055
Used unique id instead of foo
maneesht May 31, 2022
6145b2b
Used exactCount instead of Count for accumulator
maneesht Jun 1, 2022
788aaa0
Removed only
maneesht Jun 1, 2022
51d57a2
Reformatted
maneesht Jun 1, 2022
41a7303
Removed get wip
maneesht Jun 2, 2022
a58bb33
Got minimal repro
maneesht Jun 2, 2022
5d41ce0
Got minimal repro
maneesht Jun 2, 2022
bdfd40b
Fixed race condition issue
maneesht Jun 2, 2022
4ca2289
Updated tests
maneesht Jun 2, 2022
b52f468
Updated yarn lock file
maneesht Jun 3, 2022
27f9ea1
Updated module type
maneesht Jun 3, 2022
065dab0
Added comment
maneesht Jun 3, 2022
d5fef45
Merge remote-tracking branch 'origin/master' into fix-get-cache
maneesht Jun 3, 2022
74fe5e8
Removed unnecessary code
maneesht Jun 3, 2022
5858b86
Merge remote-tracking branch 'origin/master' into fix-get-cache
maneesht Jun 6, 2022
be41979
Removed unnecessary comment
maneesht Jun 6, 2022
f320a93
Removed unused imports
maneesht Jun 6, 2022
c4f8ff7
Updated test to remove redundant assertion
maneesht Jun 6, 2022
1dc798c
Moved query check
maneesht Jun 8, 2022
42e38f6
Updated tests to include example queries
maneesht Jun 8, 2022
b4c7f28
Fixed formatting
maneesht Jun 8, 2022
5559921
Added suggestions
maneesht Jun 8, 2022
52fcf72
Fixed formatting
maneesht Jun 8, 2022
c553405
Replaced let with const
maneesht Jun 8, 2022
a7e5865
Fixed tests
maneesht Jun 9, 2022
98845d1
Fixed test
maneesht Jun 9, 2022
69feeb7
Fixed formatting
maneesht Jun 9, 2022
60f9da0
Fixed tests
maneesht Jun 15, 2022
6f39fef
Removed comments
maneesht Jun 15, 2022
57a8db5
Removed version of uuid
maneesht Jun 15, 2022
31a747e
Added a new line
maneesht Jun 15, 2022
bd23960
Addressed comments
maneesht Jun 16, 2022
7e09d4b
Fixed formatting
maneesht Jun 16, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/rotten-tables-brush.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@firebase/database": patch
---

Fixed issue where `get()` saved results incorrectly for non-default queries.
9 changes: 5 additions & 4 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,15 @@
"type": "node",
"request": "launch",
"name": "RTDB Unit Tests (Node)",
"program": "${workspaceRoot}/packages/firebase/node_modules/.bin/_mocha",
"program": "${workspaceRoot}/node_modules/.bin/_mocha",
"cwd": "${workspaceRoot}/packages/database",
"args": [
"test/{,!(browser)/**/}*.test.ts",
"--file", "index.node.ts",
"--config", "../../config/mocharc.node.js"
"--file", "src/index.node.ts",
"--config", "../../config/mocharc.node.js",
],
"env": {
"TS_NODE_FILES":true,
"TS_NODE_CACHE": "NO",
"TS_NODE_COMPILER_OPTIONS" : "{\"module\":\"commonjs\"}"
},
Expand All @@ -30,7 +31,7 @@
"cwd": "${workspaceRoot}/packages/firestore",
"args": [
"--require", "babel-register.js",
"--require", "index.node.ts",
"--require", "src/index.node.ts",
"--timeout", "5000",
"test/{,!(browser|integration)/**/}*.test.ts",
"--exit"
Expand Down
3 changes: 2 additions & 1 deletion packages/database/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,8 @@
"@firebase/app": "0.7.25",
"rollup": "2.72.1",
"rollup-plugin-typescript2": "0.31.2",
"typescript": "4.2.2"
"typescript": "4.2.2",
"uuid": "^8.3.2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid bringing in an extra dep here? I see in app-check we just implemented a function ourselves, is this good enough?

export function uuidv4(): string {

If it is, maybe we can extract it into @firebase/util and share it between app-check and database (maybe in a later PR, just have a duplicate implementation in this one?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, let me just reimplement it here and add a TODO

},
"repository": {
"directory": "packages/database",
Expand Down
8 changes: 1 addition & 7 deletions packages/database/src/core/PersistentConnection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -206,12 +206,6 @@ export class PersistentConnection extends ServerActions {
onComplete: (message: { [k: string]: unknown }) => {
const payload = message['d'] as string;
if (message['s'] === 'ok') {
this.onDataUpdate_(
request['p'],
payload,
/*isMerge*/ false,
/*tag*/ null
);
deferred.resolve(payload);
} else {
deferred.reject(payload);
Expand Down Expand Up @@ -265,7 +259,7 @@ export class PersistentConnection extends ServerActions {
);
assert(
!this.listens.get(pathString)!.has(queryId),
'listen() called twice for same path/queryId.'
`listen() called twice for same path/queryId.`
);
const listenSpec: ListenSpec = {
onComplete,
Expand Down
35 changes: 28 additions & 7 deletions packages/database/src/core/Repo.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,8 @@ import {
syncTreeApplyUserOverwrite,
syncTreeCalcCompleteEventCache,
syncTreeGetServerValue,
syncTreeRemoveEventRegistration
syncTreeRemoveEventRegistration,
syncTreeRegisterQuery
} from './SyncTree';
import { Indexable } from './util/misc';
import {
Expand Down Expand Up @@ -466,16 +467,36 @@ export function repoGetValue(repo: Repo, query: QueryContext): Promise<Node> {
}
return repo.server_.get(query).then(
payload => {
const node = nodeFromJSON(payload as string).withIndex(
const node = nodeFromJSON(payload).withIndex(
query._queryParams.getIndex()
);
const events = syncTreeApplyServerOverwrite(
// if this is a filtered query, then overwrite at path
if (query._queryParams.loadsAllData()) {
syncTreeApplyServerOverwrite(repo.serverSyncTree_, query._path, node);
} else {
// Simulate `syncTreeAddEventRegistration` without events/listener setup.
jmwski marked this conversation as resolved.
Show resolved Hide resolved
// We do this (along with the syncTreeRemoveEventRegistration` below) so that
// `repoGetValue` results have the same cache effects as initial listener(s)
// updates.
const tag = syncTreeRegisterQuery(repo.serverSyncTree_, query);
syncTreeApplyTaggedQueryOverwrite(
repo.serverSyncTree_,
query._path,
node,
tag
);
// Call `syncTreeRemoveEventRegistration` with a null event registration, since there is none.
// Note: The below code essentially unregisters the query and cleans up any views/syncpoints temporarily created above.
}
const cancels = syncTreeRemoveEventRegistration(
repo.serverSyncTree_,
query._path,
node
query,
null
);
eventQueueRaiseEventsAtPath(repo.eventQueue_, query._path, events);
return Promise.resolve(node);
if (cancels.length > 0) {
repoLog(repo, 'unexpected cancel events in repoGetValue');
}
return node;
},
err => {
repoLog(repo, 'get for query ' + stringify(query) + ' failed: ' + err);
Expand Down
75 changes: 67 additions & 8 deletions packages/database/src/core/SyncTree.ts
Original file line number Diff line number Diff line change
Expand Up @@ -423,6 +423,38 @@ export function syncTreeRemoveEventRegistration(
return cancelEvents;
}

/**
* This function was added to support non-listener queries,
* specifically for use in repoGetValue. It sets up all the same
* local cache data-structures (SyncPoint + View) that are
* needed for listeners without installing an event registration.
* If `query` is not `loadsAllData`, it will also provision a tag for
* the query so that query results can be merged into the sync
* tree using existing logic for tagged listener queries.
*
* @param syncTree - Synctree to add the query to.
* @param query - Query to register
* @returns tag as a string if query is not a default query, null if query is not.
*/
export function syncTreeRegisterQuery(syncTree: SyncTree, query: QueryContext) {
jmwski marked this conversation as resolved.
Show resolved Hide resolved
jmwski marked this conversation as resolved.
Show resolved Hide resolved
const { syncPoint, serverCache, writesCache, serverCacheComplete } =
syncTreeRegisterSyncPoint(query, syncTree);
const view = syncPointGetView(
syncPoint,
query,
writesCache,
serverCache,
serverCacheComplete
);
if (!syncPoint.views.has(query._queryIdentifier)) {
syncPoint.views.set(query._queryIdentifier, view);
}
if (!query._queryParams.loadsAllData()) {
jmwski marked this conversation as resolved.
Show resolved Hide resolved
return syncTreeTagForQuery_(syncTree, query);
}
return null;
}

/**
* Apply new server data for the specified tagged query.
*
Expand Down Expand Up @@ -483,15 +515,14 @@ export function syncTreeApplyTaggedQueryMerge(
}

/**
* Add an event callback for the specified query.
*
* @returns Events to raise.
* Creates a new syncpoint for a query and creates a tag if the view doesn't exist.
* Extracted from addEventRegistration to allow `repoGetValue` to properly set up the SyncTree
* without actually listening on a query.
*/
export function syncTreeAddEventRegistration(
syncTree: SyncTree,
export function syncTreeRegisterSyncPoint(
jmwski marked this conversation as resolved.
Show resolved Hide resolved
query: QueryContext,
eventRegistration: EventRegistration
): Event[] {
syncTree: SyncTree
) {
const path = query._path;

let serverCache: Node | null = null;
Expand Down Expand Up @@ -550,6 +581,35 @@ export function syncTreeAddEventRegistration(
syncTree.tagToQueryMap.set(tag, queryKey);
}
const writesCache = writeTreeChildWrites(syncTree.pendingWriteTree_, path);
return {
syncPoint,
writesCache,
serverCache,
serverCacheComplete,
foundAncestorDefaultView,
viewAlreadyExists
};
}

/**
* Add an event callback for the specified query.
*
* @returns Events to raise.
*/
export function syncTreeAddEventRegistration(
syncTree: SyncTree,
query: QueryContext,
eventRegistration: EventRegistration
): Event[] {
const {
syncPoint,
serverCache,
writesCache,
serverCacheComplete,
viewAlreadyExists,
foundAncestorDefaultView
} = syncTreeRegisterSyncPoint(query, syncTree);

let events = syncPointAddEventRegistration(
syncPoint,
query,
Expand Down Expand Up @@ -937,7 +997,6 @@ function syncTreeSetupListener_(
const path = query._path;
const tag = syncTreeTagForQuery_(syncTree, query);
const listener = syncTreeCreateListenerForView_(syncTree, view);

const events = syncTree.listenProvider_.startListening(
syncTreeQueryForListening_(query),
tag,
Expand Down
2 changes: 1 addition & 1 deletion packages/database/src/core/util/Path.ts
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ export function pathEquals(path: Path, other: Path): boolean {
}

/**
* @returns True if this path is a parent (or the same as) other
* @returns True if this path is a parent of (or the same as) other
*/
export function pathContains(path: Path, other: Path): boolean {
let i = path.pieceNum_;
Expand Down
Loading