Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FIX: sort strings in UTF-8 encoded byte order #1967

Merged
merged 11 commits into from
Jan 15, 2025
Merged

FIX: sort strings in UTF-8 encoded byte order #1967

merged 11 commits into from
Jan 15, 2025
+212 −5

Conversation

milaGGL
Copy link
Contributor

@milaGGL milaGGL commented Jan 6, 2025 •
edited
Loading

Strings should be sorted in UTF-8 encoded byte order. Public document: https://cloud.google.com/firestore/docs/concepts/data-types#data_types

SDK sorts strings using built in comparator method, which sorts lexicographically, and leads to mismatch between server and sdk when special characters are present. This PR fixes the string order mismatches on document field, map key, and document key.

@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: firestore Issues related to the googleapis/java-firestore API. labels Jan 6, 2025
@generated-files-bot Generated Files Bot
Copy link

Warning: This pull request is touching the following templated files:

  • google-cloud-firestore-admin/src/main/java/com/google/cloud/firestore/v1/FirestoreAdminClient.java
  • google-cloud-firestore-admin/src/main/java/com/google/cloud/firestore/v1/FirestoreAdminSettings.java
  • google-cloud-firestore-admin/src/main/java/com/google/cloud/firestore/v1/package-info.java
  • google-cloud-firestore-admin/src/main/java/com/google/cloud/firestore/v1/stub/FirestoreAdminStub.java
  • google-cloud-firestore-admin/src/main/java/com/google/cloud/firestore/v1/stub/FirestoreAdminStubSettings.java
  • google-cloud-firestore-admin/src/main/java/com/google/cloud/firestore/v1/stub/GrpcFirestoreAdminCallableFactory.java
  • google-cloud-firestore-admin/src/main/java/com/google/cloud/firestore/v1/stub/GrpcFirestoreAdminStub.java
  • google-cloud-firestore-admin/src/test/java/com/google/cloud/firestore/v1/FirestoreAdminClientTest.java
  • google-cloud-firestore-admin/src/test/java/com/google/cloud/firestore/v1/MockFirestoreAdmin.java
  • google-cloud-firestore-admin/src/test/java/com/google/cloud/firestore/v1/MockFirestoreAdminImpl.java
  • google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/FirestoreClient.java
  • google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/FirestoreSettings.java
  • google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/package-info.java
  • google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/stub/FirestoreStub.java
  • google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/stub/FirestoreStubSettings.java
  • google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/stub/GrpcFirestoreCallableFactory.java
  • google-cloud-firestore/src/main/java/com/google/cloud/firestore/v1/stub/GrpcFirestoreStub.java
  • google-cloud-firestore/src/test/java/com/google/cloud/firestore/v1/FirestoreClientTest.java
  • google-cloud-firestore/src/test/java/com/google/cloud/firestore/v1/MockFirestore.java
  • google-cloud-firestore/src/test/java/com/google/cloud/firestore/v1/MockFirestoreImpl.java
  • grpc-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/FirestoreAdminGrpc.java
  • grpc-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/FirestoreGrpc.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/CollectionGroupName.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/CreateIndexRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/CreateIndexRequestOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/DatabaseName.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/DeleteIndexRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/DeleteIndexRequestOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ExportDocumentsMetadata.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ExportDocumentsMetadataOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ExportDocumentsRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ExportDocumentsRequestOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ExportDocumentsResponse.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ExportDocumentsResponseOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/Field.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/FieldName.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/FieldOperationMetadata.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/FieldOperationMetadataOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/FieldOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/FieldProto.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/FirestoreAdminProto.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/GetFieldRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/GetFieldRequestOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/GetIndexRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/GetIndexRequestOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ImportDocumentsMetadata.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ImportDocumentsMetadataOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ImportDocumentsRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ImportDocumentsRequestOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/Index.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/IndexName.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/IndexOperationMetadata.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/IndexOperationMetadataOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/IndexOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/IndexProto.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ListFieldsRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ListFieldsRequestOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ListFieldsResponse.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ListFieldsResponseOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ListIndexesRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ListIndexesRequestOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ListIndexesResponse.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ListIndexesResponseOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/LocationMetadata.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/LocationMetadataOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/LocationProto.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/OperationProto.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/OperationState.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/Progress.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/ProgressOrBuilder.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/UpdateFieldRequest.java
  • proto-google-cloud-firestore-admin-v1/src/main/java/com/google/firestore/admin/v1/UpdateFieldRequestOrBuilder.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundleElement.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundleElementOrBuilder.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundleMetadata.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundleMetadataOrBuilder.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundleProto.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundledDocumentMetadata.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundledDocumentMetadataOrBuilder.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundledQuery.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/BundledQueryOrBuilder.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/NamedQuery.java
  • proto-google-cloud-firestore-bundle-v1/src/main/java/com/google/firestore/bundle/NamedQueryOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ArrayValue.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ArrayValueOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BatchGetDocumentsRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BatchGetDocumentsRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BatchGetDocumentsResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BatchGetDocumentsResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BatchWriteRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BatchWriteRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BatchWriteResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BatchWriteResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BeginTransactionRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BeginTransactionRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BeginTransactionResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/BeginTransactionResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/CommitRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/CommitRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/CommitResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/CommitResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/CommonProto.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/CreateDocumentRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/CreateDocumentRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/Cursor.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/CursorOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DeleteDocumentRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DeleteDocumentRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/Document.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentChange.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentChangeOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentDelete.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentDeleteOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentMask.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentMaskOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentProto.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentRemove.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentRemoveOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentTransform.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/DocumentTransformOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ExistenceFilter.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ExistenceFilterOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/FirestoreProto.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/GetDocumentRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/GetDocumentRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListCollectionIdsRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListCollectionIdsRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListCollectionIdsResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListCollectionIdsResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListDocumentsRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListDocumentsRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListDocumentsResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListDocumentsResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListenRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListenRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListenResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ListenResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/MapValue.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/MapValueOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/PartitionQueryRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/PartitionQueryRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/PartitionQueryResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/PartitionQueryResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/Precondition.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/PreconditionOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/QueryProto.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/RollbackRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/RollbackRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/RunQueryRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/RunQueryRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/RunQueryResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/RunQueryResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/StructuredQuery.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/StructuredQueryOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/Target.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/TargetChange.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/TargetChangeOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/TargetOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/TransactionOptions.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/TransactionOptionsOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/UpdateDocumentRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/UpdateDocumentRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/Value.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/ValueOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/Write.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/WriteOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/WriteProto.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/WriteRequest.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/WriteRequestOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/WriteResponse.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/WriteResponseOrBuilder.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/WriteResult.java
  • proto-google-cloud-firestore-v1/src/main/java/com/google/firestore/v1/WriteResultOrBuilder.java

@milaGGL milaGGL changed the title add test sort strings in UTF-8 encoded byte order Jan 6, 2025
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Jan 7, 2025
@milaGGL milaGGL marked this pull request as ready for review January 10, 2025 15:58
@milaGGL milaGGL requested review from a team as code owners January 10, 2025 15:58
@milaGGL milaGGL mentioned this pull request Jan 13, 2025
Merged
@milaGGL milaGGL changed the title sort strings in UTF-8 encoded byte order FIX: sort strings in UTF-8 encoded byte order Jan 13, 2025
ehsannas
ehsannas approved these changes Jan 14, 2025
View reviewed changes
google-cloud-firestore/src/main/java/com/google/cloud/firestore/Order.java Outdated

private static int compareBytes(ByteString leftBytes, ByteString rightBytes) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compareByteStrings would be a better name.

@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Jan 15, 2025
@milaGGL milaGGL merged commit 4309639 into main Jan 15, 2025
23 checks passed
@milaGGL milaGGL deleted the mila/string-uses-byte-comparison branch January 15, 2025 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ehsannas ehsannas ehsannas approved these changes

Assignees
No one assigned
Labels
api: firestore Issues related to the googleapis/java-firestore API. size: m Pull request size is medium.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@milaGGL @ehsannas @cloud-java-bot