feat(firestore): Adding distance threshold and result field

[bhshkh]

The proposed changes in current PR can be used like below:

client.Collection(collName).
    FindNearest("path", []float32{5, 6, 7}, limit, DistanceMeasureEuclidean, firestore.FindNearestOptions{
				DistanceThreshold:   Ptr(20.0),
				DistanceResultField: resultField,
			}).
   

Two new fields have been added to FindNearest proto:

google-cloud-go/firestore/apiv1/firestorepb/query.pb.go

Lines 1325 to 1337 in 9a5144e

	// Optional. Optional name of the field to output the result of the vector
	// distance calculation. Must conform to [document field
	// name][google.firestore.v1.Document.fields] limitations.
	DistanceResultField string `protobuf:"bytes,5,opt,name=distance_result_field,json=distanceResultField,proto3" json:"distance_result_field,omitempty"`
	// Optional. Option to specify a threshold for which no less similar
	// documents will be returned. The behavior of the specified
	// `distance_measure` will affect the meaning of the distance threshold.
	// Since DOT_PRODUCT distances increase when the vectors are more similar,
	// the comparison is inverted.
	//
	// For EUCLIDEAN, COSINE: WHERE distance <= distance_threshold
	// For DOT_PRODUCT: WHERE distance >= distance_threshold
	DistanceThreshold *wrapperspb.DoubleValue `protobuf:"bytes,6,opt,name=distance_threshold,json=distanceThreshold,proto3" json:"distance_threshold,omitempty"`

1. DistanceResultField
2. DistanceThreshold

'DistanceThreshold' is included in 'FindNearestOptions' as a pointer to a float64 (*float64) rather than a simple float64. This distinction allows us to differentiate between cases where the value is explicitly set and cases where it defaults to 0.

If vectors are like below and the query vector is {3.0, 1.0, 2.0}

docs := []CoffeeBean{
        {
	        Name:           "Kahawa coffee beans",
	        Description:    "Information about the Kahawa coffee beans.",
	        EmbeddingField: []float32{1.0, 2.0, 3.0},
	        Color:          "red",
        },
        {
	        Name:           "Onyx coffee beans",
	        Description:    "Information about the Onyx coffee beans.",
	        EmbeddingField: []float32{4.0, 5.0, 6.0},
	        Color:          "brown",
        },
        {
	        Name:           "SleepyOwl coffee beans",
	        Description:    "Information about the Kahawa coffee beans.",
	        EmbeddingField: []float32{3.0, 1.0, 2.0},
	        Color:          "red",
        },
}

then:
-- distance threshold 0 returns:

map[color:red description:Information about the Kahawa coffee beans. embedding_field:[3 1 2] name:SleepyOwl coffee beans]

-- distance threshold unset returns:

map[color:red description:Information about the Kahawa coffee beans. embedding_field:[3 1 2] name:SleepyOwl coffee beans]
map[color:red description:Information about the Kahawa coffee beans. embedding_field:[1 2 3] name:Kahawa coffee beans]
map[color:brown description:Information about the Onyx coffee beans. embedding_field:[4 5 6] name:Onyx coffee beans]

[jba]

I believe a distance threshold of 0 is the same as unset, since distances can never be negative.
So I think using a float64 field is OK.

[bhshkh]

I believe a distance threshold of 0 is the same as unset, since distances can never be negative. So I think using a float64 field is OK.

@jba It yields different results:

If vectors are like below and the query vector is {3.0, 1.0, 2.0}

docs := []CoffeeBean{
        {
	        Name:           "Kahawa coffee beans",
	        Description:    "Information about the Kahawa coffee beans.",
	        EmbeddingField: []float32{1.0, 2.0, 3.0},
	        Color:          "red",
        },
        {
	        Name:           "Onyx coffee beans",
	        Description:    "Information about the Onyx coffee beans.",
	        EmbeddingField: []float32{4.0, 5.0, 6.0},
	        Color:          "brown",
        },
        {
	        Name:           "SleepyOwl coffee beans",
	        Description:    "Information about the Kahawa coffee beans.",
	        EmbeddingField: []float32{3.0, 1.0, 2.0},
	        Color:          "red",
        },
}

then:
-- distance threshold 0 returns:

map[color:red description:Information about the Kahawa coffee beans. embedding_field:[3 1 2] name:SleepyOwl coffee beans]

-- distance threshold unset returns:

map[color:red description:Information about the Kahawa coffee beans. embedding_field:[3 1 2] name:SleepyOwl coffee beans]
map[color:red description:Information about the Kahawa coffee beans. embedding_field:[1 2 3] name:Kahawa coffee beans]
map[color:brown description:Information about the Onyx coffee beans. embedding_field:[4 5 6] name:Onyx coffee beans]

[jba]

OK, but I still think we should put them in the options struct. They are actually options for FindNearest, that is the right place for them.
We can make the pointer stuff easier with a helper function. Here's what we did in the Gemini client: https://pkg.go.dev/github.com/google/[email protected]/genai#Ptr.

[bhshkh]

OK, but I still think we should put them in the options struct. They are actually options for FindNearest, that is the right place for them. We can make the pointer stuff easier with a helper function. Here's what we did in the Gemini client: https://pkg.go.dev/github.com/google/[email protected]/genai#Ptr.

This is awesome! Thanks for sharing.
Updated

[jba]

Looks pretty good. We should not submit until this whole feature goes GA.

[jba]

There should be an example for this in examples_test.go.

[bhshkh]

Curious why we maintain examples_test.go. Don't the sample snippets in golang-samples repo serve the same purpose ?

[jba]

Ideally there'd be one source of truth, but now they serve different purposes. The ones in example_test.go appear in pkgsite: https://pkg.go.dev/cloud.google.com/go/firestore#pkg-examples. The samples are linked from the GCP documentation.

[bhshkh]

Removed do not merge label, since feature is now GA

[daniel-sanche]

LGTM

[jba]

s/get/gets/