.Net: Cannot get the value of a token type 'False' as a number when using Chroma Memory Store

[paolosalvatori]

Describe the bug
It looks like the problem is still there. I use the latest Microsoft.SemanticKernel.Connectors.Memory.Chroma, 0.24.230911.2-preview nuget package in the 09-memory-with-chroma.ipynb sample. I get the following error:

**Error: System.Text.Json.JsonException: The JSON value could not be converted to Microsoft.SemanticKernel.Memory.MemoryRecordMetadata. Path: $.is_reference | LineNumber: 0 | BytePositionInLine: 102.
---> System.InvalidOperationException: Cannot get the value of a token type 'False' as a number.

The issue has been already reported several times: #2568, #2049 and a PR was created: #2072

To Reproduce
Steps to reproduce the behavior:

1. Go to09-memory-with-chroma.ipynb polygot notebook
2. Use the Microsoft.SemanticKernel.Connectors.Memory.Chroma, 0.24.230911.2-preview nuget package
3. Run the sample
4. See error:

**Error: System.Text.Json.JsonException: The JSON value could not be converted to Microsoft.SemanticKernel.Memory.MemoryRecordMetadata. Path: $.is_reference | LineNumber: 0 | BytePositionInLine: 102.
---> System.InvalidOperationException: Cannot get the value of a token type 'False' as a number.
at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_ExpectedNumber(JsonTokenType tokenType)
at System.Text.Json.Utf8JsonReader.TryGetInt16(Int16& value)
at Microsoft.SemanticKernel.Connectors.Memory.Chroma.ChromaBooleanConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
at System.Text.Json.Serialization.JsonConverter`1.TryReadAsObject(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state, Object& value)
at System.Text.Json.Serialization.Converters.LargeObjectWithParameterizedConstructorConverter`1.ReadAndCacheConstructorArgument(ReadStack& state, Utf8JsonReader& reader, JsonParameterInfo jsonParameterInfo)
at System.Text.Json.Serialization.Converters.ObjectWithParameterizedConstructorConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
--- End of inner exception stack trace ---
at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
at Microsoft.SemanticKernel.Connectors.Memory.Chroma.ChromaMemoryStore.GetMetadataForMemoryRecord(List`1 metadatas, Int32 recordIndex)
at Microsoft.SemanticKernel.Connectors.Memory.Chroma.ChromaMemoryStore.GetMemoryRecordFromModel(List`1 metadatas, List`1 embeddings, List`1 ids, Int32 recordIndex)
at Microsoft.SemanticKernel.Connectors.Memory.Chroma.ChromaMemoryStore.GetMemoryRecordFromQueryResultModel(ChromaQueryResultModel queryResultModel, Int32 recordIndex)
at Microsoft.SemanticKernel.Connectors.Memory.Chroma.ChromaMemoryStore.GetNearestMatchesAsync(String collectionName, ReadOnlyMemory`1 embedding, Int32 limit, Double minRelevanceScore, Boolean withEmbeddings, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Connectors.Memory.Chroma.ChromaMemoryStore.GetNearestMatchesAsync(String collectionName, ReadOnlyMemory`1 embedding, Int32 limit, Double minRelevanceScore, Boolean withEmbeddings, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at Microsoft.SemanticKernel.Memory.SemanticTextMemory.SearchAsync(String collection, String query, Int32 limit, Double minRelevanceScore, Boolean withEmbeddings, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Memory.SemanticTextMemory.SearchAsync(String collection, String query, Int32 limit, Double minRelevanceScore, Boolean withEmbeddings, CancellationToken cancellationToken)+MoveNext()
at Microsoft.SemanticKernel.Memory.SemanticTextMemory.SearchAsync(String collection, String query, Int32 limit, Double minRelevanceScore, Boolean withEmbeddings, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at System.Linq.AsyncEnumerable.<TryGetFirst>g__Core|95_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/FirstOrDefault.cs:line 130
at System.Linq.AsyncEnumerable.<TryGetFirst>g__Core|95_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/FirstOrDefault.cs:line 132
at System.Linq.AsyncEnumerable.<FirstOrDefaultAsync>g__Core|91_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/FirstOrDefault.cs:line 30
at Submission#6.<<Initialize>>d__0.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray`1 precedingExecutors, Func`2 currentExecutor, StrongBox`1 exceptionHolderOpt, Func`2 catchExceptionOpt, CancellationToken cancellationToken)**

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Platform**
 - OS: [e.g. Windows, Mac]
 - IDE: [e.g. Visual Studio, VS Code]
 - Language: [e.g. C#, Python]
 - Source: [e.g. NuGet package version 0.1.0, pip package version 0.1.0, main branch of repository]

**Additional context**
Add any other context about the problem here.

[dmytrostruk]

@paolosalvatori Good catch, thank you for creating this issue!
Unfortunately, this is happening every time when Chroma releases breaking changes, so we need to handle them on our side as well.

This PR should resolve an issue: #2796
In case you need a quick fix, you should be able to run examples with Chroma 0.4.0. I just checked notebook and integration tests and it's working as expected.

As part of my PR, I've updated notebook and Chroma README file to mention the latest Chroma version, which was used to verify Chroma connector and examples.

I noticed that Chroma notebook also contains some obsolete code, which should be updated together with SK version upgrade. We will update it as soon as new SK version will be published, including PR mentioned above.

Thanks again!

[paolosalvatori]

Thanks @dmytrostruk, will the fix ship in the next SK release? If yes, what is the ETA?

[dmytrostruk]

@paolosalvatori Preview version of package with fix should be published to NuGet next Monday (18th September).

But if you want to work with it earlier, we have nightly builds, the latest one was published 7 hours ago, which should include the fix:
https://github.com/microsoft/semantic-kernel/pkgs/nuget/Microsoft.SemanticKernel.Connectors.Memory.Chroma/127417566