Self referencing loop detected with type 'NetTopologySuite.Features.AttributesTable'. Path '[257].properties'. #46
Comments
|
Without more details about the object being serialized, I can't say anything definitive. Have you tried setting |
|
I have tried that, unfortunately it didn't help. The in memory object that some times was deserialized from the above object was the following: It didn't happen all the time so I'm not sure what caused this... Might be related to multithreading issue of some sort, not sure... |
|
I am having a similar issue on 2.0.1. The AttributesTable seems to get messed up on geoJsonReader.Read and ends up self-referencing along with inserting lower level json properties at the top level of the attributesTable. Repro here: [TestMethod]
public void test_deserialise_nested_geojson()
{
string sampleGeoJsonNested =
"{\"type\":\"FeatureCollection\",\"features\":[{\"type\":\"Feature\",\"id\":\"63a72ea5-45d6-4d4c-a77c-948e5c814317\",\"geometry\":{\"type\":\"Polygon\",\"coordinates\":[[[0,0],[1,0],[1,1],[0,0]]]},\"properties\":{\"a\":[],\"b\":[],\"c\":0.15403640270233154,\"d\":0.15403640270233154,\"e\":null,\"f\":null,\"g\":null,\"h\":null,\"i\":0,\"n\":\"2018-05-08T00:00:00\",\"o\":{\"a1\":\"2018-09-14T00:00:00\",\"b1\":86400,\"c1\":\"R6\",\"d1\":4,\"e1\":12.47,\"f1\":1563.25,\"g1\":129,\"h1\":1,\"i1\":0.23666,\"j1\":0.00056,\"k1\":0.8}}}]}";
var geoJsonReader = new GeoJsonReader();
var collection = geoJsonReader.Read<FeatureCollection>(sampleGeoJsonNested);
Assert.AreEqual(11, collection[0].Attributes.Count);
}
|
|
Yay! I'm not crazy! |
|
yep, here's the problem context should be invalidated when reading inner attributes |
|
hope #47 can fix the problem |
|
Is there a way I can test it? a temporary Nuget server I can connect to? |
@HarelM the potential fix has been merged to Version In theory, this command should be what you need to get it started: |
|
@DGuidi Thanks for the fix! I can't reproduce the error when using 2.0.2. |
|
Sorry for nagging, Any chance to release this fix? I really want to put behind me all the NTS 2.0 upgrade... |
|
Seems like there a pull request from July that seems similar to this fix, I haven't fully looked into it for comparison but the issue described and the solution seems close: #36. |
@HarelM it's there now. |
|
Great! Thanks! probably worth copying the test from #36 and closing the PR as well as I think it is the same code change. |
|
@DGuidi Is it possible that the fix is not thread safe? |
|
looking at the code, seems strange to me to see thread-safety problems in my fix. the only "variable" in the code (the serialized context) wasn't changed at all... of course if more than a serializer is used, then |
|
Thanks for the response. Unfortunately, I can't fully explain the behavior but I was able to reproduce the issue several times on my machine. |
|
if you're able to reproduce the error in your machine, you can change the code of geojson serializer and add a private guid field initialized in the constructor, and then log this guid and the thread id when the method is called. Just an idea, to see if the same object is shared between calls. If so, probably the serialization context can be the cause of your error, and I fear this require a larger refactoring. |
|
The serializer is initialized only once when the app starts and I know that it is being used from multiple threads (according to logging). |
This is definetively a problem for me. As far as I know, if the SerializationContext is somewhat shared between different threads deserializing different objects, you can see randomic errors like the ones you see. My2Cents. |
|
@DGuidi thanks a lot for the super quick response and detailed answer. |
|
actually looking better at the code I'm not sure that my suggestion is actually viable :(
this is part of your code or is elastic doing this? |
|
I'm creating a factory to be used by elastic, can be seen here: |
|
@DGuidi anything I can do to help out with this? Seems like this is happening more often than not... :-( |
|
@HarelM previous version look the using the same code to me The problem can be that in the past you haven't used nested attributes tables? |
|
I have used nested attributes table from the beginning. |
|
I've checked a bit and the code looks the same from 2.0 and previous versions. |
|
I have no clue either... :-(
Yea, it's probably called inside the elastic code...
probably...
Not sure how this would help as it will probably do the same as the elastic factory... But I don't mind trying anything at this point... |
|
Seems like this is open source and the code can be seen here: |
|
I'll try and debug this by seeing if I can dump the response I get from elastic as text if there's a problem: |
|
You can also try to open a issue in their repo: to me, this is a bug and your/nts code is doing the right job. My2cents, of course. |
|
Well, I'd be happy to be wrong and find out that the bug is in my code - that way I can easily fix it. The only idea I currently have is to try and migrate to STJ instead of Json.Net and see if something changes in the behavior... or something similar to override how elastic is using this library... |
|
I've opened a ticket to elastic as well, I hope they can have a look at it and suggest a solution of some kind. |
|
added a comment in the elastic issue, hope to be helpful |
|
Thanks for that! :-) /// <summary>
/// A JSON serializer that uses Json.NET for serialization and able to parse geojson objects
/// </summary>
public class JsonNetSerializerWithGeoJson : IElasticsearchSerializer
{
private static readonly Encoding ExpectedEncoding = new UTF8Encoding(false);
private readonly ConcurrentDictionary<string, IPropertyMapping> Properties = new ConcurrentDictionary<string, IPropertyMapping>();
private readonly JsonSerializerSettings _noneIndentedSettings;
private readonly JsonSerializerSettings _indentedSettings;
public JsonNetSerializerWithGeoJson(IConnectionSettingsValues settings)
{
_noneIndentedSettings = CreateSettings(SerializationFormatting.None, settings);
_indentedSettings = CreateSettings(SerializationFormatting.Indented, settings);
}
public IPropertyMapping CreatePropertyMapping(MemberInfo memberInfo)
{
var memberInfoString = $"{memberInfo.DeclaringType?.FullName}.{memberInfo.Name}";
if (Properties.TryGetValue(memberInfoString, out var mapping))
return mapping;
mapping = PropertyMappingFromAttributes(memberInfo);
Properties.TryAdd(memberInfoString, mapping);
return mapping;
}
public T Deserialize<T>(Stream stream)
{
if (stream == null) return default;
using var streamReader = new StreamReader(stream);
using var jsonTextReader = new JsonTextReader(streamReader);
return JsonSerializer.Create(_noneIndentedSettings).Deserialize<T>(jsonTextReader);
}
public Task<T> DeserializeAsync<T>(Stream stream, CancellationToken cancellationToken = default(CancellationToken))
{
//Json.NET does not support reading a stream asynchronously :(
var result = Deserialize<T>(stream);
return Task.FromResult(result);
}
public void Serialize(object data, Stream writableStream, SerializationFormatting formatting = SerializationFormatting.Indented)
{
var serializer = formatting == SerializationFormatting.Indented
? JsonSerializer.Create(_indentedSettings)
: JsonSerializer.Create(_noneIndentedSettings);
using var writer = new StreamWriter(writableStream, ExpectedEncoding, 1024, true);
using var jsonWriter = new JsonTextWriter(writer);
serializer.Serialize(jsonWriter, data);
writer.Flush();
jsonWriter.Flush();
}
private static IPropertyMapping PropertyMappingFromAttributes(MemberInfo memberInfo)
{
var jsonProperty = memberInfo.GetCustomAttribute<JsonPropertyAttribute>(true);
var dataMember = memberInfo.GetCustomAttribute<DataMemberAttribute>(true);
var ignoreProperty = memberInfo.GetCustomAttribute<JsonIgnoreAttribute>(true);
if (jsonProperty == null && ignoreProperty == null && dataMember == null) return null;
return new PropertyMapping { Name = jsonProperty?.PropertyName ?? dataMember?.Name, Ignore = ignoreProperty != null };
}
private JsonSerializerSettings CreateSettings(SerializationFormatting formatting, IConnectionSettingsValues connectionSettings)
{
var settings = new JsonSerializerSettings()
{
Formatting = formatting == SerializationFormatting.Indented ? Formatting.Indented : Formatting.None,
ContractResolver = new ElasticContractResolver(connectionSettings, null),
DefaultValueHandling = DefaultValueHandling.Include,
NullValueHandling = NullValueHandling.Ignore
};
foreach(var converter in GeoJsonSerializer.Create(GeometryFactory.Default, 3).Converters)
{
settings.Converters.Add(converter);
}
return settings;
}
} |
|
@DGuidi Thanks for all your help! seems like my last commit had fixed the issue. |
|
Just to be curious, you written in elastic issue
so basically I assume that in the past you haven't experienced the bug because you used a previous version of elastic code, right? |
|
No, I think the bug was there all along, I just didn't experienced it I believe. I'm using the 5.6.6 version in production for probably more than a year or two... |
|
" Again, based on a little dig in the code, to me it's a bug (or at least an unexpected behaviour) in elastic code. |
|
Multithreading issue might be fixed by #59... (just adding a reference here...) |
I'm not sure what causes this as the stack trace from my server is hot helping but since I upgraded to 2.0 I see a lot of the following error in the log file:
Self referencing loop detected with type 'NetTopologySuite.Features.AttributesTable'. Path '[x].properties'.Any change you guys know what can cause this?
The number in the [x] is a changing number.
This happens when the server returns a geojson to the client.
The following is the stacktrace:
Any help would be appreciated...
The text was updated successfully, but these errors were encountered: