Couple crashes with `java.util.ConcurrentModificationException` and `java.util.NoSuchElementException: Key null is missing in the map` #736

L-Andrade · 2024-05-29T11:19:34Z

How to reproduce

Hey! Thanks again for the library. It's been working great so far, but we've started getting a couple crashes now after pushing it to more users (nothing major yet).

I've tried to reproduce it for the last couple days, but haven't managed to.

I have a LazyColumn with a CartesianChartHost. This is the basic gist of the chart:

ProvideVicoTheme(rememberM2VicoTheme()) {
    val modelProducer = remember { CartesianChartModelProducer.build() }
    LaunchedEffect(entries) {
        modelProducer.tryRunTransaction {
            lineSeries { series(x = entries.map { it.key }, y = entries.map { it.amount }) }
        }
    }
    // ...
    CartesianChartHost(
            modifier = modifier,
            scrollState = rememberVicoScrollState(scrollEnabled = false),
            modelProducer = modelProducer,
            diffAnimationSpec = spring(stiffness = Spring.StiffnessVeryLow),
            chart = rememberCartesianChart(
                rememberLineCartesianLayer(
                    lines = listOf(rememberLineSpec(shader = shader, backgroundShader = null)),
                    axisValueOverrider = remember {
                        object : AxisValueOverrider {
                            override fun getMinY(minY: Float, maxY: Float, extraStore: ExtraStore): Float {
                                return (minY - 0.0005f).coerceAtLeast(0f)
                            }

                            override fun getMaxY(minY: Float, maxY: Float, extraStore: ExtraStore): Float {
                                return (maxY + 0.0005f).coerceAtLeast(0.1f)
                            }
                        }
                    },
                ),
                decorations = listOf(
                    rememberHorizontalLine(
                        y = { state.start },
                        line = rememberLineComponent(
                            color = MaterialTheme.colors.gray80, shape = remember { DashedShape() },
                        ),
                    ),
                ),
                persistentMarkers = // Custom logic to display some markers
            ),
            marker = primaryMarker,
            markerVisibilityListener = // Custom logic for the marker visibility
        )
}

I have removed a few bits for brevity and because I can not share the whole code. Let me know if any of it is important to figuring this issue.

The chart is only shown if there are entries to display (entries is not empty). It seems to happen after the user scrolls to the bottom of the screen (chart is at the top).

Could it be because the CartesianChartModelProducer is recreated when the chart is visible again? Since it's in a LazyColumn, the chart leaves the composition and then returns once the user sees it again. Not sure it should matter though, since the chart is also a "new" one.

I'm not entirely sure if it is an issue on our end or an issue in Vico's end. Do you folks have any idea of how we can try to replicate this issue?

Let me know if you see any issue in our implementation too, please!

Observed behavior

There are a few crashes. I have a couple stack traces, not sure if they're linked to the same issue or not

      Fatal Exception: java.util.ConcurrentModificationException:
       at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:760)
       at java.util.LinkedHashMap$LinkedValueIterator.next(LinkedHashMap.java:787)
       at kotlin.collections.CollectionsKt___CollectionsKt.mapTo(CollectionsKt___Collections.kt:1620)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer.tryUpdate(CartesianChartModelProducer.kt:59)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer.access$tryUpdate(CartesianChartModelProducer.kt:38)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer$Transaction.tryCommit(CartesianChartModelProducer.kt:206)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer.tryRunTransaction(CartesianChartModelProducer.kt:171)
       at com.getbux.android.stocks.ui.chart.ValueChartKt$ValueChart$1$1.invokeSuspend(ValueChart.kt:55)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
       at androidx.compose.ui.platform.AndroidUiDispatcher.performTrampolineDispatch(AndroidUiDispatcher.android.kt:81)
       at androidx.compose.ui.platform.AndroidUiDispatcher.access$performTrampolineDispatch(AndroidUiDispatcher.android.kt:41)
       at androidx.compose.ui.platform.AndroidUiDispatcher$dispatchCallback$1.run(AndroidUiDispatcher.android.kt:57)
       at android.os.Handler.handleCallback(Handler.java:958)
       at android.os.Handler.dispatchMessage(Handler.java:99)
       at android.os.Looper.loopOnce(Looper.java:224)
       at android.os.Looper.loop(Looper.java:318)
       at android.app.ActivityThread.main(ActivityThread.java:8677)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:561)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1013)

and

      Fatal Exception: java.util.NoSuchElementException: Key null is missing in the map.
       at kotlin.collections.MapsKt__MapWithDefaultKt.getOrImplicitDefaultNullable(MapsKt__MapWithDefault.kt:24)
       at kotlin.collections.MapsKt__MapsKt.getValue(MapsKt__Maps.kt:360)
       at com.patrykandpatrick.vico.core.cartesian.data.MutableChartValuesKt$toImmutable$1.getYRange(MutableChartValues.kt:122)
       at com.patrykandpatrick.vico.core.cartesian.layer.LineCartesianLayer.toDrawingModel(LineCartesianLayer.kt:598)
       at com.patrykandpatrick.vico.core.cartesian.layer.LineCartesianLayer.prepareForTransformation(LineCartesianLayer.kt:583)
       at com.patrykandpatrick.vico.core.cartesian.layer.LineCartesianLayer.prepareForTransformation(LineCartesianLayer.kt:73)
       at com.patrykandpatrick.vico.core.cartesian.CartesianChart$transformationPreparationModelAndLayerConsumer$1.invoke(CartesianChart.java:106)
       at com.patrykandpatrick.vico.core.cartesian.CartesianChart.consume(CartesianChart.kt:328)
       at com.patrykandpatrick.vico.core.cartesian.CartesianChart.forEachWithLayer(CartesianChart.kt:316)
       at com.patrykandpatrick.vico.core.cartesian.CartesianChart.prepareForTransformation(CartesianChart.kt:251)
       at com.patrykandpatrick.vico.compose.cartesian.CartesianChartModelProducerKt$collectAsState$2$1$2.invoke(CartesianChartModelProducer.kt:124)
       at com.patrykandpatrick.vico.compose.cartesian.CartesianChartModelProducerKt$collectAsState$2$1$2.invoke(CartesianChartModelProducer.kt:124)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer$UpdateReceiver.handleUpdate(CartesianChartModelProducer.kt:229)
       at com.patrykandpatrick.vico.core.cartesian.data.CartesianChartModelProducer.registerForUpdates(CartesianChartModelProducer.kt:146)
       at com.patrykandpatrick.vico.compose.cartesian.CartesianChartModelProducerKt$collectAsState$2$1.invokeSuspend(CartesianChartModelProducer.kt:114)
       at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
       at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:104)
       at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.java:585)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:802)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:706)
       at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:693)

Expected behavior

The app should not crash

Vico version(s)

2.0.0-alpha.20

Android version(s)

12, 13, 14

Additional information

We're actually on 2.0.0-alpha.19, but there seem to be only small changes in alpha.20.

It's quite hard to figure out where the issue is coming from. Since the stack trace does not point to our app code at all, we don't know if it's an issue with our integration or an internal Vico issue.

And since we're not familiar with Vico's codebase, it's a bit harder to try to replicate it. I've tried scrolling up and down multiple times, adding delays in multiple places, adding more charts to the screen and scrolling, configuration changes, going to the background/foreground, spamming buttons that cause the entries to change, navigating back or to another screen, and more. Let me know if you have any ideas that could replicate it, any help would be great here!

Thanks in advance

The text was updated successfully, but these errors were encountered:

Gowsky · 2024-05-30T20:15:40Z

Hello. I did some stress testing and I haven't experienced this crash.

Could it be because the CartesianChartModelProducer is recreated when the chart is visible again?

This shouldn't be the reason. Have you tried putting CartesianChartModelProducer in the viewmodel, as suggested here? The examples use LaunchedEffect for simplicity, but especially when the chart leaves and enters the composition, it's more optimal to store the producer in the viewmodel.

Do you folks have any idea of how we can try to replicate this issue?

It's hard to tell without specific information. Is there anything common between the incidents? Like low-spec devices?

Let me know if you see any issue in our implementation too, please!

I can't see any issues that could cause this.

ibcurly · 2024-05-31T03:38:09Z

I also got this crash, spent a few hours looking for the culprit no luck yet

in the below code, yRanges[axisPosition] returns null because yRanges is empty and yRanges.getValue(null) throws the NoSuchElementException with the message "Key null is missing in the map."

in MutableChartValues.kt

public fun MutableChartValues.toImmutable(): ChartValues =
    object : ChartValues {
        private val yRanges = [email protected]
        override val minX: Float = [email protected]
        override val maxX: Float = [email protected]
        override val xStep: Float = [email protected]
        override val model: CartesianChartModel = [email protected]()
        override fun getYRange(axisPosition: AxisPosition.Vertical?): ChartValues.YRange =
            yRanges[axisPosition] ?: yRanges.getValue(null)
    }

UPDATE:

I was using the chart host that takes in a CartesianChartModel instead of the CartesianChartModelProducer. After changing it to match OP's implementation with the model producer and the LaunchedEffect, I no longer see the issue, so I'm not sure why OP has the issue. 🤷

L-Andrade · 2024-05-31T10:13:31Z

Hey! Thanks for the reply

Hello. I did some stress testing and I haven't experienced this crash.

Could it be because the CartesianChartModelProducer is recreated when the chart is visible again?

This shouldn't be the reason. Have you tried putting CartesianChartModelProducer in the viewmodel, as suggested here? The examples use LaunchedEffect for simplicity, but especially when the chart leaves and enters the composition, it's more optimal to store the producer in the viewmodel.

Yes, looking further into it, it seems like putting it in the ViewModel would most likely solve it. We were just trying to avoid it since we wanted the chart implementation to be part of a UI-only module, and this way we'll also have to put it a Vico dependency in the module that the ViewModel is in. Or we will have to create a wrapper around it. Reason we wanted to keep it out of the ViewModel is in case we'd want to change the implementation or library, we would have fewer places to change and it would be more contained.

Do you folks have any idea of how we can try to replicate this issue?

It's hard to tell without specific information. Is there anything common between the incidents? Like low-spec devices?

So far only had issues with Android 12, 13, and 14. Devices are not low-spec, they're decent (most common so far is Galaxy A53 5G).

I tried to understand the internal Vico code and had an idea, but might be totally wrong. Maybe still worth checking on your side just in case though. Click here to check the details

I was taking a look at the logic in CartesianChartModelProducer, and I think there are potential race conditions. With debugging breakpoints (non-suspend, log only) we can see that registerForUpdates (suspend function) and unregisterFromUpdates can be called before, during, or after tryUpdate. This can mean that a transaction might be running when a receiver is registered/unregistered, which could cause the ConcurrentModificationException. registerForUpdates is also called inside a scope.launch - which is scheduled to be immediately run, but might there might be a delay (albeit small of course) if

To solve it, registering or unregistering a receiver would need to either:

Check and wait for the Mutex lock before setting/removing
Cancel any ongoing transaction before setting/removing

Otherwise there might be an unfortunate timing.

With that said, I still can't replicate the issue, so I'm probably still missing something! I think I will try to move it to the ViewModel and see if it's fixed, but I believe the race condition will still be possible, just less probable (almost impossible, I guess) since the transaction will most likely be done before the chart is composed, and as such the receiver will always be registered after the transaction is complete (and fewer transactions will also be done, since it will not be done every time the chart is composed), and there's a smaller chance of a transaction being ran when unregistering since there will be fewer transactions too.

Gowsky · 2024-07-05T17:15:35Z

@L-Andrade, Vico 2.0.0-alpha.22 fixes the registration problem you described. This should fix the ConcurrentModificationException. As the release notes describe, you must switch to runTransaction to benefit from the fix. tryRunTransaction is now deprecated. NoSuchElementException might be fixed as well. We can't be sure, as we're lacking the reproduction. Please let us know if you still experience it.

@ibcurly, thanks for reporting it. Unfortunately, we weren't able to reproduce it, so we'll need a MRE to fix this.

L-Andrade · 2024-07-08T09:42:11Z

Hey @Gowsky! Thanks for the update. I took a look and it looks like a nice improvement. In the meantime, we had already shipped the recommended fix of putting the transaction call in the ViewModel (we abstracted it away in a class) which seemed to fix the issue, and it makes sense since it causes much fewer transactions.

However, I saw the recommendation in the Vico docs asking to change the dispatcher to Default on the caller side. I believe this should be something done in the Vico side, following the guideline that suspend functions should be main safe from the caller:

https://developer.android.com/kotlin/coroutines/coroutines-best-practices#main-safe

Other big libraries also respect this guideline, such as Room and Retrofit. This would be a further improvement, a bit less error-prone for the library users

Thanks again!

patrickmichalik · 2024-07-19T14:26:48Z

Thanks for the update and the suggestion, @L-Andrade! Vico 2.0.0 Alpha 23 makes CartesianChartModelProducer’s suspending functions main-safe. Execution is now moved to Dispatchers.Default internally.

While, as mentioned by @Gowsky, we haven’t been able to reproduce the NoSuchElementException, we believe it stemmed from the same problem as the ConcurrentModificationException. With this problem having been resolved, neither of the two reported exceptions should occur anymore. I’ll thus be adding the “done in alpha” label. Should you face any issues, please let us know. Cheers!

L-Andrade · 2024-07-19T14:31:02Z

Thanks both! I also believe the changes fix the issue. We will update to that version when we can.

Feel free to close the issue!

Gowsky · 2024-07-31T19:08:04Z

Since Vico 1 is still supported, we keep bug reports open until they're addressed in both Vico versions. The "Done in alpha" label basically means that the issue is closed as far as Vico 2 is concerned.

mak1nt0sh · 2024-10-23T10:57:22Z

Just tried beta.2 and the crash reappeared:

java.util.NoSuchElementException: Key null is missing in the map.
	at kotlin.collections.MapsKt__MapWithDefaultKt.getOrImplicitDefaultNullable(MapWithDefault.kt:24)
	at kotlin.collections.MapsKt__MapsKt.getValue(Maps.kt:369)
	at com.patrykandpatrick.vico.core.cartesian.data.MutableCartesianChartRangesKt$toImmutable$1.getYRange(MutableCartesianChartRanges.kt:99)
	at com.patrykandpatrick.vico.core.cartesian.axis.DefaultVerticalAxisItemPlacer.getHeightMeasurementLabelValues(DefaultVerticalAxisItemPlacer.kt:56)
	at com.patrykandpatrick.vico.core.cartesian.axis.VerticalAxis.getMaxLabelHeight(VerticalAxis.kt:362)
	at com.patrykandpatrick.vico.core.cartesian.axis.VerticalAxis.updateInsets(VerticalAxis.kt:305)
	at com.patrykandpatrick.vico.core.cartesian.axis.VerticalAxis.updateInsets(VerticalAxis.kt:58)
	at com.patrykandpatrick.vico.core.cartesian.CartesianChart.prepare(CartesianChart.kt:229)
	at com.patrykandpatrick.vico.compose.cartesian.CartesianChartHostKt.CartesianChartHostImpl$lambda$23$lambda$22(CartesianChartHost.kt:195)
	at com.patrykandpatrick.vico.compose.cartesian.CartesianChartHostKt.$r8$lambda$aQQKwZOn9huYyaTRo_AvgGCegn8(Unknown Source:0)
	at com.patrykandpatrick.vico.compose.cartesian.CartesianChartHostKt$$ExternalSyntheticLambda5.invoke(D8$$SyntheticClass:0)
	at androidx.compose.ui.draw.DrawBackgroundModifier.draw(DrawModifier.kt:127)
	at androidx.compose.ui.node.LayoutNodeDrawScope.drawDirect-eZhPAX0$ui_release(LayoutNodeDrawScope.kt:110)
	at androidx.compose.ui.node.LayoutNodeDrawScope.draw-eZhPAX0$ui_release(LayoutNodeDrawScope.kt:89)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:450)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
	at androidx.compose.ui.node.LayoutNode.draw$ui_release(LayoutNode.kt:1000)
	at androidx.compose.ui.node.InnerNodeCoordinator.performDraw(InnerNodeCoordinator.kt:196)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:280)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:447)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:439)
	at androidx.compose.ui.node.LayoutNode.draw$ui_release(LayoutNode.kt:1000)
	at androidx.compose.ui.node.InnerNodeCoordinator.performDraw(InnerNodeCoordinator.kt:196)

Gowsky · 2024-10-23T18:12:22Z

Hello, @mak1nt0sh. Can you share more details and some context for the issue? Can you provide an MRE?

L-Andrade added the bug label May 29, 2024

Gowsky added the needs information label May 30, 2024

patrickmichalik added done in alpha high priority and removed needs information labels Jul 19, 2024

patrickmichalik added done in beta and removed done in alpha labels Oct 11, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Couple crashes with `java.util.ConcurrentModificationException` and `java.util.NoSuchElementException: Key null is missing in the map` #736

Couple crashes with `java.util.ConcurrentModificationException` and `java.util.NoSuchElementException: Key null is missing in the map` #736

L-Andrade commented May 29, 2024

Gowsky commented May 30, 2024

ibcurly commented May 31, 2024 •

edited

Loading

L-Andrade commented May 31, 2024

Gowsky commented Jul 5, 2024

L-Andrade commented Jul 8, 2024

patrickmichalik commented Jul 19, 2024

L-Andrade commented Jul 19, 2024

Gowsky commented Jul 31, 2024

mak1nt0sh commented Oct 23, 2024 •

edited

Loading

Gowsky commented Oct 23, 2024

Couple crashes with java.util.ConcurrentModificationException and java.util.NoSuchElementException: Key null is missing in the map #736

Couple crashes with java.util.ConcurrentModificationException and java.util.NoSuchElementException: Key null is missing in the map #736

Comments

L-Andrade commented May 29, 2024

How to reproduce

Observed behavior

Expected behavior

Vico version(s)

Android version(s)

Additional information

Gowsky commented May 30, 2024

ibcurly commented May 31, 2024 • edited Loading

L-Andrade commented May 31, 2024

Gowsky commented Jul 5, 2024

L-Andrade commented Jul 8, 2024

patrickmichalik commented Jul 19, 2024

L-Andrade commented Jul 19, 2024

Gowsky commented Jul 31, 2024

mak1nt0sh commented Oct 23, 2024 • edited Loading

Gowsky commented Oct 23, 2024

Couple crashes with `java.util.ConcurrentModificationException` and `java.util.NoSuchElementException: Key null is missing in the map` #736

Couple crashes with `java.util.ConcurrentModificationException` and `java.util.NoSuchElementException: Key null is missing in the map` #736

ibcurly commented May 31, 2024 •

edited

Loading

mak1nt0sh commented Oct 23, 2024 •

edited

Loading