Neuroglancer crashing while browsing tags on Chrome/Mac #685
Comments
|
If it helps, Chrome version is: |
|
Just looking at this a bit, this kind of annotation-rich state produces a URL that is very, very large. I can seem to copy only 629,772 characters out of the URL or history, but I don't think that is at all complete and the history is storing the whole URL. Chrome history is updating maybe once every second or two in workflows like this. It is easy to get many hundreds or thousands of MB-scale URLs stored in history in just 10 or 20 minutes of working. |
|
While stopgaps like supporting compression in the URL syntax may help, ultimately the only real solution is to store the annotation list outside of the Neuroglancer state, either in neuroglancer precomputed format if read-only access is sufficient, or in some new format/server if read-write access to the annotations is needed. |
|
Read-only access is not sufficient for these use cases, unfortunately, and there are no performance issues within neuroglancer for using it this way. The amount of time to build a novel solution that simply recreates existing functionality is not feasible with current resources, nor does it suit the use case for rapid analysis to have to maintain yet another managed server. We frequently encounter a pipeline where a person does local analysis, generates thousands of annotations to reflect that analysis, uses tags to rapidly validate results in the full context of the EM volume, and then returns the data for subsequent analysis via a state server. In many cases, thousands of points can reflect relatively only an hour or two of work to validate, and both should be visualized together and do not need to be spread across multiple sessions. Neuroglancer itself handles high-annotation states fine, as does the state-server method of saving and retrieving JSON, it is only the synchronization of state with URL that causes the memory issues. During these situations, we already rely on the state-sharing approach rather than URL copying since the URLs are too long to robustly copy, thus URL synchronization is not serving the same compatibility goal as in smaller states. While I understand this would not be a default option (particularly as it only applies if a state sharing URL is set), it feels like there could be some optional mechanism that gracefully passes from URL synchronization for modest states to truncated URLs + sharing (with a warning in the UI somehow). Or perhaps a flag in the state JSON itself about synchronization so that one could turn it off in these particular situations, again with a warning in the UI. I wonder if it wouldn't be possible to open a persistent connection to a state server ID, where rather than download it once, neuroglancer could push states (or, better, state changes) in some way to keep that state ID up to date. That way, the URL would continue to be up to date or nearly up to date for sharing purposes, but we would also get the value of the short links. It would be particularly ideal if combined with state-level undo, since this would make the URL history no longer reflect state history. |
|
I can see that this large scale annotation editing might be the source of some of the crash reports i've been hearing about, but they have been pretty wide spread at the Allen the last week or so, so i want to at least entertain the possibility that this isn't just about editing large scale annotation lists. On the annotation read/write server question, what would the skeleton of an API look like that would be easiest to implement such a feature utilizing the existing annotation infrastructure? |
|
but also is this a chrome bug or is neuroglancer abusing chrome history in some fashion that it isn't designed to do? If its about the history it shouldn't really be able memory management, unless somehow the history is stored in memory and not on disk. |
|
Neuroglancer uses Just adding an option not to store the state in the URL, such that saving to the state server would be required, would be easy enough, not sure how well that would actually address your proofreading use case. As far as automatically "synchronizing" the Neuroglancer state to a state server, and potentially storing the history, certainly that would be a useful feature. I prototyped that with Google Drive some years ago, but dropped it because limitations in the Google Drive APIs meant that sharing with another user couldn't work in a reasonable way. That could certainly be implemented with a change to your state server --- just synchronizing the latest state would probably be minimal effort, storing the history would likely be more complicated. In general I think state synchronization with or without history is useful, but for dealing with annotations, I think it would be more natural and convenient in most cases to store the annotations separately with their own history independent of the Neuroglancer state used to view those annotations, as that would allow multiple users to edit the same annotation table simultaneously. I previously discussed with Forrest and others that idea of a general "log-structured" server for Neuroglancer, where the server allows users to create "logs" that support the following operations:
This would enable read-write storage of a number of things to be implemented entirely client-side:
The key limitation would be that the total size of the log could not be too large, e.g. a few MB, or maybe 100 MB at most, since clients will always download the entire log, and then reconstruct the state at the requested timestamp by iterating over all of the entries. But that would probably still cover a lot of use cases, and by treating the log as a delta on top of some large base annotation table or base agglomeration graph you could go even further. As far as integration into Neuroglancer --- Neuroglancer already has an internal API for read-write annotation sources, which is used for a Google-internal brainmaps:// annotation source and for a DVID annotation source, but nothing currently to support the concept of a history. However, support for history needs to be solved anyway to acommodate graphene/CAVE. |
|
Hi! I’ve also encountered Chrome crashing while viewing a large number of skeletons in Neuroglancer (without annotations or proofreading). Unfortunately, I don’t have a publicly shareable link at the moment. From my experience, the first crash appeared to be partially memory-related. When loading the same number of skeletons in Safari, I received a warning pop-up stating that my system had run out of application memory. However, no such warning appeared in Chrome—it just crashed. What’s similar to Ben’s experience is that after the initial crash, Chrome started crashing frequently (every few minutes), even when I wasn’t working with any NG links. I didn’t get a chance to explore the History tab, but I’m wondering if there might be a broader underlying issue causing this behavior. |
I am having an issue with Neuroglancer crashing in Chrome on Mac.
Here is a link that at least one other person has been able to replicate the issue with. Should be able to view after agreeing to TOS, please let me know if you have any issues.
I get the crash after bouncing around the tagged annotation points (using "[" and "]") for several minutes. Unfortunately we have not yet found a way to make it happen immediately.
My Chrome crash logs look something like this:
Some observations: we think this has something to do with Chrome history. I've observed that after one of these crashes, my chrome profile can become unstable and start crashing at random times even while not using Neuroglancer (and with nothing else consuming much memory on my machine). The only solution to this I have found is to make a new chrome profile, or clear history in the current one.
Similarly, I've observed that opening Chrome history can also crash chrome after I've experienced one of these crashes in Neuroglancer.
Please let me know if there's any other information I can provide to help reproduce or debug.
The text was updated successfully, but these errors were encountered: