Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix timespan compression in backend and add bandaid compressing to front-end #4196

Merged
merged 7 commits into from
Jul 23, 2019

Conversation

philippotto
Copy link
Member

@philippotto philippotto commented Jul 22, 2019

There was a bug in the backend code which effectively disabled merging of neighboring time spans completely (annotation ids did never match due to missing casting).
This PR fixes that back end issue (thanks again, @youri-k, for finding this!) and adds a band-aid compression to the front-end (~ 9000 time entries are compressed to 9).

URL of deployed dev instance (used for testing):

  • https://___.webknossos.xyz

Steps to test:

  • I used a mock array with some time tracking data and duplicated this in code to test whether the compaction is fast (< 1 s even for 60 000 time entries)
[{"time":"PT0H0M4.798S","timestamp":1561889710979,"annotation":"5d18866601000028d9265a9c","_id":"5d188bce010000b9da2666ef","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.875S","timestamp":1561889710104,"annotation":"5d18866601000028d9265a9c","_id":"5d188bae010000aeda2666a8","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.555S","timestamp":1561889709549,"annotation":"5d18866601000028d9265a9c","_id":"5d188bae010000b5da2666a5","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M30.269S","timestamp":1561889679280,"annotation":"5d18866601000028d9265a9c","_id":"5d188bae010000b5da2666a2","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.584S","timestamp":1561889678696,"annotation":"5d18866601000028d9265a9c","_id":"5d188b8f010000b8da26667f","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.113S","timestamp":1561889678583,"annotation":"5d18866601000028d9265a9c","_id":"5d188b8f010000b8da26667e","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.267S","timestamp":1561889678316,"annotation":"5d18866601000028d9265a9c","_id":"5d188b8f010000b4da26667c","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.287S","timestamp":1561889678029,"annotation":"5d18866601000028d9265a9c","_id":"5d188b8f010000b4da26667a","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.202S","timestamp":1561889677827,"annotation":"5d18866601000028d9265a9c","_id":"5d188b8e010000b4da266678","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.075S","timestamp":1561889677752,"annotation":"5d18866601000028d9265a9c","_id":"5d188b8e010000b9da266677","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.291S","timestamp":1561889669628,"annotation":"5d18866601000028d9265a9c","_id":"5d188b86010000b4da266654","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.306S","timestamp":1561889669322,"annotation":"5d18866601000028d9265a9c","_id":"5d188b86010000afda266653","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.09S","timestamp":1561889669232,"annotation":"5d18866601000028d9265a9c","_id":"5d188b86010000aeda266652","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.203S","timestamp":1561889669029,"annotation":"5d18866601000028d9265a9c","_id":"5d188b85010000afda266651","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.343S","timestamp":1561889668686,"annotation":"5d18866601000028d9265a9c","_id":"5d188b85010000aeda266650","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.302S","timestamp":1561889668384,"annotation":"5d18866601000028d9265a9c","_id":"5d188b85010000afda26664f","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.287S","timestamp":1561889668097,"annotation":"5d18866601000028d9265a9c","_id":"5d188b85010000aeda26664e","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.304S","timestamp":1561889667793,"annotation":"5d18866601000028d9265a9c","_id":"5d188b84010000aeda26664d","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.29S","timestamp":1561889667503,"annotation":"5d18866601000028d9265a9c","_id":"5d188b84010000b9da26664b","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.286S","timestamp":1561889667217,"annotation":"5d18866601000028d9265a9c","_id":"5d188b84010000aeda266649","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.088S","timestamp":1561889667129,"annotation":"5d18866601000028d9265a9c","_id":"5d188b83010000b9da266647","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.186S","timestamp":1561889666943,"annotation":"5d18866601000028d9265a9c","_id":"5d188b83010000afda266646","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.263S","timestamp":1561889666680,"annotation":"5d18866601000028d9265a9c","_id":"5d188b83010000b4da266645","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.124S","timestamp":1561889666556,"annotation":"5d18866601000028d9265a9c","_id":"5d188b83010000b5da266643","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.198S","timestamp":1561889666358,"annotation":"5d18866601000028d9265a9c","_id":"5d188b83010000b8da266642","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.289S","timestamp":1561889666069,"annotation":"5d18866601000028d9265a9c","_id":"5d188b82010000b5da266640","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.305S","timestamp":1561889665764,"annotation":"5d18866601000028d9265a9c","_id":"5d188b82010000b8da26663f","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.306S","timestamp":1561889665458,"annotation":"5d18866601000028d9265a9c","_id":"5d188b82010000aeda26663d","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.045S","timestamp":1561889665413,"annotation":"5d18866601000028d9265a9c","_id":"5d188b82010000b5da26663c","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.246S","timestamp":1561889665167,"annotation":"5d18866601000028d9265a9c","_id":"5d188b81010000aeda26663a","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.313S","timestamp":1561889664854,"annotation":"5d18866601000028d9265a9c","_id":"5d188b81010000aeda266639","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.043S","timestamp":1561889664811,"annotation":"5d18866601000028d9265a9c","_id":"5d188b81010000b5da266637","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.464S","timestamp":1561889664347,"annotation":"5d18866601000028d9265a9c","_id":"5d188b810100009eda266636","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.2S","timestamp":1561889664147,"annotation":"5d18866601000028d9265a9c","_id":"5d188b81010000afda266634","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.274S","timestamp":1561889663873,"annotation":"5d18866601000028d9265a9c","_id":"5d188b80010000b4da266633","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.239S","timestamp":1561889663634,"annotation":"5d18866601000028d9265a9c","_id":"5d188b80010000afda266631","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.223S","timestamp":1561889663411,"annotation":"5d18866601000028d9265a9c","_id":"5d188b80010000afda266630","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.069S","timestamp":1561889663342,"annotation":"5d18866601000028d9265a9c","_id":"5d188b800100009eda26662e","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.185S","timestamp":1561889663157,"annotation":"5d18866601000028d9265a9c","_id":"5d188b7f010000b5da26662d","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.283S","timestamp":1561889662874,"annotation":"5d18866601000028d9265a9c","_id":"5d188b7f010000b4da26662b","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.062S","timestamp":1561889662812,"annotation":"5d18866601000028d9265a9c","_id":"5d188b7f010000b4da26662a","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.232S","timestamp":1561889662580,"annotation":"5d18866601000028d9265a9c","_id":"5d188b7f010000afda266629","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.263S","timestamp":1561889662317,"annotation":"5d18866601000028d9265a9c","_id":"5d188b7f0100009cda266627","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.242S","timestamp":1561889662075,"annotation":"5d18866601000028d9265a9c","_id":"5d188b7e010000b8da266625","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.55S","timestamp":1561889661525,"annotation":"5d18866601000028d9265a9c","_id":"5d188b7e010000afda266624","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M29.421S","timestamp":1561889632104,"annotation":"5d18866601000028d9265a9c","_id":"5d188b7e010000b8da266622","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M31.125S","timestamp":1561889600979,"annotation":"5d18866601000028d9265a9c","_id":"5d188b5f0100009cda2665d5","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.519S","timestamp":1561889600460,"annotation":"5d18866601000028d9265a9c","_id":"5d188b41010000adda2665ad","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.258S","timestamp":1561889600202,"annotation":"5d18866601000028d9265a9c","_id":"5d188b41010000adda2665ac","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.046S","timestamp":1561889600156,"annotation":"5d18866601000028d9265a9c","_id":"5d188b40010000b5da2665ab","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.313S","timestamp":1561889599843,"annotation":"5d18866601000028d9265a9c","_id":"5d188b40010000b4da2665aa","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.05S","timestamp":1561889599793,"annotation":"5d18866601000028d9265a9c","_id":"5d188b400100009cda2665a9","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.244S","timestamp":1561889599549,"annotation":"5d18866601000028d9265a9c","_id":"5d188b40010000abda2665a8","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.329S","timestamp":1561889599220,"annotation":"5d18866601000028d9265a9c","_id":"5d188b40010000afda2665a7","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.099S","timestamp":1561889599121,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3f010000afda2665a6","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.224S","timestamp":1561889598897,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3f010000afda2665a5","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.337S","timestamp":1561889598560,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3f0100009eda2665a4","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.256S","timestamp":1561889598304,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3f010000b4da2665a3","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.292S","timestamp":1561889598012,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3e0100009cda2665a2","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.299S","timestamp":1561889597713,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3e010000aeda2665a1","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.055S","timestamp":1561889597658,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3e010000b4da2665a0","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.274S","timestamp":1561889597384,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3e0100009eda26659f","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.286S","timestamp":1561889597098,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3e0100009eda26659e","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.289S","timestamp":1561889596809,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3d010000aeda26659d","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.052S","timestamp":1561889596757,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3d0100009cda26659c","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.245S","timestamp":1561889596512,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3d010000b4da26659b","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.3S","timestamp":1561889596212,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3d0100009eda26659a","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.079S","timestamp":1561889596133,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3c0100009cda266599","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.218S","timestamp":1561889595915,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3c0100009eda266598","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.226S","timestamp":1561889595689,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3c0100009eda266597","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.266S","timestamp":1561889595423,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3c010000afda266596","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.294S","timestamp":1561889595129,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3c0100009eda266595","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.025S","timestamp":1561889595104,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3b0100009cda266594","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.269S","timestamp":1561889594835,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3b010000b4da266593","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.262S","timestamp":1561889594573,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3b010000b4da266592","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.299S","timestamp":1561889594274,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3b0100009cda266591","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.035S","timestamp":1561889594239,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3b0100009cda266590","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.247S","timestamp":1561889593992,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3a0100009cda26658f","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.288S","timestamp":1561889593704,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3a010000b4da26658e","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.28S","timestamp":1561889593424,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3a010000afda26658d","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.232S","timestamp":1561889593192,"annotation":"5d18866601000028d9265a9c","_id":"5d188b3a010000aeda26658c","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.242S","timestamp":1561889592950,"annotation":"5d18866601000028d9265a9c","_id":"5d188b390100009eda26658b","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.232S","timestamp":1561889592718,"annotation":"5d18866601000028d9265a9c","_id":"5d188b39010000b4da26658a","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.157S","timestamp":1561889592561,"annotation":"5d18866601000028d9265a9c","_id":"5d188b390100009cda266589","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.234S","timestamp":1561889592327,"annotation":"5d18866601000028d9265a9c","_id":"5d188b39010000aeda266588","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.232S","timestamp":1561889592095,"annotation":"5d18866601000028d9265a9c","_id":"5d188b390100009eda266587","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.298S","timestamp":1561889591797,"annotation":"5d18866601000028d9265a9c","_id":"5d188b38010000aeda266586","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.082S","timestamp":1561889591715,"annotation":"5d18866601000028d9265a9c","_id":"5d188b380100009cda266585","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.428S","timestamp":1561889591287,"annotation":"5d18866601000028d9265a9c","_id":"5d188b38010000b4da266584","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.203S","timestamp":1561889591084,"annotation":"5d18866601000028d9265a9c","_id":"5d188b380100009eda266583","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.235S","timestamp":1561889590849,"annotation":"5d18866601000028d9265a9c","_id":"5d188b370100009cda266581","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.231S","timestamp":1561889590618,"annotation":"5d18866601000028d9265a9c","_id":"5d188b370100009cda26657f","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.231S","timestamp":1561889590387,"annotation":"5d18866601000028d9265a9c","_id":"5d188b37010000b4da26657e","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.367S","timestamp":1561889590020,"annotation":"5d18866601000028d9265a9c","_id":"5d188b37010000afda26657c","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.418S","timestamp":1561889589602,"annotation":"5d18866601000028d9265a9c","_id":"5d188b36010000abda26657b","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.226S","timestamp":1561889589376,"annotation":"5d18866601000028d9265a9c","_id":"5d188b36010000adda266579","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.075S","timestamp":1561889589301,"annotation":"5d18866601000028d9265a9c","_id":"5d188b36010000afda266578","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.226S","timestamp":1561889589075,"annotation":"5d18866601000028d9265a9c","_id":"5d188b35010000afda266576","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.244S","timestamp":1561889588831,"annotation":"5d18866601000028d9265a9c","_id":"5d188b35010000aeda266573","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"},{"time":"PT0H0M0.289S","timestamp":1561889588542,"annotation":"5d18866601000028d9265a9c","_id":"5d188b35010000afda266571","task_id":"5c98ed8a010000e300313b20","project_name":"AK_06469_P57_st003_st004_FluoEM_postsyn_25_03_2019","tasktype_id":"5c7e42fd010000831e9ab704","tasktype_summary":"dendrites_WITHOUT_spines_comment_events"}]
  • also I spot checked that the compress function does reasonable things (especially, that the duration of all compressed time entries equals the duration of all uncompressed time entries)

Issues:


@philippotto philippotto requested a review from daniel-wer July 22, 2019 15:26
@philippotto philippotto self-assigned this Jul 22, 2019
Copy link
Member

@daniel-wer daniel-wer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but let's still have a look at why the backend doesn't add up these entries. Afaics the tracingPauseInSeconds is configured to be 60s even. Or does the backend only use this to calculate the total time a user worked, but doesn't actually "compress" multiple time spans that are close together into one?

If there is an easy backend fix for this, I would prefer that instead of the frontend compression.

@philippotto
Copy link
Member Author

philippotto commented Jul 23, 2019

If there is an easy backend fix for this, I would prefer that instead of the frontend compression.

I totally agree! However, from my understanding, the time spans were already added to the database in that way. So, if it's a bug, we can fix, we still need a way to compress these time spans somehow. But maybe the backend should do this.. I'll try to grab @youri-k today and investigate with him :)

@philippotto philippotto requested a review from youri-k July 23, 2019 09:40
@philippotto philippotto changed the title Bandaid fix time line performance by compressing timeLogs Fix timespan compression in backend and add bandaid compressing to front-end Jul 23, 2019
@philippotto
Copy link
Member Author

@daniel-wer @youri-k and I fixed the actual back end issue. I left the front-end compression in the PR, though, since migrating the existing time spans might be too much effort for too little gain right now.

Maybe @fm3 can gauge whether it's okay to leave the uncompressed spans in the db. In any way, we can do the migration always later.

@philippotto
Copy link
Member Author

Also we tweaked the compression algorithm to compare the end timestamp (= timestamp + duration) of timespan A with the start timestamp with timespan B. Previously, only the start timestamps were compared 🙈

Copy link
Member

@daniel-wer daniel-wer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great fixes 🎉

Copy link
Contributor

@youri-k youri-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :shipit:

@philippotto philippotto merged commit 07b3f81 into master Jul 23, 2019
@normanrz normanrz deleted the user-stats-perf branch August 12, 2019 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants