Add new ARKitScenes example #1538

pablovela5620 · 2023-03-09T03:34:26Z

pip install -r examples/python/arkitscenes/requirements.txt
examples/python/arkitscenes/main.py

Checklist

I have read and agree to Contributor Guide and the Code of Conduct
I've included a screenshot or gif (if applicable)

arkitscenes-example.mov

Hey all, I'm attaching a python example for the ARKitScenes dataset. Currently, I've only included the trajectory and depth for low-res (256x192) data, but I plan on adding in

High-Resolution Depth
.ply reconstruction from ARKit Depth
~~Faro Laser point cloud~~ (Turns out to be much too large to log | 160 million points)
3d bounding boxes for objects
2d bounding boxes reprojected into relevant view for both low and high res

a few points/questions I had:

I had to include a rotation of the extrinsic/intrinsics as some captures are in landscape orientation while others are in portrait, rather than being able to modify them using the log_view_coordinate I had to do it manually since there's no connection between rotating the 2d image/depth and the camera parameters
there's no clear way to give a logged depth image a colormap via the log_depth_image like there is for back projecting

----EDIT----
I added in a few issues I found when trying to log the include .ply file #1570 and #1571
Added in 3d bounding boxes, noticed that I would need to project the 3d keypoints into the 2d frame myself for them. Not sure if this is something that we want to include in this PR or wait until there's a way to do it via the rerun-sdk natively

----EDIT 2----

Added code to reproject the oriented bounding boxes, added a ton of complexity that can be stripped out once #1581 is done

nikolausWest · 2023-03-09T21:14:14Z

I had to include a rotation of the extrinsic/intrinsics as some captures are in landscape orientation while others are in portrait, rather than being able to modify them using the log_view_coordinate I had to do it manually since there's no connection between rotating the 2d image/depth and the camera parameters

What happens if you just set the view_coordinates correctly for the 3D view wrt landscape/portrait? Is the downside just that the 2D view has the wrong orientation but everything matches up correctly otherwise?

there's no clear way to give a logged depth image a colormap via the log_depth_image like there is for back projecting

Yeah we don't yet have support for setting colormaps to depth images (in 2D) yet. They are just colored with turbo as a quick fix but all of colormapping needs a retake.

pablovela5620 · 2023-03-10T02:41:21Z

What happens if you just set the view_coordinates correctly for the 3D view wrt landscape/portrait? Is the downside just that the 2D view has the wrong orientation but everything matches up correctly otherwise?

Yep exactly that, just means the 2d view is the wrong orientation

Yeah we don't yet have support for setting colormaps to depth images (in 2D) yet. They are just colored with turbo as a quick fix but all of color mapping needs a retake.

Okay understood!

nikolausWest · 2023-03-10T10:07:31Z

Yep exactly that, just means the 2d view is the wrong orientation

I actually don't think this is the worst thing in the world. I think it's fine to leave it like that in favour of simpler code, and just leave a TODO(Pablovela): Fix orientation in 2D view once #1387 is closed.

pablovela5620 · 2023-03-10T22:11:10Z

Okay, I'll leave an example so you can see exactly what I'm talking about. I'll remove the orientation code and add TODO.

unoriented_view.mov

emilk · 2023-03-11T14:24:28Z

Very cool - I'm looking forwards to seeing this PR evolve!

pablovela5620 · 2023-03-12T15:53:33Z

Here it is with the included bounding boxes (with mesh invisible/visible)

3dbboxes.mov

nikolausWest

Great start!

I added some comment for improvements. Also, I think we should go ahead and add the high resolution depth as well.

Before merging we might want to wait for some other stuff to land but we might as well take it as far as we can first

examples/python/arkitscenes/main.py

pablovela5620 · 2023-03-21T18:52:35Z

One question I had is regarding the linter. On my local machine, just py-lint passed but on the CI it did not. I'm guessing this is because of the version of python being used (CI is 3.8 v.s. my 3.10). Is this something that should be specified here?

nikolausWest · 2023-03-21T21:26:19Z

One question I had is regarding the linter. On my local machine, just py-lint passed but on the CI it did not. I'm guessing this is because of the version of python being used (CI is 3.8 v.s. my 3.10). Is this something that should be specified here?

Yeah, Rerun supports Python 3.8+ so CI needs to run with Python 3.8 to avoid features that depend on later versions creeping in. It used to say front and center but apparently no longer. The only place it's really written down is in the pyproject.toml so it should probably say in the README as well

nikolausWest · 2023-03-21T21:29:34Z

Btw, is it a lot of work to get the Faro laser scanner in as well?

examples/python/arkitscenes/download_dataset.py

examples/python/arkitscenes/main.py

nikolausWest

I have a last fix I'd like to get in but otherwise lgtm.

I'd love a review / look from someone else in the team as well though since I think I might have become blind by now.

examples/python/arkitscenes/main.py

Co-authored-by: Nikolaus West <[email protected]>

emilk · 2023-03-28T19:11:19Z

The depth cloud rendering for the high-red depth point cloud is really slow 😬 not the fault of this PR obviously, but it makes me doubt whether it is good to have it on by default… maybe there is some low-hanging fruit there though.

EDIT: Partial fix in #1729

depth=0 is a degenerate case that causes a 1440x1920 depth-map from #1538 take 75ms (!) on my M1 MacBook Pro. With this fix this goes down to 15 ms However, if we zoom out a lot, so that the depth point cloud covers a very small part of the screen, the slowness returns. I suspect this is a GPU binning problem of some sort, where too many points in too small of a screen area causes performence issues.

nikolausWest · 2023-03-29T07:12:58Z

The depth cloud rendering for the high-red depth point cloud is really slow 😬 not the fault of this PR obviously, but it makes me doubt whether it is good to have it on by default… maybe there is some low-hanging fruit there though.

Alright, maybe we put it behind a feature flag (command line option) until we've fixed the large depth image speed

* Optimize the depth-cloud shader when depth=0 depth=0 is a degenerate case that causes a 1440x1920 depth-map from #1538 take 75ms (!) on my M1 MacBook Pro. With this fix this goes down to 15 ms However, if we zoom out a lot, so that the depth point cloud covers a very small part of the screen, the slowness returns. I suspect this is a GPU binning problem of some sort, where too many points in too small of a screen area causes performence issues. * Put happy-path first, and also handle NaNs * var -> let * Remove unnecessary parenthesis in wgsl if-statements

emilk · 2023-03-31T08:00:18Z

examples/python/arkitscenes/main.py

+    color_positions = np.linspace(0, 1, num_objects)
+    colormap = plt.cm.get_cmap("viridis")
+    color_array_float = [colormap(pos) for pos in color_positions]
+    color_list = [(int(r * 255), int(g * 255), int(b * 255), int(a * 255)) for r, g, b, a in color_array_float]


log_obb should be able to take float colors too, though I notice the type signature for it is wrong 🤦

emilk

Very cool demo.

It reveals a lot of shortcomings in Rerun. I think we should merge this and then plan in a day where we try to fix as many as possible of the reported issues.

Oh, and also: this makes me wanna create an alternative first-person eye for the 3D view, as an alternative to the current orbital eye. The former is better for when you are inside the thing you are looking at (e.g. inside a room), while the latter is better when you are observing something from the outside.

commit 409dcd8 Author: Emil Ernerfeldt <[email protected]> Date: Tue Apr 4 18:58:14 2023 +0200 Option to show scene bounding box (#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box commit 679e245 Author: Jeremy Leibs <[email protected]> Date: Tue Apr 4 12:21:07 2023 -0400 Allow torch tensors for log_rigid3 (#1769) commit b9f1380 Author: Jeremy Leibs <[email protected]> Date: Tue Apr 4 12:18:45 2023 -0400 Don't initialize an SDK session if we are only going to be launching the app (#1768) commit 6c383c3 Author: Andreas Reich <[email protected]> Date: Tue Apr 4 18:16:21 2023 +0200 Fix undo/redo selection shortcut/action changing selection history without changing selection (#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes #1172 * typo fix commit 9310bd7 Author: Clement Rey <[email protected]> Date: Tue Apr 4 17:38:59 2023 +0200 Columnar timepoints in data tables and during transport (#1767) * columnar timepoints * self review commit 1713e60 Author: Jeremy Leibs <[email protected]> Date: Tue Apr 4 10:02:22 2023 -0400 Always create the log_time timeline (#1763) commit 7892098 Author: Emil Ernerfeldt <[email protected]> Date: Tue Apr 4 15:36:39 2023 +0200 Collapse space-view by default if there is only one child (#1762) commit 9c1babe Author: Emil Ernerfeldt <[email protected]> Date: Tue Apr 4 14:59:18 2023 +0200 Python SDK: document that we also accept colors in 0-1 floats (#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format commit 1946683 Author: Emil Ernerfeldt <[email protected]> Date: Tue Apr 4 14:47:48 2023 +0200 Fix log_obb usage (#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example commit a479c0c Author: Pablo Vela <[email protected]> Date: Tue Apr 4 07:03:45 2023 -0500 Add new ARKitScenes example (#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> commit bf27c8e Author: Andreas Reich <[email protected]> Date: Tue Apr 4 10:32:26 2023 +0200 Readback depth from GPU picking (#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds commit d94ca3d Author: Emil Ernerfeldt <[email protected]> Date: Tue Apr 4 09:44:30 2023 +0200 Fix crash when trying to do picking on depth clouds

* `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]>

* node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (rerun-io#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (rerun-io#1920) * Always spawn instead of fork in multiprocessing example (rerun-io#1922) * Add `--num-frames` arg to canny (webcam) example (rerun-io#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (rerun-io#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (rerun-io#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (rerun-io#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (rerun-io#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (rerun-io#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (rerun-io#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (rerun-io#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (rerun-io#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (rerun-io#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (rerun-io#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (rerun-io#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (rerun-io#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (rerun-io#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (rerun-io#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (rerun-io#1969) * Full (experimental) WebGPU support (rerun-io#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (rerun-io#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]>

* v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (rerun-io#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (rerun-io#1920) * Always spawn instead of fork in multiprocessing example (rerun-io#1922) * Add `--num-frames` arg to canny (webcam) example (rerun-io#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (rerun-io#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (rerun-io#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (rerun-io#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (rerun-io#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (rerun-io#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (rerun-io#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (rerun-io#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (rerun-io#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (rerun-io#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (rerun-io#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (rerun-io#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (rerun-io#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (rerun-io#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (rerun-io#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (rerun-io#1969) * Full (experimental) WebGPU support (rerun-io#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (rerun-io#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix rerun lints * rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions * remove compiler warnings * fix memory leak when app is in background * Custom viewport UI (#4) * custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible * small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui * custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly. * improve auto layout to not split when only 3D or 2D view is available * MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB * Styling (#6) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Tabify all panels (except for blueprint) (#7) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive * remove bottom panel, switch to RAW imu sensors * XLink statistics initial implementation * initial xlink throughput statistics impllementation, have to glow it up a bit and maybe clean up the code * Xlink and rerun rename (#9) * Rename rerun py library to depthai_viewer * bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts * auto layout * Fix maximize not working and add maximize for Stats tab * delete profiling stuff that shouldn't have been commited * mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split * add docstring for update_tree * WIP auto layout can_create_mono_quad checker, not at all finished yet * pass lint checks and bugfixes * forgot to sort imports * Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid * pylints and fix Queue typehinting * fix doc build * try to pass pylints with py.typed * ignore misc mypy errors * forgot to run black formatter * switch back to old ci --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]>

* v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (rerun-io#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (rerun-io#1920) * Always spawn instead of fork in multiprocessing example (rerun-io#1922) * Add `--num-frames` arg to canny (webcam) example (rerun-io#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (rerun-io#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (rerun-io#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (rerun-io#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (rerun-io#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (rerun-io#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (rerun-io#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (rerun-io#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (rerun-io#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (rerun-io#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (rerun-io#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (rerun-io#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (rerun-io#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (rerun-io#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (rerun-io#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (rerun-io#1969) * Full (experimental) WebGPU support (rerun-io#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (rerun-io#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix rerun lints * rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions * remove compiler warnings * fix memory leak when app is in background * Custom viewport UI (#4) * custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible * small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui * custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly. * improve auto layout to not split when only 3D or 2D view is available * MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB * Styling (#6) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Tabify all panels (except for blueprint) (#7) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive * remove bottom panel, switch to RAW imu sensors * XLink statistics initial implementation * initial xlink throughput statistics impllementation, have to glow it up a bit and maybe clean up the code * Xlink and rerun rename (#9) * Rename rerun py library to depthai_viewer * bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts * auto layout * Fix maximize not working and add maximize for Stats tab * delete profiling stuff that shouldn't have been commited * mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split * add docstring for update_tree * WIP auto layout can_create_mono_quad checker, not at all finished yet * pass lint checks and bugfixes * forgot to sort imports * Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid * pylints and fix Queue typehinting * fix doc build * try to pass pylints with py.typed * ignore misc mypy errors * forgot to run black formatter * switch back to old ci * sync mono camera settings * fix web build * Update scripts/version_util.py --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]>

* change version to 0.0.1 * format pyproject.toml * add back check_version to version_util.py * switch to normal runners * try [email protected] * Some fixes (#11) * v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (rerun-io#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (rerun-io#1920) * Always spawn instead of fork in multiprocessing example (rerun-io#1922) * Add `--num-frames` arg to canny (webcam) example (rerun-io#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (rerun-io#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (rerun-io#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (rerun-io#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (rerun-io#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (rerun-io#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (rerun-io#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (rerun-io#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (rerun-io#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (rerun-io#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (rerun-io#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (rerun-io#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (rerun-io#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (rerun-io#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (rerun-io#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (rerun-io#1969) * Full (experimental) WebGPU support (rerun-io#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (rerun-io#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix rerun lints * rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions * remove compiler warnings * fix memory leak when app is in background * Custom viewport UI (#4) * custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible * small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui * custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly. * improve auto layout to not split when only 3D or 2D view is available * MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB * Styling (#6) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Tabify all panels (except for blueprint) (#7) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive * remove bottom panel, switch to RAW imu sensors * XLink statistics initial implementation * initial xlink throughput statistics impllementation, have to glow it up a bit and maybe clean up the code * Xlink and rerun rename (#9) * Rename rerun py library to depthai_viewer * bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts * auto layout * Fix maximize not working and add maximize for Stats tab * delete profiling stuff that shouldn't have been commited * mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split * add docstring for update_tree * WIP auto layout can_create_mono_quad checker, not at all finished yet * pass lint checks and bugfixes * forgot to sort imports * Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid * pylints and fix Queue typehinting * fix doc build * try to pass pylints with py.typed * ignore misc mypy errors * forgot to run black formatter * switch back to old ci * sync mono camera settings * fix web build * Update scripts/version_util.py --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * install depthai_viewer instead of rerun-sdk for pytest * Updated design and improvements * get rid of rerun_sdk and rerun_bindings, now just depthai_viewer and depthai_viewer_bindings get installed, so you don't clash with an existing rerun install * hopefuly fix ci * hopefuly fix ci * a bunch of fixes and improvements * disable smart layout updater for now * bugfixes * Update README.md * Patches --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]>

* v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (#1762) * Always create the log_time timeline (#1763) * Columnar timepoints in data tables and during transport (#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes #1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (#1768) * Allow torch tensors for log_rigid3 (#1769) * Option to show scene bounding box (#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (#1780) * Add typing_extensions to requirements-doc.txt (#1786) * auto_color class-ids if they are present (#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (#1806) * Compile with `panic = "abort"` (#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of https://github.com/rerun-io/rerun/issues/1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (#1821) * Use gpu picking for points, streamline/share picking code some more (#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (#1820) * Fix CI syntax error * New option to disable persistent storage (#1825) * New option to disable persistent storage * New API to reset_time (#1826) * Datastore revamp 1: new indexing model & core datastructures (#1727) * Datastore revamp 2: serialization & formatting (#1735) * Datastore revamp 3: efficient incremental stats (#1739) * Datastore revamp 4: sunset `MsgId` (#1785) * Datastore revamp 5: (#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (#1795) * Datastore revamp 7: garbage collection (#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (#1822) * Simplify point cloud builder and fix crash on too many points Fixes #1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes #1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (#1798) (#1834) (#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (#1834) * Refactor the relationship between the assorted web / websocket servers (#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (#1853) * Fix typos in notebook readme (#1852) * Fix the python build when running without web_viewer enabled (#1856) * Error instead of expect inside msg_encode. (#1857) * Restore: New API to reset_time (#1826) (#1854) * Revert "Implement billinear filtering of textures (#1850)" (#1859) This reverts commit d33dab6e7a33f82ab2513058d0f85744e3ce6ef4. * Add Restart command and keyboard shortcut for moving time to start of timeline (#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (#1863) * Gpu picking for depth clouds (#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (#1860) * Revert "Revert "Implement billinear filtering of textures (#1850)" (#1859)" This reverts commit 625d2bdd241c09ff9d0ae394ba91565fa48455ec. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (#1920) * Always spawn instead of fork in multiprocessing example (#1922) * Add `--num-frames` arg to canny (webcam) example (#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (#1934) * Add argument parsing to the rerun_demo (#1925) * More robust wait for exit condition during .serve() (#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by https://github.com/rerun-io/rerun/issues/1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (#1934) * Add argument parsing to the rerun_demo (#1925) * More robust wait for exit condition during .serve() (#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by https://github.com/rerun-io/rerun/issues/1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (#1969) * Full (experimental) WebGPU support (#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix rerun lints * rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions * remove compiler warnings * fix memory leak when app is in background * Custom viewport UI (#4) * custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible * small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui * custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly. * improve auto layout to not split when only 3D or 2D view is available * MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB * Styling (#6) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Tabify all panels (except for blueprint) (#7) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive * remove bottom panel, switch to RAW imu sensors * XLink statistics initial implementation * initial xlink throughput statistics impllementation, have to glow it up a bit and maybe clean up the code * Xlink and rerun rename (#9) * Rename rerun py library to depthai_viewer * bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts * auto layout * Fix maximize not working and add maximize for Stats tab * delete profiling stuff that shouldn't have been commited * mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split * add docstring for update_tree * WIP auto layout can_create_mono_quad checker, not at all finished yet * pass lint checks and bugfixes * forgot to sort imports * Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid * pylints and fix Queue typehinting * fix doc build * try to pass pylints with py.typed * ignore misc mypy errors * forgot to run black formatter * switch back to old ci * change version to 0.0.1 * format pyproject.toml * add back check_version to version_util.py * switch to normal runners * try [email protected] * sync mono camera settings * fix web build * Update scripts/version_util.py * Some fixes (#11) * v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (#1762) * Always create the log_time timeline (#1763) * Columnar timepoints in data tables and during transport (#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes #1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (#1768) * Allow torch tensors for log_rigid3 (#1769) * Option to show scene bounding box (#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (#1780) * Add typing_extensions to requirements-doc.txt (#1786) * auto_color class-ids if they are present (#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (#1806) * Compile with `panic = "abort"` (#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of https://github.com/rerun-io/rerun/issues/1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (#1821) * Use gpu picking for points, streamline/share picking code some more (#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (#1820) * Fix CI syntax error * New option to disable persistent storage (#1825) * New option to disable persistent storage * New API to reset_time (#1826) * Datastore revamp 1: new indexing model & core datastructures (#1727) * Datastore revamp 2: serialization & formatting (#1735) * Datastore revamp 3: efficient incremental stats (#1739) * Datastore revamp 4: sunset `MsgId` (#1785) * Datastore revamp 5: (#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (#1795) * Datastore revamp 7: garbage collection (#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (#1822) * Simplify point cloud builder and fix crash on too many points Fixes #1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes #1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (#1798) (#1834) (#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (#1834) * Refactor the relationship between the assorted web / websocket servers (#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (#1853) * Fix typos in notebook readme (#1852) * Fix the python build when running without web_viewer enabled (#1856) * Error instead of expect inside msg_encode. (#1857) * Restore: New API to reset_time (#1826) (#1854) * Revert "Implement billinear filtering of textures (#1850)" (#1859) This reverts commit d33dab6e7a33f82ab2513058d0f85744e3ce6ef4. * Add Restart command and keyboard shortcut for moving time to start of timeline (#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (#1863) * Gpu picking for depth clouds (#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (#1860) * Revert "Revert "Implement billinear filtering of textures (#1850)" (#1859)" This reverts commit 625d2bdd241c09ff9d0ae394ba91565fa48455ec. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <urho.laukkar…

* v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (#1762) * Always create the log_time timeline (#1763) * Columnar timepoints in data tables and during transport (#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes #1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (#1768) * Allow torch tensors for log_rigid3 (#1769) * Option to show scene bounding box (#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (#1780) * Add typing_extensions to requirements-doc.txt (#1786) * auto_color class-ids if they are present (#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (#1806) * Compile with `panic = "abort"` (#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of https://github.com/rerun-io/rerun/issues/1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (#1821) * Use gpu picking for points, streamline/share picking code some more (#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (#1820) * Fix CI syntax error * New option to disable persistent storage (#1825) * New option to disable persistent storage * New API to reset_time (#1826) * Datastore revamp 1: new indexing model & core datastructures (#1727) * Datastore revamp 2: serialization & formatting (#1735) * Datastore revamp 3: efficient incremental stats (#1739) * Datastore revamp 4: sunset `MsgId` (#1785) * Datastore revamp 5: (#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (#1795) * Datastore revamp 7: garbage collection (#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (#1822) * Simplify point cloud builder and fix crash on too many points Fixes #1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes #1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (#1798) (#1834) (#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (#1834) * Refactor the relationship between the assorted web / websocket servers (#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (#1853) * Fix typos in notebook readme (#1852) * Fix the python build when running without web_viewer enabled (#1856) * Error instead of expect inside msg_encode. (#1857) * Restore: New API to reset_time (#1826) (#1854) * Revert "Implement billinear filtering of textures (#1850)" (#1859) This reverts commit d33dab6e7a33f82ab2513058d0f85744e3ce6ef4. * Add Restart command and keyboard shortcut for moving time to start of timeline (#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (#1863) * Gpu picking for depth clouds (#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (#1860) * Revert "Revert "Implement billinear filtering of textures (#1850)" (#1859)" This reverts commit 625d2bdd241c09ff9d0ae394ba91565fa48455ec. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (#1920) * Always spawn instead of fork in multiprocessing example (#1922) * Add `--num-frames` arg to canny (webcam) example (#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (#1934) * Add argument parsing to the rerun_demo (#1925) * More robust wait for exit condition during .serve() (#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by https://github.com/rerun-io/rerun/issues/1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (#1934) * Add argument parsing to the rerun_demo (#1925) * More robust wait for exit condition during .serve() (#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by https://github.com/rerun-io/rerun/issues/1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (#1969) * Full (experimental) WebGPU support (#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix rerun lints * rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions * remove compiler warnings * fix memory leak when app is in background * Custom viewport UI (#4) * custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible * small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui * custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly. * improve auto layout to not split when only 3D or 2D view is available * MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB * Styling (#6) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Tabify all panels (except for blueprint) (#7) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive * remove bottom panel, switch to RAW imu sensors * XLink statistics initial implementation * initial xlink throughput statistics impllementation, have to glow it up a bit and maybe clean up the code * Xlink and rerun rename (#9) * Rename rerun py library to depthai_viewer * bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts * auto layout * Fix maximize not working and add maximize for Stats tab * delete profiling stuff that shouldn't have been commited * mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split * add docstring for update_tree * WIP auto layout can_create_mono_quad checker, not at all finished yet * pass lint checks and bugfixes * forgot to sort imports * Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid * pylints and fix Queue typehinting * fix doc build * try to pass pylints with py.typed * ignore misc mypy errors * forgot to run black formatter * switch back to old ci * change version to 0.0.1 * format pyproject.toml * add back check_version to version_util.py * switch to normal runners * try [email protected] * sync mono camera settings * fix web build * Update scripts/version_util.py * Some fixes (#11) * v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (#1762) * Always create the log_time timeline (#1763) * Columnar timepoints in data tables and during transport (#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes #1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (#1768) * Allow torch tensors for log_rigid3 (#1769) * Option to show scene bounding box (#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (#1780) * Add typing_extensions to requirements-doc.txt (#1786) * auto_color class-ids if they are present (#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (#1806) * Compile with `panic = "abort"` (#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of https://github.com/rerun-io/rerun/issues/1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (#1821) * Use gpu picking for points, streamline/share picking code some more (#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (#1820) * Fix CI syntax error * New option to disable persistent storage (#1825) * New option to disable persistent storage * New API to reset_time (#1826) * Datastore revamp 1: new indexing model & core datastructures (#1727) * Datastore revamp 2: serialization & formatting (#1735) * Datastore revamp 3: efficient incremental stats (#1739) * Datastore revamp 4: sunset `MsgId` (#1785) * Datastore revamp 5: (#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (#1795) * Datastore revamp 7: garbage collection (#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (#1822) * Simplify point cloud builder and fix crash on too many points Fixes #1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes #1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (#1798) (#1834) (#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (#1834) * Refactor the relationship between the assorted web / websocket servers (#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (#1853) * Fix typos in notebook readme (#1852) * Fix the python build when running without web_viewer enabled (#1856) * Error instead of expect inside msg_encode. (#1857) * Restore: New API to reset_time (#1826) (#1854) * Revert "Implement billinear filtering of textures (#1850)" (#1859) This reverts commit d33dab6e7a33f82ab2513058d0f85744e3ce6ef4. * Add Restart command and keyboard shortcut for moving time to start of timeline (#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (#1863) * Gpu picking for depth clouds (#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (#1860) * Revert "Revert "Implement billinear filtering of textures (#1850)" (#1859)" This reverts commit 625d2bdd241c09ff9d0ae394ba91565fa48455ec. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <urho.laukkarinen@gmail.…

pablovela5620 added 2 commits March 8, 2023 22:03

add arkitscenes python example

10b1034

add another example

f3cafc6

nikolausWest added the examples Issues relating to the Rerun examples label Mar 9, 2023

nikolausWest self-assigned this Mar 9, 2023

nikolausWest self-requested a review March 9, 2023 09:14

pablovela5620 added 3 commits March 9, 2023 08:42

fixed linter errors

0958637

add type annotations to download dataset

af8a555

remove extraneous functions, fix linter erros

90381e8

log all images rather then just those with traj

d480d95

remove orientation code for portrait mode samples

180d09f

pablovela5620 added 4 commits March 11, 2023 15:09

add mesh/annotations download

ea502be

log ply mesh

ecb5e8b

update requirements to include numpy and trimesh

ebf126b

add 3d bounding boxes to scene

23f31a3

nikolausWest requested changes Mar 14, 2023

View reviewed changes

pablovela5620 added 3 commits March 21, 2023 12:54

simplify traj_string_to_matrix

dcb7372

log mesh in glb format

a72e915

add view coords and cv2 convert

961e30c

swap mesh/bbox logging order, add todo

2c790e8

nikolausWest reviewed Mar 21, 2023

View reviewed changes

examples/python/arkitscenes/download_dataset.py Show resolved Hide resolved

nikolausWest reviewed Mar 21, 2023

View reviewed changes

examples/python/arkitscenes/main.py Outdated Show resolved Hide resolved

pablovela5620 added 5 commits March 28, 2023 09:50

comments why we return in log_annotated_bboxes

8262e65

add todos for removing once #1581

ed4df39

linter fix

a9a67d2

clear centroid labels when out of frame

0e68ddb

Merge branch 'main' into pablovela5620/arkitscenes

48c7cad

pablovela5620 marked this pull request as ready for review March 28, 2023 15:13

nikolausWest approved these changes Mar 28, 2023

View reviewed changes

examples/python/arkitscenes/main.py Outdated Show resolved Hide resolved

combine comments and clean up color hack

318866a

Co-authored-by: Nikolaus West <[email protected]>

emilk mentioned this pull request Mar 28, 2023

Optimize the depth-cloud shader when depth=0 #1729

Merged

1 task

include flag for high resolution images/depths

025903a

emilk reviewed Mar 31, 2023

View reviewed changes

emilk mentioned this pull request Mar 31, 2023

Python SDK: document that we also accept colors in 0-1 floats #1740

Merged

2 tasks

emilk approved these changes Mar 31, 2023

View reviewed changes

emilk changed the title ~~Pablovela5620/arkitscenes~~ Add new ARKitScenes example Apr 4, 2023

emilk merged commit a479c0c into rerun-io:main Apr 4, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new ARKitScenes example #1538

Add new ARKitScenes example #1538

pablovela5620 commented Mar 9, 2023 •

edited by emilk

Loading

nikolausWest commented Mar 9, 2023

pablovela5620 commented Mar 10, 2023

nikolausWest commented Mar 10, 2023 •

edited

Loading

pablovela5620 commented Mar 10, 2023

emilk commented Mar 11, 2023

pablovela5620 commented Mar 12, 2023

nikolausWest left a comment

pablovela5620 commented Mar 21, 2023

nikolausWest commented Mar 21, 2023

nikolausWest commented Mar 21, 2023

nikolausWest left a comment

emilk commented Mar 28, 2023 •

edited

Loading

nikolausWest commented Mar 29, 2023

emilk Mar 31, 2023

emilk Mar 31, 2023

emilk left a comment

Add new ARKitScenes example #1538

Add new ARKitScenes example #1538

Conversation

pablovela5620 commented Mar 9, 2023 • edited by emilk Loading

Checklist

nikolausWest commented Mar 9, 2023

pablovela5620 commented Mar 10, 2023

nikolausWest commented Mar 10, 2023 • edited Loading

pablovela5620 commented Mar 10, 2023

emilk commented Mar 11, 2023

pablovela5620 commented Mar 12, 2023

nikolausWest left a comment

Choose a reason for hiding this comment

pablovela5620 commented Mar 21, 2023

nikolausWest commented Mar 21, 2023

nikolausWest commented Mar 21, 2023

nikolausWest left a comment

Choose a reason for hiding this comment

emilk commented Mar 28, 2023 • edited Loading

nikolausWest commented Mar 29, 2023

emilk Mar 31, 2023

Choose a reason for hiding this comment

emilk Mar 31, 2023

Choose a reason for hiding this comment

emilk left a comment

Choose a reason for hiding this comment

pablovela5620 commented Mar 9, 2023 •

edited by emilk

Loading

nikolausWest commented Mar 10, 2023 •

edited

Loading

emilk commented Mar 28, 2023 •

edited

Loading