Splat batches #15

emilk · 2022-06-16T19:53:29Z

and a bunch of optimizations

* node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (rerun-io#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (rerun-io#1920) * Always spawn instead of fork in multiprocessing example (rerun-io#1922) * Add `--num-frames` arg to canny (webcam) example (rerun-io#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (rerun-io#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (rerun-io#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (rerun-io#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (rerun-io#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (rerun-io#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (rerun-io#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (rerun-io#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (rerun-io#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (rerun-io#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (rerun-io#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (rerun-io#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (rerun-io#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (rerun-io#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (rerun-io#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (rerun-io#1969) * Full (experimental) WebGPU support (rerun-io#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (rerun-io#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]>

* v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (rerun-io#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (rerun-io#1920) * Always spawn instead of fork in multiprocessing example (rerun-io#1922) * Add `--num-frames` arg to canny (webcam) example (rerun-io#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (rerun-io#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (rerun-io#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (rerun-io#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (rerun-io#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (rerun-io#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (rerun-io#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (rerun-io#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (rerun-io#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (rerun-io#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (rerun-io#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (rerun-io#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (rerun-io#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (rerun-io#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (rerun-io#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (rerun-io#1969) * Full (experimental) WebGPU support (rerun-io#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (rerun-io#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix rerun lints * rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions * remove compiler warnings * fix memory leak when app is in background * Custom viewport UI (#4) * custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible * small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui * custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly. * improve auto layout to not split when only 3D or 2D view is available * MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB * Styling (#6) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Tabify all panels (except for blueprint) (#7) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive * remove bottom panel, switch to RAW imu sensors * XLink statistics initial implementation * initial xlink throughput statistics impllementation, have to glow it up a bit and maybe clean up the code * Xlink and rerun rename (#9) * Rename rerun py library to depthai_viewer * bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts * auto layout * Fix maximize not working and add maximize for Stats tab * delete profiling stuff that shouldn't have been commited * mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split * add docstring for update_tree * WIP auto layout can_create_mono_quad checker, not at all finished yet * pass lint checks and bugfixes * forgot to sort imports * Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid * pylints and fix Queue typehinting * fix doc build * try to pass pylints with py.typed * ignore misc mypy errors * forgot to run black formatter * switch back to old ci --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]>

* v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (rerun-io#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (rerun-io#1920) * Always spawn instead of fork in multiprocessing example (rerun-io#1922) * Add `--num-frames` arg to canny (webcam) example (rerun-io#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (rerun-io#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (rerun-io#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (rerun-io#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (rerun-io#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (rerun-io#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (rerun-io#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (rerun-io#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (rerun-io#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (rerun-io#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (rerun-io#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (rerun-io#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (rerun-io#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (rerun-io#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (rerun-io#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (rerun-io#1969) * Full (experimental) WebGPU support (rerun-io#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (rerun-io#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix rerun lints * rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions * remove compiler warnings * fix memory leak when app is in background * Custom viewport UI (#4) * custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible * small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui * custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly. * improve auto layout to not split when only 3D or 2D view is available * MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB * Styling (#6) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Tabify all panels (except for blueprint) (#7) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive * remove bottom panel, switch to RAW imu sensors * XLink statistics initial implementation * initial xlink throughput statistics impllementation, have to glow it up a bit and maybe clean up the code * Xlink and rerun rename (#9) * Rename rerun py library to depthai_viewer * bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts * auto layout * Fix maximize not working and add maximize for Stats tab * delete profiling stuff that shouldn't have been commited * mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split * add docstring for update_tree * WIP auto layout can_create_mono_quad checker, not at all finished yet * pass lint checks and bugfixes * forgot to sort imports * Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid * pylints and fix Queue typehinting * fix doc build * try to pass pylints with py.typed * ignore misc mypy errors * forgot to run black formatter * switch back to old ci * sync mono camera settings * fix web build * Update scripts/version_util.py --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]>

* change version to 0.0.1 * format pyproject.toml * add back check_version to version_util.py * switch to normal runners * try [email protected] * Some fixes (#11) * v0.0.1 of DepthaiViewer, WIP (#1) * node graph scaffolding and a bit of depthai integration * remove import * left, right + depth * fix fps sliders for mono cams * partial refactor, partially added support for device selection. * pc support + currently selected device fixes * reafactor subscriptions to api, doesn't work too well, trouble with syncing the ui with the backend * Partially migrated to websockets for the api * finish moving to websockets for the config api * fix issues after merge, retry websocket connections * fix ws thread not exiting * sleep for 1 sec before trying to reconnect * Ai support + imu support + rotate camera + bug fixes (#9) * ai support + update pipeline on device select * quick fix depth * set subs when selecting device * rename to Depthai Viewer, use changed() for config ui, disable ui when device not selected * added age gender detection * added mobilenet support * pointcloud support * Toggle subs from visible (#4) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * Toggle subs from visible (#5) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * Fix ws not sending messages, fix stutter every 2s (#6) * very bad implementation of toggling subscriptions based on space view visibility TODO: less cloning and stuff * better implementation and actually seems to work * fix windows not resizing properly when size gets smaller * fix issues with 3d when resizing scene bbox, only force 2d to always have the size of self.scene_bbox * Nuked Subscriptions struct, just use Vec<ChannelId>, much nicer, much better aswell, also fixed subscriptions for channels that need some time to actually show up, it all feels a bit hacky tho * fix incorrect despawn of entity paths * remove todo comment * error handling * fix websocket not sending pipeline, hotfix more or less might keep * change back to unbounded * Imu support (#8) * add imu logging, exclude it from regular space views * fix imu logging * POC imu plotting * Imu accelerometer and gyroscope plotting * add orientation to imu log, make device id a String * fix imu charts layout * imu log add magnetometer * fix right panel ui as much as it makes sense to fix rn * sadly idk how to create Option::None in pyarrow for the magnetometer field * imu logging fixed, magnetometer value is now None when not logged * fix plot scrolling * Merge upstream (#10) * `arrow2_convert` primitive (de)serialization benchmarks (rerun-io#1742) * arrow2_convert primitive benchmarks * addressing PR comments * Fix logged obb being displayed with half of the requested size (rerun-io#1749) * benchmarks for common vector ops across `smallvec`/`tinyvec`/std (rerun-io#1747) * benchmarks for common vector ops * handle N=1 * Tracked 3D cameras lead now to on-hover rays in other space views that show the same camera but don't track it. (rerun-io#1751) In the same way as a 2D scene causes a on-hover ray in all space views that contain the space camera at which the 2D view "sits". * Improve dealing with raw buffers for texture read/write (rerun-io#1744) * Replace TextureRowDataInfo with the more versatile Texture2DBufferInfo * comment & naming fixes * `arrow2` erased refcounted clones benchmarks (rerun-io#1745) * arrow2 erased refcounted clone benchmarks * lint * addressing PR comments * dude * `arrow2` estimated_bytes_size benchmarks (rerun-io#1743) * arrow2 estimated_bytes_size benchmarks * cleanup * Fix crash when trying to do picking on depth clouds * Readback depth from GPU picking (rerun-io#1752) * gpu picking in the viewer picks up depth now * WebGL workarounds * Add new ARKitScenes example (rerun-io#1538) Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix log_obb usage (rerun-io#1761) * Make sure all log_obb uses uses half_size correctly * Remove outdated link from README * Fix docstring of save * Force named arguments of log_scalar * Add link to --memory-limit docs * update ros example * Python SDK: document that we also accept colors in 0-1 floats (rerun-io#1740) * Python SDK: document that we also accept colors in 0-1 floats * Assume float colors to be in gamma-space, and document that * Update arkitscenes example * Fix bug * typo * py-format * Collapse space-view by default if there is only one child (rerun-io#1762) * Always create the log_time timeline (rerun-io#1763) * Columnar timepoints in data tables and during transport (rerun-io#1767) * columnar timepoints * self review * Fix undo/redo selection shortcut/action changing selection history without changing selection (rerun-io#1765) * Fix undo/redo selection shortcut/action changing selection history without changing selection Fixes rerun-io#1172 * typo fix * Don't initialize an SDK session if we are only going to be launching the app (rerun-io#1768) * Allow torch tensors for log_rigid3 (rerun-io#1769) * Option to show scene bounding box (rerun-io#1770) * Include depth clouds in bounding box calculation * Don't wrap text when showing bbox in ui * Handle projective transforms * Nicer selection view: don't wrap second column too early * Add checkbox to show the scene bounding box * Fix a whole lot of crashes, all at once (rerun-io#1780) * Add typing_extensions to requirements-doc.txt (rerun-io#1786) * auto_color class-ids if they are present (rerun-io#1783) * auto_color class-ids if they are present * Update log line in segmentation demo * Avoid tuple structs * Don't run 3rd party bench suites on CI (rerun-io#1787) * dont run 3rd party bench suites on CI * typo * and other annoyances * Use copilot markers in PR template (rerun-io#1784) * Use copilot markers in PR template * remove poem Co-authored-by: Clement Rey <[email protected]> --------- Co-authored-by: Clement Rey <[email protected]> * re_format: barebone support for custom formatting (rerun-io#1776) * implement barebone support for custom formatting and apply to Tuid * unwrap * <ERR> rather than [] * use re_tuid * Always send recording_id as part of LogMsg (rerun-io#1778) * Always send recording_id as part of LogMsg * Rename build_chunk_from_components -> build_data_table_from_components * Don't make RecordingInfo optional * Always default the recording id * Log an error if we hit the initialization issue * Refactor: Add new helper crate `re_log_encoding` (rerun-io#1772) * CI: Check `rerun` with --no-default features and/or with --features sdk * Create a new helper crate re_transport containing stream_rrd_from_http * Fix warnings * Move file sink to re_transport * wasm compilation fix * Move LogMsg encoding/decoding into re_transport * Fix typo * Fix web build * Fix tests * Remove a lot of unused dependencies with `cargo machete` * Build fix * Clarify * Rename the crate to re_log_encoding * better docstring Co-authored-by: Jeremy Leibs <[email protected]> * better readme Co-authored-by: Jeremy Leibs <[email protected]> --------- Co-authored-by: Jeremy Leibs <[email protected]> * New example code for facebook research segment anything (rerun-io#1788) * New example code for facebook research segment anything * Add segmentation workaround for users still on 0.4.0 * Images should use class-id as label * Add an alternative tensor-based view * Implement `re_tuid::Tuid::random()` on web (rerun-io#1796) * Implement `re_tuid::Tuid::random()` on web * Fix bad error message * ci: fix benchmarks (rerun-io#1799) * workflow: just run --all * datastore: skip bucket permutations etc on CI * i give up, just replace re_log_types by re_log_encoding * Add `minimal_options` example (`RerunArgs`) (rerun-io#1773) * Allows connecting to remote server through rerun's RerunArgs. Co-authored-by: Clement Rey <[email protected]> * Add `pacman` support to `setup_web.sh` (rerun-io#1797) Co-authored-by: Clement Rey <[email protected]> * fix ci (cargo-deny): cargo update -p crossbeam-channel (rerun-io#1806) * Compile with `panic = "abort"` (rerun-io#1813) * Compile with `panic = "abort"` This PR sets `panic = "abort"` for both debug and release builds. This cuts down the `rerun` binary size in release builds from 29.9 MB to 22.7 MB - a 25% reduction! ## Details The default panic behavior in Rust is to unwind the stack. This leads to a lot of extra code bloat, and some missed opportunities for optimization. The benefit is that one can let a thread die without crashing the whole application, and one can use `std::panic::catch_unwind` as a kind of try-catch block. We don't make use of these features at all (at least not intentionally), and so are paying a cost for something we don't need. I would also argue that a panic SHOULD lead to a hard crash unless you are building an Erlang-like robust actor system where you use defensive programming to protect against programmer errors (all panics are programmer errors - user errors should use `Result`). * Quiet clippy * Add `rerun --strict`: crash if any warning or error is logged (rerun-io#1812) * Add `rerun --strict`: crash if any warning or error is logged Part of rerun-io#1483 * Can't doc-test private functions * Refactor: Remove `TensorTrait` (rerun-io#1819) * Refactor: Remove `TensorTrait` We don't need it anymore * End-to-end testing of python logging -> store ingestion (rerun-io#1817) * Sort the arguments to `rerun` * Pass on `LogMsg::Goodbye` just like any other message * Add `rerun --test-receive` * `just py-build --quiet` is now possible * Add scripts/run_python_e2e_test.py * replace `cargo r -p rerun` with `python3 -m rerun` * lint and explain choice of examples * Add to CI * check returncode * Fix e2e test on CI: Don't try to re-build rerun-sdk (rerun-io#1821) * Use gpu picking for points, streamline/share picking code some more (rerun-io#1814) * use gpu picking for picking points * gpu based picking no longer works like a fallback but integrates with other picking sources * fix incorrect cursor rounding for picking * refactor picking context to be a pub struct with exposed state * unify ui picking method for 2d & 3d space views * less indentation for picking method * picking rect size is dynamically chosen * fix accidental z scaling in projection correction for picking & make cropped_projection_from_projection easier to read * CI: install pip requirements for Python e2e test * Process 2d points always in batches (rerun-io#1820) * Fix CI syntax error * New option to disable persistent storage (rerun-io#1825) * New option to disable persistent storage * New API to reset_time (rerun-io#1826) * Datastore revamp 1: new indexing model & core datastructures (rerun-io#1727) * Datastore revamp 2: serialization & formatting (rerun-io#1735) * Datastore revamp 3: efficient incremental stats (rerun-io#1739) * Datastore revamp 4: sunset `MsgId` (rerun-io#1785) * Datastore revamp 5: (rerun-io#1791) * Datastore revamp 6: sunset `LogMsg` storage + save store to disk (rerun-io#1795) * Datastore revamp 7: garbage collection (rerun-io#1801) * Don't assert if inserting a rowid with a matching timepoint does not create a conflict (rerun-io#1832) * CI: Try installing the correct wheel on CI * CI: try again with the CI * re_query: up to date with latest data types and structures (rerun-io#1828) * No more raw arrays for primary components * Don't need to carry around component names no more * Cluster keys are now raw-array-less and _not_ optional anymore * that is done indeed * helpers * datastore: incremental metadata registry stats (rerun-io#1833) * add profile scopes for stats * implement incremental metadata registry stats * lint * future proofing comment * RFC: datastore state of the union & end-to-end batching (rerun-io#1610) * add batching rfc * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * Update design/batching.md Co-authored-by: Emil Ernerfeldt <[email protected]> * qa component instances vs. instance keys * no sticky * qa are there any special components & non-integer instance keys * give some clue about cell incompatibility * temporary constructs * zstd is already setup * more planning * date and links --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Python CI: use bash as shell * Fix too many points crash (rerun-io#1822) * Simplify point cloud builder and fix crash on too many points Fixes rerun-io#1779 * faster point cloud population by not chaining iterators with default values * Use GPU picking for line(like) primitives, fix `interactive` flags (rerun-io#1829) * line strip builder no longer has user data, exposes picking id instead (not implemented yet) * handle interactive object property when evaluating picking code * take line strip builder directly when building up line draw data * finish implementing picking for lines * remove unused iter_strips_with_vertices * Simplify picking handling now that there are a lot less types. Labels & textured rects are always picked now, fixes rerun-io#1021 * CI: try to fix mac wheel build * Reduce memory used by staging belts on Web (rerun-io#1836) In particular this prevents crashing with out of memory on a run-away belt memory usage caused by failure to unmap buffers. A bit concerningly, the fix uses our knowledge of how `wgpu::Device::poll` is broken in the current wgpu version. I took the opportunity to sharpens the definition of `HardwareTier` a bit. * CI: only test the x86_64 wheel on macos * Always flush when we remove a sink (rerun-io#1830) Whenever we disconnect (or implicitly disconnect by swapping a sink) we should flush the pending messages. Additionally disconnect and flush calls both require releasing the GIL (for the same reason as shutdown previously). * GPU colormapping, first step (rerun-io#1835) * Add TextureManager2D::get_or_create_with * Small code cleanup * Add code to upload a Tensor to a GPU texture * Add helper method Tensor::image_height_width_depth * Minor code cleanup (multiplicative_tint) * Hook up color textures via the new path * Refactor: introduce ColormappedTexture * Start working on an uint sampler * merge fix * Dumb colormapping of depth textures! * Use turbo for depth maps (and single-channel images :grimace:) * Use grayscale for luminance * ColorMap -> Colormap * Apply annotation context colormaps * Support sint textures too * cleanup * merge fix * Fix RGB images * More cleanup * Better error-handlign and nicer error message * Clean up the SAMPLE_TYPE with constants * Nicer shader interface * Better error handling * Remove dead code * Self-review cleanup * Fix bug in shader when sampling sint textures * Use textureSampleLevel * Apply a gamma to the image (unused as of now) * image_height_width_channels * fix various review comments * Optimize narrow_f64_to_f32s: avoid one allocation * Test and handle all tensor dtypes as images (rerun-io#1840) * Support i64 and u64 tensors * api_demo: log all image types * Don't even print out the contents of a tensor * Handle unfilterable float textures * fix typo * py-format * Simplify is_float_filterable * Add a helper function pad_and_narrow_and_cast * Reuse existing image * Exclude image_tensors demo from default api_demo * Still run all api demos in e2e test * pyformat * Install the rerun-sdk in CI using --no-index and split out linux wheel build to run first. (rerun-io#1838) * Install the rerun-sdk by the expected version * Fix comment * typo * Use --no-index when installing the rerun wheel * Use the cargo_version, not the new_version * Split dependency install into its own step * Don't use force-reinstall * Refactor setting of expected_version variable. * Use bash when setting env * Always run the linux job first and use its rrds for the other wheels * GPU tensor colormapping (rerun-io#1841) * Refactor: introduce struct SliceSelection * Refactor: use SliceSelection inside of ViewTensorState * MVP of tensor colormapping on GPU * Remove old ui code * Support 64-bit tensors by narrowing to f32 * Allow more colormap options * Clippy * Report range errors instead of ignoring them * Sort colormaps * Shorten function name * Create module gpu_bridge * Move some code around * Simnplify API * Create ViewBuilder::new * Fix missing colon in lint.py * fix typos and formatting * Disable texture filtering options for tensors for now * Update docstrings * Add profile scopes * ViewBuilder cleanup * Make ViewBuilder::setup non-Option * Remove Result from thing that cannot fail * Fix colormap numbering * review cleanup * pass in debug_name * Unify the `range` function * typo * Show previews of colormaps when selecting them (rerun-io#1846) * Make infallible version of get_or_create_texture * Show colormap previews in UI * Spelling * Implement billinear filtering of textures (rerun-io#1850) * Implement opt-in billinear filtering of textures * bilinear * MVP Support for inline-rendering of Rerun within jupyter notebooks (rerun-io#1798) (rerun-io#1834) (rerun-io#1844) * Introduce the ability to push an rrd binary via iframe.contentWindow.postMessage * New API to output the current buffered messages as a cell in jupyter * Example notebook with the cube demo * Track that we need to send another recording msg after draining the backlog * Dynamically resolve the app location based on git commit. Allow override to use self-hosted assets * Add some crude timeout logic in case the iframe fails to load * Don't persist app state in notebooks * Introduce new MemoryRecording for use with Jupyter notebooks (rerun-io#1834) * Refactor the relationship between the assorted web / websocket servers (rerun-io#1844) * Rename RemoteViewerServer to WebViewerSink * CLI arguments for specifying ports * Proper typing for the ports * Disable wheel tests for x86_64-apple-darwin (rerun-io#1853) * Fix typos in notebook readme (rerun-io#1852) * Fix the python build when running without web_viewer enabled (rerun-io#1856) * Error instead of expect inside msg_encode. (rerun-io#1857) * Restore: New API to reset_time (rerun-io#1826) (rerun-io#1854) * Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859) This reverts commit d33dab6. * Add Restart command and keyboard shortcut for moving time to start of timeline (rerun-io#1802) * Add Restart button to timeline UI. This sets the timeline back to zero. * Adds Restart Command & Timeline Command * Adds Ctrl-Shift-Space keyboard modifier * Remove restart button from timeline UI. * Use cmd(Key::LeftArrow) as key combo for restart timeline. * Add TimeControl::restart to restart the current timeline. * Use this from the kb_shortcut function * fix some code nits --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Fix shutdown race condition in `re_sdk_comms` client (rerun-io#1861) * Wait for encoder to shut down before shutting down the other threads * Remove unused dependencies (rerun-io#1863) * Gpu picking for depth clouds (rerun-io#1849) * wip * allow for hovering depth clouds via gpu picking * Use `[x, y]: [u32; 2]` as argument --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * Remove manual depth projection from car and nyud examples (rerun-io#1869) * Remove manual depth projection from car and nyud examples * revert change to cube.ipynb * revert changes to cube.ipynb * third times a charm for cube.ipynb * Improve end-to-end testing slightly (rerun-io#1862) * CI: Run e2e tests with RUST_LOG=debug * Move installing of pip packaged from CI to e2e script * Re-enable bilinear interpolation again (rerun-io#1860) * Revert "Revert "Implement billinear filtering of textures (rerun-io#1850)" (rerun-io#1859)" This reverts commit 625d2bd. * Split rectangle.wgsl into fragme/vertex parts to work around naga bug * Use GPU colormapping when showing images in the GUI (rerun-io#1865) * Cleanup: move Default close to the struct definition * Simplify code: use if-let-else-return * Simplify code: no need for Arc * Add EntityDataUi so that the Tensor ui function knows entity path * Better naming: selection -> item * Simplify code: no optional tensor stats * Less use of anyhow * Use GPU colormapping when showing tensors in GUI * Link to issue * Optimize pad_to_four_elements for debug builds * Refactor: simpler arguments to show_zoomed_image_region_area_outline * Fix missing meter argument * Refactor: break up long function * Less use of Arc * Pipe annotation context to the hover preview * Simplify `AnnotationMap::find` * Use new GPU colormapper for the hover-zoom-in tooltip * Refactor * Add helper function for turning a Tensor into an image::DynamicImage * Fix warning on web builds * Add helper function `Tensor::could_be_dynamic_image` * Implement click-to-copy and click-to-save for tensors without egui * Convert histogram to the new system * Remove the TensorImageCache * Fix TODO formatting * bug fixes and cleanups * Rename some stuff * Build-fix * Simplify some code * Turn off benchmakrs comment on each PR (rerun-io#1872) * Refactor: remove `GpuTexture2DHandle::invalid` (rerun-io#1866) * Refactor TexturedRect * Remove GpuTexture2DHandle::invalid * `GpuTexture2DHandle` is always valid * spacing * Update enumflags2 to non-yanked version (rerun-io#1874) * Update enumflags2 to non-yanked version ``` ❯ cargo update -p enumflags2 Updating crates.io index Updating enumflags2 v0.7.5 -> v0.7.7 Updating enumflags2_derive v0.7.4 -> v0.7.7 Updating proc-macro2 v1.0.47 -> v1.0.56 Updating quote v1.0.21 -> v1.0.26 Adding syn v2.0.15 ``` Unfortunately this adds the syn v2 dependency for some platforms * Updating dependencies is a valid label * cargo deny: check more platforms * fix stuff broken from merging upstream --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> * expose depth config, get available sensor resolutions for the selected device * move removing entities to a place where the removal will always be tried, not just when expanding space view header * added depth alignment * added stream enabled buttons * Depth cloud textures are now cached frame-to-frame (rerun-io#1913) * Depth cloud textures are now cached frame-to-frame Simplified logic a bit by enforcing F32 texture conversion (there was a u16 path for native only) * doc fix * naming consistency, format check, remove unnecessary scaling * improve depth cloud texture check * fixes after merging * Smooth out scroll wheel input for camera zooming (rerun-io#1920) * Always spawn instead of fork in multiprocessing example (rerun-io#1922) * Add `--num-frames` arg to canny (webcam) example (rerun-io#1923) * fix formatting issues * rerun format fix * fix spelling * lint check fixes * mypy * some more lint fixes * some more fixes for python lint checks * Collect extra egui features into the main Cargo.toml (rerun-io#1926) * just rs-run-all * `just py-run-all-{native|web|rrd}` (rerun-io#1927) * make all python examples handle unknown arguments gracefully * just py-run-all-{native|web|rrd} * bump version * comment out clang * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * change python workflow for testing purposes, remove windows and macos wheels * add back one macos so the matrix is valid * 2.5GB before GC kick in on web (rerun-io#1944) * change name to depthai-viewer * change pip install/uninstalls from rerun-sdk to depthai-viewer * change all occurances of rerun-sdk to depthai-viewer * change windows runner to windows-latest for now when using my personal gh * Release `0.5.0` (rerun-io#1919) * changelog * 0.5.0-alpha.0 * more changelog * re_format: fix implicit recursive feature flags * publish_crates.sh: fix crate ordering * Join threads at end of multi-threading example (rerun-io#1934) * Add argument parsing to the rerun_demo (rerun-io#1925) * More robust wait for exit condition during .serve() (rerun-io#1939) * More robust wait for exit condition during .serve() * lint * Use zipfile python library instead of `unzip` command in arkitscene (rerun-io#1936) * Use zipfile python library instead of `unzip` command in arkitscene Windows doesn't have unzip! * Nit: import sort order --------- Co-authored-by: Nikolaus West <[email protected]> * Fix annotation images sometimes drawn in the background. (rerun-io#1933) This caused fairly ugly rendering whenever that happened. Also cleaned up redundant image/textured_rect defintions in spatial scene buildup * Fix backslashes in arkitscene rigid transformation path (rerun-io#1938) * Fix backslashes in arkitscene rigid transformation path Should be fixed properly by rerun-io#1937 * Use PosixPath instead of .replace("\\", "/") --------- Co-authored-by: Nikolaus West <[email protected]> * changelog * Fix hover/select highlights when picking single points in a scene with multiple point clouds (rerun-io#1942) Batch vertex offset for single highlights wasn't correctly computed. Different parts of the code made different assumptions what offsets referred to * changelog * Fix hovering depth clouds (rerun-io#1943) We didn't add to `scene.primitives.image`. Instead of adding to this list, it is instead now no longer needed for picking since we can very easily query for tensor again. * changelog * 2.5GB before GC kick in on web (rerun-io#1944) * changelog * 0.5.0 --------- Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix imu plots scrolling past their container * fix bottom/top panel sizing after showing the spinner on config setting * Lint error names in `map_err` (rerun-io#1948) * Lint: Properly name errors in `map_err` * Use correct names for errors * fix config and stats tabs not being able to be viewed at the same time, renamed stats to IMU * New dispatch-only workflow for running the lint-job (rerun-io#1950) * Fix secret in dispatch_lint.yml * Only maintain a single manual-dispatch job for testing workflows * apply button, have to make it look a bit nicer, but will do that when I fix scrolling in device configuration panel * Bump hyper version due to RUSTSEC-2023-0034 (rerun-io#1951) * Add other build parameterizations to manual_dispatch.yml * Use proper if gates on the manual_dispatch.yml jobs * Add ability to save cache to manual_disaptch.yml * Standard case of inputs * Add manual step for packaging to 'manual_dispatch.yml' * add back panels when the underlying subscription reappears * Fix crash for missing class ids causing zero sized texture (rerun-io#1947) * Fix crash for missing class ids causing zero sized texture Two things fixed actually: * texture manager now checks for zero sized texture, this ripples out in a lot more error handling * class id texture texture handles not having any classes gracefully * Use Display for all errors * typo * Better naming of error * Better docs and names * Fix off-by-one error * some use of `context`, change which error is implicitly converted on texture manager2d --------- Co-authored-by: Emil Ernerfeldt <[email protected]> * fixes LR stream subscriptions (maybe breaks panels reappearing after sub is gone) (#15) * make config ui scrollable, fix padding * Move clippy_wasm/clippy.toml to under scripts (rerun-io#1949) * Move clippy_wasm/clippy.toml to under scripts This is just to clean up the root a bit, and to move it closer to where it is actually used. * Fix comment typo --------- Co-authored-by: Andreas Reich <[email protected]> * change crate version to 0.6.0-alpha.0 (rerun-io#1952) * New workflow_dispatch for building wheels for a PR * initial light mode, luxonis depthai viewer * Rename build_wheels_for_pr.yml -> manual_build_wheels_for_pr.yml * Fix run-wasm crash on trying to wait for server (rerun-io#1959) This ruined my dev experience for re_renderer examples a bit. Not sure what made the previous hack stop working, might be a timing issue. It ended up crashing the `cargo_run_wasm` web server * Update egui to latest and wgpu to 0.16 (rerun-io#1958) * update to wgpu 0.16 and egui using this version * shader fixup for type aliases and rectangle shader * shader signed/unsigned shenanigans * more signed/unsigned issues * fix texture component count * fix picking layer depth readback crash on web * patch wgpu * better texture size estimate * fix patches * Handle leaking of prerelease into alpha version (rerun-io#1953) * Make device config panel remember it's height throughout loading * hide time panel, make ai model dropdown wider * Fix incorrect memory usage stats for destroyed on-creation-mapped buffers (rerun-io#1963) We actually don't have anywhere where we discard this kind of buffer yet, but if we would the stats would be wrong (noticed while doing quick & dirty experiments on the staging belt) * New manual workflow for running benches * Introduce new reusable workflow jobs and cleanup manual trigger (rerun-io#1954) There are 8 reusable workflow "components" that we can use to build different scenarios: reusable_checks.yml - These are all the checks that run to ensure the code is formatted, reusable_bench.yml - This job runs the benchmarks to check for performance regressions. reusable_deploy_docs- This job deploys the python and rust documentation to https://ref.rerun.io reusable_build_and_test_wheels.yml - This job builds the wheels, runs the end-to-end test, and produces a sample RRD. The artifacts are accessible via GitHub artifacts, but not otherwise uploaded anywhere. reusable_upload_wheels.yml- This job uploads the wheels to google cloud reusable_build_web.yml - This job builds the wasm artifacts for the web. reusable_upload_web.yml - This job uploads the web assets to google cloud. By default this only uploads to: app.rerun.io/commit/<commit>/ reusable_pr_summary.yml - This job updates the PR summary with the results of the CI run. Example summary can be found at: https://storage.googleapis.com/rerun-builds/pull_request/1954/index.html This also introduces a manual_dispatch.yml helper as a convenience for testing these workflows and their different parameterizations. * New manual workflow for adhoc web builds * Use new CI workflows for pull-request and merge to main (rerun-io#1955) on_pull_request.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_build_and_test_wheels.yml -- Configured in a "minimal" mode with SDK includes end-to-end test and produces an rrd. - reusable_build_web.yml -- Verifies we can build the wasm - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io to confirm the demo works as well as support notebook testing. - reusable_pr_summary.yml -- Create a manifest page with a link to the on_push_main.yml includes the following pieces: - reusable_checks.yml -- Run all of the lints, code-formatting, tests, etc. - reusable_bench.yml -- Run the benchmarks - reusable_build_and_test_wheels.yml -- Builds wheels for all platforms - reusable_upload_wheel.yml -- Uploads the all the wheels to gcloud - reusable_build_web.yml -- Builds the wasm bundle - reusable_upload_web.yml -- Uploads the RRD and Wasm to app.rerun.io - reusable_pip_index.yml -- Generates a pip index page which can be used to install packages with, e.g. * Fix name of on_push_main.yml * Fix usage of long commit in generate_prerelease_pip_index.py * Try making pull-request workflows non-concurrent (rerun-io#1970) * Try making pull-request workflows non-concurrent * Concurrency groups for push_main as well * Each sub-workflow needs its own name or they fight * Another attempt to make jobs non-concurrent on a per-PR basis (rerun-io#1974) * Another attempt to make jobs non-concurrent on a per-PR basis * Move concurrency into the reusable job * Jobs with duplicated instances still need separate concurrency keys based on platform * Round to nearest color_index when doing color mapping (rerun-io#1969) * Full (experimental) WebGPU support (rerun-io#1965) * always build with unstable web sys apis * Make shader Tint friendly * expose webgl feature flag on re_renderer & re_viewer * fix bug link on negative hexadecimal * hardware tier is now created from wgpu adapter * sort out build flags for webgpu & document building webviewer * introduce shader text replacement workarounds to workaround current chrome issue * latest egui master * typo fix * doc fixes, use if cfg! instead of attribute cfg * move backend to rerun * If there's a `{{ pr-build-summary }}` in the PR description, update it. (rerun-io#1971) * If there's a `{{ pr-build-summary }}` in the PR description, update it. * Add comment to the PR template * Add pull-requests permission to pr_summary job * Run the cube notebook on PR (rerun-io#1972) * Run the cube notebook on PR * Add notebooks to the build summary * Use the new concurrency model * reformat py files * reformat * fix pylint errors * Add ability to manually run a web build to upload to an adhoc name (rerun-io#1966) * Add ability to manually run a web build to upload to an adhoc name * Pass through ADHOC_NAME * Add a concurrency criteria for the new adhoc job * Make input description more explicit * change entity paths * merged wip albedo colormap into latest depth_cloud * remove pointclouds generated in sdk * fix compiler error, trying to compile frame.close for wasm * Default to albedo texture for depth cloud, added support for mono albedo textures * restart backend on failure, added oak_cam.device.close seems to really close the cam * py lint fix * don't run notebooks * remove run-notebook dependency --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * Fix rerun lints * rename binary, always start with memory-limit, pin depthai dependencies, fix errors after changing dependency versions * remove compiler warnings * fix memory leak when app is in background * Custom viewport UI (#4) * custom blueprint panel, show logical space views for depthai-viewer users, added settings clog on top of space view tab to configure what is visible * small fixes, clear entity_paths every time to avoid displaying an unavailable entity_path in space view options ui * custom left panel to add or remove space view instances, created a new default viewport layout. Improved behaviour when a panel re appears after user selected to hide it, then if stream stops and starts again the panel will be spawned back in correctly. * improve auto layout to not split when only 3D or 2D view is available * MJPEG encode image frames if connected to a PoE device. Only add magnetometer to imu sensors list if the device has a BNO IMU. Lower the memory limit to 100MB * Styling (#6) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Tabify all panels (except for blueprint) (#7) * initial styling impl * make buttons that should be small, small * Runtime depth config and fix device selection ui * comment * Proper runtime depth config updates * switch to Yolo v8 * add comma to label for non open zoo models * split 2d + 3d cam view vertically instead of horizontally * Make the UI more configurable by converting the right panel into tabs. TODO: UX while laying out the panels. When a new space view appears only update the viewport layout, try to keep the user configured fixed function panels as they were. Just handle it in a way that is intuitive * remove bottom panel, switch to RAW imu sensors * XLink statistics initial implementation * initial xlink throughput statistics impllementation, have to glow it up a bit and maybe clean up the code * Xlink and rerun rename (#9) * Rename rerun py library to depthai_viewer * bug fixes and started working on a smart auto layout that operates on an existing tree, to preserve ui as much as possible, while also creating good layouts * auto layout * Fix maximize not working and add maximize for Stats tab * delete profiling stuff that shouldn't have been commited * mostly fix smart layout, TODO: detect when you can group mono and color 3d + 2d views into a 4 way split * add docstring for update_tree * WIP auto layout can_create_mono_quad checker, not at all finished yet * pass lint checks and bugfixes * forgot to sort imports * Fix mypy lint (TODO: Proper typing, especially in the comms from back to store to ws) other types are pretty solid * pylints and fix Queue typehinting * fix doc build * try to pass pylints with py.typed * ignore misc mypy errors * forgot to run black formatter * switch back to old ci * sync mono camera settings * fix web build * Update scripts/version_util.py --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]> * install depthai_viewer instead of rerun-sdk for pytest * Updated design and improvements * get rid of rerun_sdk and rerun_bindings, now just depthai_viewer and depthai_viewer_bindings get installed, so you don't clash with an existing rerun install * hopefuly fix ci * hopefuly fix ci * a bunch of fixes and improvements * disable smart layout updater for now * bugfixes * Update README.md * Patches --------- Co-authored-by: Clement Rey <[email protected]> Co-authored-by: benjamin de charmoy <[email protected]> Co-authored-by: Andreas Reich <[email protected]> Co-authored-by: Emil Ernerfeldt <[email protected]> Co-authored-by: Pablo Vela <[email protected]> Co-authored-by: Nikolaus West <[email protected]> Co-authored-by: Jeremy Leibs <[email protected]> Co-authored-by: h3mosphere <[email protected]> Co-authored-by: Urho Laukkarinen <[email protected]> Co-authored-by: Nikolaus West <[email protected]>

emilk added 17 commits June 15, 2022 14:01

Split out Data::Batch to a new enum, LoggedData

9af6550

Add "batch splatting"

21a975e

Optimize batch insertion

c23fa2a

Introduce IndexHash

fef2a15

Refactor: move Index to own file

0030734

struct Batch

d1710cb

Optimize storage by using hashes instead of full indices and index paths

77d85a8

Don't use three_d::Color::new

d079459

Don't box IndexKey::index

4d375a9

Fix crash

5b55449

Benchmark data insertion

f70b9fe

Remove hash from IndexPath

6079a58

Use IntMap for ObjTypePath

9747fb5

Less use of IndexKey

bf27a95

Remove IndexKey

78b4d1c

Small optimization

689d602

Clippy fixes

1487dff

emilk merged commit 7af9294 into main Jun 16, 2022

emilk deleted the splat-batch branch June 16, 2022 20:06

emilk mentioned this pull request Jan 24, 2023

egui deadlock #833

Closed

teh-cmc mentioned this pull request Jan 30, 2024

Deadlock playground: disable top-level system exec parallelism #4968

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Splat batches #15

Splat batches #15

emilk commented Jun 16, 2022

Splat batches #15

Splat batches #15

Conversation

emilk commented Jun 16, 2022