Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DDS using generic format types #11946

Merged
merged 10 commits into from
Jul 11, 2023
Merged

DDS using generic format types #11946

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
71bb34c
rs-dds-adapter sends only basic formats (no conversions)
OhadMeir Jun 20, 2023
f77c8dd
rs-dds-stream-proxy handles basic stream formats only. Not dependent …
OhadMeir Jun 22, 2023
e4c874b
Enable different frequencies to motion streams of same sensor
OhadMeir Jun 22, 2023
d8bc61c
Handle colored IR and confidence streams. Remove rs-dds-internal-data
OhadMeir Jun 22, 2023
95416c9
Disabling different frequencies to motion streams of same sensor (rev…
OhadMeir Jun 26, 2023
b1a6a16
Updating DDS unit tests for generic formats conversion (not product-i…
OhadMeir Jun 26, 2023
b761c1d
Save context json settings and use them instead of individual flags
OhadMeir Jun 26, 2023
77866e7
Handle DDS default profiles. Adapter sets defaults (might be differen…
OhadMeir Jun 28, 2023
3ed5f9e
Handle PR#11946 comments
OhadMeir Jul 6, 2023
edff921
Some more PR#11946 comments
OhadMeir Jul 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/context.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,8 @@ namespace librealsense
context::context( json const & settings )
: context()
{
_settings = settings;

_backend = platform::create_backend(); // standard type

environment::get_instance().set_time_service( _backend->create_time_service() );
Expand Down
5 changes: 4 additions & 1 deletion src/context.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
#include "core/streaming.h"

#include <vector>
#include <nlohmann/json_fwd.hpp>
#include <rsutils/json.h>
maloel marked this conversation as resolved.
Show resolved Hide resolved
#include "media/playback/playback_device.h"

namespace librealsense
Expand Down Expand Up @@ -136,6 +136,7 @@ namespace librealsense

void add_software_device(std::shared_ptr<device_info> software_device);

const nlohmann::json & get_settings() const { return _settings; }

private:
void on_device_changed(platform::backend_device_group old,
Expand All @@ -158,6 +159,8 @@ namespace librealsense
void start_dds_device_watcher( size_t message_timeout_ms );
#endif

nlohmann::json _settings; // Save operation settings

devices_changed_callback_ptr _devices_changed_callback;
std::map<int, std::weak_ptr<const stream_interface>> _streams;
std::map<int, std::map<int, std::weak_ptr<lazy<rs2_extrinsics>>>> _extrinsics;
Expand Down
31 changes: 30 additions & 1 deletion src/dds/rs-dds-device-proxy.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ dds_device_proxy::dds_device_proxy( std::shared_ptr< context > ctx, std::shared_

for( auto & sensor_info : sensor_name_to_info )
{
sensor_info.second.proxy->initialization_done( dev->device_info().product_id, dev->device_info().product_line );
sensor_info.second.proxy->initialization_done();

// Set profile's ID based on the dds_stream's ID (index already set). Connect the profile to the extrinsics graph.
for( auto & profile : sensor_info.second.proxy->get_stream_profiles() )
Expand All @@ -233,6 +233,8 @@ dds_device_proxy::dds_device_proxy( std::shared_ptr< context > ctx, std::shared_
set_profile_intrinsics( profile, stream_iter->second );

_stream_name_to_profiles[stream_iter->second->name()].push_back( profile ); // For extrinsics

tag_default_profiles( profile, stream_iter->second );
}
}

Expand Down Expand Up @@ -366,4 +368,31 @@ std::shared_ptr< dds_sensor_proxy > dds_device_proxy::create_sensor( const std::
}


// Tagging converted profiles. dds_sensor_proxy::add_video/motion_stream tagged the raw profiles.
void dds_device_proxy::tag_default_profiles( std::shared_ptr<stream_profile_interface> & profile,
maloel marked this conversation as resolved.
Show resolved Hide resolved
const std::shared_ptr< realdds::dds_stream > & stream ) const
{
auto const & dds_default_profile = stream->profiles()[stream->default_profile_index()];
maloel marked this conversation as resolved.
Show resolved Hide resolved

if( profile->get_stream_type() == to_rs2_stream_type( stream->type_string() ) &&
profile->get_format() == dds_default_profile->format().to_rs2() &&
profile->get_framerate() == dds_default_profile->frequency() )
{
auto vsp = std::dynamic_pointer_cast< video_stream_profile >( profile );
auto dds_vsp = std::dynamic_pointer_cast< realdds::dds_video_stream_profile >( dds_default_profile );
if( vsp && dds_vsp &&
( vsp->get_width() != dds_vsp->width() || vsp->get_height() != dds_vsp->height() ) )
return; // Video profiles of incompatible resolutions

profile->tag_profile( PROFILE_TAG_DEFAULT );
}
}


void dds_device_proxy::tag_profiles( stream_profiles profiles ) const
{
//Do nothing. PROFILE_TAG_DEFAULT is already added in tag_default_profiles.
}


} // namespace librealsense
6 changes: 6 additions & 0 deletions src/dds/rs-dds-device-proxy.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ class dds_device;
class dds_stream;
class dds_video_stream;
class dds_motion_stream;
class dds_stream_profile;
} // namespace realdds


Expand Down Expand Up @@ -51,7 +52,12 @@ class dds_device_proxy : public software_device
public:
dds_device_proxy( std::shared_ptr< context > ctx, std::shared_ptr< realdds::dds_device > const & dev );

void tag_default_profiles( std::shared_ptr< stream_profile_interface > & profile,
maloel marked this conversation as resolved.
Show resolved Hide resolved
const std::shared_ptr< realdds::dds_stream > & stream ) const;

std::shared_ptr< dds_sensor_proxy > create_sensor( const std::string & sensor_name );

void tag_profiles( stream_profiles profiles ) const override;
};


Expand Down
91 changes: 52 additions & 39 deletions src/dds/rs-dds-sensor-proxy.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#include <proc/spatial-filter.h>
#include <proc/temporal-filter.h>
#include <proc/threshold.h>
#include <proc/color-formats-converter.h>

#include <rsutils/json.h>

Expand Down Expand Up @@ -106,47 +107,59 @@ std::shared_ptr< stream_profile_interface > dds_sensor_proxy::add_motion_stream(
}


void dds_sensor_proxy::initialization_done( const std::string & product_id, const std::string & product_line )
void dds_sensor_proxy::initialization_done()
{
auto converters = dds_rs_internal_data::get_profile_converters( product_id, product_line );
handle_no_converters( converters );
_formats_converter.register_converters( converters );
register_basic_converters();
_profiles = _formats_converter.get_all_possible_profiles( _raw_rs_profiles );

auto tags = dds_rs_internal_data::get_profiles_tags( product_id, product_line );
tag_profiles( tags );
}


void dds_sensor_proxy::handle_no_converters( std::vector<librealsense::processing_block_factory> & converters )
void dds_sensor_proxy::register_basic_converters()
{
if( converters.empty() )
// Create "identity converter" for each raw profile
for( auto & raw_profile : _raw_rs_profiles )
converters.push_back( processing_block_factory::create_id_pbf( raw_profile->get_format(),
raw_profile->get_stream_type(),
raw_profile->get_stream_index() ) );
}
std::vector< librealsense::processing_block_factory > converters;
std::vector< processing_block_factory > tmp;

// Color
converters.push_back( processing_block_factory::create_id_pbf( RS2_FORMAT_YUYV, RS2_STREAM_COLOR ) );
converters.push_back( processing_block_factory::create_id_pbf( RS2_FORMAT_UYVY, RS2_STREAM_COLOR ) );
converters.push_back( processing_block_factory::create_id_pbf( RS2_FORMAT_RGB8, RS2_STREAM_COLOR ) );
converters.push_back( processing_block_factory::create_id_pbf( RS2_FORMAT_RGBA8, RS2_STREAM_COLOR ) );
converters.push_back( processing_block_factory::create_id_pbf( RS2_FORMAT_BGR8, RS2_STREAM_COLOR ) );
converters.push_back( processing_block_factory::create_id_pbf( RS2_FORMAT_BGRA8, RS2_STREAM_COLOR ) );

converters.push_back( { { { RS2_FORMAT_UYVY } }, { { RS2_FORMAT_RGB8, RS2_STREAM_COLOR } },
[]() { return std::make_shared< uyvy_converter >( RS2_FORMAT_RGB8 ); } } );
converters.push_back( { { { RS2_FORMAT_YUYV } }, { { RS2_FORMAT_RGB8, RS2_STREAM_COLOR } },
[]() { return std::make_shared< yuy2_converter >( RS2_FORMAT_RGB8 ); } } );

// Depth
converters.push_back( processing_block_factory::create_id_pbf( RS2_FORMAT_Z16, RS2_STREAM_DEPTH ) );

// Infrared (converter source needs type to be handled properly by formats_converter)
converters.push_back( { { { RS2_FORMAT_Y8, RS2_STREAM_INFRARED } },
{ { RS2_FORMAT_Y8, RS2_STREAM_INFRARED, 0 },
{ RS2_FORMAT_Y8, RS2_STREAM_INFRARED, 1 },
{ RS2_FORMAT_Y8, RS2_STREAM_INFRARED, 2 } },
[]() { return std::make_shared< identity_processing_block >(); } } );
converters.push_back( { { { RS2_FORMAT_Y16, RS2_STREAM_INFRARED } },
{ { RS2_FORMAT_Y16, RS2_STREAM_INFRARED, 1 },
{ RS2_FORMAT_Y16, RS2_STREAM_INFRARED, 2 } },
[]() { return std::make_shared< identity_processing_block >(); } } );

// Motion
converters.push_back( { { { RS2_FORMAT_MOTION_XYZ32F, RS2_STREAM_ACCEL } },
{ { RS2_FORMAT_MOTION_XYZ32F, RS2_STREAM_ACCEL } },
[]() { return std::make_shared< identity_processing_block >(); } } );
converters.push_back( { { { RS2_FORMAT_MOTION_XYZ32F, RS2_STREAM_GYRO } },
{ { RS2_FORMAT_MOTION_XYZ32F, RS2_STREAM_GYRO } },
[]() { return std::make_shared< identity_processing_block >(); } } );

// Confidence
converters.push_back( { { { RS2_FORMAT_RAW8, RS2_STREAM_CONFIDENCE } },
{ { RS2_FORMAT_RAW8, RS2_STREAM_CONFIDENCE } },
[]() { return std::make_shared< identity_processing_block >(); } } );


void dds_sensor_proxy::tag_profiles( const std::vector<librealsense::tagged_profile> & tags )
{
for( auto & profile : _profiles )
for( auto & tag : tags )
if( ( tag.stream == RS2_STREAM_ANY || tag.stream == profile->get_stream_type() ) &&
( tag.format == RS2_FORMAT_ANY || tag.format == profile->get_format() ) &&
( tag.fps == -1 || tag.fps == profile->get_framerate() ) &&
( tag.stream_index == -1 || tag.stream_index == profile->get_stream_index() ) )
{
if( auto vp = dynamic_cast< video_stream_profile_interface * >( profile.get() ) )
{ // For video the resolution should match as well
if( ( tag.width == -1 || tag.width == vp->get_width() ) &&
( tag.height == -1 || tag.height == vp->get_height() ) )
profile->tag_profile( tag.tag );
}
else
profile->tag_profile( tag.tag );
}
_formats_converter.register_converters( converters );
}


Expand Down Expand Up @@ -466,6 +479,11 @@ void dds_sensor_proxy::stop()
{
auto & dds_stream = _streams[sid_index( profile->get_unique_id(), profile->get_stream_index() )];

dds_stream->stop_streaming();
dds_stream->close();

_streaming_by_name[dds_stream->name()].syncer.on_frame_ready( nullptr );

if( auto dds_video_stream = std::dynamic_pointer_cast< realdds::dds_video_stream >( dds_stream ) )
{
dds_video_stream->on_data_available( nullptr );
Expand All @@ -476,11 +494,6 @@ void dds_sensor_proxy::stop()
}
else
throw std::runtime_error( "Unsupported stream type" );

dds_stream->stop_streaming();
dds_stream->close();

_streaming_by_name[dds_stream->name()].syncer.on_frame_ready( nullptr );
}

// Resets frame source. Nullify streams on_data_available before calling stop.
Expand Down
6 changes: 2 additions & 4 deletions src/dds/rs-dds-sensor-proxy.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ class dds_sensor_proxy : public software_sensor
std::shared_ptr<stream_profile_interface> add_video_stream( rs2_video_stream video_stream, bool is_default ) override;
std::shared_ptr<stream_profile_interface> add_motion_stream( rs2_motion_stream motion_stream, bool is_default ) override;
// Not adding streams or profiles after this
void initialization_done( const std::string & product_id, const std::string & product_line );
void initialization_done();

void open( const stream_profiles & profiles ) override;
void start( frame_callback_ptr callback ) override;
Expand All @@ -84,9 +84,7 @@ class dds_sensor_proxy : public software_sensor
const std::map< sid_index, std::shared_ptr< realdds::dds_stream > > & streams() const { return _streams; }

private:
void handle_no_converters( std::vector<librealsense::processing_block_factory> & converters );

void tag_profiles( const std::vector<librealsense::tagged_profile> & tags );
void register_basic_converters();

std::shared_ptr< realdds::dds_video_stream_profile >
find_profile( sid_index sidx, realdds::dds_video_stream_profile const & profile ) const;
Expand Down
12 changes: 12 additions & 0 deletions src/device.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
#include "core/video.h"
#include "core/motion.h"
#include "device.h"

#include <rsutils/string/from.h>
#include <rsutils/json.h>

using namespace librealsense;

Expand Down Expand Up @@ -428,3 +430,13 @@ void device::stop_activity() const
}
}
}

bool device::should_use_basic_formats() const
{
if( _context )
{
return rsutils::json::get< bool >( _context->get_settings(), std::string("use-basic-formats", 17), false);
}

return false;
}
2 changes: 2 additions & 0 deletions src/device.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@ class device : public virtual device_interface, public info_container

bool device_changed_notifications_on() const { return _device_changed_notifications; }

bool should_use_basic_formats() const;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things: :)

  • Not sure about the name - if we stuck to the existing convention, it would be use_basic_formats_on()?
  • Should we really expose a function from the device? I'd bet the sensor pretty much knows how to get the context? If so, why not query the context directly and no go through the device? It's really not a device function, is it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sensor uses it's device "owner" to access the context, no direct access.
Conceptually we can make this a per sensor decision, but I don't really see a case where it is not an application wide resolution. It is whether we convert at server side or host side and the CPU/bandwidth resources needed.

Copy link
Collaborator

@maloel maloel Jul 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would argue it's not actually per device, though. It's per context.
I think the sensor should query the context.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated sensor to parse this parameter


uint16_t _pid;
protected:
int add_sensor(const std::shared_ptr<sensor_interface>& sensor_base);
Expand Down
Loading