Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache Descriptor Set Path #202

Merged
merged 7 commits into from
Jul 30, 2024
Merged

Cache Descriptor Set Path #202

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
32ca9a4
Caching Descriptor Set Location
ifadams Jul 25, 2024
9f2f8bb
Update src/RSCommand.h
ifadams Jul 26, 2024
85f9db2
Automated updates: Format and/or coverage
sys-vdms Jul 26, 2024
e502e34
Automated updates: Format and/or coverage
sys-vdms Jul 26, 2024
643831c
relocating cache map to subclass for clarity
ifadams Jul 29, 2024
0c4fec7
Automated updates: Format and/or coverage
sys-vdms Jul 29, 2024
ec1b3b2
Automated updates: Format and/or coverage
sys-vdms Jul 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/coverage/cpp.develop.coverage_report.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ src/BackendNeo4j.cc 121 0 0% 4,6-17,20,24,2
src/BlobCommand.cc 87 66 75% 76,130-132,136-139,145,147,165,186-189,192-196,202
src/BoundingBoxCommand.cc 180 4 2% 45,49,51,53-54,56-59,62,64-67,70-73,76,83,87,90-91,93-97,101,103,105,114,118,122-123,125-132,137-138,140-144,147-150,152,154-160,162-165,167-169,171-173,176-177,179-181,183-184,186-187,190,193,196-197,199,201-204,206-210,213,215-219,222-223,225-227,229-237,240-244,246,251-256,259-261,263,265-266,268,270,272-274,276-277,281-283,286,288,292-294,296,298,300-303,307-308,310-313,316-319,321-326,329-330,335,338-339,341
src/CommunicationManager.cc 46 0 0% 42-43,46-47,49-50,52-54,57,61-66,68-71,73-81,84,86-88,93,96-97,100-101,105,107-108,110,113-116
src/DescriptorsCommand.cc 594 107 18% 56,61,63-67,69-73,75,77,80-81,84,86-87,90,92-93,96-100,103,106,109,156-158,162,176-180,220-231,241,255-257,261,276-283,285,297-300,305,310-314,330,338,340-345,348-351,354,357-358,360-364,367,370,372-373,376-378,380-381,383-384,387-388,390-392,398,402,404,406,408-409,412,414-417,423-424,427,429-430,432,434-435,438,441,443,445,448,450-455,457-461,463,466,469-470,473-474,485,488,491,493-495,503,507,509,512,514-517,520-525,527-531,533,536,538,540,543,546-547,549,551,553-558,561,563-565,568-569,571,573,575-576,578,580,582-584,586,588-593,598-599,602,604,606,613-614,617,621,623,626,628-631,634-638,640-644,646,648-651,653-655,658,662-671,676-677,680,687,689,692-693,696,700,706,708,711,714-715,719,724,726,728,731,734-735,737-740,744,748-749,751,753-755,757,759-760,762,764-766,768-770,775-777,780-781,783,786-790,794,797,801,803-804,806-807,809,811-812,814-816,818,823,825-826,828-830,832-833,837,839-842,844-847,852,855-857,859,861,863-866,868-872,875-876,879,881,883,885-889,892-893,896-899,901,903-910,915,917,919-920,923-926,928,930,932-939,943-948,954,957,959,961-964,967,969,972-975,977-981,987,989,991-994,999,1001,1003-1004,1007-1009,1011,1013,1015-1018,1021-1022,1024-1025,1027,1029-1030,1032-1038,1042-1043,1047-1048,1050-1051,1059-1060,1063-1064,1066,1068-1075,1079,1083-1084,1088-1092,1097-1098,1100
src/DescriptorsCommand.cc 602 107 17% 56,63-68,73,75-79,81-85,87,89,92-93,96,98-99,102,104-105,108-114,117,120,123,170-172,176,190-194,234-245,255,269-271,275,290-297,299,311-314,319,324-328,344,352,354-359,362-365,368,371-372,374-378,381,384,386-387,390-392,394-395,397-398,401-402,404-406,412,416,418,420,422-423,426,428-431,437-438,441,443-444,446,448-449,452,455,457,459,462,464-469,471-475,477,480,483-484,487-488,499,502,505,507-509,517,521,523,526,528-531,534-539,541-545,547,550,552,554,557,560-561,563,565,567-572,575,577-579,582-583,585,587,589-590,592,594,596-598,600,602-607,612-613,616,618,620,627-628,631,635,637,640,642-645,648-652,654-658,660,662-665,667-669,672,676-685,690-691,694,701,703,706-707,710,714,720,722,725,728-729,733,738,740,742,745,748-749,751-754,758,762-763,765,767-769,771,773-774,776,778-780,782-784,789-791,794-795,797,800-804,808,811,815,817-818,820-821,823,825-826,828-830,832,837,839-840,842-844,846-847,851,853-856,858-861,866,869-871,873,875,877-880,882-886,889-890,893,895,897,899-903,906-907,910-913,915,917-924,929,931,933-934,937-940,942,944,946-953,957-962,968,971,973,975-978,981,983,986-989,991-995,1001,1003,1005-1008,1013,1015,1017-1018,1021-1023,1025,1027,1029-1032,1035-1036,1038-1039,1041,1043-1044,1046-1052,1056-1057,1061-1062,1064-1065,1073-1074,1077-1078,1080,1082-1089,1093,1097-1098,1102-1106,1111-1112,1114
src/DescriptorsManager.cc 24 19 79% 49-50,57-58,73
src/ExceptionsCommand.cc 6 0 0% 35-40
src/ImageCommand.cc 322 157 48% 55,59,63,65,67-69,71,73-76,78,81,86,88-89,97,99,106,109,111-112,114-115,117-118,120-121,124,151,162-163,174-175,177,182-185,195-196,198,203-206,221-229,231-233,246-247,257-267,269-270,272-273,278,286,297,304,308,311,313,315,337,339-340,343-348,350,352,374-376,379-381,385-388,394,396,403-406,420,427,433-436,440-441,452-455,458-463,468-470,481-484,489-493,498-499,501-502,504-508,511,513-517,520-523,526-527,530,532,537
Expand All @@ -25,7 +25,7 @@ src/PMGDQueryHandler.cc 623 517 83% 82-84,166-167,
src/QueryHandlerBase.cc 32 6 18% 26-29,35-36,38,42-43,45,47-48,52,56-58,60-62,64-66,68-69,72-73
src/QueryHandlerExample.cc 33 18 54% 65-67,75-78,84-85,89-92,94-95
src/QueryHandlerNeo4j.cc 139 0 0% 53,55-56,58,60-62,64-65,67,70-76,80-81,83-87,91,93,95-100,104-108,111-115,119-126,129-132,134-136,139-147,149-153,159,162,169,172-175,177-179,181,184-187,189-190,192,194,197,199,201-204,207-208,210,212-213,216,218,222-223,225,229,231-232,235,237-240,243-247,250-254,256-257,261-268,271-274,277
src/QueryHandlerPMGD.cc 344 226 65% 102-104,112-115,130-131,135-139,142-146,150-157,160-163,165-167,176-178,182-184,202-204,209-211,226-232,236-238,255,257-266,295-297,343-345,347-349,352-353,355-358,381-382,384-385,394,400-412,414-416,423-431,468,470,525-526,528-529
src/QueryHandlerPMGD.cc 344 226 65% 108-110,118-121,136-137,141-145,148-152,156-163,166-169,171-173,182-184,188-190,208-210,215-217,232-238,242-244,261,263-272,301-303,349-351,353-355,358-359,361-364,387-388,390-391,400,406-418,420-422,429-437,474,476,531-532,534-535
src/QueryMessage.cc 12 0 0% 37-39,42-43,45-46,48,51-54
src/RSCommand.cc 144 105 72% 65-67,73-74,98,100-101,103,110,131,134-138,141,172-174,176,178-181,188,262,285,287-289,291-297,301
src/SearchExpression.cc 99 38 38% 59,132-133,135,137-139,143,146,148-153,157,160,168-170,177,180-181,183-185,188,192-195,197,201,217-222,224-225,227,235-240,243,247-249,252-256,263,276,284-285
Expand Down Expand Up @@ -57,5 +57,5 @@ utils/src/comm/Exception.cc 6 0 0% 35-40
utils/src/stats/SystemStats.cc 250 249 99% 453
utils/src/timers/TimerMap.cc 82 75 91% 126,151,153,155-158
------------------------------------------------------------------------------
TOTAL 10068 6488 64%
TOTAL 10076 6488 64%
------------------------------------------------------------------------------
2 changes: 1 addition & 1 deletion .github/coverage/cpp.develop.coverage_value.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
64.4418
64.3906
14 changes: 14 additions & 0 deletions src/DescriptorsCommand.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,18 @@ DescriptorsCommand::DescriptorsCommand(const std::string &cmd_name)
std::string DescriptorsCommand::get_set_path(PMGDQuery &query_tx,
const std::string &set_name,
int &dim) {

// Check cache for descriptor set, if its found set dimensions and return
// path,
// otherwise we go forward and query PMGD to locate the descriptor set
auto element = _desc_set_locator.find(set_name);
std::string mapped_path;
if (element != _desc_set_locator.end()) {
mapped_path = element->second;
dim = _desc_set_dims[set_name];
return mapped_path;
}

// Will issue a read-only transaction to check
// if the Set exists
PMGDQuery query(query_tx.get_pmgd_qh());
Expand Down Expand Up @@ -97,6 +109,8 @@ std::string DescriptorsCommand::get_set_path(PMGDQuery &query_tx,
assert(ent.isMember(VDMS_DESC_SET_PATH_PROP));
std::string set_path = ent[VDMS_DESC_SET_PATH_PROP].asString();
dim = ent[VDMS_DESC_SET_DIM_PROP].asInt();
_desc_set_dims[set_name] = dim;
_desc_set_locator[set_name] = set_path;
return set_path;
}

Expand Down
6 changes: 6 additions & 0 deletions src/QueryHandlerPMGD.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,12 @@ using namespace VDMS;

std::unordered_map<std::string, RSCommand *> QueryHandlerPMGD::_rs_cmds;

// Static globals for use in looking up descriptor set locations, defined in
// RSCommand.h
tbb::concurrent_unordered_map<std::string, std::string>
RSCommand::_desc_set_locator;
tbb::concurrent_unordered_map<std::string, int> RSCommand::_desc_set_dims;

void QueryHandlerPMGD::init() {
DescriptorsManager::init();

Expand Down
7 changes: 6 additions & 1 deletion src/RSCommand.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
*
* The MIT License
*
* @copyright Copyright (c) 2017 Intel Corporation
* @copyright Copyright (c) 2024 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"),
Expand All @@ -30,6 +30,7 @@
*/

#pragma once
#include "tbb/concurrent_unordered_map.h"
#include <atomic>
#include <string>
#include <unordered_map>
Expand All @@ -49,6 +50,10 @@ class RSCommand {
const std::string _cmd_name;
std::map<std::string, int> _valid_params_map;

static tbb::concurrent_unordered_map<std::string, std::string>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to place this DescriptorSet specific cache in the base RSCommand class? Won't it be better placed in DescriptorsCommand class?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the time there was a specific reason (actually did the dev work weeks ago) and was having some compilation issues with it in the subclass.

I can't for the life of me remember the specifics though.... if its causing some heartburn I can try moving it back to the subclass. There's some vaguely potential value of having it in the super class for more general access for some hand-wavey other future dev but from a functional standpoint right now, meh

_desc_set_locator;
static tbb::concurrent_unordered_map<std::string, int> _desc_set_dims;

template <typename T>
T get_value(const Json::Value &json, const std::string &key, T def = T());

Expand Down