Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1987: LB: Rename LB Spec to LB Config #1989

Merged
merged 2 commits into from
Oct 12, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/md/lb-manager.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,19 @@ instrumentation of work and communication performed by collection elements.
To run a load balancer at runtime:

- Pass `--vt_lb --vt_lb_name=<LB>` as a command line argument
- Write a LB specification file `--vt_lb --vt_lb_file_name=<FILE>`
- Write a LB config file `--vt_lb --vt_lb_file_name=<FILE>`
- One can also pass `--vt_lb_self_migration` as a command line argument to allow load balancer to migrate objects to the same node

Note that one should use either `--vt_lb_name` or `--vt_lb_file_name` option, not both.

\section lb-specification-file LB Specification File
\section lb-config-file LB Config File

The LB specification file allows users to specify which load balancer along with
The LB config file allows users to specify which load balancer along with
which LB-specific configuration parameters are passed to the load balancer
instance for any given phase. The order of the LB phase specification lines in
the file disambiguates lines---higher precedence for earlier lines.

The format of the LB specification file is:
The format of the LB config file is:

\code
[%] <$phase> <$lbname> [$LB-specific-arg-1] ... [$LB-specific-arg-N]
Expand All @@ -42,7 +42,7 @@ balancer are allowed to customize how the load balancer is run with the format
of `key=value`. These arguments are the equivalent of passing
`--vt_lb_args="A=test B=test2"` on the command line.

The following is an example LB specification:
The following is an example LB config:

\code
%10 TemperedLB c=1 k=5 f=2 i=10
Expand All @@ -51,7 +51,7 @@ The following is an example LB specification:
120 GreedyLB c=0 k=2 f=3 i=3
\endcode

To print LB specification during startup, use `--vt_lb_show_spec` command line flag.
To print LB config during startup, use `--vt_lb_show_config` command line flag.

\section load-balancers Load balancers

Expand Down
2 changes: 1 addition & 1 deletion src/vt/collective/collective_ops.cc
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ void printOverwrittens(
printIfOverwritten(vt_trace_event_polling);
printIfOverwritten(vt_trace_irecv_polling);
printIfOverwritten(vt_lb);
printIfOverwritten(vt_lb_show_spec);
printIfOverwritten(vt_lb_show_config);
printIfOverwritten(vt_lb_quiet);
printIfOverwritten(vt_lb_file_name);
printIfOverwritten(vt_lb_name);
Expand Down
4 changes: 2 additions & 2 deletions src/vt/configs/arguments/app_config.h
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ struct AppConfig {
bool vt_trace_irecv_polling = false;

bool vt_lb = false;
bool vt_lb_show_spec = false;
bool vt_lb_show_config = false;
bool vt_lb_quiet = false;
std::string vt_lb_file_name = "";
std::string vt_lb_name = "NoLB";
Expand Down Expand Up @@ -309,7 +309,7 @@ struct AppConfig {
| vt_trace_irecv_polling

| vt_lb
| vt_lb_show_spec
| vt_lb_show_config
| vt_lb_quiet
| vt_lb_file_name
| vt_lb_name
Expand Down
12 changes: 6 additions & 6 deletions src/vt/configs/arguments/args.cc
Original file line number Diff line number Diff line change
Expand Up @@ -459,11 +459,11 @@ void addLbArgs(CLI::App& app, AppConfig& appConfig) {
/*
* Flags for enabling load balancing and configuring it
*/
auto lb = "Enable load balancing";
auto lb_args = "Arguments pass to LB: \"x=0 y=1\"; try --vt_help_lb_args";
auto lb_quiet = "Silence load balancing output";
auto lb_file_name = "LB specification file to read";
auto lb_show_spec = "Show LB specification during startup";
auto lb = "Enable load balancing";
auto lb_args = "Arguments pass to LB: \"x=0 y=1\"; try --vt_help_lb_args";
auto lb_quiet = "Silence load balancing output";
auto lb_file_name = "LB config file to read";
auto lb_show_config = "Show LB config during startup";
auto lb_name = "Name of the load balancer to use";
auto lb_interval = "Load balancing interval";
auto lb_keep_last_elm = "Do not migrate last element in collection";
Expand All @@ -483,7 +483,7 @@ void addLbArgs(CLI::App& app, AppConfig& appConfig) {
auto s = app.add_flag("--vt_lb", appConfig.vt_lb, lb);
auto t1 = app.add_flag("--vt_lb_quiet", appConfig.vt_lb_quiet, lb_quiet);
auto u = app.add_option("--vt_lb_file_name", appConfig.vt_lb_file_name, lb_file_name)->capture_default_str()->check(CLI::ExistingFile);
auto u1 = app.add_flag("--vt_lb_show_spec", appConfig.vt_lb_show_spec, lb_show_spec);
auto u1 = app.add_flag("--vt_lb_show_config", appConfig.vt_lb_show_config, lb_show_config);
auto v = app.add_option("--vt_lb_name", appConfig.vt_lb_name, lb_name)->capture_default_str();
auto v1 = app.add_option("--vt_lb_args", appConfig.vt_lb_args, lb_args)->capture_default_str();
auto w = app.add_option("--vt_lb_interval", appConfig.vt_lb_interval, lb_interval)->capture_default_str();
Expand Down
6 changes: 3 additions & 3 deletions src/vt/runtime/runtime_banner.cc
Original file line number Diff line number Diff line change
Expand Up @@ -306,10 +306,10 @@ void Runtime::printStartupBanner() {
auto f11 = opt_on("--vt_lb_file_name", f12);
fmt::print("{}\t{}{}", vt_pre, f11, reset);

if (getAppConfig()->vt_lb_show_spec) {
auto s = opt_on("--vt_lb_show_spec", "Showing LB specification");
if (getAppConfig()->vt_lb_show_config) {
auto s = opt_on("--vt_lb_show_config", "Showing LB specification");
fmt::print("{}\t{}", vt_pre, s);
fmt::print(vrt::collection::balance::ReadLBSpec::toString());
fmt::print(vrt::collection::balance::ReadLBConfig::toString());
}
} else {
auto a3 = fmt::format("Load balancer name: \"{}\"", getAppConfig()->vt_lb_name);
Expand Down
18 changes: 9 additions & 9 deletions src/vt/vrt/collection/balance/baselb/baselb.cc
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ std::shared_ptr<const balance::Reassignment> BaseLB::startLB(

runInEpochCollective("BaseLB::startLB -> runLB", [this,total_load]{
getArgs(phase_);
inputParams(spec_entry_.get());
inputParams(config_entry_.get());
runLB(total_load);
});

Expand Down Expand Up @@ -106,18 +106,18 @@ void BaseLB::importProcessorData(
void BaseLB::getArgs(PhaseType phase) {
using namespace balance;

bool has_spec = ReadLBSpec::openSpec(theConfig()->vt_lb_file_name);
if (has_spec) {
auto spec = ReadLBSpec::entry(phase);
if (spec) {
spec_entry_ = std::make_unique<SpecEntry>(*spec);
bool has_config = ReadLBConfig::openConfig(theConfig()->vt_lb_file_name);
if (has_config) {
auto config = ReadLBConfig::entry(phase);
if (config) {
config_entry_ = std::make_unique<ConfigEntry>(*config);
} else {
vtAssert(false, "Error no spec found, which must exist");
vtAssert(false, "Error no config found, which must exist");
}
} else {
auto const args = theConfig()->vt_lb_args;
spec_entry_ = std::make_unique<SpecEntry>(
ReadLBSpec::makeSpecFromParams(args)
config_entry_ = std::make_unique<ConfigEntry>(
ReadLBConfig::makeConfigFromParams(args)
);
}
}
Expand Down
16 changes: 8 additions & 8 deletions src/vt/vrt/collection/balance/baselb/baselb.h
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ struct BaseLB {
void transferMigrations(TransferMsg<TransferVecType>* msg);
void finalize(CountMsg* msg);

virtual void inputParams(balance::SpecEntry* spec) = 0;
virtual void inputParams(balance::ConfigEntry* config) = 0;
virtual void runLB(TimeType total_load) = 0;

StatisticMapType const* getStats() const {
Expand All @@ -152,14 +152,14 @@ struct BaseLB {
void getArgs(PhaseType phase);

protected:
double start_time_ = 0.0f;
ElementCommType const* comm_data = nullptr;
objgroup::proxy::Proxy<BaseLB> proxy_ = {};
PhaseType phase_ = 0;
std::unique_ptr<balance::SpecEntry> spec_entry_ = nullptr;
double start_time_ = 0.0f;
ElementCommType const* comm_data = nullptr;
objgroup::proxy::Proxy<BaseLB> proxy_ = {};
PhaseType phase_ = 0;
std::unique_ptr<balance::ConfigEntry> config_entry_ = nullptr;
// Observer only - LBManager owns the instance
balance::LoadModel* load_model_ = nullptr;
bool comm_aware_ = false;
balance::LoadModel* load_model_ = nullptr;
bool comm_aware_ = false;

protected:
/**
Expand Down
12 changes: 6 additions & 6 deletions src/vt/vrt/collection/balance/greedylb/greedylb.cc
Original file line number Diff line number Diff line change
Expand Up @@ -121,17 +121,17 @@ Default: scatter
return keys_help;
}

void GreedyLB::inputParams(balance::SpecEntry* spec) {
void GreedyLB::inputParams(balance::ConfigEntry* config) {
auto keys_help = getInputKeysWithHelp();

std::vector<std::string> allowed;
for (auto&& elm : keys_help) {
allowed.push_back(elm.first);
}
spec->checkAllowedKeys(allowed);
min_threshold = spec->getOrDefault<double>("min", greedy_threshold_p);
max_threshold = spec->getOrDefault<double>("max", greedy_max_threshold_p);
auto_threshold = spec->getOrDefault<bool>("auto", greedy_auto_threshold_p);
config->checkAllowedKeys(allowed);
min_threshold = config->getOrDefault<double>("min", greedy_threshold_p);
max_threshold = config->getOrDefault<double>("max", greedy_max_threshold_p);
auto_threshold = config->getOrDefault<bool>("auto", greedy_auto_threshold_p);

balance::LBArgsEnumConverter<DataDistStrategy> strategy_converter_(
"strategy", "DataDistStrategy", {
Expand All @@ -140,7 +140,7 @@ void GreedyLB::inputParams(balance::SpecEntry* spec) {
{DataDistStrategy::bcast, "bcast"}
}
);
strat_ = strategy_converter_.getFromSpec(spec, strat_);
strat_ = strategy_converter_.getFromConfig(config, strat_);
}

void GreedyLB::runLB(TimeType total_load) {
Expand Down
2 changes: 1 addition & 1 deletion src/vt/vrt/collection/balance/greedylb/greedylb.h
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ struct GreedyLB : LoadSamplerBaseLB {

void init(objgroup::proxy::Proxy<GreedyLB> in_proxy);
void runLB(TimeType total_load) override;
void inputParams(balance::SpecEntry* spec) override;
void inputParams(balance::ConfigEntry* config) override;

static std::unordered_map<std::string, std::string> getInputKeysWithHelp();

Expand Down
12 changes: 6 additions & 6 deletions src/vt/vrt/collection/balance/hierarchicallb/hierlb.cc
Original file line number Diff line number Diff line change
Expand Up @@ -125,19 +125,19 @@ Default: LoadOverLessThan
return keys_help;
}

void HierarchicalLB::inputParams(balance::SpecEntry* spec) {
void HierarchicalLB::inputParams(balance::ConfigEntry* config) {
auto keys_help = getInputKeysWithHelp();

std::vector<std::string> allowed;
for (auto&& elm : keys_help) {
allowed.push_back(elm.first);
}
spec->checkAllowedKeys(allowed);
min_threshold = spec->getOrDefault<double>("min", hierlb_threshold_p);
max_threshold = spec->getOrDefault<double>("max", hierlb_max_threshold_p);
auto_threshold = spec->getOrDefault<bool>("auto", hierlb_auto_threshold_p);
config->checkAllowedKeys(allowed);
min_threshold = config->getOrDefault<double>("min", hierlb_threshold_p);
max_threshold = config->getOrDefault<double>("max", hierlb_max_threshold_p);
auto_threshold = config->getOrDefault<bool>("auto", hierlb_auto_threshold_p);

std::string extract = spec->getOrDefault<std::string>(
std::string extract = config->getOrDefault<std::string>(
"strategy", "LoadOverLessThan"
);
if (extract.compare("LoadOverLessThan") == 0) {
Expand Down
2 changes: 1 addition & 1 deletion src/vt/vrt/collection/balance/hierarchicallb/hierlb.h
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ struct HierarchicalLB : LoadSamplerBaseLB {

void init(objgroup::proxy::Proxy<HierarchicalLB> in_proxy);
void runLB(TimeType total_load) override;
void inputParams(balance::SpecEntry* spec) override;
void inputParams(balance::ConfigEntry* config) override;

static std::unordered_map<std::string, std::string> getInputKeysWithHelp();

Expand Down
16 changes: 8 additions & 8 deletions src/vt/vrt/collection/balance/lb_args_enum_converter.h
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ namespace vt { namespace vrt { namespace collection { namespace balance {
*
* \brief A VT component that converts enumerated values to their
* stringifications for the purposes of reading LB arguments from the LB
* spec file.
* config file.
*/
template <typename T>
struct LBArgsEnumConverter {
Expand All @@ -67,7 +67,7 @@ struct LBArgsEnumConverter {
/**
* \brief Construct a converter for a specific enumerated type
*
* \param[in] par_name name of the LB arg in the spec file
* \param[in] par_name name of the LB arg in the config file
* \param[in] enum_type name of the enumerated type being converted
* \param[in] enum_to_str unordered map of enumerated values to their strings
*/
Expand Down Expand Up @@ -129,22 +129,22 @@ struct LBArgsEnumConverter {
}

/**
* \brief Read string from the spec and convert to enum
* \brief Read string from the config and convert to enum
*
* Read the string from the spec entry and convert it to an enumerated value
* Read the string from the config entry and convert it to an enumerated value
* using the reverse of the mapping provided at construction.
*/
T getFromSpec(balance::SpecEntry* spec, T def_value) const {
std::string spec_value = spec->getOrDefault<std::string>(
T getFromConfig(balance::ConfigEntry* config, T def_value) const {
std::string config_value = config->getOrDefault<std::string>(
par_name_, getString(def_value)
);
return getEnum(spec_value);
return getEnum(config_value);
}

private:
EnumToStrMap enum_to_str_; //< 1 to 1 mapping of enumerated values to strings
StrToEnumMap str_to_enum_; //< 1 to 1 mapping of strings to enumerated values
std::string par_name_; //< parameter to look for in the spec file
std::string par_name_; //< parameter to look for in the config file
std::string enum_type_; //< name of the enumerated type for error handling
};

Expand Down
4 changes: 2 additions & 2 deletions src/vt/vrt/collection/balance/lb_invoke/lb_manager.cc
Original file line number Diff line number Diff line change
Expand Up @@ -119,9 +119,9 @@ LBType LBManager::decideLBToRun(PhaseType phase, bool try_file) {

auto& spec_file = theConfig()->vt_lb_file_name;
if (spec_file != "" and try_file) {
bool const has_spec = ReadLBSpec::openSpec(spec_file);
bool const has_spec = ReadLBConfig::openConfig(spec_file);
if (has_spec) {
the_lb = ReadLBSpec::getLB(phase);
the_lb = ReadLBConfig::getLB(phase);
}
} else {
auto interval = theConfig()->vt_lb_interval;
Expand Down
2 changes: 1 addition & 1 deletion src/vt/vrt/collection/balance/offlinelb/offlinelb.h
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ struct OfflineLB : BaseLB {

void init(objgroup::proxy::Proxy<OfflineLB> in_proxy);
void runLB(TimeType) override;
void inputParams(balance::SpecEntry* spec) override { }
void inputParams(balance::ConfigEntry* config) override { }

static std::unordered_map<std::string, std::string> getInputKeysWithHelp() {
return std::unordered_map<std::string, std::string>{};
Expand Down
8 changes: 4 additions & 4 deletions src/vt/vrt/collection/balance/randomlb/randomlb.cc
Original file line number Diff line number Diff line change
Expand Up @@ -83,16 +83,16 @@ Default: 123456789
return keys_help;
}

void RandomLB::inputParams(balance::SpecEntry* spec) {
void RandomLB::inputParams(balance::ConfigEntry* config) {
auto keys_help = getInputKeysWithHelp();

std::vector<std::string> allowed;
for (auto&& elm : keys_help) {
allowed.push_back(elm.first);
}
spec->checkAllowedKeys(allowed);
seed_ = spec->getOrDefault<int32_t>("seed", seed_);
randomize_seed_ = spec->getOrDefault<bool>("randomize_seed", randomize_seed_);
config->checkAllowedKeys(allowed);
seed_ = config->getOrDefault<int32_t>("seed", seed_);
randomize_seed_ = config->getOrDefault<bool>("randomize_seed", randomize_seed_);
}

void RandomLB::runLB(TimeType) {
Expand Down
2 changes: 1 addition & 1 deletion src/vt/vrt/collection/balance/randomlb/randomlb.h
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ struct RandomLB : BaseLB {

void init(objgroup::proxy::Proxy<RandomLB> in_proxy);
void runLB(TimeType) override;
void inputParams(balance::SpecEntry* spec) override;
void inputParams(balance::ConfigEntry* config) override;

static std::unordered_map<std::string, std::string> getInputKeysWithHelp();

Expand Down
Loading