Feature/SK-732 | Introduces FedYogi and FedAdaGrad aggregators #580
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 10 commits
March 25, 2024 15:09
niklastheman
approved these changes
Apr 19, 2024
Wrede
changed the title
Wrede
requested changes
Apr 23, 2024
| @@ -102,36 +114,128 @@ def combine_models(self, helper=None, delete_models=True): | |||
| "AGGREGATOR({}): Error encoutered while processing model update {}, skipping this update.".format(self.name, e)) | |||
| self.model_updates.task_done() | |||
|
|
|||
| model = self.serveropt_adam(helper, pseudo_gradient, model_old) | |||
| if params['serveropt'] == 'adam': | |||
There was a problem hiding this comment.
I would also catch KeyError if something unexpected changes, like default_params
added 12 commits
April 26, 2024 13:18
Wrede
requested changes
Apr 30, 2024
docs/aggregators.rst
Outdated
| see compute package). The base class implements a default callback that checks that all metadata assumed by the aggregation algorithms FedAvg and FedOpt | ||
| is present in the metadata. However, the callback could also be used to implement custom preprocessing and additional checks including strategies | ||
| The ``on_model_update`` callback recieves the model update messages from clients (including all metadata) and can be used to perform validation and | ||
| potential transformation of the model update before it is places on the aggregation queue (see image above). |
docs/aggregators.rst
Outdated
| The ``on_model_update`` callback recieves the model update messages from clients (including all metadata) and can be used to perform validation and | ||
| potential transformation of the model update before it is places on the aggregation queue (see image above). | ||
| The base class implements a default callback that checks that all metadata assumed by the aggregation algorithms FedAvg and FedOpt | ||
| is available. The callback could also be used to implement custom pre-processing and additional checks including strategies |
| :param params: Additional key-word arguments. | ||
| :type params: dict | ||
| :param parameters: Aggregator hyperparameters. | ||
| :type parameters: `fedn.utils.parmeters.Parameters`, optional |
There was a problem hiding this comment.
:class: then reference to the class
| @@ -10,8 +10,12 @@ class Aggregator(AggregatorBase): | |||
|
|
|||
There was a problem hiding this comment.
class name should be somethiing more related to FedOpt? like FedOptAggregator?
There was a problem hiding this comment.
No, this is how the plugin architecture is implemented, it has to be called Aggregator.
| if key not in params: | ||
| params[key] = value | ||
| # Define parameter schema | ||
| parameter_schema = { |
There was a problem hiding this comment.
I would have this a required attribute of AggregatorBase, even if it can be empty
There was a problem hiding this comment.
Let's think about this for a bit longer, I think we should discuss this whole Parameter thing in more depth and make sure we have something that we want to use across the whole code-base.
| elif parameters['serveropt'] == 'yogi': | ||
| model = self.serveropt_yogi(helper, pseudo_gradient, model_old, parameters) | ||
| elif parameters['serveropt'] == 'adagrad': | ||
| model = self.serveropt_adagrad(helper, pseudo_gradient, model_old, parameters) | ||
| else: | ||
| logger.error("Unsupported server optimizer passed to FedOpt.") | ||
| return |
|
I have activated the docs for this branch: https://fedn.readthedocs.io/en/feature-sk-732/aggregators.html |
Wrede
approved these changes
May 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Introduces new features related to model aggregation: