Develop #418
Merged
Develop #418
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 tasks
ahellander
added a commit
that referenced
this pull request
May 30, 2023
…459) * Add instructions to connect examples to distributed deployment * Update README.md * Update README.md * Update README.md * Formatting and sorting imports in Python files * fix * Fix sorting * inference rest * add inference to validation channel * fixing formatting * formatting * merge option for entripoints * add task type * clean up requirement.txt file/dir * Make default compose a development compose * Add dev mount to pytorch example * Status type for inference * improve entrypoint * fix sorting * Remove license * add stuff to test * add python version to matrix * fix * remove version from name * Increase sleep for reducers and clients * Improve logs * Always print logs * Fix PyTorch example data mount path in compose file Fix PyTorch example data mount path in override compose file. * mets many python versions * quotes * Fix CI sleep time * Add Python versiong * don't fail fast * remove python 3.10 * remove python 3.10 * fix numpy for py 3.7 * Inference CI * minor * fix * fix * fix * fix * fix * fix * fix * reduce CI time * fix conflict * Initial implementation toggle ssl for REST service * Removed unused reducer inference interface mockup * Removed geoip2 dependency * Dockerfile update, install developer tools * Draft implementation * Remove mocked inference endpoint in restservice * Develop (#418) * validate user-specified name (#415) * Delete old Docker-related files (#412) Co-authored-by: Marco Capuccini <[email protected]> * fix code-checks * insecure mode in ci (http) * secure option to package download and checksum * work in progress * fix flake8 warning * Remove Talisman * bugfix, combiner now correctly uses secure flag in connector * Revert accidetal change to compose file * sort import * Changed combiner ssl default config to False * Fixed code checks * Code checks * Add docstings in connecy.py * Add docstings in certificatemanager * Docstrings * Changed some parameter names in reducer CLI * Default no-ssl for REST, ssl for gRPC * Fix code check * Harmoize option names between combiner and reducer * Add help text for combiner options * Make --secure option flag * Works to disable secure grpc * Added back use of copy * Remove possibility to generate cert for reducer * Default to insecure gRPC setting * Fix code scanning alerts * Initial refactor * Initial refactor reducer * Introduce base class for controller * More refactoring and cleaning * refactored look-aside loadbalancer * Refactored load-balancer * Fixed code checks * latest * work in progress * Fixed code checks * Update control page * added metadata field to modelupdaterequest * Client passes on metadata dict with model update * Latest * Latest * latest * Refactor aggregation * Fix * Add docstring for load_model_update * Extract model update metadata and make available in aggregator * Added some docstrings * More docstrings * Renamed aggregator files and base class * suppress LOG status messages in stdout * Introduce policy for when to trigger aggregation at combiner * Latest * Added files * Fixes * Fixed broken congig file generation. * Added option to parse client name from config file * Flattened client config file, generalized so that all settings can be passed in the file * Fixed file generation * Latest * Updated config template * Removed mongotracing in control, will refactor to have all tracing data in one collection * Refactored combiner job submit * Remove psutil tracing * Refactor tracer * cleaning * get latest round refactored * Enable early termination by default * Removed unused round_config object * Remove printout of sensitive information * Remove old control, make new version default * Remove unused code * Changed default name for fedn network in config template * Cleaning, docstrings * bugfix * Variable name changes * Removed old combine models implementation * bugfix * Add a hook to validate the model update before putting it on the aggregation queue * Validate metadata on model 'update * Validate metadata on model 'update * incremental weighted average in new style aggregator * small cleaning in control form * Added instructions in controller form, rearranged menu items * latest * Resolve merge conflicts * Added back accidentally removed file * Conflict resolution * Remove unused readme file * More merging * latest * Fixed round_config regression * Controller polls db instead of combiners * More api docs * Add infer_instruct * Cleaning * Added training metadata for keras example * work in progress db cleanup * Refactor * More refactoring in db backend * Remove 'control' setting from reducer config file * Flatten combiner config * Flatten combiner config * Flatten combiner config * Harmonize CLI option names * Refactor helpers * Refactor helpers * Refactor helpers * Refactor helpers * Refactor helpers * Plugin arch for helpers * Updated UI config * Raise exception if misconfigured helper * Added tracing of sessions in the db * Update version to 0.5-dev * Updated torch version * Updated torch version * bugfix * Skip osx tests * latest * change helper name * fix formatting and syntax * fix formatting and syntax errors * update ci new db * fix round_id key and equal weight to reduce models * save helper for metrics and metadata * improve readability and add test for fedavg * update doc strings for client and combiner * Resolve conflict * formatting * add id to logging * extra logging and doc strings --------- Co-authored-by: mcapuccini <[email protected]> Co-authored-by: Andreas Hellander <[email protected]> Co-authored-by: Fredrik Wrede <[email protected]>
ahellander
added a commit
that referenced
this pull request
Jan 29, 2024
* Increase sleep for reducers and clients * Improve logs * Always print logs * Fix PyTorch example data mount path in compose file Fix PyTorch example data mount path in override compose file. * mets many python versions * quotes * Fix CI sleep time * Add Python versiong * don't fail fast * remove python 3.10 * remove python 3.10 * fix numpy for py 3.7 * Inference CI * minor * fix * fix * fix * fix * fix * fix * fix * reduce CI time * fix conflict * Initial implementation toggle ssl for REST service * Removed unused reducer inference interface mockup * Removed geoip2 dependency * Dockerfile update, install developer tools * Draft implementation * Remove mocked inference endpoint in restservice * Develop (#418) * validate user-specified name (#415) * Delete old Docker-related files (#412) Co-authored-by: Marco Capuccini <[email protected]> * fix code-checks * insecure mode in ci (http) * secure option to package download and checksum * work in progress * fix flake8 warning * Remove Talisman * bugfix, combiner now correctly uses secure flag in connector * Revert accidetal change to compose file * sort import * Changed combiner ssl default config to False * Fixed code checks * Code checks * Add docstings in connecy.py * Add docstings in certificatemanager * Docstrings * Changed some parameter names in reducer CLI * Default no-ssl for REST, ssl for gRPC * Fix code check * Harmoize option names between combiner and reducer * Add help text for combiner options * Make --secure option flag * Works to disable secure grpc * Added back use of copy * Remove possibility to generate cert for reducer * Default to insecure gRPC setting * Fix code scanning alerts * Initial refactor * Initial refactor reducer * Introduce base class for controller * More refactoring and cleaning * refactored look-aside loadbalancer * Refactored load-balancer * Fixed code checks * latest * work in progress * Fixed code checks * Update control page * added metadata field to modelupdaterequest * Client passes on metadata dict with model update * Latest * Latest * latest * Refactor aggregation * Fix * Add docstring for load_model_update * Extract model update metadata and make available in aggregator * Added some docstrings * More docstrings * Renamed aggregator files and base class * suppress LOG status messages in stdout * Introduce policy for when to trigger aggregation at combiner * Latest * Added files * Fixes * Fixed broken congig file generation. * Added option to parse client name from config file * Flattened client config file, generalized so that all settings can be passed in the file * Fixed file generation * Latest * Updated config template * Removed mongotracing in control, will refactor to have all tracing data in one collection * Refactored combiner job submit * Remove psutil tracing * Refactor tracer * cleaning * get latest round refactored * Enable early termination by default * Removed unused round_config object * Remove printout of sensitive information * Remove old control, make new version default * Remove unused code * Changed default name for fedn network in config template * Cleaning, docstrings * bugfix * Variable name changes * Removed old combine models implementation * bugfix * Add a hook to validate the model update before putting it on the aggregation queue * Validate metadata on model 'update * Validate metadata on model 'update * incremental weighted average in new style aggregator * small cleaning in control form * Added instructions in controller form, rearranged menu items * latest * Resolve merge conflicts * Added back accidentally removed file * Conflict resolution * Remove unused readme file * More merging * latest * Fixed round_config regression * Controller polls db instead of combiners * More api docs * Add infer_instruct * Cleaning * Added training metadata for keras example * work in progress db cleanup * Refactor * More refactoring in db backend * Remove 'control' setting from reducer config file * Flatten combiner config * Flatten combiner config * Flatten combiner config * Harmonize CLI option names * Refactor helpers * Refactor helpers * Refactor helpers * Refactor helpers * Refactor helpers * Plugin arch for helpers * Updated UI config * Raise exception if misconfigured helper * Added tracing of sessions in the db * Update version to 0.5-dev * Updated torch version * Updated torch version * bugfix * Skip osx tests * latest * change helper name * fix formatting and syntax * fix formatting and syntax errors * update ci new db * fix round_id key and equal weight to reduce models * save helper for metrics and metadata * improve readability and add test for fedavg * update doc strings for client and combiner * Resolve conflict * formatting * add id to logging * extra logging and doc strings * work in progress * Refactor of controller * Refactor of controller * Refactor polling in control * Refactor polling in control * Refactor polling in control * Functioning * start on new simulation example * update * Updated test * Fix typos * Removed accidentally committed files * update api * added new async-simulation example * rename example * latest * Updates after code review * Resolved merge conflicts * Updated docstrings * Fixed docstrings * Fixes * Fixed code check * use setter * latest * removed script for combiners * Fix numpyarrayhelper * work in progress * Use latest mongodb and bump version number * Fixed bug in client * Client sends model only once, combiner deletes staged model after training round * Cleaned up new example/test * Change naming of temp storage class member in modelservice, for clarity * Make detach() public * Renamed some methods in client for clarity * refactored set_model to avoide code duplication on client * Refactored modelservice for code reuse * Fix dashboard package upload * Fix default helper in session * Delete combiner level model from minio after reduce * delete combiner models from minio by default * code checks * changes following review --------- Co-authored-by: mcapuccini <[email protected]> Co-authored-by: Andreas Hellander <[email protected]> Co-authored-by: Fredrik Wrede <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description