NUS Privacy Meter (Run outside of OpenFL or incorporate a new task)? #111
Replies: 7 comments 8 replies
-
It is an interesting question. I think the objective of the integration is to allow parties to measure their information leakage during the FL. Thus, at a high level, the local parties will use the privacy meter. I think the local parties' dataset should contain two groups of data:
To measure the information leakage, the local parties will partition the training dataset and test dataset into two parts:
Based on this setup, local parties can measure their information leakage in different scenarios.
About the privacy meter API: Please feel free to comment on it if I missed anything or anything is unclear. |
Beta Was this translation helpful? Give feedback.
-
I think one way to proceed with the integration is to add the ability to read the saved data in ML Privacy Meter, and perform the attacks there. As you mentioned we can also trigger the FL training within ML Privacy Meter. But if the FL training is triggered using ML Privacy Meter, would it be done on the aggregator? If this is the case, will we be able to train the attack models locally on the collaborators? Perhaps we can add another |
Beta Was this translation helpful? Give feedback.
-
Taking scenario 1 (from Hongyan's comments above) as the first use case (if this makes sense to all), and taking the idea from our recent meeting that we will not concatenate data over rounds but rather train an attack model for each round of training, let's spell out some specifics to agree on. Basic idea: The Privacy Meter tasks will be run as additional nodes of the federation (one additional node for each collaborator if we want to measure leakage for all collaborators in the federation. In the end the aggregator (and companion database) will track the work to be done (and completed work results) for the collaborator training as well as the privacy meter work. @psfoley @alexey-gruzdev the privacy meter companion task for a given collaborator will require access to local model updates from its collaborator from each round, otherwise as I understand it, it is an independent task as far as info during the FL workflow (it will however use all of the collaborator data and it will use a version (child class?) of the model object - with additional methods including the ability to retrieve intermediate layer per example gradients for example). Each collaborator will have a companion instance of the Privacy meter (another participant of the federation) that will utilize the four data partitions if the corresponding collaborator data that Hongyan specified above (which I'll name): Task 1 for a PM (Privacy Meter) instance Task 2 for a PM (Privacy Meter) instance @psfoley @alexey-gruzdev maybe now you can start to describe at a high level how you image the work would go to write a model object (other code) and task related code to help NUS to see what will be involved (as well as questions for them related to this)? |
Beta Was this translation helpful? Give feedback.
-
Proposed solution to reduce compute time and storage requirements:Here 'privacy meter' is analogous to the 'attack model' we have been discussing before. Since we aren't attacking the federation but evaluating the privacy risk, we can use this term instead to avoid any confusion.
Proposed changes to the ML Privacy Meter API:
|
Beta Was this translation helpful? Give feedback.
-
Hi all, the changes being made on ML Privacy Meter can be tracked here: https://github.com/amad-person/ml_privacy_meter/tree/federated_learning. Using ML Privacy Meter within openfl:
|
Beta Was this translation helpful? Give feedback.
-
Here are the tasks that get done each round:
|
Beta Was this translation helpful? Give feedback.
-
What attacks will we use? Will use black box features (with a stretch of using white box features as well) |
Beta Was this translation helpful? Give feedback.
-
Hi all,
This discussion is intended to address the first question regarding the exploration of the ML Privacy Meter during federated training using OpenFederatedLearning. What will this integration look like at a high level? Below I take attack scenario 1 (from Aadyaa's presentation) as a first case to discuss.
@psfoley suggested that contributions to OpenFL can enable integration of the Privacy Meter with relative ease, through the addition of a task to be performed by each collaborator at each round. As I understand it, at a high level the new task would be the following:
After local training has ended, collect the locally trained model outputs on three groups of data:
Model outputs from groups 1 and 2 (from a single collaborator, collected over all rounds) forms the training set and outputs from group 3 (from the same collaborator, again over all rounds trained) forms the test set for the an attack model [i.e. one attack model for each collaborator running this task]. Training of the attack model does not commence until all FL training is done, and so maybe the modifications to OpenFL will only be needed for this data collection. How the Privacy Meter will trigger this execution during its work flow is still in question for me (I have not looked at that code yet), but my initial guess is that it will trigger FL training, then consume the data mentioned above by reading off of disk upon completion of training?
Does this sound right? Questions? Comments?
Beta Was this translation helpful? Give feedback.
All reactions