diff --git a/README.md b/README.md index c2dcf0f..32b9ba1 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,27 @@ -# autodp: Automating differential privacy computation +# autodp: Automating differential privacy computation -### Highlights: +### Highlights + +* Advanced DP techniques (e.g., Renyi DP, Moments Accountant, f-DP) working behind the scene. +* Easily customizable. Bring your own mechanism in any way you want. +* Strong composition over heterogeneous mechanisms. + +### All new autodp "Mechanism" API +The three main classes of the autodp 'mechanism' API. + +### New features that come with the new API + +1. Object oriented design: see check out ```autodp_core.py``` +2. Zoos are open with many private animals: ```mechanism_zoo```, ```transformer_zoo```, ```calibrator_zoo```. +3. Added support for f-DP and privacy profile alongside RDP. +4. Stronger RDP to (eps,delta)-DP conversion. +5. Privacy amplification by X. +6. Exactly tight privacy accounting for Gaussian mechanisms and their compositions. +7. Interpretable privacy guarantee via Hypothesis testing interpretation for any Mechanism. + + +The new API makes it extremely easy to obtain state-of-the-art privacy guarantees for your favorite randomized mechanisms, with just a few lines of codes. -1. An RDP (Renyi Differential Privacy) based analytical Moment Accountant implementation that is numerically stable. -2. Supports privacy amplification for generic RDP algorithm for subsampling without replacement and poisson sampling. -3. Stronger composition than the optimal composition using only (ε,δ)-DP. -4. A privacy calibrator that numerically calibrates noise to privacy requirements using RDP. -5. Bring Your Own Mechanism: Just implement the RDP of your own DP algorithm as a function. ### How to use? @@ -14,38 +29,60 @@ It's easy. Just run: ``` pip install autodp ``` -Then follow the Jupyter notebooks in the `tutorials` folder to get started. +or + +``` +pip3 install autodp +``` +Check out the Jupyter notebooks in the `tutorials` folder to get started. #### Notes: * ```pip``` should automatically install all the dependences for you. -* Currently we support only Python3. -* You might need to run ```pip3 install autodp --upgrade``` +* Currently we support only Python3. +* You might need to run ```pip install autodp --upgrade``` + + +### To use the current version at the master branch + +Install it locally by: +``` +pip install -e . +``` ### Research Papers: * Yu-Xiang Wang, Borja Balle, and Shiva Kasiviswanathan. (2019) ["Subsampled Renyi Differential Privacy and Analytical Moments Accountant."](https://arxiv.org/abs/1808.00087). in AISTATS-2019 (**Notable Paper Award**). * Yuqing Zhu, Yu-Xiang Wang. (2019) ["Poisson Subsampled Renyi Differential Privacy"](http://proceedings.mlr.press/v97/zhu19c.html). ICML-2019. + * Yuqing Zhu, Yu-Xiang Wang. (2020) ["Improving Sparse Vector Technique with Renyi Differential Privacy"](https://papers.nips.cc/paper/2020/hash/e9bf14a419d77534105016f5ec122d62-Abstract.html). in NeurIPS-2020. -### Examples： +### How to Contribute? -Composing Subsampled Gaussian Mechanisms (high noise)Composing Subsampled Gaussian Mechanisms (low noise) +Follow the standard practice. Fork the repo, create a branch, develop the edit and send a pull request. One of the maintainers are going to review the code and merge the PR. Alternatively, please feel free to creat issues to report bugs, provide comments and suggest new features. -**Figure 1**: Composing subsampled Gaussian Mechanisms. *Left*: High noise setting with σ=5, γ=0.001, δ=1e-8. *Right*: Low noise setting with σ=0.5, γ=0.001, δ=1e-8. +At the moment, contributions to examples, tutorials, as well as the RDP of currently unsupported mechanisms are most welcome (add them to ```RDP_bank.py```)! +Also, you may add new mechanisms to ``mechanism_zoo.py``. Contributions to ``transformer_zoo.py`` and ``calibrator_zoo.py`` are trickier, please email us! +Please explain clearly what the contribution is about in the PR and attach/cite papers whenever appropriate. -Composing Subsampled Laplace Mechanisms (high noise)Composing Subsampled Laplace Mechanisms (low noise) -**Figure 2**: Composing subsampled Laplace Mechanisms. *Left*: High noise setting with b=2, γ=0.001, δ=1e-8. *Right*: Low noise setting with b=0.5, γ=0.001, δ=1e-8. +### Legacy: the moments accountant API from autodp v.0.11 is still supported: +1. An RDP (Renyi Differential Privacy) based analytical Moment Accountant implementation that is numerically stable. +2. Supports privacy amplification for generic RDP algorithm for subsampling without replacement and poisson sampling. +3. Stronger composition than the optimal composition using only (ε,δ)-DP. +4. A privacy calibrator that numerically calibrates noise to privacy requirements using RDP. +5. Bring Your Own Mechanism: Just implement the RDP of your own DP algorithm as a function. -### How to Contribute? -Follow the standard practice. Fork the repo, create a branch, develop the edit and send a pull request. One of the maintainers are going to review the code and merge the PR. Alternatively, please feel free to creat issues to report bugs, provide comments and suggest new features. -At the moment, contributions to examples, tutorials, as well as the RDP of currently unsupported mechanisms are most welcome (add them to ```RDP_bank.py```)! Please explain clearly what the contribution is about in the PR and attach/cite papers whenever appropriate. +### Examples： +Composing Subsampled Gaussian Mechanisms (high noise)Composing Subsampled Gaussian Mechanisms (low noise) +**Figure 1**: Composing subsampled Gaussian Mechanisms. *Left*: High noise setting with σ=5, γ=0.001, δ=1e-8. *Right*: Low noise setting with σ=0.5, γ=0.001, δ=1e-8. +Composing Subsampled Laplace Mechanisms (high noise)Composing Subsampled Laplace Mechanisms (low noise) +**Figure 2**: Composing subsampled Laplace Mechanisms. *Left*: High noise setting with b=2, γ=0.001, δ=1e-8. *Right*: Low noise setting with b=0.5, γ=0.001, δ=1e-8. diff --git a/autodp/autodp_core.py b/autodp/autodp_core.py new file mode 100644 index 0000000..c25cd52 --- /dev/null +++ b/autodp/autodp_core.py @@ -0,0 +1,304 @@ +""" +Core classes for autodp: + +Mechanism --- A `mechanism' describes a randomized algorithm and its privacy properties. + All `mechanism's (e.g., those in the `mechanism_zoo' module) inherit this class. + +Transformer --- A transformer takes one or a list of mechanism and outputs another mechanism + All `transformer's (e.g., those in the `transformer_zoo' module, e.g., amplificaiton + by sampling, shuffling, and composition) inherit this class. + +Calibrator --- A `calibrator' takes a mechanism with parameters (e.g. noise level) and automatically + choose those parameters to achieve a prespecified privacy budget. + All `calibrator's (e.g., the Analytical Gaussian Mechanism calibration, and others + in `calibrator_zoo'inherit this class) + +""" + +import numpy as np +from autodp import converter + + +class Mechanism(): + """ + The base mechanism will use typically two functions to describe the mechanism + + # Attributes (actually functions as well): + # 1: Approximate DP: epsilon as a function of delta + # 2. Renyi DP: RDP epsilon as a function of \alpha + # 3. Approximate RDP: approximate RDP. RDP conditioning on a failure probability delta0. + # 4. f-DP: Type II error as a function of Type I error. You can get that from Approximate-DP + # or FDP directly. + # 5. epsilon: Pure DP bound. If not infinity, then the mechanism satisfies pure DP. + # 6. delta0: Failure probability which documents the delta to use for approximate RDP + # in the case when there are no information available about the failure event. + # 7. local_flag: Indicates whether the guarantees are intended to be for local differential privacy + # 8. group_size: Integer measuring the granuality of DP. Default is 1. + # 9. replace_one: Flag indicating whether this is for add-remove definition of DP + # or replace-one version of DP, default is False + + # CDP and approximate-CDP will be subsumed in RDP bounds + # + # If we specify RDP only then it will propagate the RDP calculations to approximate-DP + # and to f-DP + # If we specify pure-DP only then it propagates to RDP, Approximate-DP, f-DP and so on. + # If we specify approximate-DP only, then it implies an approximate RDP bound with \delta_0. + # If we specify f-DP only then it propagates to other specifications. + # If we specify multiple calculations, then it will take the minimum of all of them + # in each category + """ + + + def __init__(self): + # Initialize everything with trivial (non-private) defaults + def RenyiDP(alpha): + return np.inf + + def approxRDP(delta, alpha): + return np.inf + + def approxDP(delta): + return np.inf + + def fDP(fpr): + fnr = 0.0 + return fnr + + self.RenyiDP = RenyiDP + self.approxRDP = approxRDP + self.approxDP = approxDP + self.fDP = fDP + + self.eps_pureDP = np.inf # equivalent to RenyiDP(np.inf) and approxDP(0). + + self.delta0 = np.inf # indicate the smallest allowable \delta0 in approxRDP that is not inf + + self.group_size = 1 # transformation that increases group size. + self.replace_one = False # + self.local_flag = False # for the purpose of implementating local DP. + # We can convert localDP to curator DP by parallel composition and by shuffling. + self.updated = False # if updated, then when getting eps, we know that directly getting + # approxDP is the tightest possible. + + def get_approxDP(self, delta): + # Output eps as a function of delta + return self.approxDP(delta) + + def get_approxRDP(self, delta, alpha): + # Output eps as a function of delta and alpha + return self.approxRDP(delta, alpha) + + def get_RDP(self, alpha): + # Output RDP eps as a function of alpha + return self.RenyiDP(alpha) + + def get_fDP(self, fpr): + # Output false negative rate as a function of false positive rate + return self.fDP(fpr) + + def get_pureDP(self): + return self.eps_pureDP + + def get_eps(self, delta): + # Get the smallest eps fo multiple calculations + eps = [self.get_pureDP(), self.get_approxDP(delta)] + # add get eps from RDP and get eps from approx RDP + # and check the 'updated' flag. if updated, no need to do much + return np.min(eps) + + def propagate_updates(self, func, type_of_update, + delta0=0, + BBGHS_conversion=True, + fDP_based_conversion=False): + # This function receives a new description of the mechanisms and updates all functions + # based on what is new by calling converters. + + if type_of_update == 'pureDP': + # function is one number + eps = func + self.eps_pureDP = np.minimum(eps, self.eps_pureDP) + + approxdp_new = converter.puredp_to_approxdp(eps) + self.approxDP = converter.pointwise_minimum(approxdp_new, self.approxDP) + rdp_new = converter.puredp_to_rdp(eps) + self.RenyiDP = converter.pointwise_minimum(rdp_new, self.RenyiDP) + fdp_new = converter.puredp_to_fdp(eps) + self.fDP = converter.pointwise_maximum(fdp_new, self.fDP) + + self.approxRDP = converter.approxdp_func_to_approxrdp(self.approxDP) + + self.delta0 = 0 # the minimum non-trivial approximate RDP is now 0 + + # lambda x: np.maximum(fdp_new(x),self.fDP(x)) + elif type_of_update == 'approxDP': + # func will be a tuple of two numbers + eps = func[0] + delta = func[1] + + self.approxRDP = converter.pointwise_minimum_two_arguments(self.approxRDP, + converter.approxdp_to_approxrdp(eps, delta)) + + def approx_dp_func(delta1): + if delta1 >= delta: + return eps + else: + return np.inf + + self.approxDP = converter.pointwise_minimum(self.approxDP, approx_dp_func) + self.fDP = converter.pointwise_maximum(self.fDP, converter.approxdp_to_fdp(eps, delta)) + + self.delta0 = np.minimum(self.delta0, delta) + elif type_of_update == 'approxDP_func': + # func outputs eps as a function of delta + # optional input delta0, telling us from where \epsilon becomes infinity + + self.delta0 = np.minimum(delta0, self.delta0) + self.fDP = converter.pointwise_maximum(self.fDP, converter.approxdp_func_to_fdp(func)) + self.approxRDP = converter.pointwise_minimum_two_arguments(self.approxRDP, + converter.approxdp_func_to_approxrdp(func)) + self.approxDP = converter.pointwise_minimum(self.approxDP, func) + + elif type_of_update == 'RDP': + # function output RDP eps as a function of alpha + self.RenyiDP = converter.pointwise_minimum(self.RenyiDP, func) + + if fDP_based_conversion: + + fdp_log, fdp_grad_log = converter.rdp_to_fdp_and_fdp_grad_log(func) + + self.fDP = converter.pointwise_maximum(self.fDP, converter.rdp_to_fdp(self.RenyiDP)) + + # # --------- debugging code below ----------------- + # + # def fdp_grad(x): + # return -np.exp(fdp_grad_log(np.log(x)))[0] + # + # def plot_fdp(x): + # grad = fdp_grad(x) + # y = self.fDP(x) + # + # def tangent_line(u): + # return y + grad*(u-x) + # + # import matplotlib.pyplot as plt + # + # fpr_list, fnr_list = self.plot_fDP() + # plt.figure(1) + # plt.plot(fpr_list, fnr_list) + # plt.plot(fpr_list, tangent_line(fpr_list)) + # plt.show() + # + # plot_fdp(0.01) + # # ------------------------------------------------ + + self.approxDP = converter.pointwise_minimum(self.approxDP, + converter.fdp_fdp_grad_to_approxdp( + fdp_log, fdp_grad_log, + log_flag=True)) + + # self.approxDP = converter.pointwise_minimum(self.approxDP, + # converter.fdp_to_approxdp(self.fDP)) + else: + self.approxDP = converter.pointwise_minimum(self.approxDP, + converter.rdp_to_approxdp(self.RenyiDP, BBGHS_conversion=BBGHS_conversion)) + self.fDP = converter.pointwise_maximum(self.fDP, + converter.approxdp_func_to_fdp( + self.approxDP)) + + + elif type_of_update == 'fDP': + # f-DP, input is Type I error or fpr, output is Type II error or fnr + self.fDP = converter.pointwise_maximum(self.fDP, func) + + self.approxDP = converter.pointwise_minimum(self.approxDP, + converter.fdp_to_approxdp(func)) + self.approxRDP = converter.pointwise_minimum(self.approxRDP, + converter.approxdp_func_to_approxrdp( + self.approxDP)) + elif type_of_update == 'fDP_and_grad': + # the input will be two functions + fdp = func[0] + fdp_grad = func[1] + self.fdp = converter.pointwise_maximum(self.fDP, fdp) + self.approxDP = converter.pointwise_minimum(self.approxDP, + converter.fdp_fdp_grad_to_approxdp(fdp, + fdp_grad,log_flag=False)) + self.approxRDP = converter.pointwise_minimum(self.approxRDP, + converter.approxdp_func_to_approxrdp( + self.approxDP)) + elif type_of_update == 'fDP_and_grad_log': + # the input will be two functions + fun1 = func[0] + fun2 = func[1] + fdp = lambda x: 1 - np.exp(fun1(np.log(x))) + self.fDP = converter.pointwise_maximum(self.fDP, fdp) + self.approxDP = converter.pointwise_minimum(self.approxDP, + converter.fdp_fdp_grad_to_approxdp(fun1, + fun2, + log_flag=True)) + self.approxRDP = converter.pointwise_minimum(self.approxRDP, + converter.approxdp_func_to_approxrdp( + self.approxDP)) + elif type_of_update == 'approxRDP': + # func is a function of alpha and delta + self.delta0 = np.minimum(delta0, self.delta0) + self.approxRDP = converter.pointwise_minimum_two_arguments(self.approxRDP, func) + # TODO: Write a function that converts approximateRDP to approximateDP + + # TODO: Write a function that converts approximateRDP to fDP. + else: + print(type_of_update, ' not recognized.') + + # Plotting functions: returns lines for people to plot outside + + # Plot FNR against FPR --- hypothesis testing interpretation of DP + def plot_fDP(self, length=101): + fpr_list = np.linspace(0, 1, length) + fnr_list = np.array([self.get_fDP(fpr) for fpr in fpr_list]) + return fpr_list, fnr_list + + # Plot RDP function + def plot_RDP(self, alphamax=101): + alpha_list = np.linspace(1, alphamax, alphamax) + RDP_list = np.array([self.get_RDP(alpha) for alpha in alpha_list]) + return alpha_list, RDP_list + + +class Transformer(): + """ + A transformer is a callable object that takes one or more mechanism as input and + **transform** them into a new mechanism + """ + + def __init__(self): + self.name = 'generic_transformer' + self.unary_operator = False # If true it takes one mechanism as an input, + # otherwise it could take many, e.g., composition + self.preprocessing = False # Relevant if unary is true, it specifies whether the operation + # is before or after the mechanism, e.g., amplification by sampling is before applying the + # mechanism, "amplification by shuffling" is after applying the LDP mechanisms + self.transform = lambda x: x + + def __call__(self, *args, **kwargs): + return self.transform(*args, **kwargs) + + +class Calibrator(): + """ + A calibrator calibrates noise (or other parameters) meet a pre-scribed privacy budget + """ + + def __init__(self): + self.name = 'generic_calibrator' + + self.eps_budget = np.inf + self.delta_budget = 1.0 + + self.obj_func = lambda x: 0 + + self.calibrate = lambda x: x + # Input privacy budget, a mechanism with params, output a set of params that works + # while minimizing the obj_func as much as possible + + def __call__(self, *args, **kwargs): + return self.calibrate(*args, **kwargs) diff --git a/autodp/calibrator_zoo.py b/autodp/calibrator_zoo.py new file mode 100644 index 0000000..1344fa1 --- /dev/null +++ b/autodp/calibrator_zoo.py @@ -0,0 +1,182 @@ +""" +'calibrator_zoo' module implements a number of ways to choose parameters of a mechanism to achieve +a pre-defined privacy guarantee. + +For instance, we could calibrate noise to sensitivity in Laplace mechanism to achieve any given \eps + +All calibrators inherit the autodp_core.calibrator + +""" + +from math import exp, sqrt +from scipy.special import erf +from autodp.autodp_core import Calibrator +from scipy.optimize import minimize_scalar, root_scalar + +class eps_delta_calibrator(Calibrator): + """ + Noting there exists multiple parameters in mechanisms. we must specify which parameter requires optimization. + """ + def __init__(self): + Calibrator.__init__(self) + self.name = 'eps_delta_calibrator' + + # Update the function that is callable + + self.calibrate = self.param_from_eps_delta + + def param_from_eps_delta(self,mech_class,eps,delta, bounds,name=None): + def get_eps(x,delta): + + mech = mech_class(x) + return mech.get_approxDP(delta) + + def err(x): + return abs(eps-get_eps(x,delta)) + + results = minimize_scalar(err, method='bounded', bounds=bounds) + if results.success and results.fun < 1e-3: + if name: + mech = mech_class(results.x,name=name) + else: + mech = mech_class(results.x) + return mech + else: + raise RuntimeError(f"eps_delta_calibrator fails to find a parameter: {results.message}") + + +class generalized_eps_delta_calibrator(Calibrator): + """ + Noting there exists multiple parameters in mechanisms. we must specify which parameter requires optimization. + """ + def __init__(self): + Calibrator.__init__(self) + self.name = 'generalized_eps_delta_calibrator' + + # Update the function that is callable + + self.calibrate = self.param_from_eps_delta + + def param_from_eps_delta(self,mech_class,eps,delta, bounds, params=None, para_name=None, name=None): + """ + params defines all parameters, include those need to optimize over + para_name is the parameter that we need to tune + + for example, in subsampleGaussian mechanism + params = {'prob':prob, 'sigma':sigma, 'coeff':coeff}, where coeff and sigma are given as the fixed input + """ + def get_eps(x,delta): + if params is None: + # params is None implies the mechanism only has one parameter + mech = mech_class(x) + else: + params[para_name] = x + mech = mech_class(params) + return mech.get_approxDP(delta) + + def err(x): + return abs(eps-get_eps(x,delta)) + + results = minimize_scalar(err, method='bounded', bounds=bounds) + if results.success and results.fun < 1e-3: + + if params is None: + mech = mech_class(results.x, name=name) + else: + params[para_name] = results.x + mech = mech_class(params,name=name) + return mech + else: + raise RuntimeError(f"eps_delta_calibrator fails to find a parameter: {results.message}") + + + +class ana_gaussian_calibrator(Calibrator): + """ Calibrate a Gaussian perturbation for differential privacy using the analytic Gaussian mechanism of [Balle and Wang, ICML'18] + + Modified from https://github.com/BorjaBalle/analytic-gaussian-mechanism/blob/master/agm-example.py + + Arguments: + epsilon : target epsilon (epsilon > 0) + delta : target delta (0 < delta < 1) + tol : error tolerance for binary search (tol > 0) + Output: + params : a dictionary that contains field `sigma' --- the standard deviation of Gaussian noise needed to achieve + (epsilon,delta)-DP under global sensitivity 1 + + Note that this one does not support composition or subsample + """ + + + def __init__(self): + Calibrator.__init__(self) + self.name = 'ana_gaussian_calibrator' + + # Update the function that is callable + + self.calibrate = self.param_from_eps_delta + + def param_from_eps_delta(self,mech_class,eps,delta,name=None, tol=1.e-12): + def Phi(t): + return 0.5 * (1.0 + erf(float(t) / sqrt(2.0))) + + def caseA(epsilon, s): + return Phi(sqrt(epsilon * s)) - exp(epsilon) * Phi(-sqrt(epsilon * (s + 2.0))) + + def caseB(epsilon, s): + return Phi(-sqrt(epsilon * s)) - exp(epsilon) * Phi(-sqrt(epsilon * (s + 2.0))) + + def doubling_trick(predicate_stop, s_inf, s_sup): + while (not predicate_stop(s_sup)): + s_inf = s_sup + s_sup = 2.0 * s_inf + return s_inf, s_sup + + def binary_search(predicate_stop, predicate_left, s_inf, s_sup): + s_mid = s_inf + (s_sup - s_inf) / 2.0 + while (not predicate_stop(s_mid)): + if (predicate_left(s_mid)): + s_sup = s_mid + else: + s_inf = s_mid + s_mid = s_inf + (s_sup - s_inf) / 2.0 + return s_mid + + delta_thr = caseA(eps, 0.0) + + if (delta == delta_thr): + alpha = 1.0 + sigma = alpha / sqrt(2.0 * eps) + + else: + if (delta > delta_thr): + predicate_stop_DT = lambda s: caseA(eps, s) >= delta + function_s_to_delta = lambda s: caseA(eps, s) + predicate_left_BS = lambda s: function_s_to_delta(s) > delta + function_s_to_alpha = lambda s: sqrt(1.0 + s / 2.0) - sqrt(s / 2.0) + + else: + predicate_stop_DT = lambda s: caseB(eps, s) <= delta + function_s_to_delta = lambda s: caseB(eps, s) + predicate_left_BS = lambda s: function_s_to_delta(s) < delta + function_s_to_alpha = lambda s: sqrt(1.0 + s / 2.0) + sqrt(s / 2.0) + + predicate_stop_BS = lambda s: abs(function_s_to_delta(s) - delta) <= tol + + s_inf, s_sup = doubling_trick(predicate_stop_DT, 0.0, 1.0) + s_final = binary_search(predicate_stop_BS, predicate_left_BS, s_inf, s_sup) + alpha = function_s_to_alpha(s_final) + sigma = alpha / sqrt(2.0 * eps) + + + if name: + mech = mech_class(sigma,name=name) + else: + mech = mech_class(sigma) + return mech + + + +#TODO: implement other generic and specialized privacycalibrators. +#TODO: deprecate the API from "privacy_calibrator.py" + diff --git a/autodp/converter.py b/autodp/converter.py new file mode 100644 index 0000000..274377d --- /dev/null +++ b/autodp/converter.py @@ -0,0 +1,819 @@ +# This module implements all known conversions from DP + + + +import numpy as np +from autodp import utils +import math +from autodp import rdp_bank +from scipy.optimize import minimize_scalar, root_scalar + + + +def puredp_to_rdp(eps): + # From pure dp to RDP + assert(eps >= 0) + + def rdp(alpha): + assert(alpha >= 0) + if alpha==1: + # Calculate this by l'Hospital rule + return eps*(math.cosh(eps)-1)/math.sinh(eps) + elif np.isinf(alpha): + return eps + elif alpha>1: + # in the proof of Lemma 4 of Bun et al. (2016) + s, mag = utils.stable_log_diff_exp(utils.stable_log_sinh(alpha*eps), + utils.stable_log_sinh((alpha-1)*eps)) + return (mag - utils.stable_log_sinh(eps))/(alpha-1) + else: + return min(alpha * eps * eps /2, eps*(math.cosh(eps)-1)/math.sinh(eps)) + + return rdp + +def puredp_to_fdp(eps): + # From Wasserman and Zhou + def fdp(fpr): + return np.max(np.array([0, 1-np.exp(eps)*fpr, np.exp(-eps)*(1-fpr)])) + return fdp + +def puredp_to_approxdp(eps): + # Convert pureDP to approx dp + # Page 3 of https://eprint.iacr.org/2018/277.pdf + def approxdp(delta): + s,mag = utils.stable_log_diff_exp(eps, np.log(delta)) + return mag + return approxdp + +def rdp_to_approxdp(rdp, alpha_max=np.inf, BBGHS_conversion=True): + # from RDP to approx DP + # alpha_max is an optional input which sometimes helps avoid numerical issues + # By default, we are using the RDP to approx-DP conversion due to BBGHS'19's Theorem 21 + # paper: https://arxiv.org/pdf/1905.09982.pdf + # if you need to use the simpler RDP to approxDP conversion for some reason, turn the flag off + + def approxdp(delta): + """ + approxdp outputs eps as a function of delta based on rdp calculations + + :param delta: + :return: the \epsilon with a given delta + """ + + if delta < 0 or delta > 1: + print("Error! delta is a probability and must be between 0 and 1") + if delta == 0: + return rdp(np.inf) + else: + def fun(x): # the input the RDP's \alpha + if x <= 1: + return np.inf + else: + if BBGHS_conversion: + return np.maximum(rdp(x) + np.log((x-1)/x) + - (np.log(delta) + np.log(x))/(x-1), 0) + else: + return np.log(1 / delta) / (x - 1) + rdp(x) + + results = minimize_scalar(fun, method='Brent', bracket=(1,2), bounds=[1, alpha_max]) + if results.success: + return results.fun + else: + # There are cases when certain \delta is not feasible. + # For example, let p and q be uniform the privacy R.V. is either 0 or \infty and unless all \infty + # events are taken cared of by \delta, \epsilon cannot be < \infty + return np.inf + return approxdp + + +def single_rdp_to_fdp(alpha, rho): + assert (alpha >= 0.5) + + def fdp(x): + assert (0 <= x <= 1) + + if x == 0: + return 1 + elif x == 1: + return 0 + + if alpha == 1: + # in this case rho is the KL-divergence + def fun(y): + assert (0 <= y <= 1 - x) + if y == 0: + if x == 1: + return 0 + else: + return np.inf + elif y == 1: + if x == 0: + return 0 + else: + return np.inf + diff1 = (x * (np.log(x) - np.log(1 - y)) + + (1 - x) * (np.log(1 - x) - np.log(y)) - rho) + diff2 = (y * (np.log(y) - np.log(1 - x)) + + (1 - y) * (np.log(1 - y) - np.log(x)) - rho) + + return np.maximum(diff1, diff2) + else: + # This is the general case for Renyi Divergence with \alpha > 1 or \alpha <1 + # find y such that + # log(x^alpha (1-y)^{1-alpha} + (1-x)^alpha y^{1-alpha}) = rho(alpha-1)) + # and log(y^alpha (1-x)^{1-alpha}) + (1-y)^alpha x^{1-alpha} = rho(alpha-1)) + def fun(y): + if y == 0: + if x == 1: + return 0 + else: + return np.inf + elif y == 1: + if x == 0: + return 0 + else: + return np.inf + + diff1 = (utils.stable_logsumexp_two(alpha * np.log(x) + (1 - alpha) * np.log(1 - y), + alpha * np.log(1 - x) + (1 - alpha) * np.log(y)) + - rho * (alpha - 1)) + diff2 = (utils.stable_logsumexp_two(alpha * np.log(y) + (1 - alpha) * np.log(1 - x), + alpha * np.log(1 - y) + (1 - alpha) * np.log(x)) + - rho * (alpha - 1)) + if alpha > 1: + return np.maximum(diff1, diff2) + else: # alpha < 1 + # Notice that the sign of the inequality is toggled + return np.minimum(diff1, diff2) + + + def normal_equation(y): + # for finding the root + return abs(fun(y)) + + # there are two roots, we care about the roots smaller than 1-x + results = minimize_scalar(normal_equation, bounds=[0, 1-x], method='bounded', + options={'xatol':1e-9*(1-x)}) + if results.success: + return results.x + else: + return 0.0 + return fdp + + +def rdp_to_fdp(rdp, alpha_max=np.inf): + # RDP function to FDP. + # maximize the fdp over alpha + def fdp(x): + assert (0 <= x <= 1) + if x == 0: + return 1 + elif x == 1: + return 0 + + def fun(alpha): + if alpha < 0.5: + return np.inf + else: + single_fdp = single_rdp_to_fdp(alpha, rdp(alpha)) + return -single_fdp(x) + + # This will use brent to start with 1,2. + results = minimize_scalar(fun, bracket=(0.5, 2), bounds=(0.5, alpha_max)) + if results.success: + return -results.fun + else: + return 0.0 + return fdp + + +def single_rdp_to_fdp_and_fdp_grad_log(alpha, rho): + # Return two functions + # the first function outputs log(1-fdp(x)) as a function of logx + # the second function outputs log(-partial fdp(x)) as a function of logx + # The format of the output of the second function is an interval. + assert (alpha >= 0.5) + + def diff1_KL(logx,u): + assert(logx < u < 0) + return (np.exp(logx) * (logx - u) + + (1 - np.exp(logx)) * (np.log(1 - np.exp(logx)) - np.log(1-np.exp(u))) - rho) + + def diff2_KL(logx,u): + return ((1 - np.exp(u)) * (np.log(1 - np.exp(u)) - np.log(1 - np.exp(logx))) + + np.exp(u) * (u - logx) - rho) + + def diff1_general(logx,u): + return (utils.stable_logsumexp_two(alpha * logx + (1 - alpha) * u, + alpha * np.log(1 - np.exp(logx)) + + (1 - alpha) * np.log(1 - np.exp(u))) + - rho * (alpha - 1)) + + def diff2_general(logx,u): + return (utils.stable_logsumexp_two(alpha * np.log(1-np.exp(u)) + + (1 - alpha) * np.log(1 - np.exp(logx)), + alpha * u + (1 - alpha) * logx) - rho * (alpha - 1)) + + def grad1_KL(logx,u): + mag1 = np.log(u - logx + np.log(1-np.exp(logx)) - np.log(1-np.exp(u))) + s, mag2 = utils.stable_log_diff_exp(np.log(1-np.exp(logx))- np.log(1-np.exp(u)),logx - u) + return mag1 - mag2 + # return (logx - u - np.log(1-np.exp(logx)) + # + np.log(1-np.exp(u))) / ((1-np.exp(logx))/(1-np.exp(u)) + # - np.exp(logx)/np.exp(u)) + + def grad2_KL(logx,u): + mag1 = np.log(u - logx + np.log(1-np.exp(logx)) - np.log(1-np.exp(u))) + s, mag2 = utils.stable_log_diff_exp(u-logx, np.log(1-np.exp(u))- np.log(1-np.exp(logx))) + return mag2 - mag1 + # return ((1-np.exp(u))/(1-np.exp(logx)) + # - np.exp(u)/np.exp(logx)) / (u - logx + # - np.log(1-np.exp(u)) + np.log(1-np.exp(logx))) + + def grad1_general(logx,u): + #return - grad1_general(np.log(1-np.exp(u)), np.log(1-np.exp(logx))) + + s, mag = utils.stable_log_diff_exp(alpha * (np.log(1 - np.exp(logx)) + - np.log(1 - np.exp(u))), alpha * (logx-u)) + if alpha > 1: + s, mag1 = utils.stable_log_diff_exp((alpha-1) * (np.log(1 - np.exp(logx)) + - np.log(1 - np.exp(u))), + (alpha-1) * (logx-u)) + return np.log(alpha)-np.log(alpha-1) + mag1 - mag + else: + s, mag1 = utils.stable_log_diff_exp((alpha-1) * (logx-u), + (alpha-1) * (np.log(1 - np.exp(logx)) + - np.log(1 - np.exp(u)))) + return np.log(alpha)-np.log(1-alpha) + mag1 - mag + + def grad2_general(logx,u): + s, mag = utils.stable_log_diff_exp(alpha * (u - logx), + alpha * (np.log(1 - np.exp(u)) + - np.log(1 - np.exp(logx)))) + if alpha > 1: + s, mag2 = utils.stable_log_diff_exp((alpha-1) * (u - logx), + (alpha-1) * (np.log(1 - np.exp(u)) + - np.log(1 - np.exp(logx)))) + return (np.log(1-1.0/alpha)) + mag - mag2 + else: # if alpha < 1 + s, mag2 = utils.stable_log_diff_exp((alpha-1) * (np.log(1 - np.exp(u)) + - np.log(1 - np.exp(logx))), + (alpha - 1) * (u - logx)) + return np.log(1.0/alpha - 1) + mag - mag2 + + def log_one_minus_fdp(logx): + #assert (0 <= x <= 1) + assert(logx <= 0) + + if logx == 0: # x==1, f(x) should be 0 + return 0 + elif np.isneginf(logx): # x = 0, f(x) should be 1 + return -np.inf + + # Now define the non-linear equation ``fun'' + # such that the u such that fun(u) = 0 gives log(1-f(x)) + + if alpha == 1: + # in this case rho is the KL-divergence + def fun(u): + assert( u >= logx) + # assert (0 <= y <= 1 - x) + if u == 0: #y == 0: + if logx == 0: #x == 1: + return 0 + else: + return np.inf + elif np.isneginf(u): #y == 1: + if np.isneginf(logx): #x == 0: + return 0 + else: + return np.inf + + diff1 = diff1_KL(logx,u) + #diff1 = (x * (np.log(x) - np.log(1 - y)) + # + (1 - x) * (np.log(1 - x) - np.log(y)) - rho) + diff2 = diff2_KL(logx,u) + + #diff2 = (y * (np.log(y) - np.log(1 - x)) + # + (1 - y) * (np.log(1 - y) - np.log(x)) - rho) + + return np.maximum(diff1, diff2) + else: + # This is the general case for Renyi Divergence with \alpha > 1 or \alpha <1 + # find y such that + # log(x^alpha (1-y)^{1-alpha} + (1-x)^alpha y^{1-alpha}) = rho(alpha-1)) + # and log(y^alpha (1-x)^{1-alpha}) + (1-y)^alpha x^{1-alpha} = rho(alpha-1)) + def fun(u): + assert( u >= logx) + if u == 0: #y == 0: + if logx == 0: #x == 1: + return 0 + else: + return np.inf + elif np.isneginf(u): #y == 1: + if np.isneginf(logx): #x == 0: + return 0 + else: + return np.inf + + # diff1 = (utils.stable_logsumexp_two(alpha * np.log(x) + (1 - alpha) * np.log(1 - y), + # alpha * np.log(1 - x) + (1 - alpha) * np.log(y)) + # - rho * (alpha - 1)) + diff1 = diff1_general(logx,u) + # diff2 = (utils.stable_logsumexp_two(alpha * np.log(y) + (1 - alpha) * np.log(1 - x), + # alpha * np.log(1 - y) + (1 - alpha) * np.log(x)) + # - rho * (alpha - 1)) + diff2 = diff2_general(logx,u) + if alpha > 1: + return np.maximum(diff1, diff2) + else: # alpha < 1 + # Notice that the sign of the inequality is toggled + return np.minimum(diff1, diff2) + + def normal_equation(u): + # for finding the root + return abs(fun(u)) + + # there are two roots, we care about the roots smaller than 1-x + results = minimize_scalar(normal_equation, bounds=[logx,0], method='bounded', + options={'xatol':1e-8}) + if results.success: + return results.x + else: + return 0.0 + + + + def log_neg_partial_fdp(logx): + assert(logx <= 0) + + if np.isneginf(logx): # x = 0, the gradient is negative infinity unless alpha = +inf + # but alpha = +inf won't be passed into here. + return np.inf, np.inf + elif logx == 0: # x = 1 + return 0, 0 + + u = log_one_minus_fdp(logx) + # Find which leg is active, and output the log (- subgradient) + tol = 1e-5 + + grad_l = np.inf + grad_h = 0 + + if alpha == 1: + err = min(abs(diff1_KL(logx, u)), abs(diff2_KL(logx,u))) + if err > tol: + print('no solution found!') + + if abs(diff1_KL(logx,u)) <= tol: + grad_l = grad1_KL(logx,u) + grad_h = grad_l + if abs(diff2_KL(logx,u)) <= tol: + grad = grad2_KL(logx, u) + grad_l = min(grad,grad_l) + grad_h = max(grad,grad_h) + else: + err = min(abs(diff1_general(logx, u)), abs(diff2_general(logx,u))) + if err > tol: + print('no solution found!') + if abs(diff1_general(logx,u)) <= tol: + grad_l = grad1_general(logx,u) + grad_h = grad_l + if abs(diff2_general(logx,u)) <= tol: + grad = grad2_general(logx,u) + grad_l = min(grad,grad_l) + grad_h = max(grad,grad_h) + return [grad_l,grad_h] + + + # ------------ debugging -------------------------- + # def fdp(x): + # return 1- np.exp(log_one_minus_fdp(np.log(x))) + # + # fdp_ref = single_rdp_to_fdp(alpha, rho) + # + # import matplotlib.pyplot as plt + # + # fpr_list = np.linspace(0,1,100) + # plt.figure(1) + # fnr1 = [fdp(x) for x in fpr_list] + # fnr2 = [fdp_ref(x) for x in fpr_list] + # plt.plot(fpr_list, fnr1) + # plt.plot(fpr_list, fnr2) + # + # x = 0.01 + # u = log_one_minus_fdp(np.log(x)) + # log_minus_grad_ref = grad2_general(np.log(x), u) + # + # log_minus_grad = log_neg_partial_fdp(np.log(x)) + # + # + # def grad2_general_new(x,y): + # return - alpha*((x/(1-y))**(alpha-1) - ((1-x)/y)**(alpha-1)) / (1-alpha) / ( - (x/(1-y))**alpha + ((1-x)/y)**alpha) + # + # def Fxy(x,y): + # return (x/(1-y))**alpha * (1-y) + ((1-x)/y)**alpha * y - np.exp((alpha-1)*rho) + # + # y = 1-np.exp(u) + # + # grad_ref = grad2_general_new(x,y) + # + # grad = -np.exp(log_minus_grad) + # grad = grad[0] + # + # def tangent_line(v): + # return y + grad * (v - x) + # + # plt.plot(fpr_list, tangent_line(fpr_list)) + # + # plt.ylim([0,1]) + # plt.xlim([0,1]) + # + # plt.show() + + + return log_one_minus_fdp, log_neg_partial_fdp + + +def rdp_to_fdp_and_fdp_grad_log(rdp, alpha_max=np.inf): + # Return the a function that outputs the minimum of + # log(1-fdp_alpha(x)) and the corresponding log(-partial fdp_alpha(x)) at the optimal \alpha. + + # This, when plugged into the standard machinery, would allow a more direct conversion from RDP. + + def log_one_minus_fdp(logx): + assert (logx <= 0) + if np.isneginf(logx):# x == 0: + return [-np.inf, np.inf] # y = 1, log (1-y) = -np.inf, alpha is inf (pure DP) + elif logx == 0: + return [0, np.inf] + + def fun(alpha): + if alpha < 0.5: + return np.inf + else: + log_one_minus_fdp_alpha, tmp = single_rdp_to_fdp_and_fdp_grad_log(alpha, rdp(alpha)) + return log_one_minus_fdp_alpha(logx) + + # This will use brent to start with 1,2. + results = minimize_scalar(fun, bracket=(0.5, 2), bounds=(0.5, alpha_max)) + if results.success: + return [results.fun, results.x] + else: + return [log_one_minus_fdp(results.x), results.x] + + def log_one_minus_fdp_only(logx): + res = log_one_minus_fdp(logx) + return res[0] + + def log_neg_partial_fdp(logx): + assert (logx <=0) + if np.isneginf(logx):# x == 0: + tmp = rdp(np.inf) + return [tmp, np.inf] # y = 1, log (1-y) = -np.inf, alpha is inf (pure DP) + elif logx == 0: + tmp = rdp(np.inf) + return [-np.inf, -tmp] + + # The following implements the more generic case + # when we need to find the alpha that is active + + res = log_one_minus_fdp(logx) + best_alpha = res[1] + tmp, log_neg_partial_fdp_alpha = single_rdp_to_fdp_and_fdp_grad_log(best_alpha, + rdp(best_alpha)) + return log_neg_partial_fdp_alpha(logx) + + return log_one_minus_fdp_only, log_neg_partial_fdp + + +def approxdp_to_approxrdp(eps,delta): + # from a single eps,delta calculation to an approxdp function + def approxrdp(alpha, delta1): + if delta1 >= delta: + rdp = puredp_to_rdp(eps) + return rdp(alpha) + else: + return np.infty + return approxrdp + + +def approxdp_func_to_approxrdp(eps_func): + # from an approximate_dp function to approxrdp function + def approxrdp(alpha, delta): + rdp = puredp_to_rdp(eps_func(delta)) + return rdp(alpha) + + return approxrdp + + + +def approxdp_to_fdp(eps, delta): + # from a single eps, delta approxdp to fdp + assert(eps >= 0 and 0 <= delta <= 1) + + def fdp(fpr): + assert(0 <= fpr <= 1) + if fpr == 0: # deal with log(0) below + return 1-delta + elif np.isinf(eps): + return 0 + else: + return np.max(np.array([0, 1-delta-np.exp(eps)*fpr, np.exp(-eps)*(1-delta-fpr)])) + return fdp + + +def approxdp_func_to_fdp(func, delta_func=False): + """ + from an approxdp function to fdp + :param func: epsilon as a function of delta by default. + :param delta_func: if the flag is True, then 'func' is a delta as a function of epsilon. + :return: fdp function + """ + # + # By default, logdelta_func is False, and func is eps as a function of delta + # fpr = maximize_{delta} approxdp_to_fdp(eps(delta),delta)(fpr) + # if delta_func is True, it means that 'func' is a delta as a function of eps, then + # fpr = maximize_{delta} approxdp_to_fdp(eps,delta(eps))(fpr) + if delta_func: + def fdp(fpr): + + assert(0 <= fpr <= 1) + if fpr == 1: + return 0 + + def fun(eps): + fdp_eps = approxdp_to_fdp(eps, func(eps)) + fnr = fdp_eps(fpr) + return -fnr + + results = minimize_scalar(fun, bounds=[0, +np.inf], options={'disp': False}) + if results.success: + return -results.fun + else: + return 0 + else: + def fdp(fpr): + assert(0 <= fpr <= 1) + if fpr == 1: + return 0 + + def fun(delta): + fdp_delta = approxdp_to_fdp(func(delta), delta) + fnr = fdp_delta(fpr) + return -fnr + + results = minimize_scalar(fun, method='Bounded', bounds=[0, 1-fpr], + options={'disp': False}) + if results.success: + return -results.fun + else: + return 0 + return fdp + + + + +def fdp_fdp_grad_to_approxdp(fdp, fdp_grad, log_flag = False): + # when there is a dedicated implementation of fdp_grad + + # If the log flag is False, then + # fdp takes x \in [0,1] and output f(x) + # fdp_grad takes x \in [0,1] and output the subdifferential as an interval [grad_l, grad_h] + + + # If log_flag is True, then it indicates that + # 1. the first argument denotes log(1-fdp) as a function of logx + # 2. the second argument denotes log(- partial fdp) as a function of logx + + if log_flag: + fun1 = fdp + fun2 = fdp_grad + else: + def fun1(logx): + assert(logx <= 0) + if np.isneginf(logx): # x == 0 + return np.log(1-fdp(0)) + elif logx == 0: # x == 1 + return 1.0 + + def fun2(logx): + assert(logx <= 0) + if np.isneginf(logx): + grad_l, grad_h = fdp_grad(0) + else: + grad_l, grad_h = fdp_grad(np.exp(logx)) + log_neg_grad_l = np.log(-grad_l) + log_neg_grad_h = np.log(-grad_h) + + if log_neg_grad_l > log_neg_grad_h: + # in case the order is swapped + tmp = log_neg_grad_h + log_neg_grad_h = log_neg_grad_l + log_neg_grad_l = tmp + + return log_neg_grad_l, log_neg_grad_h + + def find_logx(delta): + def fun(logx): + if np.isneginf(logx): + output = np.log(delta) - fun1(logx) + return output,output + else: + log_neg_grad_l, log_neg_grad_h = fun2(logx) + log_one_minus_f = fun1(logx) + low = utils.stable_logsumexp_two(log_neg_grad_l + logx, + np.log(delta)) - log_one_minus_f + high = utils.stable_logsumexp_two(log_neg_grad_h + logx, + np.log(delta)) - log_one_minus_f + return low, high + + def normal_equation(logx): + if logx > 0: + return np.inf + low, high = fun(logx) + if low <= 0 <= high: + return 0 + else: + return min(abs(high),abs(low)) + + def normal_equation_loglogx(loglogx): + logx = np.exp(loglogx) + return normal_equation(logx) + + # find x such that y = 1-\delta + tmp = fun1(np.log(1 - delta)) + if abs(tmp) < 1e-5: + bound1 = np.log(-tmp - tmp**2 / 2 - tmp**3 / 6) + else: + bound1 = np.log(1-np.exp(fun1(np.log(1-delta)))) + #results = minimize_scalar(normal_equation, bounds=[-np.inf,0], bracket=[-1,-2]) + results = minimize_scalar(normal_equation, method="Bounded", bounds=[bound1,0], + options={'xatol': 1e-10, 'maxiter': 500, 'disp': 0}) + if results.success: + if abs(results.fun) > 1e-4 and abs(results.x)>1e-10: + # This means that we hit xatol (x is close to 0, but + # the function value is not close to 0) In this case let's do an even larger search. + raise RuntimeError("'find_logx' fails to find the tangent line.") + else: + return results.x + else: + raise RuntimeError(f"'find_logx' fails to find the tangent line: {results.message}") + + def approxdp(delta): + if delta == 0: + logx = -np.inf + log_neg_grad_l, log_neg_grad_h = fun2(logx) + return log_neg_grad_l + elif delta == 1: + return 0.0 + else: + logx = find_logx(delta) + log_one_minus_f = fun1(logx) + # log_neg_grad_l, log_neg_grad_h = fun2(logx) + s, mag = utils.stable_log_diff_exp(log_one_minus_f,np.log(delta)) + eps = mag - logx + if eps < 0: + return 0.0 + else: + return eps + + #approxdp(1e-3) + + return approxdp + + + + # def findx(delta): + # + # def fun(x): + # # if log_flag: + # # if x == 0: + # # return np.log(delta) - fun1(0) + # # else: + # # return utils.stable_logsumexp_two(fun2(x) + np.log(x), np.log(delta)) - fun1(x) + # #else: + # fx = fdp(x) + # if x == 0: + # output = np.log(delta) - np.log(1 - fx) + # return output + # else: + # grad_l, grad_h = fdp_grad(x) + # return utils.stable_logsumexp_two(np.log(-fdp_grad(x)) + np.log(x), + # np.log(delta)) - np.log(1-fx) + # + # def normal_equation(x): + # return abs(fun(x)) + # results = minimize_scalar(normal_equation, method="Bounded", bounds=[0,1], + # options={'xatol': min(1e-10,1e-3*delta), 'maxiter': 500, 'disp': 0}) + # if results.success: + # return results.x + # else: + # return None + # + # def approxdp(delta): + # x = findx(delta) - min(1e-10,1e-3*delta) + # if log_flag: + # return fun2(x) + # else: + # return np.log(-fdp_grad(x)) + # return approxdp + + + +def fdp_to_approxdp(fdp): + # Check out Proposition 2.12 of Dong, Roth and Su + # if given a symmetric fdp function f, + # its convex conjugate with some manipulation defines the \delta as a function of \epsilon. + # How to calculate convex conjugates? Can we just do numerical computation? + # How to ensure the symmetry of fdp function f? + # One way we can go is to define an fdp_bank where we code up the conjugate pairs in analytical form + # This allows one to more easily convert approxdp function to fdp + + fstar = conjugate(fdp) + # def delta_from_fdp(eps): + # return 1 + fstar(-np.exp(eps)) + # + # approxdp = numerical_inverse(delta_from_fdp) + + # def neg_log_one_plus_fstar_neg_input(x): + # return -np.log(1 + fstar(-x)) + # + # exp_eps = numerical_inverse(neg_log_one_plus_fstar_neg_input) + # def approxdp(delta): + # return np.log(exp_eps(-np.log(delta))) + + def neg_fstar_neg_input(x): + return -fstar(-x) + + exp_eps = numerical_inverse(neg_fstar_neg_input,[0,1]) + def approxdp(delta): + return np.log(exp_eps(1-delta)) + + return approxdp + + +def numerical_inverse(f, bounds=None): + # of a scalar, monotonic function + def inv_f(y): + if bounds: + if y > bounds[1] or y < bounds[0]: + raise ValueError(f'y value {y} is out of bounds [{bounds[0]},{bounds[1]}].') + + def fun(x): + return f(x) - y + + # The domain should be encoded in the definition of f directly. + def normal_equation(x): + return abs(fun(x)) + + results = minimize_scalar(normal_equation, bounds=[1,np.inf], bracket=[1,2]) + + + #results = root_scalar(fun, options={'disp': False}) + if results.success: + return results.x + else: + raise RuntimeError(f"Failed to invert function {f} at {y}: {results.message}") + + return inv_f + + +def approxdp_from_its_inverse(delta_func): + # Convert delta as a function of epsilon to epsilon as a function of delta + return numerical_inverse(delta_func, bounds=[0,1]) + + + +## Utility functions + +def conjugate(f,tol=1e-10): + # numerically evaluate convex conjugate of a convex function f: [0,1] --> [0,1] + # domain of the function y is [0,1] + def fstar(x): + def fun(y): + return -y*x + f(y) + results = minimize_scalar(fun, method='Bounded', bounds=(0, 1), + options={'disp': False,'xatol':tol}) + if results.success: + return -(results.fun + tol) + # output an upper bound + else: + raise RuntimeError(f"Failed to conjugate function {f} at {x}: {results.message}") + return fstar + + +def pointwise_minimum(f1, f2): + def min_f1_f2(x): + return np.minimum(f1(x), f2(x)) + return min_f1_f2 + +def pointwise_minimum_two_arguments(f1, f2): + def min_f1_f2(x, y): + return np.minimum(f1(x, y), f2(x, y)) + return min_f1_f2 + +def pointwise_maximum(f1, f2): + def max_f1_f2(x): + return np.maximum(f1(x), f2(x)) + return max_f1_f2 diff --git a/autodp/dp_bank.py b/autodp/dp_bank.py index e2d09f6..8b47603 100644 --- a/autodp/dp_bank.py +++ b/autodp/dp_bank.py @@ -8,7 +8,8 @@ """ import numpy as np -from autodp import rdp_acct, rdp_bank, utils +from autodp import dp_acct, rdp_acct, rdp_bank, utils +from scipy.special import comb from scipy.stats import norm from scipy.optimize import minimize_scalar, root_scalar @@ -76,12 +77,45 @@ def fun(x): return np.inf else: return get_logdelta_ana_gaussian(sigma, x) - np.log(delta) - # The following by default uses the 'secant' method for finding - results = root_scalar(fun, x0=0, x1=5) + + eps_upperbound = 1/2/sigma**2+1/sigma*np.sqrt(2*np.log(1/delta)) + results = root_scalar(fun,bracket=[0, eps_upperbound]) if results.converged: return results.root else: - return None + raise RuntimeError(f"Failed to find epsilon: {results.flag}") + +def eps_generalized_gaussian(x, sigma, delta,k, c, c_tilde): + """ + submodule for generalized SVT with Gaussian noise + we want to partition c into [c/c'] parts, each part using (k choose c') + need to check whether (k choose c') > log(1/delta') + k is the maximam number of queries to answer for each chunk + x is log delta for each chunk, it needs to be negative + :param x: + :param sigma: + :param delta: + :return: + """ + acct = dp_acct.DP_acct() + per_delta = np.exp(x) # per_delta for each c' chunk + coeff = comb(k,c_tilde) + assert per_delta < 1.0/(coeff) + #compute the eps per step with 1/(sigma_1**2) + sqrt(2/simga_1**2 *(log k + log(1/epr_delta))) + # compose eps for each chunk + while c: + if c>= c_tilde: + c = c - c_tilde + else: + # the remaining part c // c_tilde + c = 0 + c_tilde = c + coeff = comb(k, c_tilde) + per_eps = (1.0+c_tilde) / (2*sigma ** 2) + np.sqrt((1.0 +c_tilde) / (2*sigma ** 2) * (np.log(coeff) - x)) + acct.update_DPlosses(per_eps, per_delta) + + compose_eps = acct.get_eps(delta) + return compose_eps def get_eps_laplace(b,delta): diff --git a/autodp/fdp_bank.py b/autodp/fdp_bank.py new file mode 100644 index 0000000..38f91a6 --- /dev/null +++ b/autodp/fdp_bank.py @@ -0,0 +1,87 @@ +# This module implements f-DP function templates for various mechanisms +# Note that f-DP is equivalent to the family of (eps,delta)-DP described by the functions +# delta(eps) or eps(delta), generic conversion is available from the converter module +# however it is numerically better if we implement and support cases when fdp is natural + +import numpy as np +from scipy.stats import norm +from autodp import utils + + +def fDP_gaussian(params, fpr): + """ + :param params: + 'sigma' --- is the normalized noise level: std divided by global L2 sensitivity + :param fpr: False positive rate --- input to the fDP function + :return: Evaluation of the fnr lower bound supported by the gaussian mechanism + """ + sigma = params['sigma'] + # assert(sigma > 0) + assert(sigma >= 0) + if sigma == 0: + return 0 + else: + return norm.cdf(norm.ppf(1-fpr)-1/sigma) + + +def fdp_grad_gaussian(params,fpr): + """ + :param params: + 'sigma' --- is the normalized noise level: std divided by global L2 sensitivity + :param fpr: False positive rate --- input to the fDP function + :return: Evaluation of derivative of the Tradeoff function at input fpr + """ + sigma = params['sigma'] + # assert(sigma > 0) + assert(sigma >= 0) + if sigma == 0: + return 0 + else: + return -norm.pdf(norm.ppf(1-fpr)-1/sigma)/norm.pdf(norm.ppf(1-fpr)) + +def log_one_minus_fdp_gaussian(params, logfpr): + """ + :param params: + 'sigma' --- is the normalized noise level: std divided by global L2 sensitivity + :param logfpr: log of False positive rate --- input to the fDP function + :return: log(1-f(x)). + """ + sigma = params['sigma'] + # assert(sigma > 0) + assert(sigma >= 0) + if sigma == 0: + return 0 + else: + if np.isneginf(logfpr): + return -np.inf + else: + + norm_ppf_one_minus_fpr = utils.stable_norm_ppf_one_minus_x(logfpr) + + return norm.logsf(norm_ppf_one_minus_fpr-1/sigma) + + + + +def log_neg_fdp_grad_gaussian(params, logfpr): + """ + :param params: + 'sigma' --- is the normalized noise level: std divided by global L2 sensitivity + :param logfpr: log of False positive rate --- input to the fDP function + :return: log(-partial f(x)) + """ + sigma = params['sigma'] + # assert(sigma > 0) + assert(sigma >= 0) + if sigma == 0: + return 0 + else: + if np.isneginf(logfpr): # == 0: + return np.inf, np.inf + elif logfpr == 0: #fpr == 1: + return -np.inf, -np.inf + else: + norm_ppf_one_minus_fpr = utils.stable_norm_ppf_one_minus_x(logfpr) + grad = -(norm_ppf_one_minus_fpr + - 1 / sigma) ** 2 / 2 + norm_ppf_one_minus_fpr ** 2 / 2 + return grad, grad diff --git a/autodp/mechanism_zoo.py b/autodp/mechanism_zoo.py new file mode 100644 index 0000000..384ecc5 --- /dev/null +++ b/autodp/mechanism_zoo.py @@ -0,0 +1,304 @@ +""" +'mechanism_zoo' module implements popular DP mechanisms with their privacy guarantees + +""" +import math + +from autodp.autodp_core import Mechanism +from autodp import rdp_bank, dp_bank, fdp_bank, utils +from autodp import transformer_zoo + +from scipy.optimize import minimize_scalar + + +# Example of a specific mechanism that inherits the Mechanism class +class GaussianMechanism(Mechanism): + def __init__(self, sigma, name='Gaussian', + RDP_off=False, approxDP_off=False, fdp_off=True, + use_basic_RDP_to_approxDP_conversion=False, + use_fDP_based_RDP_to_approxDP_conversion=False): + # the sigma parameter is the std of the noise divide by the l2 sensitivity + Mechanism.__init__(self) + + self.name = name # When composing + self.params = {'sigma': sigma} # This will be useful for the Calibrator + # TODO: should a generic unspecified mechanism have a name and a param dictionary? + + self.delta0 = 0 + if not RDP_off: + new_rdp = lambda x: rdp_bank.RDP_gaussian({'sigma': sigma}, x) + if use_fDP_based_RDP_to_approxDP_conversion: + # This setting is slightly more complex, which involves converting RDP to fDP, + # then to eps-delta-DP via the duality + self.propagate_updates(new_rdp, 'RDP', fDP_based_conversion=True) + elif use_basic_RDP_to_approxDP_conversion: + self.propagate_updates(new_rdp, 'RDP', BBGHS_conversion=False) + else: + # This is the default setting with fast computation of RDP to approx-DP + self.propagate_updates(new_rdp, 'RDP') + + if not approxDP_off: # Direct implementation of approxDP + new_approxdp = lambda x: dp_bank.get_eps_ana_gaussian(sigma, x) + self.propagate_updates(new_approxdp,'approxDP_func') + + if not fdp_off: # Direct implementation of fDP + fun1 = lambda x: fdp_bank.log_one_minus_fdp_gaussian({'sigma': sigma}, x) + fun2 = lambda x: fdp_bank.log_neg_fdp_grad_gaussian({'sigma': sigma}, x) + self.propagate_updates([fun1,fun2],'fDP_and_grad_log') + # overwrite the fdp computation with the direct computation + self.fdp = lambda x: fdp_bank.fDP_gaussian({'sigma': sigma}, x) + + # the fDP of gaussian mechanism is equivalent to analytical calibration of approxdp, + # so it should have been automatically handled numerically above + + + # Discussion: Sometimes delta as a function of eps has a closed-form solution + # while eps as a function of delta does not + # Shall we represent delta as a function of eps instead? + + +class ExactGaussianMechanism(Mechanism): + """ + The Gaussian mechanism to use in practice with tight direct computation of everything + """ + def __init__(self, sigma=None, name='Gaussian'): + # the sigma parameter is the std of the noise divide by the l2 sensitivity + Mechanism.__init__(self) + + self.name = name # When composing + self.params = {'sigma': sigma} # This will be useful for the Calibrator + self.delta0 = 0 + if sigma is not None: + new_rdp = lambda x: rdp_bank.RDP_gaussian({'sigma': sigma}, x) + self.propagate_updates(new_rdp, 'RDP') + # Overwrite the approxDP and fDP with their direct computation + self.approxDP = lambda x: dp_bank.get_eps_ana_gaussian(sigma, x) + self.fDP = lambda x: fdp_bank.fDP_gaussian({'sigma': sigma}, x) + + +class LaplaceMechanism(Mechanism): + """ + param params: + 'b' --- is the is the ratio of the scale parameter and L1 sensitivity + """ + def __init__(self, b=None, name='Laplace'): + + Mechanism.__init__(self) + + self.name = name + self.params = {'b': b} # This will be useful for the Calibrator + self.delta0 = 0 + if b is not None: + new_rdp = lambda x: rdp_bank.RDP_laplace({'b': b}, x) + self.propagate_updates(new_rdp, 'RDP') + + +class RandresponseMechanism(Mechanism): + + """ + param params: + 'p' --- is the Bernoulli probability p of outputting the truth. + """ + + def __init__(self, p=None, name='Randresponse'): + Mechanism.__init__(self) + + self.name = name + self.params = {'p': p} # This will be useful for the Calibrator + self.delta0 = 0 + if p is not None: + new_rdp = lambda x: rdp_bank.RDP_randresponse({'p': p}, x) + self.propagate_updates(new_rdp, 'RDP') + + +class PureDP_Mechanism(Mechanism): + def __init__(self, eps, name='PureDP'): + # the eps parameter is the pure DP parameter of this mechanism + Mechanism.__init__(self) + + self.name = name # Used for generating new names when composing + self.params = {'eps': eps} # + + self.propagate_updates(eps, 'pureDP') + + # ------- I verified that the following options give the same results ---- + # def new_rdp(x): + # return rdp_bank.RDP_pureDP({'eps': eps}, x) + # + # if use_basic_RDP_to_approxDP_conversion: + # self.propagate_updates(new_rdp, 'RDP', BBGHS_conversion=False) + # else: + + + # self.propagate_updates(new_rdp, 'RDP') + + + +class SubsampleGaussianMechanism(Mechanism): + """ + This one is used as an example for calibrator with subsampled Gaussian mechanism + """ + def __init__(self,params,name='SubsampleGaussian'): + Mechanism.__init__(self) + self.name=name + self.params={'prob':params['prob'],'sigma':params['sigma'],'coeff':params['coeff']} + # create such a mechanism as in previously + subsample = transformer_zoo.AmplificationBySampling() # by default this is using poisson sampling + mech = GaussianMechanism(sigma=params['sigma']) + + # Create subsampled Gaussian mechanism + SubsampledGaussian_mech = subsample(mech, params['prob'], improved_bound_flag=True) + + # Now run this for niter iterations + compose = transformer_zoo.Composition() + mech = compose([SubsampledGaussian_mech], [params['coeff']]) + + # Now we get it and let's extract the RDP function and assign it to the current mech being constructed + rdp_total = mech.RenyiDP + self.propagate_updates(rdp_total, type_of_update='RDP') + + +class ComposedGaussianMechanism(Mechanism): + """ + This one is used as an example for calibrator with composed Gaussian mechanism + """ + def __init__(self,params,name='SubsampleGaussian'): + Mechanism.__init__(self) + self.name=name + self.params={'sigma':params['sigma'],'coeff':params['coeff']} + # create such a mechanism as in previously + + mech = GaussianMechanism(sigma=params['sigma']) + # Now run this for coeff iterations + compose = transformer_zoo.Composition() + mech = compose([mech], [params['coeff']]) + + # Now we get it and let's extract the RDP function and assign it to the current mech being constructed + rdp_total = mech.RenyiDP + self.propagate_updates(rdp_total, type_of_update='RDP') + + + +class NoisyScreenMechanism(Mechanism): + """ + The data-dependent RDP of ``Noisy Screening" (Theorem 7 in Private-kNN (CPVR-20)) + This mechanism is also used in Figure 2(a) in NIPS-20 + """ + def __init__(self,params,name='NoisyScreen'): + Mechanism.__init__(self) + self.name=name + self.params={'logp':params['logp'],'logq':params['logq']} + # create such a mechanism as in previously + + new_rdp = lambda x: rdp_bank.RDP_noisy_screen({'logp': params['logp'], 'logq': params['logq']}, x) + self.propagate_updates(new_rdp, 'RDP') + + +class GaussianSVT_Mechanism(Mechanism): + """ + Gaussian SVT proposed by NeurIPS-20 + parameters k and sigma + k is the maximum length before the algorithm stops + rdp_c_1 = True indicates we use RDP-based Gaussian-SVT with c=1, else c>1 + + """ + def __init__(self,params,name='GaussianSVT', rdp_c_1=True): + Mechanism.__init__(self) + self.name=name + if rdp_c_1 == True: + self.name = name + 'c_1' + self.params = {'sigma': params['sigma'], 'k': params['k'], 'margin':params['margin']} + new_rdp = lambda x: rdp_bank.RDP_gaussian_svt_c1(self.params, x) + else: + self.name = name + 'c>1' + self.params = {'sigma':params['sigma'],'k':params['k'], 'c':params['c']} + new_rdp = lambda x: rdp_bank.RDP_gaussian_svt_cgreater1(self.params, x) + self.propagate_updates(new_rdp, 'RDP') + +class LaplaceSVT_Mechanism(Mechanism): + """ + Laplace SVT (c>=1) used in NeurIPS-20 + parameters k and sigma + k is the maximum length before the algorithm stops + We provide the RDP implementation and pure-DP implementation + """ + def __init__(self,params,name='GaussianSVT'): + Mechanism.__init__(self) + self.name=name + self.params={'b':params['b'],'k':params['k'], 'c':params['c']} + + new_rdp = lambda x: rdp_bank.RDP_svt_laplace(self.params, x) + self.propagate_updates(new_rdp, 'RDP') + + +class StageWiseMechanism(Mechanism): + """ + The StageWise generalized SVT is proposed by Zhu et.al., NeurIPS-20 + used for Sparse vector technique with Gaussian Noise + + c is the number of tops (composition) + k is the maximum limit for each chunk, e.g., the algorithm restarts whenever it encounters a top or reaches k limit. + """ + def __init__(self, params=None,approxDP_off=False, name='StageWiseMechanism'): + # the sigma parameter is the std of the noise divide by the l2 sensitivity + Mechanism.__init__(self) + + self.name = name # When composing + self.params = {'sigma': params['sigma'], 'k':params['k'], 'c':params['c']} + self.delta0 = 0 + + if not approxDP_off: # Direct implementation of approxDP + new_approxdp = lambda x: dp_bank.eps_generalized_gaussian(x, **params) + self.propagate_updates(new_approxdp, 'approxDP_func') + + +# # Example 1: Short implementation of noisy gradient descent mechanism as a composition of GMs +# class NoisyGD_mech(GaussianMechanism): +# def __init__(self,sigma_list,name='NoisyGD'): +# GaussianMechanism.__init__(self, sigma=np.sqrt(np.sum(sigma_list)),name=name) +# self.params = {'sigma_list':sigma_list} +# +# # The user could log sigma_list and then just declare a NoisyGD_mech object. +# mech = NoisyGD_mech(sigma_list) +# mech.get_approxDP(delta=1e-6) +# +# +# # Example 2: Implementing NoisySGD from basic building blocks +# subsample = Transformers.Subsample(prob=0.01) +# mech = Mechanisms.GaussianMechanism(sigma=5.0) +# # Create subsampled Gaussian mechanism +# SubsampledGaussian_mech = subsample(mech) +# +# # Now run this for 100 iterations +# compose = Transformers.Composition() +# NoisySGD_mech = compose(mechanism_list = [SubsampledGaussian_mech],coeffs_list=[100]) +# +# +# # Example 3: You could also package this together by defining a NoisySGD mechanism +# class NoisySGD_mech(Mechanism): +# def __init__(self,prob,sigma,niter,name='NoisySGD'): +# Mechanism.__init__() +# self.name=name +# self.params={'prob':prob,'sigma':sigma,'niter':niter} +# +# rdp = rdp_bank.subsampled_gaussian({'prob':params['prob'],'sigma':params['sigma']}) +# self.propagate_updates(rdp,type_of_update='RDP') +# +# +# # Example 4: Online decision. Hetereogenous sigma decided online +# # (maybe as a function of computed eps) +# # Alternatively if we want to do it via composition, so we can make online decision about +# # the sigma in the sigma_list +# +# delta = 1e-6 +# online_sgd = Mechanisms.SubsampledGaussian_mech(prob=prob,sigma=sigma) +# compose = Transformers.Composition() +# for i in range(niter): +# eps = online_ngd.get_approxDP(delta) +# #determine the next prob, sigma +# prob, sigma = func(eps) +# mech_cur = Mechanisms.SubsampledGaussian_mech(prob=prob, sigma=sigma) +# online_ngd = compose([online_ngd, mech_cur]) +# +# # The above is quite general and can be viewed as a privacy accountant + diff --git a/autodp/privacy_calibrator.py b/autodp/privacy_calibrator.py index a78eedb..0c40231 100644 --- a/autodp/privacy_calibrator.py +++ b/autodp/privacy_calibrator.py @@ -29,7 +29,8 @@ def subsample_epsdelta(eps,delta,prob): See the proof of (b) """ - + if prob == 0: + return 0,0 return np.log(1+prob*(np.exp(eps)-1)), prob*delta @@ -95,7 +96,7 @@ def func(x): count = count + 1 if count >=maxiter: # infeasible - return None + raise ValueError('Infeasible privacy parameters for given RDP function and parameter bounds.') root = brentq(func, a, b) diff --git a/autodp/rdp_acct.py b/autodp/rdp_acct.py index f705bf3..4ca5d8d 100644 --- a/autodp/rdp_acct.py +++ b/autodp/rdp_acct.py @@ -4,6 +4,11 @@ In particular it supports amplification of RDP by subsampling without replacement and the amplification of RDP by poisson sampling, but unfortunately not (yet) together. + + +* The RDP accountant API is deprecated. +* All functionality of an RDP accountant can be achieved via the Mechanism API. + """ @@ -68,58 +73,88 @@ def fast_subsampled_cgf_upperbound(func, mm, prob, deltas_local): return np.inf if mm == 1: return 0 - secondterm = np.minimum(np.minimum((2) * np.log(np.exp(func(np.inf)) - 1) - + np.minimum(func(2), np.log(4)), - np.log(2) + func(2)), - np.log(4) + 0.5 * deltas_local[int(2 * np.floor(2 / 2.0)) - 1] - + 0.5 * deltas_local[int(2 * np.ceil(2 / 2.0)) - 1] - ) + 2 * np.log(prob) + np.log(mm) + np.log(mm - 1) - np.log(2) - if mm == 2: - return utils.stable_logsumexp([0, secondterm]) + secondterm = 2 * np.log(prob) + + np.log(mm) + np.log(mm - 1) - np.log(2) \ + + np.mininum(np.log(4) + func(2.0) + np.log(1 - np.exp(-func(2.0))), + func(2.0) + np.mininum(np.log(2), + 2 * (eps_inf + np.log(1 - np.exp(-eps_inf))))) - # approximate the remaining terms using a geometric series - logratio1 = np.log(prob) + np.log(mm) + func(mm) - logratio2 = logratio1 + np.log(np.exp(func(np.inf)) - 1) - logratio = np.minimum(logratio1, logratio2) - if logratio1 > logratio2: - coeff = 1 - else: - coeff = 2 - - - if mm == 3: - return utils.stable_logsumexp([0, secondterm, np.log(coeff) + 3 * logratio]) - - # Calculate the sum of the geometric series starting from the third term. This is a total of mm-2 terms. - if logratio < 0: - geometric_series_bound = np.log(coeff) + 3 * logratio - np.log(1 - np.exp(logratio)) \ - + np.log(1 - np.exp((mm - 2) * logratio)) - elif logratio > 0: - geometric_series_bound = np.log(coeff) + 3 * logratio + (mm-2) * logratio - np.log(np.exp(logratio) - 1) - else: - geometric_series_bound = np.log(coeff) + np.log(mm - 2) + # secondterm = np.minimum(np.minimum((2) * np.log(np.exp(func(np.inf)) - 1) + # + np.minimum(func(2), np.log(4)), + # np.log(2) + func(2)), + # np.log(4) + 0.5 * deltas_local[int(2 * np.floor(2 / 2.0)) - 1] + # + 0.5 * deltas_local[int(2 * np.ceil(2 / 2.0)) - 1] + # ) + 2 * np.log(prob) + np.log(mm) + np.log(mm - 1) - np.log(2) - # we will approximate using (1+h)^mm - logh1 = np.log(prob) + func(mm - 1) + if mm == 2: + return utils.stable_logsumexp([0, secondterm]) - logh2 = logh1 + np.log(np.exp(func(np.inf)) - 1) + # approximate the remaining terms using a geometric series or binomial series - binomial_series_bound1 = np.log(2) + mm * utils.stable_logsumexp_two(0, logh1) - binomial_series_bound2 = mm * utils.stable_logsumexp_two(0, logh2) + log_exp_eps_minus_one = func(np.inf) + np.log(1 - np.exp(-func(np.inf))) - tmpsign, binomial_series_bound1 \ - = utils.stable_sum_signed(True, binomial_series_bound1, False, np.log(2) - + utils.stable_logsumexp([0, logh1 + np.log(mm), 2 * logh1 + np.log(mm) - + np.log(mm - 1) - np.log(2)])) - tmpsign, binomial_series_bound2 \ - = utils.stable_sum_signed(True, binomial_series_bound2, False, - utils.stable_logsumexp([0, logh2 + np.log(mm), 2 * logh2 + np.log(mm) - + np.log(mm - 1) - np.log(2)])) - remainder = np.min([geometric_series_bound, binomial_series_bound1, binomial_series_bound2]) + if mm == 3: + return utils.stable_logsumexp([0, secondterm, (3 * (np.log(prob) + np.log(mm)) + + 2*func(mm) + + np.minumum(np.log(2), + 3 * log_exp_eps_minus_one))]) - return utils.stable_logsumexp([0, secondterm, remainder]) + logratio1 = np.log(prob) + np.log(mm) + func(mm) + logratio2 = logratio1 + log_exp_eps_minus_one + + s, mag = utils.stable_log_diff_exp(1,logratio1) + s, mag2 = utils.stable_log_diff_exp(1, (mm-3)*logratio1) + remaining_terms1 = (np.log(2) + 3 * (np.log(prob) + np.log(mm)) + 2*func(mm) + + mag2 - mag) + + s, mag = utils.stable_log_diff_exp(1,logratio2) + s, mag2 = utils.stable_log_diff_exp(1, (mm-3)*logratio2) + + remaining_terms2 = (3 * (np.log(prob) + np.log(mm) + log_exp_eps_minus_one) + 2*func(mm) + + mag2 - mag) + + return utils.stable_logsumexp([0, secondterm, np.minimum(remaining_terms1,remaining_terms2)]) + + # logratio = np.minimum(logratio1, logratio2) + # if logratio1 > logratio2: + # coeff = 1 + # else: + # coeff = 2 + # + # + # if mm == 3: + # return utils.stable_logsumexp([0, secondterm, np.log(coeff) + 3 * logratio]) + # + # # Calculate the sum of the geometric series starting from the third term. This is a total of mm-2 terms. + # if logratio < 0: + # geometric_series_bound = np.log(coeff) + 3 * logratio - np.log(1 - np.exp(logratio)) \ + # + np.log(1 - np.exp((mm - 2) * logratio)) + # elif logratio > 0: + # geometric_series_bound = np.log(coeff) + 3 * logratio + (mm-2) * logratio - np.log(np.exp(logratio) - 1) + # else: + # geometric_series_bound = np.log(coeff) + np.log(mm - 2) + # + # # we will approximate using (1+h)^mm + # logh1 = np.log(prob) + func(mm - 1) + # + # logh2 = logh1 + np.log(np.exp(func(np.inf)) - 1) + # + # binomial_series_bound1 = np.log(2) + mm * utils.stable_logsumexp_two(0, logh1) + # binomial_series_bound2 = mm * utils.stable_logsumexp_two(0, logh2) + # + # tmpsign, binomial_series_bound1 \ + # = utils.stable_sum_signed(True, binomial_series_bound1, False, np.log(2) + # + utils.stable_logsumexp([0, logh1 + np.log(mm), 2 * logh1 + np.log(mm) + # + np.log(mm - 1) - np.log(2)])) + # tmpsign, binomial_series_bound2 \ + # = utils.stable_sum_signed(True, binomial_series_bound2, False, + # utils.stable_logsumexp([0, logh2 + np.log(mm), 2 * logh2 + np.log(mm) + # + np.log(mm - 1) - np.log(2)])) + # + # remainder = np.min([geometric_series_bound, binomial_series_bound1, binomial_series_bound2]) + # + # return utils.stable_logsumexp([0, secondterm, remainder]) @@ -211,7 +246,7 @@ def __init__(self, m=100, tol=0.1, m_max=500, m_lin_max=10000, approx = False, v self.m_max = m_max # An upper bound of the quadratic dependence self.m_lin_max = m_lin_max # An upper bound of the linear dependence. self.verbose = verbose - self.approx = approx + self.approx = approx # If true, use the fast k-term approximation self.lambs = np.linspace(1, self.m, self.m).astype(int) # Corresponds to \alpha = 2,3,4,5,.... for RDP self.alphas = np.linspace(1, self.m, self.m).astype(int) @@ -391,7 +426,7 @@ def fun_int(i): # the input is RDP's \alpha in integer if results.success: return results.fun else: - return None + raise RuntimeError(f"Optimal RDP order not found: {results.message}") #return fun(bestint) if bestint == 0: @@ -410,7 +445,7 @@ def fun_int(i): # the input is RDP's \alpha in integer # There are cases when certain \delta is not feasible. # For example, let p and q be uniform the privacy R.V. is either 0 or \infty and unless all \infty # events are taken cared of by \delta, \epsilon cannot be < \infty - return -1 + return np.inf def compose_mechanism(self, func, coeff=1.0): self.flag = False @@ -440,8 +475,24 @@ def compose_mechanism(self, func, coeff=1.0): self.RDP_inf += func(np.inf) * coeff #795010 #imple 100 - def compose_subsampled_mechanism(self, func, prob, coeff=1.0): - # This function is for subsample without replacements. + def compose_subsampled_mechanism(self, func, prob, coeff=1.0, improved_bound_flag=False): + """ + # This function is for subsample without replacements + :param func: RDP function of the mechanism before amplification by sampling + :param prob: proportion of the data to sample + :param coeff: number of times the subsampled mechanism is being composed. + :param improved_bound_flag: + - If True, then it uses Theorem 27 of https://arxiv.org/pdf/1808.00087.pdf + - If False (default value), it uses Theorem 9 of https://arxiv.org/pdf/1808.00087.pdf + To qualify for the improved bound, the mechanism needs to have a pair of neighboring + datasets that is worst for all Renyi-divergence and Pearson-Vajda divergence; + Also, the RDP bound needs to be tight (see Definition 26 from the same paper). + Gaussian mechanism, Laplace mechanism and many others satisfy this condition. + + :return: nothing (updates to the RDP accountant's attribute) + """ + + # (find a random subset of proportion prob) self.flag = False self.flag_subsample = True if (func, prob) in self.idxhash: @@ -451,52 +502,86 @@ def compose_subsampled_mechanism(self, func, prob, coeff=1.0): # also update the integer CGFs self.RDPs_int += self.cache[(func, prob)] * coeff else: - def cgf(x): return x * func(x+1) - # we need forward differences of thpe exp(cgf) - # The following line is the numericall y stable way of implementing it. - # The output is in polar form with logarithmic magnitude - deltas, signs_deltas = utils.get_forward_diffs(cgf,self.m) - #deltas1, signs_deltas1 = get_forward_diffs_direct(func, self.m) + if not improved_bound_flag: + def subsample_func_int(x): + # output the cgf of the subsampled mechanism + mm = int(x) + eps_inf = func(np.inf) + + moments_two = 2 * np.log(prob) + utils.logcomb(mm,2) \ + + np.minimum(np.log(4) + func(2.0) + np.log(1-np.exp(-func(2.0))), + func(2.0) + np.minimum(np.log(2), + 2 * (eps_inf+np.log(1-np.exp(-eps_inf))))) + moment_bound = lambda j: np.minimum(j * (eps_inf + np.log(1-np.exp(-eps_inf))), + np.log(2)) + cgf(j - 1) \ + + j * np.log(prob) + utils.logcomb(mm, j) + moments = [moment_bound(j) for j in range(3, mm + 1, 1)] + return np.minimum((x-1)*func(x), utils.stable_logsumexp([0,moments_two] + moments)) + else: + # we need forward differences of exp(cgf) + # The following line is the numerically stable way of implementing it. + # The output is in polar form with logarithmic magnitude + deltas, signs_deltas = utils.get_forward_diffs(cgf, self.m) - #tmp = deltas-deltas1 + #deltas1, signs_deltas1 = get_forward_diffs_direct(func, self.m) - self.deltas_cache[(func,prob)] = [deltas,signs_deltas] + #tmp = deltas-deltas1 - def subsample_func_int(x): - # This function evaluates teh CGF at alpha = x, i.e., lamb = x- 1 - deltas_local, signs_deltas_local = self.deltas_cache[(func,prob)] - if np.isinf(func(x)): - return np.inf + self.deltas_cache[(func,prob)] = [deltas,signs_deltas] - mm = int(x) + def subsample_func_int(x): + # This function evaluates teh CGF at alpha = x, i.e., lamb = x- 1 + deltas_local, signs_deltas_local = self.deltas_cache[(func,prob)] + if np.isinf(func(x)): + return np.inf + + mm = int(x) + eps_inf = func(np.inf) + + moments_two = 2 * np.log(prob) + utils.logcomb(mm, 2) \ + + np.minimum( + np.log(4) + func(2.0) + np.log(1 - np.exp(-func(2.0))), + func(2.0) + np.minimum(np.log(2), + 2 * (eps_inf + np.log(1 - np.exp(-eps_inf))))) + + moment_bound = lambda j: np.minimum(np.log(4) + 0.5*deltas_local[int(2*np.floor(j/2.0))-1] + + 0.5*deltas_local[int(2*np.ceil(j/2.0))-1], + np.minimum(j * (eps_inf + np.log(1 - np.exp(-eps_inf))), + np.log(2)) + + cgf(j - 1)) \ + + j * np.log(prob) + utils.logcomb(mm, j) + + moment_bound_linear = lambda j: np.minimum(j * (eps_inf + np.log(1-np.exp(-eps_inf))), + np.log(2)) + cgf(j - 1) \ + + j * np.log(prob) + utils.logcomb(mm, j) - fastupperbound = fast_subsampled_cgf_upperbound(func, mm, prob, deltas_local) - fastupperbound2 = general_upperbound(func, mm, prob) - if self.approx ==True: - if fastupperbound2 <0: - print('general rdp is negative',x) - return fastupperbound2 + fastupperbound = fast_subsampled_cgf_upperbound(func, mm, prob, deltas_local) - if mm <= self.alphas[-1]: # compute the bound exactly. Requires book keeping of O(x^2) + if mm <= self.alphas[-1]: # compute the bound exactly. Requires book keeping of O(x^2) + # + # moments = [ np.minimum(np.minimum((j)*np.log(np.exp(func(np.inf))-1) + np.minimum(cgf(j-1),np.log(4)), + # np.log(2) + cgf(j-1)), + # np.log(4) + 0.5*deltas_local[int(2*np.floor(j/2.0))-1] + # + 0.5*deltas_local[int(2*np.ceil(j/2.0))-1]) + j*np.log(prob) + # +self.logBinomC[int(mm), j] for j in range(2,int(mm+1),1)] + moments = [moment_bound(j) for j in range(3, mm + 1, 1)] - moments = [ np.minimum(np.minimum((j)*np.log(np.exp(func(np.inf))-1) + np.minimum(cgf(j-1),np.log(4)), - np.log(2) + cgf(j-1)), - np.log(4) + 0.5*deltas_local[int(2*np.floor(j/2.0))-1] - + 0.5*deltas_local[int(2*np.ceil(j/2.0))-1]) + j*np.log(prob) - +self.logBinomC[int(mm), j] for j in range(2,int(mm+1),1)] + return np.minimum(fastupperbound, utils.stable_logsumexp([0, moments_two] + moments)) + elif mm <= self.m_lin_max: # compute the bound with stirling approximation. Everything is O(x) now. + # moment_bound = lambda j: np.minimum(j * np.log(np.exp(func(np.inf)) - 1) + # + np.minimum(cgf(j - 1), np.log(4)), np.log(2) + # + cgf(j - 1)) + j * np.log(prob) + utils.logcomb(mm, j) + # moments = [moment_bound(j) for j in range(2,mm+1,1)] - return np.minimum(fastupperbound, utils.stable_logsumexp([0]+moments)) - elif mm <= self.m_lin_max: # compute the bound with stirling approximation. Everything is O(x) now. - moment_bound = lambda j: np.minimum(j * np.log(np.exp(func(np.inf)) - 1) - + np.minimum(cgf(j - 1), np.log(4)), np.log(2) - + cgf(j - 1)) + j * np.log(prob) + utils.logcomb(mm, j) - moments = [moment_bound(j) for j in range(2,mm+1,1)] - return np.minimum(fastupperbound, utils.stable_logsumexp([0]+ moments)) - else: # Compute the O(1) upper bound - return fastupperbound + + moments = [moment_bound_linear(j) for j in range(3, mm + 1, 1)] + + return np.minimum(fastupperbound, utils.stable_logsumexp([0, moments_two] + moments)) + else: # Compute the O(1) upper bound + return fastupperbound @@ -517,9 +602,9 @@ def subsample_func(x): return np.minimum(epsinf, subsample_func_int(x) / (x-1) ) xc = math.ceil(x) xf = math.floor(x) - return np.minimum( - epsinf, - ((x-xf)*subsample_func_int(xc) + (1-(x-xf))*subsample_func_int(xf)) / (x-1) + return np.min( + [epsinf,func(x), + ((x-xf)*subsample_func_int(xc) + (1-(x-xf))*subsample_func_int(xf)) / (x-1)] ) @@ -546,38 +631,8 @@ def subsample_func(x): eps, delta = subsample_epsdelta(func(np.inf), 0, prob) self.RDP_inf += eps * coeff - - # mm = np.max(self.alphas) - # - # jvec = np.arange(2, mm+1) # - # logterm3plus = np.zeros_like(results) - # for j in jvec: - # logterm3plus[j-2] = (np.minimum(np.minimum(j * np.log(np.exp(func(np.inf)) - 1) - # + np.minimum(np.log(4),cgf(j-1)), np.log(2) + cgf(j-1)), - # np.log(4) + 0.5 * deltas[int(2 * np.floor(j / 2.0))-1] - # + 0.5 * deltas[int(2 * np.ceil(j / 2.0))-1]) - # + j * np.log(prob)) - # - # for alpha in range(2, mm+1): - # if np.isinf(logterm3plus[alpha-1]): - # results[alpha-1] = np.inf - # else: - # tmp = utils.stable_logsumexp(logterm3plus[0:alpha-1] + self.logBinomC[alpha, 2:(alpha+1)]) - # results[alpha-1] = utils.stable_logsumexp_two(0, tmp) / (1.0*alpha-1) - # - # results[0] = results[1] # Provide the trivial upper bound of RDP at alpha = 1 --- the KL privacy. - # - # self.cache[(func,prob)] = results # save in cache - # self.RDPs_int += results - # - # # For debugging: The following 'results1' should be the same as 'results' above. - # # results1 = np.zeros_like(self.RDPs_int, float) - # # for j in range(self.m): - # # results1[j] = subsample_func(j+1) - # - # eps, delta = subsample_epsdelta(func(np.inf), 0, prob) - # self.RDP_inf += eps - + def compose_subsampled_mechanisms_lowerbound(self, func, prob, coeff=1.0): + self.compose_poisson_subsampled_mechanisms(func, prob, coeff=coeff) def compose_poisson_subsampled_mechanisms(self, func, prob, coeff=1.0): # This function implements the lower bound for subsampled RDP. @@ -633,6 +688,8 @@ def subsample_func(x): # linear interpolation upper bound return np.inf if prob == 1.0: return func(x) + if prob == 0: + return 0 epsinf, tmp = subsample_epsdelta(func(np.inf),0,prob) @@ -731,6 +788,10 @@ def subsample_func_int(x): def subsample_func(x): # linear interpolation upper bound + if prob == 0: + return 0 + if prob == 1.0: + return func(x) epsinf, tmp = subsample_epsdelta(func(np.inf),0,prob) if np.isinf(x): diff --git a/autodp/rdp_bank.py b/autodp/rdp_bank.py index 2516319..429f366 100644 --- a/autodp/rdp_bank.py +++ b/autodp/rdp_bank.py @@ -291,6 +291,89 @@ def extrapolate(a, b): (np.sum(np.log(sigma)) - extrapolate(np.sum(np.log(sigma2)), np.sum(np.log(sigma1))))) + +def RDP_svt_laplace(params, alpha): + """ + Laplace-SVT (via RDP), used in NeurIPS-20 + :param b is the noise scale for rho + :param params: + :param alpha: + :return: + """ + b = params['b'] + k = params['k'] # the algorithm stops either k is achieved or c is achieved + c = max(params['c'], 1) + + alpha = 1.0 * alpha + if alpha <= 1: + eps_1 = (1 / b + np.exp(-1 / b) - 1) + elif np.isinf(alpha): + eps_1 = 1 / b + else: # alpha > 1 + eps_1 = utils.stable_logsumexp_two((alpha - 1.0) / b + np.log(alpha / (2.0 * alpha - 1)), + -1.0 * alpha / b + np.log((alpha - 1.0) / (2.0 * alpha - 1))) / (alpha - 1) + + eps_2 = 1 / b # infinity rdp on nu + c_log_n_c = c * np.log(k / c) + tilde_eps = eps_2 * (c + 1) # eps_infinity + ret_rdp = min(c * eps_2 + eps_1, c_log_n_c * 1.0 / (alpha - 1) + eps_1 * (c + 1)) + ret_rdp = min(ret_rdp, 0.5 * alpha * tilde_eps ** 2) + if np.isinf(alpha) or alpha == 1: + return ret_rdp + # The following is sinh-based method + tilde_eps = eps_2 * (c + 1) + cdp_bound = np.sinh(alpha * tilde_eps) - np.sinh((alpha - 1) * tilde_eps) + cdp_bound = cdp_bound / np.sinh(tilde_eps) + cdp_bound = 1.0 / (alpha - 1) * np.log(cdp_bound) + return min(ret_rdp, cdp_bound) + + +def RDP_gaussian_svt_cgreater1(params, alpha): + """ + This is for gaussian-svt with c>1 + k is the maximum length before svt stops + :param params: + :param alpha: + :return: + """ + sigma = params['sigma'] + c = max(params['c'], 1) + k = params['k'] # the algorithm stops either k is achieved or c is achieved + rdp_rho = 0.5 / (sigma ** 2) * alpha + c_log_n_c = c * np.log(k / c) + ret_rdp = c_log_n_c * 1.0 / (alpha - 1) + rdp_rho * (c + 1) + return ret_rdp + + +def RDP_gaussian_svt_c1(params, alpha): + """ + This is for gaussian-svt with c=1 + k is the maximum length before svt stops + :param params: + :param alpha: + :return: + """ + sigma = params['sigma'] + k = params['k'] + margin = params['margin'] + c = 1 + + + rdp_rho = 0.5 / (sigma ** 2) * alpha + + ret_rdp = np.log(k) / (alpha - 1) + rdp_rho * 2 + if alpha == 1: + return ret_rdp * c + ################ Implement corollary 15 in NeurIPS-20 + inside_part = np.log(2 * np.sqrt(3) * math.pi * (1 + 9 * margin ** 2 / (sigma ** 2))) + moment_term = utils.stable_logsumexp_two(0, inside_part + margin ** 2 * 1.0 / (sigma ** 2)) + moment_term = moment_term / (2.0 * (alpha - 1)) + moment_based = moment_term + rdp_rho * 2 + + return min(moment_based, ret_rdp) * c + + + def RDP_pureDP(params,alpha): """ This function generically converts pure DP to Renyi DP. diff --git a/autodp/transformer_zoo.py b/autodp/transformer_zoo.py new file mode 100644 index 0000000..cb014c5 --- /dev/null +++ b/autodp/transformer_zoo.py @@ -0,0 +1,208 @@ +# Example of a specific transformer that outputs the composition + + +from autodp.autodp_core import Mechanism, Transformer +import math +import numpy as np + +from autodp import mechanism_zoo, rdp_acct + + +# The generic composition class +class Composition(Transformer): + """ Composition is a transformer that takes a list of Mechanisms and number of times they appear, + and output a Mechanism that represents the composed mechanism""" + def __init__(self): + Transformer.__init__(self) + self.name = 'Composition' + + # Update the function that is callable + self.transform = self.compose + + def compose(self, mechanism_list, coeff_list, RDP_compose_only=True): + # Make sure that the mechanism has a unique list + # for example, if there are two Gaussian mechanism with two different sigmas, call it + # Gaussian1, and Gaussian2 + + # if RDP_compose_only is true, we use only RDP composition. + + + newmech = Mechanism() + + # update the functions + def newrdp(x): + return sum([c * mech.RenyiDP(x) for (mech, c) in zip(mechanism_list, coeff_list)]) + newmech.propagate_updates(newrdp, 'RDP') + + # TODO: the fDP_based_conversion sometimes fails due to undefined RDP with alpha < 1 + + newmech.eps_pureDP = sum([c * mech.eps_pureDP for (mech, c) + in zip(mechanism_list, coeff_list)]) + newmech.delta0 = max([mech.delta0 for (mech, c) + in zip(mechanism_list, coeff_list)]) + + if not RDP_compose_only: # Also do KOV-composition while optimizing over \delta parameter + # TODO: Also implement the KOV-composition here and propagate the updates + # TODO: How do we generically compose eps(delta) functions? + # TODO: How do we generically compose approximate RDP functions + # TODO: How do we generically compose fDP? (efficiently) + pass + + # Other book keeping + newmech.name = self.update_name(mechanism_list, coeff_list) + # keep track of all parameters of the composed mechanisms + newmech.params = self.update_params(mechanism_list) + + return newmech + + def update_name(self,mechanism_list, coeff_list): + separator = ', ' + s = separator.join([mech.name + ': ' + str(c) for (mech, c) + in zip(mechanism_list, coeff_list)]) + + return 'Compose:{'+ s +'}' + + def update_params(self, mechanism_list): + params = {} + for mech in mechanism_list: + params_cur = {mech.name+':'+k: v for k,v in mech.params.items()} + params.update(params_cur) + return params + + + +# composition of only Gaussian mechanisms +class ComposeGaussian(Composition): + """ CompositionGaussian is a specialized composation function of ONLY Guassian mechanisms + output a Mechanism that represents the composed mechanism""" + def __init__(self): + Composition.__init__(self) + self.name = 'ComposeGaussian' + + def compose(self, mechanism_list, coeff_list): + # Make sure that the list contains only Gaussian mechanisms + for mech in mechanism_list: + assert(isinstance(mech, mechanism_zoo.GaussianMechanism) + or isinstance(mech, mechanism_zoo.ExactGaussianMechanism)) + + # Directly compose the distribution of privacy-loss random variables. + # Sum of Gaussians are gaussians + tmp = 0 + for mech, coeff in zip(mechanism_list,coeff_list): + tmp += mech.params['sigma']**(-2)*coeff + + newmech = mechanism_zoo.ExactGaussianMechanism(sigma=math.sqrt(1/tmp)) + + # Other book keeping + newmech.name = self.update_name(mechanism_list, coeff_list) + + # keep track of all parameters of the composed mechanisms + newmech.params = self.update_params(mechanism_list) + + return newmech + + + +class AmplificationBySampling(Transformer): + def __init__(self, PoissonSampling=True): + # By default, poisson sampling is used: sample a dataset by selecting each data point iid + # If PoissonSampling is set to False, then it chooses a random subset with size prob * n + Transformer.__init__(self) + if PoissonSampling: + self.name = 'PoissonSample' + else: + self.name = 'Subsample' + self.PoissonSampling = PoissonSampling + self.unary_operator = True + self.preprocessing = True # Sampling happen before the mechanism is applied + + # Update the function that is callable + self.transform = self.amplify + + def amplify(self, mechanism, prob, improved_bound_flag=False): + # If you know that your mechanism + # - (for PoissonSampling) satisfies the the conditions in Theorem 8 of http://proceedings.mlr.press/v97/zhu19c/zhu19c.pdf + # - or (for subsampling) satisfies the conditions of Theorem 27 of https://arxiv.org/pdf/1808.00087.pdf + # then you may switch general_bound_flag to False to get a tighter bound. + + # Else, for all mechanisms with RDP bounds, the general upper bounds are used by default. + + newmech = Mechanism() + + # privacy amplification via approx-dp + + + + # Amplification of RDP + # propagate to approxDP as well. + + if self.PoissonSampling: + assert not mechanism.replace_one, "mechanism's replace_one notion of DP is " \ + "incompatible with Privacy Amplification " \ + "by Poisson sampling" + # check that the input mechanism uses the standard add-or-remove notion of DP. + # If not, there actually isn't a way to convert it from replace-one notation, + # unless a "dummy" user exists in the space. + newmech.replace_one = False + + else: # if we want subsampled DP + assert mechanism.replace_one, "mechanism's add-remove notion of DP is " \ + "incompatible with Privacy Amplification " \ + "by subsampling without replacements" + # TODO: implement a transformer that convert add/remove to replace_one notion of DP. + newmech.replace_one = True + + if prob == 0: + new_approxDP = lambda delta:0 + else: + new_approxDP = lambda delta: np.log(1 + prob*(np.exp(mechanism.approxDP(delta/prob))-1)) + newmech.approxDP = new_approxDP + + acct = rdp_acct.anaRDPacct() + if self.PoissonSampling: + if improved_bound_flag: + acct.compose_poisson_subsampled_mechanisms(mechanism.RenyiDP,prob) + else: + acct.compose_poisson_subsampled_mechanisms1(mechanism.RenyiDP,prob) + else: # subsampling + if improved_bound_flag: + acct.compose_subsampled_mechanism(mechanism.RenyiDP, prob, improved_bound_flag=True) + else: + acct.compose_subsampled_mechanism(mechanism.RenyiDP, prob) + + + acct.build_zeroth_oracle() + new_rdp = acct.evalRDP + newmech.propagate_updates(new_rdp,'RDP') + + #TODO: Implement the amplification of f-DP + # propagate to approxDP, or simply get the f-DP from approximate-DP. + + + # book keeping + key = self.name + '_' + str(prob) + num = 0 + newname = self.name + + # the following handles the case when key is already in the params + while key in mechanism.params: + num = num+1 + newname = self.name+str(num) + key = newname + '_' + str(prob) + + newmech.name = newname +':'+mechanism.name + newmech.params = mechanism.params + new_params = {newname:prob} + newmech.params.update(new_params) + + return newmech + + + + +# TODO: implement other transformers: +# - amplification by shuffling +# - parallel composition +# - group composition +# - private selection of private candidates +# - amplification by overwhelmingly large-probability event. \ No newline at end of file diff --git a/autodp/utils.py b/autodp/utils.py index 82074f9..44c5788 100644 --- a/autodp/utils.py +++ b/autodp/utils.py @@ -1,5 +1,6 @@ import numpy as np from scipy.special import gammaln, comb +from scipy.stats import norm import math def stable_logsumexp(x): @@ -28,6 +29,22 @@ def stable_log_diff_exp(x, y): return s, mag +def stable_log_sinh(x): + assert(x >= 0) + s, mag = stable_log_diff_exp(x, -x) + return np.log(0.5) + mag + +def _log1mexp(x): + """ from pate Numerically stable computation of log(1-exp(x)).""" + if x < -1: + return math.log1p(-math.exp(x)) + elif x < 0: + return math.log(-math.expm1(x)) + elif x == 0: + return -np.inf + else: + raise ValueError("Argument must be non-positive.") + def stable_sum_signed(xs, x, ys, y): # x and y are log abs, xs,ys are signs @@ -49,7 +66,6 @@ def stable_inplace_diff_in_log(vec, signs, n=-1): `vec` and `signs` jointly describe a vector of real numbers' sign and abs in log scale. Output: The first n-1 dimension of vec and signs will store the log-abs and sign of the difference. - """ # # And the first n-1 dimension of signs with the sign of the differences. @@ -71,6 +87,22 @@ def stable_inplace_diff_in_log(vec, signs, n=-1): vec[j] = stable_logsumexp_two(vec[j], vec[j + 1]) signs[j] = signs[j + 1] +def stable_norm_ppf_one_minus_x(logx): + """ + This function compute the normal inverse CDF at 1-x by taking logx as an input. + Numerical stability has been taken into account with two levels of approximations. + :param logx: + :return: ppf(1-e^logx) + """ + if logx < -30: + # asymptotic approximation + norm_ppf_one_minus_x = np.sqrt(-2 * logx - np.log(-2*logx) - np.log(2*np.pi)) + # # if logx < -10: + # # norm_ppf_one_minus_x = norm.ppf(-logx - logx ** 2 / 2 - logx ** 3 / 6) + else: + norm_ppf_one_minus_x = norm.ppf(1 - np.exp(logx)) + return norm_ppf_one_minus_x + def get_forward_diffs(fun, n): """ This is the key function for computing up to nth order forward difference evaluated at 0""" @@ -96,6 +128,22 @@ def get_forward_diffs(fun, n): +def subsample_epsdelta(eps,delta,prob): + """ + :param eps: privacy loss eps of the base mechanism + :param delta: privacy loss delta of the base mechanism + :param prob: subsampling probability + :return: Amplified eps and delta + This result applies to both subsampling with replacement and Poisson subsampling. + The result for Poisson subsmapling is due to Theorem 1 of : + Li, Ninghui, Qardaji, Wahbeh, and Su, Dong. On sampling, anonymization, and differential privacy or, + k-anonymization meets differential privacy + The result for Subsampling with replacement is due to: + Jon Ullman's lecture notes: http://www.ccs.neu.edu/home/jullman/PrivacyS17/HW1sol.pdf + See the proof of (b) + """ + + return np.log(1+prob*(np.exp(eps)-1)), prob*delta @@ -164,6 +212,7 @@ def get_forward_diffs_direct(fun,n): def logcomb(n, k): + return (gammaln(n+1) - gammaln(n-k+1) - gammaln(k+1)) def get_binom_coeffs(sz): @@ -222,4 +271,4 @@ def RDP_linear_interpolation(func,x): return np.minimum( epsinf, ((x - xf) * (xc-1)* func(xc) + (1 - (x - xf)) * (xf-1)*func(xf)) / (x - 1) - ) \ No newline at end of file + ) diff --git a/example/example_amplification_by_sampling.py b/example/example_amplification_by_sampling.py new file mode 100644 index 0000000..96f7319 --- /dev/null +++ b/example/example_amplification_by_sampling.py @@ -0,0 +1,112 @@ + +from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism +from autodp.transformer_zoo import Composition, AmplificationBySampling +import matplotlib.pyplot as plt + +sigma1 = 5.0 +sigma2 = 8.0 + +gm1 = ExactGaussianMechanism(sigma1,name='GM1') +gm2 = ExactGaussianMechanism(sigma2,name='GM2') +SVT = PureDP_Mechanism(eps=0.1,name='SVT') + +# run gm1 for 3 rounds +# run gm2 for 5 times +# run SVT for once + +# compose them with the transformation: compose. +compose = Composition() + +poisson_sample = AmplificationBySampling(PoissonSampling=True) +subsample = AmplificationBySampling(PoissonSampling=False) + +prob = 0.1 +coeffs = [30,50,10] + +composed_mech = compose([gm1, gm2, SVT], coeffs) + +composed_poissonsampled_mech = compose([poisson_sample(gm1,prob), + poisson_sample(gm2,prob), + poisson_sample(SVT,prob)], + coeffs) + +composed_poissonsampled_mech1 = compose([poisson_sample(gm1,prob,improved_bound_flag=True), + poisson_sample(gm2,prob,improved_bound_flag=True), + poisson_sample(SVT,prob,improved_bound_flag=True)], + coeffs) + + + +# Now let's do subsampling. First we need to use replace-one version of the base mechanisms. +gm1.replace_one = True +gm2.replace_one = True +SVT.replace_one = True + +composed_subsampled_mech = compose([subsample(gm1,prob), + subsample(gm2,prob), + subsample(SVT,prob)], + coeffs) + +composed_subsampled_mech1 = compose([subsample(gm1,prob,improved_bound_flag=True), + subsample(gm2,prob,improved_bound_flag=True), + subsample(SVT,prob,improved_bound_flag=True)], + coeffs) + +# Query for eps given delta +delta1 = 1e-6 +eps1 = composed_mech.get_approxDP(delta1) + +delta2 = 1e-4 +eps2 = composed_mech.get_approxDP(delta2) + + +# Get name of the composed object, a structured description of the mechanism generated automatically +print('---------------------------------------------------') +print('Mechanism name is \"', composed_mech.name,'\"') +print('Parameters are: ',composed_mech.params) +print('epsilon(delta) = ', eps1, ', at delta = ', delta1) +print('epsilon(delta) = ', eps2, ', at delta = ', delta2) + + +eps1a = composed_poissonsampled_mech.get_approxDP(delta1) +eps2a = composed_poissonsampled_mech.get_approxDP(delta2) + +eps1aa = composed_poissonsampled_mech1.get_approxDP(delta1) +eps2aa = composed_poissonsampled_mech1.get_approxDP(delta2) + +# Get name of the composed object, a structured description of the mechanism generated automatically +print('---------------------------------------------------') +print('Mechanism name is \"', composed_poissonsampled_mech.name,'\"') +print('Parameters are: ',composed_poissonsampled_mech.params) +print('epsilon(delta) = ', eps1a, ', at delta = ', delta1) +print('epsilon(delta) = ', eps2a, ', at delta = ', delta2) +print('------- If qualified for the improved bounds --------') +print('epsilon(delta) = ', eps1aa, ', at delta = ', delta1) +print('epsilon(delta) = ', eps2aa, ', at delta = ', delta2) + + +eps1b = composed_subsampled_mech.get_approxDP(delta1) +eps2b = composed_subsampled_mech.get_approxDP(delta2) + +eps1bb = composed_subsampled_mech1.get_approxDP(delta1) +eps2bb = composed_subsampled_mech1.get_approxDP(delta2) + +# Get name of the composed object, a structured description of the mechanism generated automatically +print('---------------------------------------------------') +print('Mechanism name is \"', composed_subsampled_mech.name,'\"') +print('Parameters are: ',composed_subsampled_mech.params) +print('epsilon(delta) = ', eps1b, ', at delta = ', delta1) +print('epsilon(delta) = ', eps2b, ', at delta = ', delta2) +print('------- If qualified for the improved bounds --------') +print('epsilon(delta) = ', eps1bb, ', at delta = ', delta1) +print('epsilon(delta) = ', eps2bb, ', at delta = ', delta2) + +# # Get hypothesis testing interpretation so we can directly plot it +# fpr_list, fnr_list = composed_mech.plot_fDP() +# +# plt.figure(figsize = (6,6)) +# plt.plot(fpr_list,fnr_list) +# plt.xlabel('Type I error') +# plt.ylabel('Type II error') +# plt.show() + diff --git a/example/example_calibrator.py b/example/example_calibrator.py new file mode 100644 index 0000000..a2a98b8 --- /dev/null +++ b/example/example_calibrator.py @@ -0,0 +1,66 @@ +import sys +import os +from autodp.calibrator_zoo import eps_delta_calibrator,generalized_eps_delta_calibrator, ana_gaussian_calibrator +from autodp import rdp_bank +from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism,SubsampleGaussianMechanism, GaussianMechanism, ComposedGaussianMechanism, LaplaceMechanism +from autodp.transformer_zoo import Composition, AmplificationBySampling + +""" +Try calibrating noise to privacy budgets. +Cases 1: single parameter, no subsample or composition + +""" + +calibrate = eps_delta_calibrator() +ana_calibrate = ana_gaussian_calibrator() +eps = 0.1 +delta = 1e-6 + +mech1 = calibrate(ExactGaussianMechanism,eps,delta,[0,100],name='GM') +mech2 = ana_calibrate(ExactGaussianMechanism, eps, delta, name='Ana_GM') +print(mech1.name, mech1.params, mech1.get_approxDP(delta)) +print(mech2.name, mech2.params, mech2.get_approxDP(delta)) + + +""" + +Cases 2: Test calibration with Gaussian under composition, coeff is the number of composition +We now have multiple parameters --- params['coeff'] and params['sigma']. +The coeff is fixed and the calibrator optimize over sigma. We use para_name to denote the parameter that we want to optimize. + +""" +coeff = 20 +general_calibrate = generalized_eps_delta_calibrator() +params = {} +params['sigma'] = None +params['coeff'] = 20 + +mech3 = general_calibrate(ComposedGaussianMechanism, eps, delta, [0,1000],params=params,para_name='sigma', name='Composed_Gaussian') +print(mech3.name, mech3.params, mech3.get_approxDP(delta)) + +""" +Cases 3: Test calibration with SubsampledGaussian +We now have three parameters --- params['coeff'], params['prob'] and params['sigma']. +The coeff and prob are fixed and the calibrator optimize over sigma. We use para_name to denote the parameter that we want to optimize. + +""" +params['prob'] = 0.01 +mech4 = general_calibrate(SubsampleGaussianMechanism, eps, delta, [0,1000],params=params,para_name='sigma', name='Subsampled_Gaussian') +print(mech4.name, mech4.params, mech4.get_approxDP(delta)) + + + + +""" + +Cases 4: single parameter for Laplace mechanism, no subsample or composition + +""" + +calibrate = generalized_eps_delta_calibrator() + +eps = 0.1 +delta = 1e-6 +mech = calibrate(LaplaceMechanism,eps,delta,[0,100],name='Laplace') +print(mech.name, mech.params, mech.get_approxDP(delta)) + diff --git a/example/example_composition.py b/example/example_composition.py new file mode 100644 index 0000000..7b537d8 --- /dev/null +++ b/example/example_composition.py @@ -0,0 +1,44 @@ + +from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism +from autodp.transformer_zoo import Composition +import matplotlib.pyplot as plt + +sigma1 = 5.0 +sigma2 = 8.0 + +gm1 = ExactGaussianMechanism(sigma1,name='GM1') +gm2 = ExactGaussianMechanism(sigma2,name='GM2') +SVT = PureDP_Mechanism(eps=0.1,name='SVT') + +# run gm1 for 3 rounds +# run gm2 for 5 times +# run SVT for once + +# compose them with the transformation: compose. +compose = Composition() +composed_mech = compose([gm1, gm2, SVT], [3, 5, 1]) + +# Query for eps given delta +delta1 = 1e-6 +eps1 = composed_mech.get_approxDP(delta1) + +delta2 = 1e-4 +eps2 = composed_mech.get_approxDP(delta2) + + +# Get name of the composed object, a structured description of the mechanism generated automatically +print('Mechanism name is \"', composed_mech.name,'\"') +print('Parameters are: ',composed_mech.params) +print('epsilon(delta) = ', eps1, ', at delta = ', delta1) +print('epsilon(delta) = ', eps2, ', at delta = ', delta2) + + +# Get hypothesis testing interpretation so we can directly plot it +fpr_list, fnr_list = composed_mech.plot_fDP() + +plt.figure(figsize = (6,6)) +plt.plot(fpr_list,fnr_list) +plt.xlabel('Type I error') +plt.ylabel('Type II error') +plt.show() + diff --git a/example/example_fdp_vs_rdp.py b/example/example_fdp_vs_rdp.py new file mode 100644 index 0000000..05584e6 --- /dev/null +++ b/example/example_fdp_vs_rdp.py @@ -0,0 +1,69 @@ +import numpy as np +from autodp.mechanism_zoo import GaussianMechanism, PureDP_Mechanism + +# Example 1: Gaussian mechanism + +sigma = 2.0 + + +gm0 = GaussianMechanism(sigma,name='GM0',approxDP_off=True, use_basic_RDP_to_approxDP_conversion=True) +gm1 = GaussianMechanism(sigma,name='GM1',approxDP_off=True) +gm1b = GaussianMechanism(sigma,name='GM1b',approxDP_off=True, use_fDP_based_RDP_to_approxDP_conversion=True) +gm2 = GaussianMechanism(sigma,name='GM2',RDP_off=True) +gm3 = GaussianMechanism(sigma,name='GM3',RDP_off=True, approxDP_off=True, fdp_off=False) + + + +eps = np.sqrt(2)/sigma # Aligning the variance of the laplace mech and gaussian mech +laplace = PureDP_Mechanism(eps,name='Laplace') + +label_list = ['naive_RDP_conversion','BBGHS_RDP_conversion','Our new method', + 'exact_eps_delta_DP','exact_fdp',r'laplace mech ($b = \sqrt{2}/\sigma$)'] + + +import matplotlib.pyplot as plt + + + +fpr_list, fnr_list = gm0.plot_fDP() +fpr_list1, fnr_list1 = gm1.plot_fDP() +fpr_list1b, fnr_list1b = gm1b.plot_fDP() +fpr_list2, fnr_list2 = gm2.plot_fDP() +fpr_list3, fnr_list3 = gm3.plot_fDP() +fpr_list4, fnr_list4 = laplace.plot_fDP() + +plt.figure(figsize=(4,4)) +plt.plot(fpr_list,fnr_list) +plt.plot(fpr_list1,fnr_list1) +plt.plot(fpr_list1b,fnr_list1b) +plt.plot(fpr_list2, fnr_list2) +plt.plot(fpr_list3, fnr_list3,':') +plt.plot(fpr_list4, fnr_list4,'-.') +plt.legend(label_list) +plt.xlabel('Type I error') +plt.ylabel('Type II error') +plt.savefig('rdp2fdp.pdf') +plt.show() + + + +delta = 1e-3 + + +eps3 = gm3.approxDP(delta) +eps0 = gm0.approxDP(delta) +eps1 = gm1.approxDP(delta) +eps1b = gm1b.approxDP(delta) + +eps2 = gm2.approxDP(delta) + +eps4 = laplace.approxDP(delta) + +epsilons = [eps0,eps1,eps1b,eps2,eps3,eps4] + +print(epsilons) + +plt.bar(label_list,epsilons) +plt.xticks(rotation=45, ha="right") +plt.show() + diff --git a/example/example_gaussian.py b/example/example_gaussian.py new file mode 100644 index 0000000..fbf5b6a --- /dev/null +++ b/example/example_gaussian.py @@ -0,0 +1,56 @@ + +from autodp.mechanism_zoo import ExactGaussianMechanism +from autodp.transformer_zoo import Composition, ComposeGaussian +import matplotlib.pyplot as plt + +sigma1 = 5.0 +sigma2 = 8.0 + +gm1 = ExactGaussianMechanism(sigma1,name='GM1') +gm2 = ExactGaussianMechanism(sigma2,name='GM2') + +# run gm1 for 3 rounds +# run gm2 for 5 times + +# compose them with the transformation: compose and +rdp_compose = Composition() +rdp_composed_mech = rdp_compose([gm1, gm2], [3, 5]) + +compose = ComposeGaussian() +composed_mech = compose([gm1, gm2], [3, 5]) + + + +# Query for eps given delta +delta1 = 1e-6 +eps1 = composed_mech.get_approxDP(delta1) +eps1b = rdp_composed_mech.get_approxDP(delta1) + +delta2 = 1e-4 +eps2 = composed_mech.get_approxDP(delta2) +eps2b = rdp_composed_mech.get_approxDP(delta2) + +# Get name of the composed object, a structured description of the mechanism generated automatically +print('Mechanism name is \"', composed_mech.name,'\"') +print('Parameters are: ',composed_mech.params) + +print('Generic composition: epsilon(delta) = ', eps1b, ', at delta = ', delta1) +print('Gaussian composition: epsilon(delta) = ', eps1, ', at delta = ', delta1) + +print('Generic composition: epsilon(delta) = ', eps2b, ', at delta = ', delta2) +print('Gaussian composition: epsilon(delta) = ', eps2, ', at delta = ', delta2) + + + +# Get hypothesis testing interpretation so we can directly plot it +fpr_list, fnr_list = composed_mech.plot_fDP() +fpr_list, fnr_list2 = rdp_composed_mech.plot_fDP() + +plt.figure(figsize = (6,6)) +plt.plot(fpr_list,fnr_list,label='Gaussian_composition') +plt.plot(fpr_list,fnr_list2,label='Generic_composition') +plt.xlabel('Type I error') +plt.ylabel('Type II error') +plt.legend() +plt.show() + diff --git a/example/example_puredp.py b/example/example_puredp.py new file mode 100644 index 0000000..28f2212 --- /dev/null +++ b/example/example_puredp.py @@ -0,0 +1,46 @@ +from autodp.mechanism_zoo import PureDP_Mechanism + +from autodp.transformer_zoo import Composition + +# Example: pure DP mechanism and composition of it + +eps = 0.3 + + +mech = PureDP_Mechanism(eps, name='Laplace') + +import matplotlib.pyplot as plt + +fpr_list, fnr_list = mech.plot_fDP() +plt.figure(1) +plt.plot(fpr_list,fnr_list,label='fdp_of_laplace') + + +delta = 1e-6 +epslist = [mech.get_approxDP(delta)] + +# declare a transformation to handle composition +compose = Composition() + +for i in range(2,11): + mech_composed = compose([mech], [i]) + epslist.append(mech_composed.get_approxDP(delta)) + fpr_list, fnr_list = mech_composed.plot_fDP() + plt.plot(fpr_list, fnr_list, label='fdp_of_'+str(i)+'laplace') + +plt.legend() +plt.xlabel('Type I error') +plt.ylabel('Type II error') +plt.show() +# we could specify parameters of the composition, e.g. using RDP composition, using KOV and so on + + + +plt.figure(2) +plt.plot(range(1,11),epslist) +plt.xlabel('number of times compose') +plt.ylabel(r'$\epsilon$ at $\delta = 1e-6$') +plt.show() + + + diff --git a/figures/autodp_design.png b/figures/autodp_design.png new file mode 100644 index 0000000..cb37fc2 Binary files /dev/null and b/figures/autodp_design.png differ diff --git a/papers/NeurIPS20-SVT/exp2_align_var.py b/papers/NeurIPS20-SVT/exp2_align_var.py new file mode 100644 index 0000000..cd17a3b --- /dev/null +++ b/papers/NeurIPS20-SVT/exp2_align_var.py @@ -0,0 +1,154 @@ +import numpy as np +import math +import scipy +from autodp import rdp_bank, dp_bank, fdp_bank, utils +from autodp.mechanism_zoo import LaplaceMechanism, LaplaceSVT_Mechanism,StageWiseMechanism +from autodp.transformer_zoo import Composition +import matplotlib.pyplot as plt +from scipy.stats import norm, laplace +from scipy.special import comb +import matplotlib.font_manager as fm +from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism,SubsampleGaussianMechanism, GaussianMechanism, ComposedGaussianMechanism,GaussianSVT_Mechanism, NoisyScreenMechanism +from autodp.transformer_zoo import Composition, AmplificationBySampling + + +""" +This experiment corresponding to exp 2 in NeurIPS-20 (Figure 2 (a)) +We evaluate SVT variants with the same variance of noise by comparing the composed privacy loss for finishing a fixed length sequence of queries. + +rho is from Lap(lambda) -> eps_rho = 1/lambda +nu is from Lap(2lambda) -> eps_nu = 1/lambda +eps = (c+1)/lambda, lambda = (c+1)/eps + +To align variance between Gaussian-bassed and Laplace-based approaches, we set sigma_1 = sqrt(2) * lambda_rho +""" + +delta = 1e-6 +lambda_rho = 120 +lambda_nu = 240 +sigma_1 = lambda_rho*np.sqrt(2) +sigma_2 = 2*sigma_1 +eps_1 = 1.0 / lambda_rho +n = 100000 #the length of the fixed query +margin = 1000 + + + +def exp_2a(): + + eps_a = [] #standard SVT + eps_e = [] #Laplace-SVT (via RDP) + eps_g = [] #Gaussian SVT c>1 + eps_g_c = [] #c=1 for Gaussian SVT + eps_i = [] + eps_kov = [] #generalized SVT + eps_noisy = [] + k_list = [int(1.4**i) for i in range(int(math.floor(math.log(n,1.4)))+1)] + print(len(k_list)) + + query = np.zeros(n) + rho = np.random.normal(scale=sigma_1) + lap_rho = np.random.laplace(loc=0.0, scale=lambda_rho) + + """ + compute eps for noisy screening + p = Prob[ nu > Margin] + q = Prob[ nu - 1 > margin] + + count_gau counts #tops in Gaussian-SVT + count_lap counts #tops in Laplace-SVT + """ + + count_gau = 0 + count_lap = 0 + + # the following is for data-dependent screening in CVPR-20 + p = scipy.stats.norm.logsf(margin, scale=sigma_2) + q = scipy.stats.norm.logsf(margin + 1, scale=sigma_2) + params = {} + params['logp'] = p + params['logq'] = q + per_screen_mech = NoisyScreenMechanism(params, name='NoisyScreen') + per_gaussian_mech = ExactGaussianMechanism(sigma_2,name='GM1') + index = [] + compose = Composition() + for idx, qu in enumerate(query): + nu = np.random.normal(scale=sigma_2) + lap_nu = np.random.laplace(loc=0.0, scale=lambda_nu) + if nu >= rho + margin: + count_gau += 1 + if lap_nu >= lap_rho + margin: + count_lap += 1 + count_gau = max(count_gau, 1) + count_lap = max(count_lap, 1) + + if idx in k_list: + index.append(idx) + print('number of queries passing threshold', count_gau) + + #eps_a records the standard SVT + eps_a.append(eps_1 * count_lap + eps_1) + # compose data-dependent screening + screen_mech = compose([per_screen_mech], [idx]) + gaussian_mech = compose([per_gaussian_mech], [idx]) + + # standard SVT with RDP calculation + param_lap_svt = {} + param_lap_svt['b'] = lambda_rho + param_lap_svt['k'] = idx + param_lap_svt['c'] = count_lap + lapsvtrdp_mech = LaplaceSVT_Mechanism(param_lap_svt) + eps_e.append(lapsvtrdp_mech.get_approxDP(delta)) + + # stage-wise generalized SVT, k is the maximum length of each chunk + k = int(idx / np.sqrt(count_gau)) + generalized_mech = StageWiseMechanism({'sigma':sigma_1,'k':k, 'c':count_gau}) + eps_kov.append(generalized_mech.get_approxDP(delta)) + + # Gaussian-SVT c>1 with RDP, k is the total length before algorithm stops + gaussianSVT_c = GaussianSVT_Mechanism({'sigma':sigma_1,'k':idx, 'c':count_gau}, rdp_c_1=False) + eps_g.append(gaussianSVT_c.get_approxDP(delta)) + + #Gaussian-SVT with c=1, we use average_k as the approximate maximum length of each chunk, margin is used in Proposition 10 + average_k = int(idx / max(count_gau, 1)) + params_SVT = {} + params_SVT['k'] = average_k + params_SVT['sigma'] = sigma_1 + params_SVT['margin'] = margin + per_gaussianSVT_mech = GaussianSVT_Mechanism(params_SVT) + gaussianSVT_mech = compose([per_gaussianSVT_mech],[max(count_gau, 1)]) + eps_g_c.append(gaussianSVT_mech.get_approxDP(delta)) + + eps_i.append(gaussian_mech.get_approxDP(delta)) # Gaussian Mechanism + eps_noisy.append(screen_mech.get_approxDP(delta)) + + import matplotlib + import matplotlib.pyplot as plt + + font = {'family': 'times', + 'weight': 'bold', + 'size': 18} + + props = fm.FontProperties(family='Gill Sans', fname='/Library/Fonts/GillSans.ttc') + f, ax = plt.subplots() + plt.figure(num=0, figsize=(12, 8), dpi=80, facecolor='w', edgecolor='k') + plt.loglog(index, eps_a, '-r', linewidth=2) + plt.loglog(index, eps_e, '--g^', linewidth=2) + plt.loglog(index, eps_g, '-c^', linewidth=2) + plt.loglog(index, eps_g_c, '-bs', linewidth=2) + plt.loglog(index, eps_i, '--k', linewidth=2) + plt.loglog(index, eps_noisy, color='brown', linewidth=2) + plt.loglog(index, eps_kov, color='hotpink', linewidth=2) + plt.legend( + ['Laplace-SVT (Pure-DP from Lyu et al., 2017)', 'Laplace-SVT (via RDP)', 'Gaussian-SVT c>1 (RDP by Theorem 11)', + 'Gaussian-SVT c=1 (RDP by Theorem 8)', 'Gaussian Mechanism', 'Noisy Screening (data-dependent RDP)', + 'Stage-wise generalized SVT'], loc='best', fontsize=17) + plt.grid(True) + plt.xticks(fontsize=20) + plt.yticks(fontsize=20) + plt.xlabel(r'Iterations', fontsize=20) + plt.ylabel(r'$\epsilon$', fontsize=20) + ax.set_title('Title', fontproperties=props) + + plt.savefig('exp2a.pdf', bbox_inches='tight') +exp_2a() \ No newline at end of file diff --git a/test/unit_test_approxdp_to_fdp_conversion.py b/test/unit_test_approxdp_to_fdp_conversion.py new file mode 100644 index 0000000..0aa4533 --- /dev/null +++ b/test/unit_test_approxdp_to_fdp_conversion.py @@ -0,0 +1,43 @@ +from autodp.mechanism_zoo import GaussianMechanism +from autodp.fdp_bank import fDP_gaussian + +import numpy as np + +from absl.testing import absltest +from absl.testing import parameterized + +params = [0.05, 0.1, 0.2, 0.5,1.0, 2.0,5.0, 10.0] + + +def _fdp_conversion(sigma): + + # Using the log(1-f(fpr)) and log(- \partial f(fpr)) that are implemented dedicatedly + + fpr_list = np.linspace(0, 1, 21) + + # analytical gaussian implementation (privacy profile) + gm2 = GaussianMechanism(sigma, name='GM2', RDP_off=True) + + # direct f-DP implementation + fdp = lambda x: fDP_gaussian({'sigma': sigma},x) + + fdp_direct = fdp(fpr_list) + + # the fdp is converted by numerical methods from privacy profile. + fdp_converted = np.array([gm2.get_fDP(fpr) for fpr in fpr_list]) + + return fdp_direct - fdp_converted + + + +class Test_approxDP2fDP_Conversion(parameterized.TestCase): + + @parameterized.parameters(p for p in params) + def test_fdp_conversion(self, sigma): + max_diff = _fdp_conversion(sigma) + self.assertSequenceAlmostEqual(max_diff, np.zeros_like(max_diff), places=4) + + +if __name__ == '__main__': + absltest.main() + diff --git a/test/unit_test_calibrator.py b/test/unit_test_calibrator.py new file mode 100644 index 0000000..772dac4 --- /dev/null +++ b/test/unit_test_calibrator.py @@ -0,0 +1,84 @@ +import math +from autodp.calibrator_zoo import eps_delta_calibrator,generalized_eps_delta_calibrator, ana_gaussian_calibrator +from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism,SubsampleGaussianMechanism, GaussianMechanism, ComposedGaussianMechanism, LaplaceMechanism +from autodp.transformer_zoo import Composition, AmplificationBySampling +from absl.testing import absltest +from absl.testing import parameterized +""" +Unit tests for calibrating noise to privacy budgets. +Cases 1: single parameter, no subsample or composition + +""" +def testcase_single_para(): + + test_case = [] + # each test_case = {'noise_para','delta','eps'} range sigma from 1 to 100 + sigma_list = [int(1.6**i) for i in range(int(math.floor(math.log(100,1.6)))+1)] + for sigma in sigma_list: + for delta in [ 1e-3, 1e-4,1e-5,1e-6]: + gm = ExactGaussianMechanism(sigma, name = 'GM') + cur_test = {'sigma':sigma, 'delta':delta,'eps':gm.get_approxDP(delta)} + test_case.append(cur_test) + return test_case + +def _single_para(eps, delta,params_name='sigma', bounds=[0,100]): + """ + test calibrator + """ + calibrate = eps_delta_calibrator() + mech1 = calibrate(ExactGaussianMechanism,eps,delta,[0,100],name='GM') + return mech1.params[params_name] +tol = 0.01 +class TestGaussianMechanism(parameterized.TestCase): + params = testcase_single_para() + @parameterized.parameters(p for p in params) + def test_single_para(self,eps,delta, sigma): + autodp_value = _single_para(eps, delta, params_name='sigma') + self.assertGreaterEqual(autodp_value, (1.-tol)*sigma) + + +""" +Unit tests for calibrating noise to privacy budgets. +Cases 2: multi-parameter + +""" +def testcase_multi_para(): + """ + create test cases when there are multi parameters (e.g., #coeff and sigma in Composed Gaussian mechanism) + """ + + test_case = [] + # each test_case = {'noise_para','delta','coeff','eps'} range sigma from 1 to 100, coeff is the number of composition + sigma_list = [int(1.6**i) for i in range(int(math.floor(math.log(100,1.6)))+1)] + for sigma in sigma_list: + for coeff in [1, 10, 100, 1000]: + for delta in [ 1e-3, 1e-4,1e-5, 1e-6]: + gm = ExactGaussianMechanism(sigma, name = 'GM') + compose = Composition() + composed_mech = compose([gm], [coeff]) + cur_test = {'sigma':sigma, 'delta':delta,'coeff':coeff, 'eps':composed_mech.get_approxDP(delta)} + test_case.append(cur_test) + return test_case +def _multi_para(eps, delta,coeff, params_name='sigma', bounds=[0,100]): + """ + test calibrator with multi-parameters + """ + general_calibrate = generalized_eps_delta_calibrator() + params = {} + params[params_name] = None + params['coeff'] = coeff + mech1 = general_calibrate(ComposedGaussianMechanism,eps,delta,[0,1000], params=params, para_name = params_name,name='GM') + + return mech1.params[params_name] +class TestGeneralizedMechanism(parameterized.TestCase): + """ + We test generalized calibrator that deals with compostion coeff + """ + params = testcase_multi_para() + @parameterized.parameters(p for p in params) + + def test_single_para(self, eps, delta,coeff, sigma): + autodp_value = _multi_para(eps=eps, delta=delta,coeff=coeff, params_name='sigma') + self.assertGreaterEqual(autodp_value, (1. - tol) * sigma) + + diff --git a/test/unit_test_dp_sgd_poisson_sampling_add_remove.py b/test/unit_test_dp_sgd_poisson_sampling_add_remove.py new file mode 100644 index 0000000..0113aac --- /dev/null +++ b/test/unit_test_dp_sgd_poisson_sampling_add_remove.py @@ -0,0 +1,1375 @@ +from autodp.mechanism_zoo import ExactGaussianMechanism +from autodp.transformer_zoo import Composition, AmplificationBySampling + +from absl.testing import absltest +from absl.testing import parameterized + +# Parameters to test and values produced by tensorflow-privacy: +# https://github.com/tensorflow/privacy/blob/master/tensorflow_privacy/privacy/analysis/rdp_accountant.py +params = ( + {'order': 1.25, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 39.47435359280898}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 90.7896596280238}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 104.21069366522143}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 193.85977308534916}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 219.07906690372957}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 494.8831442377909}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 1245.2029477229287}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 3195.321731874551}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 6395.3585686314445}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 394.74353592808984}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 907.896596280238}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 1042.1069366522142}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 1938.5977308534916}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 2190.7906690372956}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 4948.831442377908}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 12452.029477229287}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 31953.217318745512}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 63953.58568631444}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 3947.435359280898}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 9078.96596280238}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 10421.069366522142}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 19385.977308534915}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 21907.906690372958}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 49488.314423779084}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 124520.29477229287}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 319532.1731874551}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 639535.8568631444}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 39474.353592808984}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 90789.6596280238}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 104210.69366522142}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 193859.77308534915}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 219079.06690372957}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 494883.1442377909}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 1245202.9477229286}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 3195321.731874551}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 6395358.568631444}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 197371.7679640449}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 453948.298140119}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 521053.46832610713}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 969298.8654267458}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 1095395.3345186478}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 2474415.7211889545}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 6226014.738614643}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 15976608.659372754}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 31976792.843157224}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 394743.5359280898}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 907896.596280238}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 1042106.9366522143}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 1938597.7308534917}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 2190790.6690372955}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 4948831.442377909}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 12452029.477229286}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 31953217.31874551}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 63953585.68631445}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 0.1103603174246269}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 6.790784234628705}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 9.710698228804564}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 25.859773085349214}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 30.079066903729597}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 74.883144237791}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 195.20294772292905}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 507.3217318745518}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 1019.3585686314451}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 1.103603174246269}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 67.90784234628705}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 97.10698228804563}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 258.5977308534921}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 300.79066903729597}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 748.83144237791}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 1952.0294772292905}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 5073.217318745518}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 10193.58568631445}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 11.03603174246269}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 679.0784234628705}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 971.0698228804564}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 2585.9773085349216}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 3007.90669037296}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 7488.3144237791}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 19520.294772292906}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 50732.17318745518}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 101935.85686314451}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 110.3603174246269}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 6790.784234628705}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 9710.698228804564}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 25859.773085349214}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 30079.066903729596}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 74883.144237791}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 195202.94772292906}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 507321.7318745518}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 1019358.568631445}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 551.8015871231345}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 33953.92117314353}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 48553.49114402282}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 129298.86542674607}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 150395.334518648}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 374415.721188955}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 976014.7386146453}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 2536608.659372759}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 5096792.843157225}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 1103.603174246269}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 67907.84234628706}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 97106.98228804563}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 258597.73085349213}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 300790.669037296}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 748831.44237791}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 1952029.4772292906}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 5073217.318745518}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 10193585.68631445}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.0018977736416478479}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.005345502314345245}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.0083541177519165}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 1.8618755130319669}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 3.0791796010526946}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 14.883144237791036}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 45.202947722929075}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 123.32173187455179}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 251.35856863144505}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 0.01897773641647848}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 0.053455023143452456}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 0.08354117751916501}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 18.61875513031967}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 30.791796010526944}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 148.83144237791038}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 452.02947722929076}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 1233.2173187455178}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 2513.5856863144504}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 0.1897773641647848}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 0.5345502314345245}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 0.8354117751916501}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 186.1875513031967}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 307.91796010526946}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 1488.3144237791037}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 4520.2947722929075}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 12332.173187455179}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 25135.856863144505}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 1.897773641647848}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 5.345502314345246}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 8.354117751916501}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 1861.8755130319669}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 3079.1796010526946}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 14883.144237791037}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 45202.94772292907}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 123321.73187455178}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 251358.56863144506}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 9.48886820823924}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 26.727511571726225}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 41.7705887595825}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 9309.377565159833}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 15395.898005263472}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 74415.72118895518}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 226014.73861464538}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 616608.6593727589}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 1256792.8431572253}, + {'order': 1.25, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 18.97773641647848}, + {'order': 2.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 53.45502314345245}, + {'order': 2.25, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 83.541177519165}, + {'order': 4.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 18618.755130319667}, + {'order': 4.5, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 30791.796010526945}, + {'order': 10.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 148831.44237791037}, + {'order': 25.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 452029.47722929076}, + {'order': 64.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 1233217.3187455179}, + {'order': 128.0, 'q': 0.01, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 2513585.6863144506}, + {'order': 1.25, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.00010539800475567958}, + {'order': 2.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.00017181342207453428}, + {'order': 2.25, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.0001945383384777645}, + {'order': 4.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.00036315404891075525}, + {'order': 4.5, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.00041492706734509965}, + {'order': 10.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.03827041889494864}, + {'order': 25.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 7.70294772682218}, + {'order': 64.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 27.32173187455178}, + {'order': 128.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 59.358568631445074}, + {'order': 1.25, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.001053980047556796}, + {'order': 2.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.0017181342207453426}, + {'order': 2.25, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.0019453833847776448}, + {'order': 4.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.0036315404891075526}, + {'order': 4.5, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.004149270673450996}, + {'order': 10.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.38270418894948643}, + {'order': 25.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 77.0294772682218}, + {'order': 64.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 273.2173187455178}, + {'order': 128.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 593.5856863144508}, + {'order': 1.25, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.010539800475567958}, + {'order': 2.0, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.017181342207453428}, + {'order': 2.25, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.01945383384777645}, + {'order': 4.0, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.03631540489107552}, + {'order': 4.5, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.041492706734509964}, + {'order': 10.0, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 3.8270418894948643}, + {'order': 25.0, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 770.294772682218}, + {'order': 64.0, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 2732.173187455178}, + {'order': 128.0, 'q': 0.01, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 5935.856863144508}, + {'order': 1.25, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0.10539800475567958}, + {'order': 2.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0.1718134220745343}, + {'order': 2.25, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0.1945383384777645}, + {'order': 4.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0.36315404891075526}, + {'order': 4.5, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0.41492706734509965}, + {'order': 10.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 38.27041889494864}, + {'order': 25.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 7702.94772682218}, + {'order': 64.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 27321.73187455178}, + {'order': 128.0, 'q': 0.01, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 59358.56863144507}, + {'order': 1.25, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 0.5269900237783979}, + {'order': 2.0, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 0.8590671103726714}, + {'order': 2.25, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 0.9726916923888225}, + {'order': 4.0, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 1.8157702445537762}, + {'order': 4.5, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 2.0746353367254984}, + {'order': 10.0, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 191.3520944747432}, + {'order': 25.0, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 38514.7386341109}, + {'order': 64.0, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 136608.6593727589}, + {'order': 128.0, 'q': 0.01, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 296792.84315722535}, + {'order': 1.25, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 1.0539800475567958}, + {'order': 2.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 1.7181342207453427}, + {'order': 2.25, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 1.945383384777645}, + {'order': 4.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 3.6315404891075524}, + {'order': 4.5, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 4.149270673450997}, + {'order': 10.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 382.7041889494864}, + {'order': 25.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 77029.4772682218}, + {'order': 64.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 273217.3187455178}, + {'order': 128.0, 'q': 0.01, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 593585.6863144507}, + {'order': 1.25, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 1.7710637990830963e-05}, + {'order': 2.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 2.8402138324210935e-05}, + {'order': 2.25, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 3.197693198178044e-05}, + {'order': 4.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 5.715580737173236e-05}, + {'order': 4.5, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 6.440025440052329e-05}, + {'order': 10.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.00014562314921461514}, + {'order': 25.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.0003829780935648833}, + {'order': 64.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 3.321746408681008}, + {'order': 128.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 11.358568631446696}, + {'order': 1.25, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.00017710637990830963}, + {'order': 2.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.00028402138324210935}, + {'order': 2.25, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.0003197693198178044}, + {'order': 4.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.0005715580737173236}, + {'order': 4.5, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.0006440025440052329}, + {'order': 10.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.0014562314921461514}, + {'order': 25.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.003829780935648833}, + {'order': 64.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 33.217464086810075}, + {'order': 128.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 113.58568631446695}, + {'order': 1.25, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.0017710637990830964}, + {'order': 2.0, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.0028402138324210933}, + {'order': 2.25, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.003197693198178044}, + {'order': 4.0, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.005715580737173236}, + {'order': 4.5, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.006440025440052329}, + {'order': 10.0, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.014562314921461513}, + {'order': 25.0, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.03829780935648833}, + {'order': 64.0, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 332.17464086810077}, + {'order': 128.0, 'q': 0.01, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 1135.8568631446697}, + {'order': 1.25, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.017710637990830964}, + {'order': 2.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.028402138324210936}, + {'order': 2.25, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.03197693198178044}, + {'order': 4.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.05715580737173236}, + {'order': 4.5, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.06440025440052329}, + {'order': 10.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.14562314921461514}, + {'order': 25.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.38297809356488327}, + {'order': 64.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 3321.746408681008}, + {'order': 128.0, 'q': 0.01, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 11358.568631446697}, + {'order': 1.25, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 0.08855318995415482}, + {'order': 2.0, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 0.14201069162105467}, + {'order': 2.25, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 0.1598846599089022}, + {'order': 4.0, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 0.2857790368586618}, + {'order': 4.5, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 0.32200127200261647}, + {'order': 10.0, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 0.7281157460730757}, + {'order': 25.0, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 1.9148904678244165}, + {'order': 64.0, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 16608.732043405038}, + {'order': 128.0, 'q': 0.01, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 56792.84315723348}, + {'order': 1.25, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 0.17710637990830963}, + {'order': 2.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 0.28402138324210935}, + {'order': 2.25, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 0.3197693198178044}, + {'order': 4.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 0.5715580737173236}, + {'order': 4.5, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 0.6440025440052329}, + {'order': 10.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 1.4562314921461514}, + {'order': 25.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 3.829780935648833}, + {'order': 64.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 33217.464086810076}, + {'order': 128.0, 'q': 0.01, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 113585.68631446696}, + {'order': 1.25, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 6.280881026314128e-07}, + {'order': 2.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 1.005016203373883e-06}, + {'order': 2.25, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 1.1306714441161602e-06}, + {'order': 4.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 2.0104337956990166e-06}, + {'order': 4.5, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 2.261850947807683e-06}, + {'order': 10.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 5.029098115867569e-06}, + {'order': 25.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 1.259163330476561e-05}, + {'order': 64.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 3.2361212985515655e-05}, + {'order': 128.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 6.514386177732818e-05}, + {'order': 1.25, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 6.280881026314128e-06}, + {'order': 2.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 1.005016203373883e-05}, + {'order': 2.25, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 1.1306714441161603e-05}, + {'order': 4.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 2.0104337956990168e-05}, + {'order': 4.5, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 2.2618509478076832e-05}, + {'order': 10.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 5.029098115867569e-05}, + {'order': 25.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0001259163330476561}, + {'order': 64.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.00032361212985515655}, + {'order': 128.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0006514386177732818}, + {'order': 1.25, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 6.280881026314128e-05}, + {'order': 2.0, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.0001005016203373883}, + {'order': 2.25, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.00011306714441161601}, + {'order': 4.0, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.00020104337956990167}, + {'order': 4.5, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.0002261850947807683}, + {'order': 10.0, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.000502909811586757}, + {'order': 25.0, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.001259163330476561}, + {'order': 64.0, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.0032361212985515656}, + {'order': 128.0, 'q': 0.01, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.006514386177732818}, + {'order': 1.25, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.0006280881026314128}, + {'order': 2.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.001005016203373883}, + {'order': 2.25, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.0011306714441161603}, + {'order': 4.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.0020104337956990165}, + {'order': 4.5, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.002261850947807683}, + {'order': 10.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.005029098115867569}, + {'order': 25.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.01259163330476561}, + {'order': 64.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.032361212985515656}, + {'order': 128.0, 'q': 0.01, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.06514386177732817}, + {'order': 1.25, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.0031404405131570637}, + {'order': 2.0, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.005025081016869415}, + {'order': 2.25, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.005653357220580801}, + {'order': 4.0, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.010052168978495084}, + {'order': 4.5, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.011309254739038415}, + {'order': 10.0, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.025145490579337845}, + {'order': 25.0, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.06295816652382805}, + {'order': 64.0, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.16180606492757826}, + {'order': 128.0, 'q': 0.01, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.3257193088866409}, + {'order': 1.25, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.006280881026314127}, + {'order': 2.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.01005016203373883}, + {'order': 2.25, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.011306714441161603}, + {'order': 4.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.020104337956990168}, + {'order': 4.5, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.02261850947807683}, + {'order': 10.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.05029098115867569}, + {'order': 25.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.1259163330476561}, + {'order': 64.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.3236121298551565}, + {'order': 128.0, 'q': 0.01, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.6514386177732818}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 47.52136461284061}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 94.008535452892}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 107.1076819076028}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 196.00569030192798}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 221.14834421971628}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 496.67140858493997}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 1246.879445548381}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 3196.956716420516}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 6396.980679283347}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 475.2136461284061}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 940.0853545289201}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 1071.076819076028}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 1960.0569030192798}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 2211.4834421971627}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 4966.714085849399}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 12468.79445548381}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 31969.56716420516}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 63969.80679283347}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 4752.136461284061}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 9400.853545289201}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 10710.76819076028}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 19600.569030192797}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 22114.834421971627}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 49667.140858494}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 124687.9445548381}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 319695.6716420516}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 639698.0679283347}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 47521.3646128406}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 94008.535452892}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 107107.68190760279}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 196005.69030192797}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 221148.34421971627}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 496671.40858493996}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 1246879.445548381}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 3196956.716420516}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 6396980.679283347}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 237606.82306420302}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 470042.67726446}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 535538.409538014}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 980028.45150964}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 1105741.7210985813}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 2483357.0429247}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 6234397.227741905}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 15984783.58210258}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 31984903.396416735}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 475213.64612840605}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 940085.35452892}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 1071076.819076028}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 1960056.90301928}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 2211483.4421971627}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 4966714.0858494}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 12468794.45548381}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 31969567.16420516}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 63969806.79283347}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 0.8332213070393273}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 10.008580353418688}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 12.607682080965713}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 28.005690301928013}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 32.1483442197163}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 76.67140858494001}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 196.87944554838126}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 508.9567164205166}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 1020.9806792833471}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 8.332213070393273}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 100.08580353418688}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 126.07682080965714}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 280.0569030192801}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 321.483442197163}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 766.7140858494001}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 1968.7944554838125}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 5089.567164205166}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 10209.806792833471}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 83.32213070393273}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 1000.8580353418688}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 1260.7682080965712}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 2800.569030192801}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 3214.83442197163}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 7667.140858494001}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 19687.944554838126}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 50895.67164205166}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 102098.06792833471}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 833.2213070393273}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 10008.580353418689}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 12607.682080965713}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 28005.690301928014}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 32148.3442197163}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 76671.40858494001}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 196879.44554838125}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 508956.7164205166}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 1020980.6792833471}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 4166.106535196636}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 50042.90176709344}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 63038.410404828566}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 140028.45150964006}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 160741.7210985815}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 383357.0429247001}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 984397.2277419063}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 2544783.5821025833}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 5104903.396416735}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 8332.213070393273}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 100085.80353418688}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 126076.82080965713}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 280056.9030192801}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 321483.442197163}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 766714.0858494001}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 1968794.4554838126}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 5089567.1642051665}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 10209806.79283347}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.029632735891697154}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.12574712688706774}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.2394299356010559}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 4.00584939067033}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 5.148364567706042}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 16.671408584940014}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 46.879445548381256}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 124.95671642051657}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 252.98067928334717}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 0.29632735891697154}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 1.2574712688706775}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 2.394299356010559}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 40.058493906703305}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 51.48364567706042}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 166.71408584940013}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 468.79445548381256}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 1249.5671642051657}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 2529.8067928334717}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 2.9632735891697153}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 12.574712688706773}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 23.94299356010559}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 400.584939067033}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 514.8364567706042}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 1667.1408584940013}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 4687.944554838125}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 12495.671642051657}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 25298.067928334716}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 29.632735891697155}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 125.74712688706774}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 239.42993560105592}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 4005.84939067033}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 5148.364567706042}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 16671.408584940014}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 46879.445548381256}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 124956.71642051658}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 252980.67928334718}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 148.16367945848577}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 628.7356344353387}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 1197.1496780052796}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 20029.246953351652}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 25741.82283853021}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 83357.04292470007}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 234397.22774190627}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 624783.5821025829}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 1264903.3964167358}, + {'order': 1.25, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 296.32735891697155}, + {'order': 2.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 1257.4712688706775}, + {'order': 2.25, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 2394.299356010559}, + {'order': 4.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 40058.493906703305}, + {'order': 4.5, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 51483.64567706042}, + {'order': 10.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 166714.08584940014}, + {'order': 25.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 468794.45548381255}, + {'order': 64.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 1249567.1642051658}, + {'order': 128.0, 'q': 0.05, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 2529806.7928334717}, + {'order': 1.25, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.002485316298319207}, + {'order': 2.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.004286504370418927}, + {'order': 2.25, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.004957724285961285}, + {'order': 4.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.011416268872652872}, + {'order': 4.5, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.01437879709925404}, + {'order': 10.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 1.674060563785817}, + {'order': 25.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 9.379445549128421}, + {'order': 64.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 28.95671642051658}, + {'order': 128.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 60.98067928334716}, + {'order': 1.25, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.02485316298319207}, + {'order': 2.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.042865043704189275}, + {'order': 2.25, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.049577242859612854}, + {'order': 4.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.11416268872652872}, + {'order': 4.5, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.1437879709925404}, + {'order': 10.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 16.74060563785817}, + {'order': 25.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 93.79445549128421}, + {'order': 64.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 289.5671642051658}, + {'order': 128.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 609.8067928334716}, + {'order': 1.25, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.2485316298319207}, + {'order': 2.0, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.4286504370418927}, + {'order': 2.25, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.4957724285961285}, + {'order': 4.0, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 1.1416268872652873}, + {'order': 4.5, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 1.437879709925404}, + {'order': 10.0, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 167.4060563785817}, + {'order': 25.0, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 937.9445549128421}, + {'order': 64.0, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 2895.671642051658}, + {'order': 128.0, 'q': 0.05, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 6098.067928334715}, + {'order': 1.25, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 2.485316298319207}, + {'order': 2.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 4.286504370418927}, + {'order': 2.25, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 4.957724285961286}, + {'order': 4.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 11.416268872652871}, + {'order': 4.5, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 14.37879709925404}, + {'order': 10.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 1674.060563785817}, + {'order': 25.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 9379.44554912842}, + {'order': 64.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 28956.716420516583}, + {'order': 128.0, 'q': 0.05, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 60980.679283347155}, + {'order': 1.25, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 12.426581491596036}, + {'order': 2.0, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 21.432521852094638}, + {'order': 2.25, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 24.788621429806426}, + {'order': 4.0, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 57.08134436326436}, + {'order': 4.5, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 71.8939854962702}, + {'order': 10.0, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 8370.302818929085}, + {'order': 25.0, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 46897.22774564211}, + {'order': 64.0, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 144783.5821025829}, + {'order': 128.0, 'q': 0.05, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 304903.3964167358}, + {'order': 1.25, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 24.85316298319207}, + {'order': 2.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 42.865043704189276}, + {'order': 2.25, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 49.57724285961285}, + {'order': 4.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 114.16268872652871}, + {'order': 4.5, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 143.7879709925404}, + {'order': 10.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 16740.60563785817}, + {'order': 25.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 93794.45549128421}, + {'order': 64.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 289567.1642051658}, + {'order': 128.0, 'q': 0.05, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 609806.7928334716}, + {'order': 1.25, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.0004388760120985163}, + {'order': 2.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.0007098115658748468}, + {'order': 2.25, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.0008014493223874764}, + {'order': 4.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.0014625632129495554}, + {'order': 4.5, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.0016581138661999515}, + {'order': 10.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.004044112497576389}, + {'order': 25.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.07545190786474157}, + {'order': 64.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 4.9567192096225}, + {'order': 128.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 12.98067928334747}, + {'order': 1.25, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.004388760120985163}, + {'order': 2.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.007098115658748468}, + {'order': 2.25, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.008014493223874763}, + {'order': 4.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.014625632129495554}, + {'order': 4.5, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.016581138661999516}, + {'order': 10.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.04044112497576389}, + {'order': 25.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.7545190786474156}, + {'order': 64.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 49.567192096225}, + {'order': 128.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 129.8067928334747}, + {'order': 1.25, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.04388760120985163}, + {'order': 2.0, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.07098115658748469}, + {'order': 2.25, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.08014493223874763}, + {'order': 4.0, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.14625632129495553}, + {'order': 4.5, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.16581138661999514}, + {'order': 10.0, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.40441124975763887}, + {'order': 25.0, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 7.545190786474157}, + {'order': 64.0, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 495.67192096225006}, + {'order': 128.0, 'q': 0.05, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 1298.067928334747}, + {'order': 1.25, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.4388760120985163}, + {'order': 2.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.7098115658748468}, + {'order': 2.25, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 0.8014493223874763}, + {'order': 4.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 1.4625632129495554}, + {'order': 4.5, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 1.6581138661999515}, + {'order': 10.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 4.044112497576389}, + {'order': 25.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 75.45190786474157}, + {'order': 64.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 4956.719209622501}, + {'order': 128.0, 'q': 0.05, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 12980.67928334747}, + {'order': 1.25, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 2.1943800604925814}, + {'order': 2.0, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 3.549057829374234}, + {'order': 2.25, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 4.007246611937382}, + {'order': 4.0, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 7.312816064747777}, + {'order': 4.5, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 8.290569330999757}, + {'order': 10.0, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 20.220562487881946}, + {'order': 25.0, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 377.2595393237078}, + {'order': 64.0, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 24783.5960481125}, + {'order': 128.0, 'q': 0.05, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 64903.39641673735}, + {'order': 1.25, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 4.388760120985163}, + {'order': 2.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 7.098115658748468}, + {'order': 2.25, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 8.014493223874764}, + {'order': 4.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 14.625632129495553}, + {'order': 4.5, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 16.581138661999514}, + {'order': 10.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 40.44112497576389}, + {'order': 25.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 754.5190786474157}, + {'order': 64.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 49567.192096225}, + {'order': 128.0, 'q': 0.05, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 129806.7928334747}, + {'order': 1.25, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 1.56975532377146e-05}, + {'order': 2.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 2.512510207234621e-05}, + {'order': 2.25, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 2.8269123305335597e-05}, + {'order': 4.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 5.029837790076196e-05}, + {'order': 4.5, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 5.659924543164964e-05}, + {'order': 10.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.00012610908358120653}, + {'order': 25.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0003175729623224624}, + {'order': 64.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0008288460377922641}, + {'order': 128.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.001713513235768133}, + {'order': 1.25, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.000156975532377146}, + {'order': 2.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0002512510207234621}, + {'order': 2.25, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.00028269123305335597}, + {'order': 4.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0005029837790076196}, + {'order': 4.5, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0005659924543164964}, + {'order': 10.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0012610908358120653}, + {'order': 25.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.003175729623224624}, + {'order': 64.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.00828846037792264}, + {'order': 128.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.01713513235768133}, + {'order': 1.25, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.00156975532377146}, + {'order': 2.0, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.002512510207234621}, + {'order': 2.25, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.0028269123305335597}, + {'order': 4.0, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.005029837790076196}, + {'order': 4.5, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.005659924543164964}, + {'order': 10.0, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.012610908358120652}, + {'order': 25.0, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.03175729623224624}, + {'order': 64.0, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.0828846037792264}, + {'order': 128.0, 'q': 0.05, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.1713513235768133}, + {'order': 1.25, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.0156975532377146}, + {'order': 2.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.02512510207234621}, + {'order': 2.25, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.028269123305335597}, + {'order': 4.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.050298377900761956}, + {'order': 4.5, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.05659924543164964}, + {'order': 10.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.12610908358120654}, + {'order': 25.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.3175729623224624}, + {'order': 64.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.8288460377922641}, + {'order': 128.0, 'q': 0.05, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 1.7135132357681329}, + {'order': 1.25, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.078487766188573}, + {'order': 2.0, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.12562551036173103}, + {'order': 2.25, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.141345616526678}, + {'order': 4.0, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.2514918895038098}, + {'order': 4.5, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.2829962271582482}, + {'order': 10.0, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.6305454179060326}, + {'order': 25.0, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 1.587864811612312}, + {'order': 64.0, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 4.144230188961321}, + {'order': 128.0, 'q': 0.05, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 8.567566178840664}, + {'order': 1.25, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.156975532377146}, + {'order': 2.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.25125102072346206}, + {'order': 2.25, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.282691233053356}, + {'order': 4.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.5029837790076196}, + {'order': 4.5, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.5659924543164964}, + {'order': 10.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 1.2610908358120652}, + {'order': 25.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 3.175729623224624}, + {'order': 64.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 8.288460377922641}, + {'order': 128.0, 'q': 0.05, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 17.13513235768133}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 50.9870847446919}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 95.3948298140119}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 108.35534683261069}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 196.92988654267458}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 222.03953345186477}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 497.44157211889546}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 1247.6014738614642}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 3197.6608659372755}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 6397.679284315721}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 509.870847446919}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 953.948298140119}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 1083.553468326107}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 1969.2988654267458}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 2220.395334518648}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 4974.415721188954}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 12476.014738614642}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 31976.608659372756}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 63976.79284315721}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 5098.70847446919}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 9539.48298140119}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 10835.534683261068}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 19692.988654267458}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 22203.953345186477}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 49744.15721188955}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 124760.14738614642}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 319766.0865937275}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 639767.928431572}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 50987.0847446919}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 95394.8298140119}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 108355.3468326107}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 196929.8865426746}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 222039.53345186476}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 497441.5721188955}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 1247601.4738614643}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 3197660.8659372753}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 6397679.284315721}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 254935.4237234595}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 476974.1490700595}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 541776.7341630535}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 984649.4327133729}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 1110197.6672593239}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 2487207.8605944775}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 6238007.3693073215}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 15988304.329686377}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 31988396.421578605}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 509870.847446919}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 953948.298140119}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 1083553.468326107}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 1969298.8654267457}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 2220395.3345186478}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 4974415.721188955}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 12476014.738614643}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 31976608.659372754}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 63976792.84315721}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 1.804874412576095}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 11.394840954932146}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 13.855346885245137}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 28.929886542674605}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 33.03953345186479}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 77.44157211889551}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 197.60147386146454}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 509.6608659372759}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 1021.6792843157225}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 18.048744125760948}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 113.94840954932147}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 138.55346885245137}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 289.29886542674603}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 330.3953345186479}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 774.4157211889551}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 1976.0147386146455}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 5096.608659372759}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 10216.792843157225}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 180.4874412576095}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 1139.4840954932147}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 1385.5346885245137}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 2892.9886542674603}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 3303.953345186479}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 7744.157211889551}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 19760.147386146455}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 50966.08659372759}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 102167.92843157225}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 1804.874412576095}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 11394.840954932146}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 13855.346885245137}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 28929.886542674605}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 33039.533451864794}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 77441.5721188955}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 197601.47386146453}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 509660.8659372759}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 1021679.2843157225}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 9024.372062880475}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 56974.20477466073}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 69276.73442622568}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 144649.43271337304}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 165197.66725932396}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 387207.86059447756}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 988007.3693073227}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 2548304.3296863795}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 5108396.421578612}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 18048.74412576095}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 113948.40954932146}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 138553.46885245136}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 289298.8654267461}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 330395.3345186479}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 774415.7211889551}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 1976014.7386146453}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 5096608.659372759}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 10216792.843157224}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.09126475987344684}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.42916959059789994}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.7688994086444522}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 4.929960718132623}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 6.039543080827824}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 17.441572118895508}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 47.601473861464534}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 125.66086593727589}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 253.67928431572255}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 0.9126475987344684}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 4.291695905978999}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 7.688994086444522}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 49.29960718132623}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 60.395430808278235}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 174.41572118895508}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 476.01473861464535}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 1256.608659372759}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 2536.7928431572254}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 9.126475987344683}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 42.916959059789995}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 76.88994086444522}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 492.9960718132623}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 603.9543080827824}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 1744.1572118895508}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 4760.147386146454}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 12566.086593727588}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 25367.928431572254}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 91.26475987344685}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 429.16959059789997}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 768.8994086444521}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 4929.960718132623}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 6039.543080827823}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 17441.572118895507}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 47601.47386146453}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 125660.86593727588}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 253679.28431572256}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 456.32379936723424}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 2145.8479529895}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 3844.497043222261}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 24649.803590663116}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 30197.71540413912}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 87207.86059447753}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 238007.36930732266}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 628304.3296863794}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 1268396.4215786129}, + {'order': 1.25, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 912.6475987344685}, + {'order': 2.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 4291.695905979}, + {'order': 2.25, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 7688.994086444522}, + {'order': 4.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 49299.60718132623}, + {'order': 4.5, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 60395.43080827824}, + {'order': 10.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 174415.72118895507}, + {'order': 25.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 476014.7386146453}, + {'order': 64.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 1256608.6593727588}, + {'order': 128.0, 'q': 0.1, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 2536792.8431572258}, + {'order': 1.25, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.009408440726353454}, + {'order': 2.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.017036863236176647}, + {'order': 2.25, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.02009616460080257}, + {'order': 4.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.05867260696008058}, + {'order': 4.5, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.08420910671682878}, + {'order': 10.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 2.4428163733756034}, + {'order': 25.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 10.101473861818455}, + {'order': 64.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 29.66086593727589}, + {'order': 128.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 61.67928431572254}, + {'order': 1.25, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.09408440726353454}, + {'order': 2.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.17036863236176647}, + {'order': 2.25, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.2009616460080257}, + {'order': 4.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.5867260696008058}, + {'order': 4.5, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.8420910671682879}, + {'order': 10.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 24.428163733756033}, + {'order': 25.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 101.01473861818455}, + {'order': 64.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 296.6086593727589}, + {'order': 128.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 616.7928431572254}, + {'order': 1.25, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 0.9408440726353453}, + {'order': 2.0, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 1.7036863236176647}, + {'order': 2.25, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 2.009616460080257}, + {'order': 4.0, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 5.867260696008058}, + {'order': 4.5, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 8.420910671682877}, + {'order': 10.0, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 244.28163733756034}, + {'order': 25.0, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 1010.1473861818455}, + {'order': 64.0, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 2966.086593727589}, + {'order': 128.0, 'q': 0.1, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 6167.928431572254}, + {'order': 1.25, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 9.408440726353454}, + {'order': 2.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 17.036863236176647}, + {'order': 2.25, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 20.09616460080257}, + {'order': 4.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 58.67260696008058}, + {'order': 4.5, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 84.20910671682879}, + {'order': 10.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 2442.8163733756032}, + {'order': 25.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 10101.473861818455}, + {'order': 64.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 29660.86593727589}, + {'order': 128.0, 'q': 0.1, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 61679.28431572254}, + {'order': 1.25, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 47.042203631767265}, + {'order': 2.0, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 85.18431618088323}, + {'order': 2.25, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 100.48082300401285}, + {'order': 4.0, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 293.3630348004029}, + {'order': 4.5, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 421.0455335841439}, + {'order': 10.0, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 12214.081866878018}, + {'order': 25.0, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 50507.369309092275}, + {'order': 64.0, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 148304.32968637944}, + {'order': 128.0, 'q': 0.1, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 308396.4215786127}, + {'order': 1.25, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 94.08440726353453}, + {'order': 2.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 170.36863236176646}, + {'order': 2.25, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 200.9616460080257}, + {'order': 4.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 586.7260696008058}, + {'order': 4.5, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 842.0910671682879}, + {'order': 10.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 24428.163733756035}, + {'order': 25.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 101014.73861818455}, + {'order': 64.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 296608.6593727589}, + {'order': 128.0, 'q': 0.1, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 616792.8431572254}, + {'order': 1.25, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.001737654849037913}, + {'order': 2.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.002836228266263719}, + {'order': 2.25, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.0032125121598688307}, + {'order': 4.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.006003282964489696}, + {'order': 4.5, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.006856283806419175}, + {'order': 10.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.01862425425819884}, + {'order': 25.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.7514802200125161}, + {'order': 64.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 5.6608672584152}, + {'order': 128.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 13.679284315722684}, + {'order': 1.25, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.017376548490379132}, + {'order': 2.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.02836228266263719}, + {'order': 2.25, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.03212512159868831}, + {'order': 4.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.06003282964489696}, + {'order': 4.5, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.06856283806419175}, + {'order': 10.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.1862425425819884}, + {'order': 25.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 7.514802200125161}, + {'order': 64.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 56.608672584152}, + {'order': 128.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 136.79284315722686}, + {'order': 1.25, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.17376548490379132}, + {'order': 2.0, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.2836228266263719}, + {'order': 2.25, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.32125121598688305}, + {'order': 4.0, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.6003282964489696}, + {'order': 4.5, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.6856283806419174}, + {'order': 10.0, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 1.862425425819884}, + {'order': 25.0, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 75.14802200125162}, + {'order': 64.0, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 566.08672584152}, + {'order': 128.0, 'q': 0.1, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 1367.9284315722684}, + {'order': 1.25, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 1.7376548490379131}, + {'order': 2.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 2.8362282662637193}, + {'order': 2.25, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 3.212512159868831}, + {'order': 4.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 6.003282964489696}, + {'order': 4.5, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 6.8562838064191745}, + {'order': 10.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 18.62425425819884}, + {'order': 25.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 751.4802200125162}, + {'order': 64.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 5660.8672584152}, + {'order': 128.0, 'q': 0.1, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 13679.284315722685}, + {'order': 1.25, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 8.688274245189564}, + {'order': 2.0, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 14.181141331318596}, + {'order': 2.25, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 16.062560799344155}, + {'order': 4.0, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 30.01641482244848}, + {'order': 4.5, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 34.281419032095876}, + {'order': 10.0, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 93.1212712909942}, + {'order': 25.0, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 3757.4011000625806}, + {'order': 64.0, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 28304.336292076}, + {'order': 128.0, 'q': 0.1, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 68396.42157861342}, + {'order': 1.25, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 17.37654849037913}, + {'order': 2.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 28.362282662637192}, + {'order': 2.25, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 32.12512159868831}, + {'order': 4.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 60.03282964489696}, + {'order': 4.5, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 68.56283806419175}, + {'order': 10.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 186.2425425819884}, + {'order': 25.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 7514.802200125161}, + {'order': 64.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 56608.672584152}, + {'order': 128.0, 'q': 0.1, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 136792.84315722683}, + {'order': 1.25, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 6.276771861735142e-05}, + {'order': 2.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.00010049662088720303}, + {'order': 2.25, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.00011308433028369105}, + {'order': 4.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.00020135853558812981}, + {'order': 4.5, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.00022663139483825034}, + {'order': 10.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.000506162129856179}, + {'order': 25.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.001283133366967998}, + {'order': 64.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0034110637562587477}, + {'order': 128.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.007298431011732186}, + {'order': 1.25, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0006276771861735142}, + {'order': 2.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0010049662088720303}, + {'order': 2.25, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0011308433028369106}, + {'order': 4.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.002013585355881298}, + {'order': 4.5, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0022663139483825034}, + {'order': 10.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.0050616212985617895}, + {'order': 25.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.01283133366967998}, + {'order': 64.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.034110637562587476}, + {'order': 128.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.07298431011732186}, + {'order': 1.25, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.006276771861735142}, + {'order': 2.0, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.010049662088720303}, + {'order': 2.25, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.011308433028369105}, + {'order': 4.0, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.02013585355881298}, + {'order': 4.5, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.022663139483825034}, + {'order': 10.0, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.0506162129856179}, + {'order': 25.0, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.12831333669679978}, + {'order': 64.0, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.34110637562587476}, + {'order': 128.0, 'q': 0.1, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.7298431011732186}, + {'order': 1.25, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.06276771861735142}, + {'order': 2.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.10049662088720303}, + {'order': 2.25, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.11308433028369105}, + {'order': 4.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.20135853558812983}, + {'order': 4.5, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.22663139483825034}, + {'order': 10.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.5061621298561789}, + {'order': 25.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 1.283133366967998}, + {'order': 64.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 3.4110637562587476}, + {'order': 128.0, 'q': 0.1, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 7.298431011732186}, + {'order': 1.25, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.3138385930867571}, + {'order': 2.0, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.5024831044360152}, + {'order': 2.25, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 0.5654216514184552}, + {'order': 4.0, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 1.006792677940649}, + {'order': 4.5, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 1.1331569741912517}, + {'order': 10.0, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 2.530810649280895}, + {'order': 25.0, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 6.41566683483999}, + {'order': 64.0, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 17.05531878129374}, + {'order': 128.0, 'q': 0.1, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 36.49215505866093}, + {'order': 1.25, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 0.6276771861735142}, + {'order': 2.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 1.0049662088720304}, + {'order': 2.25, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 1.1308433028369105}, + {'order': 4.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 2.013585355881298}, + {'order': 4.5, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 2.2663139483825034}, + {'order': 10.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 5.06162129856179}, + {'order': 25.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 12.83133366967998}, + {'order': 64.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 34.11063756258748}, + {'order': 128.0, 'q': 0.1, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 72.98431011732185}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 54.4528141427903}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 96.78112417513178}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 109.6030117576186}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 197.85408278342118}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 222.93072268401326}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 498.2117356528509}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 1248.3235021745475}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 3198.3650154540346}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1, 'tfp_rdp_value': 6398.377889348097}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 544.528141427903}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 967.8112417513178}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 1096.030117576186}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 1978.5408278342118}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 2229.3072268401324}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 4982.117356528509}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 12483.235021745475}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 31983.650154540344}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10, 'tfp_rdp_value': 63983.778893480965}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 5445.28141427903}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 9678.112417513177}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 10960.30117576186}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 19785.408278342118}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 22293.072268401327}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 49821.17356528509}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 124832.35021745475}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 319836.50154540344}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.1, 'steps': 100, 'tfp_rdp_value': 639837.7889348096}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 54452.8141427903}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 96781.12417513179}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 109603.0117576186}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 197854.08278342118}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 222930.72268401325}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 498211.7356528509}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 1248323.5021745474}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 3198365.015454035}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.1, 'steps': 1000, 'tfp_rdp_value': 6398377.889348097}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 272264.0707139515}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 483905.6208756589}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 548015.058788093}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 989270.4139171059}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 1114653.6134200662}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 2491058.6782642547}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 6241617.5108727375}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 15991825.077270173}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.1, 'steps': 5000, 'tfp_rdp_value': 31991889.446740482}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 544528.141427903}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 967811.2417513177}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 1096030.117576186}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 1978540.8278342118}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 2229307.2268401324}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 4982117.356528509}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 12483235.021745475}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 31983650.154540345}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.1, 'steps': 10000, 'tfp_rdp_value': 63983778.893480964}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 3.4988089172054977}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 12.781126875972346}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 15.10301177558829}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 29.8540827834212}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 33.93072268401329}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 78.211735652851}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 198.3235021745478}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 510.3650154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1, 'tfp_rdp_value': 1022.3778893480978}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 34.98808917205498}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 127.81126875972346}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 151.0301177558829}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 298.540827834212}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 339.3072268401329}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 782.1173565285101}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 1983.235021745478}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 5103.650154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10, 'tfp_rdp_value': 10223.778893480978}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 349.88089172054976}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 1278.1126875972345}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 1510.301177558829}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 2985.40827834212}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 3393.072268401329}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 7821.1735652851}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 19832.35021745478}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 51036.50154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.25, 'steps': 100, 'tfp_rdp_value': 102237.78893480978}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 3498.808917205498}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 12781.126875972346}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 15103.011775588291}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 29854.082783421203}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 33930.72268401329}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 78211.735652851}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 198323.5021745478}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 510365.0154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.25, 'steps': 1000, 'tfp_rdp_value': 1022377.8893480978}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 17494.04458602749}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 63905.63437986173}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 75515.05887794145}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 149270.413917106}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 169653.61342006645}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 391058.678264255}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 991617.510872739}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 2551825.077270176}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.25, 'steps': 5000, 'tfp_rdp_value': 5111889.446740489}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 34988.08917205498}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 127811.26875972346}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 151030.1177558829}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 298540.827834212}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 339307.2268401329}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 782117.35652851}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 1983235.021745478}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 5103650.154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.25, 'steps': 10000, 'tfp_rdp_value': 10223778.893480979}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 0.2693155798829554}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 1.1454723378159812}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 1.7160611134797041}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 5.85411562333582}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 6.930726961789178}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 18.211735652851}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 48.32350217454782}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 126.3650154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1, 'tfp_rdp_value': 254.37788934809794}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 2.693155798829554}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 11.454723378159812}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 17.16061113479704}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 58.541156233358194}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 69.30726961789178}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 182.11735652851002}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 483.2350217454782}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 1263.650154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10, 'tfp_rdp_value': 2543.7788934809796}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 26.93155798829554}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 114.54723378159812}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 171.60611134797043}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 585.411562333582}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 693.0726961789178}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 1821.1735652851}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 4832.350217454782}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 12636.50154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.5, 'steps': 100, 'tfp_rdp_value': 25437.788934809792}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 269.3155798829554}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 1145.4723378159813}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 1716.061113479704}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 5854.11562333582}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 6930.726961789178}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 18211.735652851}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 48323.50217454782}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 126365.01545403521}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.5, 'steps': 1000, 'tfp_rdp_value': 254377.88934809793}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 1346.577899414777}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 5727.361689079906}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 8580.30556739852}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 29270.5781166791}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 34653.63480894589}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 91058.67826425501}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 241617.5108727391}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 631825.077270176}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.5, 'steps': 5000, 'tfp_rdp_value': 1271889.4467404897}, + {'order': 1.25, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 2693.155798829554}, + {'order': 2.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 11454.723378159812}, + {'order': 2.25, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 17160.61113479704}, + {'order': 4.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 58541.1562333582}, + {'order': 4.5, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 69307.26961789178}, + {'order': 10.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 182117.35652851002}, + {'order': 25.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 483235.0217454782}, + {'order': 64.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 1263650.154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 0.5, 'steps': 10000, 'tfp_rdp_value': 2543778.8934809794}, + {'order': 1.25, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.034642316647709226}, + {'order': 2.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.06647221890559733}, + {'order': 2.25, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.08026393760128477}, + {'order': 4.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.29132308006676827}, + {'order': 4.5, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0.429697843698239}, + {'order': 10.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 3.212286119795191}, + {'order': 25.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 10.82350217470511}, + {'order': 64.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 30.365015454035202}, + {'order': 128.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 62.37788934809791}, + {'order': 1.25, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.34642316647709226}, + {'order': 2.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.6647221890559734}, + {'order': 2.25, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 0.8026393760128476}, + {'order': 4.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 2.913230800667683}, + {'order': 4.5, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 4.29697843698239}, + {'order': 10.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 32.122861197951906}, + {'order': 25.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 108.2350217470511}, + {'order': 64.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 303.650154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10, 'tfp_rdp_value': 623.7788934809791}, + {'order': 1.25, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 3.4642316647709226}, + {'order': 2.0, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 6.647221890559734}, + {'order': 2.25, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 8.026393760128476}, + {'order': 4.0, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 29.132308006676826}, + {'order': 4.5, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 42.9697843698239}, + {'order': 10.0, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 321.2286119795191}, + {'order': 25.0, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 1082.350217470511}, + {'order': 64.0, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 3036.50154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 1.0, 'steps': 100, 'tfp_rdp_value': 6237.788934809791}, + {'order': 1.25, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 34.64231664770922}, + {'order': 2.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 66.47221890559733}, + {'order': 2.25, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 80.26393760128477}, + {'order': 4.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 291.3230800667683}, + {'order': 4.5, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 429.697843698239}, + {'order': 10.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 3212.2861197951906}, + {'order': 25.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 10823.50217470511}, + {'order': 64.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 30365.0154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 62377.88934809791}, + {'order': 1.25, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 173.21158323854613}, + {'order': 2.0, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 332.36109452798667}, + {'order': 2.25, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 401.3196880064238}, + {'order': 4.0, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 1456.6154003338413}, + {'order': 4.5, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 2148.489218491195}, + {'order': 10.0, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 16061.430598975954}, + {'order': 25.0, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 54117.51087352555}, + {'order': 64.0, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 151825.077270176}, + {'order': 128.0, 'q': 0.2, 'sigma': 1.0, 'steps': 5000, 'tfp_rdp_value': 311889.44674048957}, + {'order': 1.25, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 346.42316647709225}, + {'order': 2.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 664.7221890559733}, + {'order': 2.25, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 802.6393760128476}, + {'order': 4.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 2913.2308006676826}, + {'order': 4.5, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 4296.97843698239}, + {'order': 10.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 32122.86119795191}, + {'order': 25.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 108235.0217470511}, + {'order': 64.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 303650.154540352}, + {'order': 128.0, 'q': 0.2, 'sigma': 1.0, 'steps': 10000, 'tfp_rdp_value': 623778.8934809791}, + {'order': 1.25, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.006824777906499827}, + {'order': 2.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.011296964989239926}, + {'order': 2.25, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.012858854549446597}, + {'order': 4.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.02497156532232035}, + {'order': 4.5, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.028878485101326103}, + {'order': 10.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 0.09784245743693407}, + {'order': 25.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 1.459148041400704}, + {'order': 64.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 6.365016041204642}, + {'order': 128.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1, 'tfp_rdp_value': 14.377889348097982}, + {'order': 1.25, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.06824777906499827}, + {'order': 2.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.11296964989239926}, + {'order': 2.25, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.12858854549446597}, + {'order': 4.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.2497156532232035}, + {'order': 4.5, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.288784851013261}, + {'order': 10.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 0.9784245743693407}, + {'order': 25.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 14.59148041400704}, + {'order': 64.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 63.65016041204642}, + {'order': 128.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10, 'tfp_rdp_value': 143.77889348097983}, + {'order': 1.25, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 0.6824777906499827}, + {'order': 2.0, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 1.1296964989239926}, + {'order': 2.25, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 1.2858854549446597}, + {'order': 4.0, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 2.497156532232035}, + {'order': 4.5, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 2.8878485101326103}, + {'order': 10.0, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 9.784245743693406}, + {'order': 25.0, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 145.9148041400704}, + {'order': 64.0, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 636.5016041204642}, + {'order': 128.0, 'q': 0.2, 'sigma': 2.0, 'steps': 100, 'tfp_rdp_value': 1437.7889348097983}, + {'order': 1.25, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 6.824777906499827}, + {'order': 2.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 11.296964989239926}, + {'order': 2.25, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 12.858854549446598}, + {'order': 4.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 24.971565322320348}, + {'order': 4.5, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 28.878485101326103}, + {'order': 10.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 97.84245743693407}, + {'order': 25.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 1459.1480414007042}, + {'order': 64.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 6365.0160412046425}, + {'order': 128.0, 'q': 0.2, 'sigma': 2.0, 'steps': 1000, 'tfp_rdp_value': 14377.889348097982}, + {'order': 1.25, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 34.123889532499135}, + {'order': 2.0, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 56.48482494619963}, + {'order': 2.25, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 64.29427274723298}, + {'order': 4.0, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 124.85782661160174}, + {'order': 4.5, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 144.3924255066305}, + {'order': 10.0, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 489.21228718467034}, + {'order': 25.0, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 7295.7402070035205}, + {'order': 64.0, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 31825.08020602321}, + {'order': 128.0, 'q': 0.2, 'sigma': 2.0, 'steps': 5000, 'tfp_rdp_value': 71889.4467404899}, + {'order': 1.25, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 68.24777906499827}, + {'order': 2.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 112.96964989239926}, + {'order': 2.25, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 128.58854549446596}, + {'order': 4.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 249.71565322320347}, + {'order': 4.5, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 288.784851013261}, + {'order': 10.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 978.4245743693407}, + {'order': 25.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 14591.480414007041}, + {'order': 64.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 63650.16041204642}, + {'order': 128.0, 'q': 0.2, 'sigma': 2.0, 'steps': 10000, 'tfp_rdp_value': 143778.8934809798}, + {'order': 1.25, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0002509008471776042}, + {'order': 2.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0004019259003295167}, + {'order': 2.25, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.00045234870800072715}, + {'order': 4.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0008064504053491498}, + {'order': 4.5, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0009079912638877577}, + {'order': 10.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.002035935314438469}, + {'order': 25.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.005219181973251608}, + {'order': 64.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.014331998148967217}, + {'order': 128.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.03281124876177494}, + {'order': 1.25, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.002509008471776042}, + {'order': 2.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.004019259003295167}, + {'order': 2.25, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.004523487080007271}, + {'order': 4.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.008064504053491498}, + {'order': 4.5, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.009079912638877577}, + {'order': 10.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.020359353144384692}, + {'order': 25.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.052191819732516076}, + {'order': 64.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.14331998148967218}, + {'order': 128.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10, 'tfp_rdp_value': 0.3281124876177494}, + {'order': 1.25, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.02509008471776042}, + {'order': 2.0, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.04019259003295167}, + {'order': 2.25, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.04523487080007271}, + {'order': 4.0, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.08064504053491497}, + {'order': 4.5, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.09079912638877577}, + {'order': 10.0, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.20359353144384693}, + {'order': 25.0, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 0.5219181973251608}, + {'order': 64.0, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 1.4331998148967218}, + {'order': 128.0, 'q': 0.2, 'sigma': 10.0, 'steps': 100, 'tfp_rdp_value': 3.2811248761774943}, + {'order': 1.25, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.2509008471776042}, + {'order': 2.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.4019259003295167}, + {'order': 2.25, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.45234870800072713}, + {'order': 4.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.8064504053491498}, + {'order': 4.5, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0.9079912638877577}, + {'order': 10.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 2.0359353144384693}, + {'order': 25.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 5.219181973251608}, + {'order': 64.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 14.331998148967218}, + {'order': 128.0, 'q': 0.2, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 32.81124876177494}, + {'order': 1.25, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 1.254504235888021}, + {'order': 2.0, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 2.0096295016475834}, + {'order': 2.25, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 2.2617435400036356}, + {'order': 4.0, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 4.0322520267457485}, + {'order': 4.5, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 4.539956319438788}, + {'order': 10.0, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 10.179676572192346}, + {'order': 25.0, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 26.095909866258037}, + {'order': 64.0, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 71.6599907448361}, + {'order': 128.0, 'q': 0.2, 'sigma': 10.0, 'steps': 5000, 'tfp_rdp_value': 164.0562438088747}, + {'order': 1.25, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 2.509008471776042}, + {'order': 2.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 4.019259003295167}, + {'order': 2.25, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 4.523487080007271}, + {'order': 4.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 8.064504053491497}, + {'order': 4.5, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 9.079912638877577}, + {'order': 10.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 20.35935314438469}, + {'order': 25.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 52.19181973251607}, + {'order': 64.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 143.3199814896722}, + {'order': 128.0, 'q': 0.2, 'sigma': 10.0, 'steps': 10000, 'tfp_rdp_value': 328.1124876177494}, + {'order': 2.0, 'q': 0.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 2.5, 'q': 0.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 4.0, 'q': 0.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 4.5, 'q': 0.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 8.0, 'q': 0.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 8.5, 'q': 0.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 2.0, 'q': 0.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 2.5, 'q': 0.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 4.0, 'q': 0.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 4.5, 'q': 0.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 8.0, 'q': 0.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 8.5, 'q': 0.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 2.0, 'q': 0.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 2.5, 'q': 0.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 4.0, 'q': 0.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 4.5, 'q': 0.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 8.0, 'q': 0.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 8.5, 'q': 0.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0}, + {'order': 2.0, 'q': 0.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 2.5, 'q': 0.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 4.0, 'q': 0.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 4.5, 'q': 0.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 8.0, 'q': 0.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 8.5, 'q': 0.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 0}, + {'order': 2.0, 'q': 1.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 1.0}, + {'order': 2.5, 'q': 1.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 1.25}, + {'order': 4.0, 'q': 1.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 2.0}, + {'order': 4.5, 'q': 1.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 2.25}, + {'order': 8.0, 'q': 1.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 4.0}, + {'order': 8.5, 'q': 1.0, 'sigma': 1.0, 'steps': 1, 'tfp_rdp_value': 4.25}, + {'order': 2.0, 'q': 1.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 1000.0}, + {'order': 2.5, 'q': 1.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 1250.0}, + {'order': 4.0, 'q': 1.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 2000.0}, + {'order': 4.5, 'q': 1.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 2250.0}, + {'order': 8.0, 'q': 1.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 4000.0}, + {'order': 8.5, 'q': 1.0, 'sigma': 1.0, 'steps': 1000, 'tfp_rdp_value': 4250.0}, + {'order': 2.0, 'q': 1.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.01}, + {'order': 2.5, 'q': 1.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0125}, + {'order': 4.0, 'q': 1.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.02}, + {'order': 4.5, 'q': 1.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0225}, + {'order': 8.0, 'q': 1.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.04}, + {'order': 8.5, 'q': 1.0, 'sigma': 10.0, 'steps': 1, 'tfp_rdp_value': 0.0425}, + {'order': 2.0, 'q': 1.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 10.0}, + {'order': 2.5, 'q': 1.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 12.5}, + {'order': 4.0, 'q': 1.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 20.0}, + {'order': 4.5, 'q': 1.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 22.5}, + {'order': 8.0, 'q': 1.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 40.0}, + {'order': 8.5, 'q': 1.0, 'sigma': 10.0, 'steps': 1000, 'tfp_rdp_value': 42.5}) + +# Relative tolerance for lower bound testing +rel_tol = 0.01 + +def _dp_sgd_poisson_rdp(q, sigma, steps, order): + sample = AmplificationBySampling(PoissonSampling=True) + compose = Composition() + gaussian_mechanism = ExactGaussianMechanism(sigma) + dp_sgd = compose([sample(gaussian_mechanism, q)], [steps]) + return dp_sgd.get_RDP(order) + + +class TestDPSGDPoisson(parameterized.TestCase): + + @parameterized.parameters(p for p in params) + def test_RDP_values_lowerbound(self, q, sigma, steps, order, tfp_rdp_value): + autodp_rdp_value = _dp_sgd_poisson_rdp(q, sigma, steps, order) + self.assertGreaterEqual(autodp_rdp_value, (1. - rel_tol) * tfp_rdp_value) + + +if __name__ == '__main__': + absltest.main() \ No newline at end of file diff --git a/test/unit_test_fdp_to_approxdp_conversion.py b/test/unit_test_fdp_to_approxdp_conversion.py new file mode 100644 index 0000000..f718d05 --- /dev/null +++ b/test/unit_test_fdp_to_approxdp_conversion.py @@ -0,0 +1,47 @@ +from autodp.mechanism_zoo import GaussianMechanism +from autodp.dp_bank import get_eps_ana_gaussian + +import numpy as np + +from absl.testing import absltest +from absl.testing import parameterized + +params = [0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0] + + +def _fdp_conversion(sigma): + + delta_list = [0,1e-8, 1e-6, 1e-4, 1e-2, 0.3, 0.5, 1] + + # f-DP implementation + gm3 = GaussianMechanism(sigma, name='GM3', RDP_off=True, approxDP_off=True, fdp_off=False) + + # direct approxdp implementation + agm = lambda x: get_eps_ana_gaussian(sigma, x) + + eps_direct = np.array([agm(delta) for delta in delta_list]) + + # the fdp is converted by numerical methods from privacy profile. + eps_converted = np.array([gm3.get_approxDP(delta) for delta in delta_list]) + max_diff = eps_direct - eps_converted + + rel_diff = max_diff / (eps_direct+1e-10) + + if np.isinf(eps_direct[0]) and np.isinf(eps_converted[0]): + rel_diff[0] = 0 + return rel_diff + + +_fdp_conversion(1.0) + +class Test_approxDP2fDP_Conversion(parameterized.TestCase): + + @parameterized.parameters(p for p in params) + def test_fdp_conversion(self, sigma): + max_diff = _fdp_conversion(sigma) + self.assertSequenceAlmostEqual(max_diff, np.zeros_like(max_diff), places=2) + + +if __name__ == '__main__': + absltest.main() + diff --git a/tutorials/DP_vs_CDP_vs_RDP.ipynb b/tutorials/legacy/DP_vs_CDP_vs_RDP.ipynb similarity index 100% rename from tutorials/DP_vs_CDP_vs_RDP.ipynb rename to tutorials/legacy/DP_vs_CDP_vs_RDP.ipynb diff --git a/tutorials/pure-dp-approximation-scheme.ipynb b/tutorials/legacy/pure-dp-approximation-scheme.ipynb similarity index 100% rename from tutorials/pure-dp-approximation-scheme.ipynb rename to tutorials/legacy/pure-dp-approximation-scheme.ipynb diff --git a/tutorials/tutorial_autodp.ipynb b/tutorials/legacy/tutorial_legacy_api.ipynb similarity index 100% rename from tutorials/tutorial_autodp.ipynb rename to tutorials/legacy/tutorial_legacy_api.ipynb diff --git a/tutorials/tutorial_privacy_calibrator.ipynb b/tutorials/legacy/tutorial_privacy_calibrator.ipynb similarity index 100% rename from tutorials/tutorial_privacy_calibrator.ipynb rename to tutorials/legacy/tutorial_privacy_calibrator.ipynb diff --git a/tutorials/tutorial_calibrator.ipynb b/tutorials/tutorial_calibrator.ipynb new file mode 100644 index 0000000..cf0f48a --- /dev/null +++ b/tutorials/tutorial_calibrator.ipynb @@ -0,0 +1,196 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "import sys\n", + "import os\n", + "from autodp.calibrator_zoo import eps_delta_calibrator,generalized_eps_delta_calibrator, ana_gaussian_calibrator\n", + "from autodp import rdp_bank\n", + "from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism,SubsampleGaussianMechanism, GaussianMechanism, ComposedGaussianMechanism, LaplaceMechanism\n", + "from autodp.transformer_zoo import Composition, AmplificationBySampling\n", + "%matplotlib inline \n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 1: calibrating noise to privacy budgets.\n", + "### Single parameter, composition is not applicable" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "GM {'sigma': 36.30468875627065} 0.10000000492560457\n", + "Ana_GM {'sigma': 36.304691899114694} 0.09999999565548193\n" + ] + } + ], + "source": [ + "calibrate = eps_delta_calibrator()\n", + "ana_calibrate = ana_gaussian_calibrator()\n", + "eps = 0.1\n", + "delta = 1e-6\n", + "\n", + "mech1 = calibrate(ExactGaussianMechanism,eps,delta,[0,100],name='GM')\n", + "mech2 = ana_calibrate(ExactGaussianMechanism, eps, delta, name='Ana_GM')\n", + "print(mech1.name, mech1.params, mech1.get_approxDP(delta))\n", + "print(mech2.name, mech2.params, mech2.get_approxDP(delta))\n", + "\n", + "# privacy budget is (0.1, 1e-6), calibrate noise sigma\n", + "# mech1 is for Gaussian mechanism and mech2 is for analytical Gaussian mechanism\n" + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Example 2: Calibration with Gaussian mechanism under composition" + ], + "metadata": { + "collapsed": false + } + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Composed_Gaussian {'sigma': 176.48801940301632, 'coeff': 20} 0.09999999894380397\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.8/site-packages/scipy/optimize/optimize.py:2522: RuntimeWarning: invalid value encountered in double_scalars\n", + " w = xb - ((xb - xc) * tmp2 - (xb - xa) * tmp1) / denom\n" + ] + } + ], + "source": [ + "#We now have multiple parameters --- params['coeff'] and params['sigma'].\n", + "\n", + "\n", + "coeff = 20\n", + "general_calibrate = generalized_eps_delta_calibrator()\n", + "params = {}\n", + "params['sigma'] = None\n", + "params['coeff'] = 20\n", + "\n", + "mech3 = general_calibrate(ComposedGaussianMechanism, eps, delta, [0,1000],params=params,para_name='sigma', name='Composed_Gaussian')\n", + "print(mech3.name, mech3.params, mech3.get_approxDP(delta))\n", + "#coeff is the number of composition. The calibrator calibrates the noise with other parameters (coeff) fixed.\n", + "#[0,1000] is the range of sigma.\n" + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Example 3: calibration with SubsampledGaussian mechanism" + ], + "metadata": { + "collapsed": false + } + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Subsampled_Gaussian {'prob': 0.01, 'sigma': 3.205947497273216, 'coeff': 20} 0.09999976859421401\n" + ] + } + ], + "source": [ + "# We now have three parameters --- params['coeff'], params['prob'] and params['sigma'].\n", + "# The coeff and prob are fixed and the calibrator optimize over sigma. We use para_name to denote the parameter that we want to optimize over.\n", + "\n", + "params['prob'] = 0.01\n", + "mech4 = general_calibrate(SubsampleGaussianMechanism, eps, delta, [0,1000],params=params,para_name='sigma', name='Subsampled_Gaussian')\n", + "print(mech4.name, mech4.params, mech4.get_approxDP(delta))\n" + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Example 4: Calibration with single parameter for Laplace mechanism" + ], + "metadata": { + "collapsed": false + } + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "pycharm": { + "name": "#%%\n" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Laplace {'b': 9.99979887663798} 0.10000001127407157\n" + ] + } + ], + "source": [ + "calibrate = generalized_eps_delta_calibrator()\n", + "\n", + "eps = 0.1\n", + "delta = 1e-6\n", + "mech = calibrate(LaplaceMechanism,eps,delta,[0,100],name='Laplace')\n", + "print(mech.name, mech.params, mech.get_approxDP(delta))\n", + "#[0, 100] is the range of laplace parameter b.\n", + "# to calibrate the noise for the composed Laplace mechanism, we can define a new composed Lapalce mechanism in mechanism.zoo (similar with Example 2)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.5" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} \ No newline at end of file diff --git a/tutorials/tutorial_new_api.ipynb b/tutorials/tutorial_new_api.ipynb new file mode 100644 index 0000000..c8a965a --- /dev/null +++ b/tutorials/tutorial_new_api.ipynb @@ -0,0 +1,793 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism\n", + "from autodp.transformer_zoo import Composition\n", + "import matplotlib.pyplot as plt\n", + "%matplotlib inline \n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 1: RDP composition of a heteogeneous sequence of mechanisms" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "sigma1 = 5.0\n", + "sigma2 = 8.0\n", + "\n", + "gm1 = ExactGaussianMechanism(sigma1,name='GM1')\n", + "gm2 = ExactGaussianMechanism(sigma2,name='GM2')\n", + "SVT = PureDP_Mechanism(eps=0.1,name='SVT')\n", + "\n", + "# run gm1 for 3 rounds\n", + "# run gm2 for 5 times\n", + "# run SVT for once\n", + "\n", + "# compose them with the transformation: compose.\n", + "compose = Composition()\n", + "composed_mech = compose([gm1, gm2, SVT], [3, 5, 1])\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Mechanism name is \" Compose:{GM1: 3, GM2: 5, SVT: 1} \"\n", + "Parameters are: {'GM1:sigma': 5.0, 'GM2:sigma': 8.0, 'SVT:eps': 0.1}\n", + "epsilon(delta) = 2.18001192542518 , at delta = 1e-06\n", + "epsilon(delta) = 1.689983703842748 , at delta = 0.0001\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.8/site-packages/scipy/optimize/optimize.py:2522: RuntimeWarning: invalid value encountered in double_scalars\n", + " w = xb - ((xb - xc) * tmp2 - (xb - xa) * tmp1) / denom\n" + ] + } + ], + "source": [ + "# Query for eps given delta\n", + "delta1 = 1e-6\n", + "eps1 = composed_mech.get_approxDP(delta1)\n", + "\n", + "delta2 = 1e-4\n", + "eps2 = composed_mech.get_approxDP(delta2)\n", + "\n", + "\n", + "# Get name of the composed object, a structured description of the mechanism generated automatically\n", + "print('Mechanism name is \\\"', composed_mech.name,'\\\"')\n", + "print('Parameters are: ',composed_mech.params)\n", + "print('epsilon(delta) = ', eps1, ', at delta = ', delta1)\n", + "print('epsilon(delta) = ', eps2, ', at delta = ', delta2)\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAFzCAYAAAAzNA41AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAucElEQVR4nO3dd3hVZb728e8vhQRIqEmoCaEEkF4CKk0sWFDBrqhjHVFULMw4x3md43h0jnMce0HFLo4iig0VxYY0QQi9CYQeioTeIeV5/9jbczIMhA3Jysre+/5cVy6z9l7Z615SblZ7HnPOISIi0SvG7wAiIuIvFYGISJRTEYiIRDkVgYhIlFMRiIhEORWBiEiUi/M7wPFKSUlxmZmZfscQEQkrs2bN2uKcSz3Se2FXBJmZmeTk5PgdQ0QkrJjZmqO9p1NDIiJRTkUgIhLlVAQiIlFORSAiEuVUBCIiUU5FICIS5VQEIiJRTkUgIhLlVAQiIlHOsyIwszfMbLOZLTzK+2Zmz5lZrpnNN7MuXmUREZGj8/KI4C3g3FLePw/ICn4NBl7yMIuIiByFZ0XgnJsEbCtllYHASBcwHahlZg28yrN59wHGL9rk1ceLiIQtP68RNALWlVjOC772b8xssJnlmFlOfn7+CW3sg5nruPWdWeTvPnhCPy8iEqnC4mKxc+4V51y2cy47NfWIo6geU99WaQBMWnZiRSIiEqn8LIL1QHqJ5cbB1zzRpkENUpMTmLB0s1ebEBEJS34WwVjguuDdQ6cAO51zG73aWEyMcVrLVCYv30JhUbFXmxERCTte3j46CpgGtDKzPDO72cxuM7PbgquMA1YCucCrwO1eZflN31ap7NxfwLy8HV5vSkQkbHg2Q5lzbtAx3nfAHV5t/0h6t0glxuDHpfl0bVKnIjctIlJphcXF4vJSs1o8XTJq8+NSXTAWEflNVBUBBE4PLVi/U7eRiogERWER6DZSEZGSoq4I2jSoQUqSbiMVEflN1BWBbiMVEflXUVcEAP3apLFzfwHTVm71O4qIiO+isgj6tkojOTGOT+Z49iCziEjYiMoiSIyP5fz2DRi/cBP7DhX6HUdExFdRWQQAF3VuxN5DRXy7+Fe/o4iI+Cpqi6B7Zh0a1kzkU50eEpEoF7VFEBNjDOzciEnLt7Bljx4uE5HoFbVFAHBx50YUFTu+mLfB7ygiIr6J6iJoWS+ZNg1q8MlcFYGIRK+oLgIIHBXMW7eD3M17/I4iIuKLqC+Cizo3Ii7GGD1zrd9RRER8EfVFkJqcQL829RgzK4+DhUV+xxERqXBRXwQAg7pnsH1fAd8s0jMFIhJ9VARArxYpNK5dlVEzdHpIRKKPioDAMwVXdUvnpxVbWbN1r99xREQqlIog6PLsdGJjjPdnrvM7iohIhVIRBNWrkcgZrdP4MCePAs1TICJRREVQwtXdM9iy5yDjF23yO4qISIVREZTQp2UqGXWq8fZPq/2OIiJSYVQEJcTGGNed2oSZq7ezaMNOv+OIiFQIFcFhLu+aTtX4WEb+tMbvKCIiFUJFcJia1eK5qHMjPp27nh37DvkdR0TEcyqCI7i+RxMOFhYzWreSikgUUBEcQev6NTi5aR3emb6GomLndxwREU+pCI7ihh6Z5G3frzmNRSTiqQiOol+bejSuXZXXp6z0O4qIiKdUBEcRFxvDTT2bMnP1duau2+F3HBERz6gISnFFt3SSE+N4bbKOCkQkcqkISpGUEMfV3TP4auEm8rbv8zuOiIgnVATHcH2PTADemrra1xwiIl5RERxDw1pVOb99A96fuY5dBwr8jiMiUu5UBCG4pXcz9hwsZNTPmsFMRCKPiiAE7RvXpFeLFF6bsooDBZrgXkQii4ogREP6Nid/90E+mbPe7ygiIuVKRRCiHs3r0qFxTUZMXKFhJ0QkoqgIQmRmDDmtOau37uPrhZrBTEQih4rgOJzdtj7NUqrz8sQVOKejAhGJDCqC4xAbY9x6WjMWrN/J5OVb/I4jIlIuVATH6aLOjWhQM5EXfsj1O4qISLlQERynhLhYbu3TjBmrtzF95Va/44iIlJmK4ARc1T2DlKQEHRWISERQEZyAxPhYBvdpypTcLcxeu93vOCIiZaIiOEHXnNyE2tXief775X5HEREpExXBCaqeEMfNvZoyYWk+C/J2+h1HROSEqQjK4LoemdSsGs+z3y/zO4qIyAlTEZRBjcR4ft+rKd8t2cz8vB1+xxEROSGeFoGZnWtmS80s18zuP8L7GWY2wczmmNl8M+vvZR4v3NAzk1rV4nnmO10rEJHw5FkRmFksMBw4D2gDDDKzNoet9hfgA+dcZ+Aq4EWv8nglOTGeW3o344dfNmuSexEJS14eEXQHcp1zK51zh4D3gYGHreOAGsHvawIbPMzjmet7ZFK7WjzPfKdrBSISfrwsgkbAuhLLecHXSnoIuNbM8oBxwFAP83gmKSGOwX2a8+PSfD1XICJhx++LxYOAt5xzjYH+wDtm9m+ZzGywmeWYWU5+fn6FhwzFdac2ISWpCk9+s9TvKCIix8XLIlgPpJdYbhx8raSbgQ8AnHPTgEQg5fAPcs694pzLds5lp6amehS3bKonxDGkbwum5m7lpxUamVREwoeXRTATyDKzpmZWhcDF4LGHrbMWOBPAzE4iUASV85/8Ibjm5Awa1EzkifFLNV+BiIQNz4rAOVcI3AmMB5YQuDtokZk9bGYDgqv9AbjFzOYBo4AbXBj/DZoYH8vQM7KYvXYHE5Zu9juOiEhILNz+3s3OznY5OTl+xziqgqJiznpqItWqxPHl0F7ExJjfkUREMLNZzrnsI73n98XiiBMfG8O9Z7VkycZdfLlgo99xRESOSUXggQs7NqR1/WSe/GYpBUXFfscRESmVisADsTHGfee0YvXWfYyeue7YPyAi4iMVgUfOaJ1Gt8zaPPv9cvYdKvQ7jojIUakIPGJm3H9ea/J3H+SNKav8jiMiclQqAg91bVKHfm3qMWLiSrbvPeR3HBGRI1IReOxP57Ri76FCntdE9yJSSakIPJZVL5nLu6bzzvTVrNm61+84IiL/RkVQAYad3ZK4mBj+MV4D0olI5aMiqAD1aiRyS++mfDl/I3M0TLWIVDIqggoy+LTmpCRV4dFxSzQgnYhUKiqCCpKUEMc9Z7Vk5urtjF/0q99xRET+l4qgAl3VLZ0WaUn8z1dLOFSooSdEpHJQEVSguNgYHuh/Equ37uOd6Wv8jiMiAqgIKlzfVqn0zkrh2e+W6SEzEakUVAQVzMz4y/lt2HOwkGe/X+53HBERFYEfWtVP5qruGfxz+hpW5O/xO46IRDkVgU+G9WtJYnwsf/tisd9RRCTKqQh8kpKUwNAzWjBhab7mNxYRX6kIfHRjz6Y0TanOI18s1kxmIuIbFYGPqsTF8JfzT2Jl/l5GTtPtpCLiDxWBz85onUaflqk8890ytu456HccEYlCKgKfmRkPXnAS+w8V8cQ3y/yOIyJRSEVQCbRIS+b6Hpm8P3MtC/J2+h1HRKKMiqCSuPusLOpWT+DBsQspLtbopCJScVQElUSNxHjuP681c9bu4KPZeX7HEZEooiKoRC7p3IguGbV47Otf2HWgwO84IhIlVASVSEyM8fDAdmzde4inv9WFYxGpGCqCSqZdo5pc3T2DkdPWsGTjLr/jiEgUUBFUQved04qaVeP5z0914VhEvKciqIRqVavC/ee2JmfNdl04FhHPqQgqqcu6NqZLRi3+56tf2LlPF45FxDsqgkoqJsZ45KJ2bN93iMe/+cXvOCISwVQElVjbhjW5vkcm7/68lrnrdvgdR0QilIqgkhvWryVpyQn8v48XUKihqkXEAyqCSi45MZ6/XtiWxRt38baGqhYRD6gIwsB57erTt1UqT32zlI079/sdR0QijIogDJgZjwxsR2Gx47/Gao5jESlfKoIwkV6nGneflcXXizbxzaJNfscRkQiiIggjt/RuRuv6yfx17CL2HCz0O46IRAgVQRiJj43h0Uvas2nXAZ4Yv9TvOCISIVQEYaZLRm1+d0oT3p62Ws8WiEi5UBGEofvOaUVacgL3fzSfAj1bICJlpCIIQ8mJ8Tw8sB2/bNrNK5NW+h1HRMKciiBMndO2Pv3b1+fZ75ezIn+P33FEJIypCMLYQwPakhgXw58/XqB5C0TkhKkIwlhaciJ/Ob8NM1ZtY9TMtX7HEZEwpSIIc5dnN6Zni7r8fdwvbNih4SdE5PipCMKcmfH3iztQVOx44JMFOKdTRCJyfFQEESCjbjXuO6cVE5bm8+nc9X7HEZEwoyKIENf3yKRrk9o8NHYxm3cf8DuOiIQRT4vAzM41s6Vmlmtm9x9lnSvMbLGZLTKz97zME8liY4zHLu3A/oIiHvx0kU4RiUjIPCsCM4sFhgPnAW2AQWbW5rB1soA/Az2dc22Be7zKEw1apCVx71kt+XrRJr6Yv9HvOCISJrw8IugO5DrnVjrnDgHvAwMPW+cWYLhzbjuAc26zh3miwi29m9IxvRYPfraQ/N0H/Y4jImHAyyJoBKwrsZwXfK2klkBLM5tqZtPN7NwjfZCZDTazHDPLyc/P9yhuZIiLjeGJyzqw92AR//npQp0iEpFj8vticRyQBfQFBgGvmlmtw1dyzr3inMt2zmWnpqZWbMIwlFUvmXv7BU4Rfa5TRCJyDKUWgZnFmtmEE/zs9UB6ieXGwddKygPGOucKnHOrgGUEikHKqOQpIt1FJCKlKbUInHNFQLGZ1TyBz54JZJlZUzOrAlwFjD1snU8JHA1gZikEThVpOM1yEBcbw5OXd2T/oSL+/JEeNBORowvl1NAeYIGZvW5mz/32dawfcs4VAncC44ElwAfOuUVm9rCZDQiuNh7YamaLgQnAfc65rSe2K3K4FmlJ3HdOK77/ZTNjZuX5HUdEKik71r8Uzez6I73unHvbk0THkJ2d7XJycvzYdFgqLnZc9ep0lmzYxdf39qFRrap+RxIRH5jZLOdc9pHeO+YRQfAv/FHArODXe36VgBy/mBjjics6UuQcfxozT8NVi8i/OWYRmFlfYDmBh8NeBJaZWR9vY0l5yqhbjQfOP4mpuVsZOW2133FEpJIJ5RrBk8DZzrnTnHN9gHOAp72NJeXt6u4Z9G2Vyt+/+oXczZrRTET+TyhFEO+cW/rbgnNuGRDvXSTxgpnxj0s7ULVKLMM+mKtJ70Xkf4VSBLPM7DUz6xv8ehXQ1dowlFYjkUcvbs/8vJ288EOu33FEpJIIpQhuAxYDdwW/FgNDvAwl3unfvgEXd27ECxNymbN2u99xRKQSOOaTxcA859xTzrlLgl9PO+c0mlkYe2hAW+rXSOTe0XPZe7DQ7zgi4rNQnixeamYZFZRHKkDNqvE8eUVH1mzbxyNfLPY7joj4LC6EdWoDi8xsBrD3txedcwOO/iNS2Z3SrC639mnOyxNXcHrrNM5pW9/vSCLik1CK4D89TyG+GNavJZOW5XP/R/PpnF6LtBqJfkcSER+Eco1ghHNu4uFfFZRPPFQlLobnBnVif0ERf/hQTx2LRCtdI4hyLdKS+cv5bZi8fAtvTF3ldxwR8YGuEQjXnJzBxGX5/OPrpZzavC5tG57IqOMiEq5CGX30tCO97tfpIY0+6o1tew9x7jOTSE6M4/OhvahWJZR/I4hIuCjr6KMTgdUEhpqYSGDCmdnlmlB8V6d6FZ6+shMrt+zVLaUiUSaU0UdvAcYAI4IvNSIws5hEmJ4tUrjttOaMmrGOLzXXsUjUCGWIiTuAnsAuAOfcciDNy1Din2H9WtIpvRb3fzyfddv2+R1HRCpAKEVw0Dl36LcFM4sDdJ9hhIqPjeH5QZ3Bwd3vz9EopSJRIJQimGhm/w+oamb9gA+Bz72NJX5Kr1ONRy9pz+y1O3j622V+xxERj4VSBPcD+cAC4FZgHPAXL0OJ/y7s2JBB3dN5aeIKJi/P9zuOiHgolLuGip1zrzrnLnfOXRb8XqeGosCDF7QlKy2Je0fPZfPuA37HERGPhHJEIFGqapVYhl/dhT0HC7l39FyKNASFSERSEUipsuol8/CAdkzN3crwCZrVTCQSqQjkmC7PbswlnRvxzHfL+GnFFr/jiEg5O+o4Amb2OaXcJqqxhqKHmfHIRe2Yl7eDu0bNZdzdvUhL1pDVIpGitAFlnqiwFFLpVU+I48VrujJw+BTueX8u79x8MrEx5ncsESkHRy0CzTkgh2tVP5mHB7bjT2Pm8+z3yxnWr6XfkUSkHJR2amgBpZ8a6uBJIqnUrshOZ8aqbTz/w3K6ZNSibyuNNiIS7ko7NXRBhaWQsPLIwHYsXL+Te0fP5cu7etOwVlW/I4lIGRz1riHn3JrSvioypFQuVavE8uI1XSgoctzx3mwOFWo8IpFwpttH5YQ0S03isUs7MGftDh4dt8TvOCJSBioCOWHnd2jATT2b8tZPq/ls7nq/44jICQqpCMysqpm18jqMhJ8/929Nt8za3P/RApb9utvvOCJyAkKZoexCYC7wdXC5k5mN9TiXhIn42BheuLoL1RPiuO2dWew+UOB3JBE5TqEcETwEdAd2ADjn5gJNPUskYadejUReuLoza7bt448fzkOD04qEl1CKoMA5t/Ow1/QnXf7FKc3q8ufzWjN+0a+8NHGF33FE5DiEUgSLzOxqINbMsszseeAnj3NJGLq5V1Mu7NiQJ8Yv1WQ2ImEklCIYCrQFDgKjCExif4+HmSRMmRmPXdqerLRkho6aw7pt+/yOJCIhCGWGsn3OuQeAM4HTnXMPOOc0XZUcUbUqcYz4XVeKih23vjOL/YeK/I4kIscQyl1D3YLjDs0HFpjZPDPr6n00CVeZKdV5blBnlmzaxf0fz9fFY5FKLpRTQ68DtzvnMp1zmcAdwJueppKwd3qrNP54dis+m7uB1yav8juOiJQilCIocs5N/m3BOTcFKPQukkSK2/s257x29fn7V0uYslwzm4lUVqEUwUQzG2Fmfc3sNDN7EfjRzLqYWRevA0r4MjOeuLwjWWnJ3PHebNZs3et3JBE5AjvW+Vszm1DK2845d0b5Ripddna2y8nJqchNShmt3bqPAcOnkJacwMe39yQpobTRz0XEC2Y2yzmXfaT3QvkTeZZzTrd+yAnLqFuN4Vd34bo3ZjBs9FxevrYrMZrmUqTSCOXU0HIze9zMTvI8jUSsni1SeKD/SXyz+Fee+W6Z33FEpIRQiqAjsAx43cymm9lgM6vhcS6JQDf2zOTyro157odcvpi/we84IhJ01CIwszgA59xu59yrzrkewH8AfwU2mtnbZtaignJKBDAz/nZxO7Kb1OaPH85j4frDh7ASET+UdkQwA8DMYs1sgJl9CjwDPAk0Az4HxnkdUCJLQlwsL/+uK3WrJ/D7t3PYvEsPqYv4LaRrBMBA4DHnXGfn3FPOuV+dc2MIzlEgcjxSkhJ49bpsdh0o4JZ3ZnGgQPciiPiptCJIM7NhwBvAYuBUMxv22xeAc+6u0j7czM41s6Vmlmtm95ey3qVm5szsiLc2SeRp07AGT1/Zifl5O7hvjIahEPFTaUUQCySV+G/yYV+lMrNYYDhwHtAGGGRmbY6wXjJwN/Dz8YaX8HZO2/r86ZzWfD5vA89+v9zvOCJRq7TnCDY65x4uw2d3B3KdcysBzOx9AqeYFh+23iPAY8B9ZdiWhKnbTmvG8s27eea75TRLTWJAx4Z+RxKJOqUdEZT1iZ9GwLoSy3nB1/5vA4EhKtKdc1+W9kHBW1ZzzCwnP18TnkQSM+Pvl7SnW2bgTqJZa7b7HUkk6pRWBGd6uWEziwGeAv5wrHWdc68457Kdc9mpqalexhIfJMTFMuJ32dSvkcjgkTma0Eakgh21CJxz28r42euB9BLLjYOv/SYZaEdgALvVwCnAWF0wjk51qlfhjRu6UVBUzI1vzWTn/gK/I4lEjVBuHz1RM4EsM2tqZlWAq4Cxv73pnNvpnEspMc/BdGCAc04jykWpFmlJvHxtV1Zv2csd786moKjY70giUcGzInDOFQJ3AuOBJcAHzrlFZvawmQ3warsS3nq0SOHRS9ozJXcLD3yyQLeVilQAT8cDds6N47Cnj51zDx5l3b5eZpHwcUV2Ouu27eP5H3JpUrc6d5yukUxEvKSB4aVSGtavJWu37ePx8UtpXLsqAzs1OvYPicgJURFIpWRm/OOyDmzaeYA/fDCPGlXjOb1Vmt+xRCKSlxeLRcokIS6WV6/PplX9ZIb8cxY5q8t6I5uIHImKQCq1GonxvH1TdxrWrMqNb81k0QYNXS1S3lQEUumlJCUw8ubuJCXEcdWI6UxYutnvSCIRRUUgYaFx7Wp8NKQH6XWqcfNbM3lt8krdWipSTlQEEjYa1qrKmCGncnab+vztyyXcN2Y+Bws1l4FIWakIJKxUqxLHi9d04a4zWjBmVh6DXpnO5t2a5UykLFQEEnZiYoxhZ7di+NVdWLJxNwNfmMqCPF1EFjlRKgIJW+d3aMCYIacSY8ZlL//EZ3PXH/uHROTfqAgkrLVtWJPP7uxJx/Ra3P3+XB4dt4SiYl1EFjkeKgIJeylJCbz7+5O57tQmvDJpJTe8OYMd+w75HUskbKgIJCLEx8bw8MB2PHZpe35euY0LX5jCko27/I4lEhZUBBJRruyWwehbT+FQYTGXvPgTY+dt8DuSSKWnIpCI0zmjNp8P7UW7RjW4a9Qc/vbFYgo1yY3IUakIJCKlJSfy7u9P4YYembw2ZRXXvv4zW/Yc9DuWSKWkIpCIVSUuhocGtOWpKzoyZ+0OLnhuCrPXbvc7lkiloyKQiHdJl8Z8fHsP4uOMK0dMY+S01RqnSKQEFYFEhbYNa/LFnb3pnZXKg58tYtgH89h3qNDvWCKVgopAokbNavG8dl02w/q15NO567l4+E+syN/jdywR36kIJKrExBh3nZnF2zd2Z/PuAwx8YSrjFmz0O5aIr1QEEpX6tEzly7t6k1Uvidvfnc1/fb6IQ4W6xVSik4pAolbDWlUZPfhUburZlDenruaKEdNYv2O/37FEKpyKQKJalbgYHrywDS9d04UVm/fQ/9nJfL/kV79jiVQoFYEIcF77Bnw+tBeNalXl5rdzeHTcEgr0NLJECRWBSFBmSnU+vr0HvzslMIrplTpVJFFCRSBSQmJ8LI9c1I4Xru7Msl8Dp4q+WbTJ71ginlIRiBzBBR0a8sXQXmTUqcbgd2bx0NhFHCws8juWiCdUBCJHkZlSnTFDTuWGHpm89dNqLnnxJ1Zt2et3LJFypyIQKUVCXCwPDWjLq9dls37Hfi54bjKfzMnzO5ZIuVIRiISgX5t6fHV3b9o2rMm9o+cxbPRc9hzUWEUSGVQEIiFqULMq791yMveclcWnc9dz/nOTmbduh9+xRMpMRSByHOJiY7jnrJaMvvVUCgqLufSln3jpxxUUF2tYawlfKgKRE9Atsw5f3d2Hs9vW47Gvf+Ga135m4049cyDhSUUgcoJqVotn+NVd+MelHZi7bgfnPTuZrzSSqYQhFYFIGZgZV3RL58u7As8cDHl3Nn8aM4+9upAsYURFIFIOmqUm8dGQHtxxenM+nJVH/+cma35kCRsqApFyEh8bw33ntGb04FMpLHJc/vI0nvp2mQavk0pPRSBSzro3rcNX9/RmYKeGPPf9ci57SVNiSuWmIhDxQI3EeJ66ohMvXtOFNdv2cf5zkxk5bTXO6TZTqXxUBCIe6t++AePv6cPJTevy4GeLuO6NGbrNVCodFYGIx+rVSOStG7vxt4vakbN6O+c8PYlP56zX0YFUGioCkQpgZlx7ShPG3d2brHrJ3DN6Lre/O5utew76HU1ERSBSkZqmVOeDW0/l/vNa8/2SzZz99CS+XqiH0MRfKgKRChYbY9x2WnM+H9qLBrUSue2fs7n7/Tns2HfI72gSpVQEIj5pVT+ZT27vyb1nteTL+Rvp9/Qkvl38q9+xJAqpCER8FB8bw91nZfHZnT2pW70Kt4zM4d7Rc3V0IBVKRSBSCbRtWJOxd/birjOz+HzeBvo9PYnxizb5HUuihIpApJKoEhfDsH4t+fSOnqQmJXDrO7MYOmqO7iwSz6kIRCqZdo1q8tmdPflDv5Z8vTBw7WDsvA167kA842kRmNm5ZrbUzHLN7P4jvD/MzBab2Xwz+97MmniZRyRcxMfGMPTMLL4Y2pv0OtW4a9Qcbhk5i193HfA7mkQgz4rAzGKB4cB5QBtgkJm1OWy1OUC2c64DMAb4h1d5RMJRq/rJfDykBw/0P4nJy/M566mJjJqxVlNjSrny8oigO5DrnFvpnDsEvA8MLLmCc26Cc25fcHE60NjDPCJhKTbGuKVPM8bf04d2DWvy548XMOjV6azastfvaBIhvCyCRsC6Est5wdeO5mbgKw/ziIS1zJTqvHfLyTx2aXsWb9zFOc9MYviEXM13IGVWKS4Wm9m1QDbw+FHeH2xmOWaWk5+fX7HhRCoRM+PKbhl8P+w0zjopjcfHL+XC56cwd90Ov6NJGPOyCNYD6SWWGwdf+xdmdhbwADDAOXfE++Scc68457Kdc9mpqamehBUJJ2k1Ennxmq68el02O/YVcPGLU3lo7CJ2HyjwO5qEIS+LYCaQZWZNzawKcBUwtuQKZtYZGEGgBDZ7mEUkIvVrU49vh/XhulOa8Pa01fR7ahJfL9ykW03luHhWBM65QuBOYDywBPjAObfIzB42swHB1R4HkoAPzWyumY09yseJyFEkJ8bzXwPb8fGQHtSqFs9t/5zFLSNzyNu+79g/LAJYuP3LITs72+Xk5PgdQ6RSKigq5s2pq3j62+UA3Nsvixt7NiU+tlJcDhQfmdks51z2kd7T7w6RCBIfG8PgPs35dlgferaoy6PjfuHC56eQs3qb39GkElMRiESgxrWr8ep12Yz4XVd27S/gspen8R9j5rNtr0Y1lX+nIhCJUGbGOW3r8+2w07i1TzM+mp3HGU/+qCeT5d+oCEQiXPWEOP7c/yS+vKs3LdOS+fPHC7jkpZ9YuH6n39GkklARiESJVvWTGX3rKTx1RUfytu/nwhem8JdPF2gSHFERiEQTM+OSLo354Y+ncUOPTEbNWMcZTwYGsivS6aKopSIQiUI1EuP564Vt+WJoL1qkJfHnjxdw8YtTmbN2u9/RxAcqApEodlKDGowefArPXtWJX3cd4OIXf+IPH8xj827NexBNVAQiUc7MGNipET/8oS9D+jbn83kbOOOJiYyYuIJDhRrZNBqoCEQECNxd9B/ntmb8vX3o3rQOf//qF855ZhLfL/lVYxdFOBWBiPyLpinVeeOGbrx5YzfM4Oa3c7j+zZks/3W339HEIyoCETmi01ulMf6ePvznBW2Yu3Y75z47mQc/W6inkyOQikBEjio+NoabezXlx/tO55qTM3j357X0fXwCr01eqesHEURFICLHVKd6FR4e2I6v7u5Np4za/O3LJfR7eqLmPogQKgIRCVnLesmMvKk7b93YjSqxMdz2z1lcOWI68zRVZlhTEYjIcevbKo2v7u7Nf1/cjpVb9jBw+FTuGjWHdds0GU440sQ0IlImuw8UMGLiSl6dvBLn4PoeTbjj9BbUqlbF72hSQmkT06gIRKRcbNy5n6e+WcaY2XkkJ8Rxx+ktuL5HJonxsX5HEzRDmYhUgAY1q/L45R0Zd1dvujSpzd+/+oUznviRMbPyNKBdJaciEJFydVKDGrx1Y3feu+VkUpIT+OOH8+j/7GS+W6wnlCsrFYGIeKJH8xQ+u6Mnw6/uwqGiYn4/MofLX57GjFWaP7myURGIiGfMjPM7NOCbe/vw3xe3Y+22fVwxYho3vDmDRRs0Q1ploYvFIlJh9h8q4u1pq3npxxXs3F/A+R0aMKxfS5qnJvkdLeLpriERqVR27i/gtckreX3KKg4UFHFJl8bcfWYW6XWq+R0tYqkIRKRS2rrnIC/9uIKR09dQXOy4ols6Q89oQYOaVf2OFnFUBCJSqW3aeYDhE3J5f+ZazIyru2cwpG9z6tVI9DtaxFARiEhYyNu+j+e/z2XM7DziYoyrT85gyGnNSVMhlJmKQETCytqt+3j+h+V8PGc9cTHGIB0hlJmKQETC0pqte3nhh1w+nrOe2BhjULd0bj2tOQ1r6RrC8VIRiEhYW7t1H8Mn5PLR7DzM4LKu6dzet7nuMjoOKgIRiQjrtu3j5Ykr+DAnjyLnuKhTI24/vbmeQwiBikBEIsrGnft5ZdJKRs1Yy8HCYvq3a8CQvs1p16im39EqLRWBiESkLXsO8vqUVfxz2hp2HyykT8tUbu/bnJOb1sHM/I5XqagIRCSi7TpQwDvT1vDGlFVs3XuIzhm1uO205vQ7qR4xMSoEUBGISJQ4UFDEhznrGDFpJXnb99M8tTqD+zTjos6NSIiL7glyVAQiElUKi4r5csFGXpm0kkUbdpGanMANPTK55uSMqJ1CU0UgIlHJOcfU3K2MmLSCycu3UDU+liuyG3NTr6Y0qVvd73gVSkUgIlHvl027eG3yKj6bu57CYke/k+pxc6+mdI+SC8sqAhGRoF93HeCdaWt49+c1bN9XQNuGNbixZ1Mu7Nggoq8jqAhERA6z/1ARn8xZz5tTV7F88x5SkqpwdfcMrjmlSUSOaaQiEBE5CuccU3K38ObU1UxYuplYM85tV5/re2SS3aR2xJw2Kq0I4io6jIhIZWJm9M5KpXdWKqu37OWf09cwOmcdX8zfSOv6yfzu1CZc1KkR1RMi969LHRGIiBxm36FCxs7dwMhpa1i8cRdJCXFc1Lkh15zchJMa1PA73gnRqSERkRPgnGP22h28+/Mavpi/kUOFxXTOqMWgbhlc0LEB1aqEz1GCikBEpIx27DvEmFl5jJqxlhX5e0lKiGNAp4ZcmZ1Oh8Y1K/21BBWBiEg5cc4xc/V23p+xlnELN3KgoJjW9ZO5PDudizo1pG5Sgt8Rj0hFICLigV0HChg7dwMf5qxjXt5O4mKM01uncVnXxpzeKo0qcTF+R/xfKgIREY8t3bSbj2bn8cmc9eTvPkitavGc374Bl3RpRJcM/29DVRGIiFSQwqJiJudu4dM56xm/aBMHCoppVKsqF3ZsyICODTmpQbIvpaAiEBHxwZ6DhXyzaBNj521g8vItFBU7mqVU5/wODejfvgGt61dcKagIRER8tnXPQb5auIlxCzYyfeVWih00qVuNc9vW5+y29emcXsvTSXR8KwIzOxd4FogFXnPO/c9h7ycAI4GuwFbgSufc6tI+U0UgIuEuf/dBvlm8ifGLfuWn3C0UFjtSkqrQt1UaZ7ROo2eLFGpWjS/XbfpSBGYWCywD+gF5wExgkHNucYl1bgc6OOduM7OrgIudc1eW9rkqAhGJJDv3F/Dj0s388Mtmflyaz879BcQYdEyvRe8WKWRn1qFTRi1qJJatGPwaa6g7kOucWxkM8T4wEFhcYp2BwEPB78cAL5iZuXA7XyUicoJqVo1nYKdGDOzUiMKiYuas28HkZflMzt3CCxNyKXZgBi1SkxjWryXntW9Q7hm8LIJGwLoSy3nAyUdbxzlXaGY7gbrAlpIrmdlgYDBARkaGV3lFRHwVFxtDt8w6dMusw7CzW7H7QAHz1u1kztrtzF67nWoeDXwXFgNlOOdeAV6BwKkhn+OIiFSI5MR4emWl0CsrxdPtePnY23ogvcRy4+BrR1zHzOKAmgQuGouISAXxsghmAllm1tTMqgBXAWMPW2cscH3w+8uAH3R9QESkYnl2aih4zv9OYDyB20ffcM4tMrOHgRzn3FjgdeAdM8sFthEoCxERqUCeXiNwzo0Dxh322oMlvj8AXO5lBhERKV3lGRpPRER8oSIQEYlyKgIRkSinIhARiXIqAhGRKKciEBGJcioCEZEopyIQEYlyKgIRkSgXdlNVmlk+sOYEfzyFw4a4jgLa5+igfY4OZdnnJs651CO9EXZFUBZmlnO0GXoilfY5Omifo4NX+6xTQyIiUU5FICIS5aKtCF7xO4APtM/RQfscHTzZ56i6RiAiIv8u2o4IRETkMBFZBGZ2rpktNbNcM7v/CO8nmNno4Ps/m1mmDzHLVQj7PMzMFpvZfDP73sya+JGzPB1rn0usd6mZOTML+ztMQtlnM7si+Gu9yMzeq+iM5S2E39sZZjbBzOYEf3/39yNneTGzN8xss5ktPMr7ZmbPBf9/zDezLmXeqHMuor4ITIu5AmgGVAHmAW0OW+d24OXg91cBo/3OXQH7fDpQLfj9kGjY5+B6ycAkYDqQ7XfuCvh1zgLmALWDy2l+566AfX4FGBL8vg2w2u/cZdznPkAXYOFR3u8PfAUYcArwc1m3GYlHBN2BXOfcSufcIeB9YOBh6wwE3g5+PwY408ysAjOWt2Pus3NugnNuX3BxOtC4gjOWt1B+nQEeAR4DDlRkOI+Ess+3AMOdc9sBnHObKzhjeQtlnx1QI/h9TWBDBeYrd865SQTmcD+agcBIFzAdqGVmDcqyzUgsgkbAuhLLecHXjriOc64Q2AnUrZB03ghln0u6mcC/KMLZMfc5eMic7pz7siKDeSiUX+eWQEszm2pm083s3ApL541Q9vkh4FozyyMwR/rQionmm+P9835Mnk5eL5WPmV0LZAOn+Z3FS2YWAzwF3OBzlIoWR+D0UF8CR32TzKy9c26Hn6E8Ngh4yzn3pJmdCrxjZu2cc8V+BwsXkXhEsB5IL7HcOPjaEdcxszgCh5NbKySdN0LZZ8zsLOABYIBz7mAFZfPKsfY5GWgH/GhmqwmcSx0b5heMQ/l1zgPGOucKnHOrgGUEiiFchbLPNwMfADjnpgGJBMbkiVQh/Xk/HpFYBDOBLDNramZVCFwMHnvYOmOB64PfXwb84IJXYcLUMffZzDoDIwiUQLifN4Zj7LNzbqdzLsU5l+mcyyRwXWSAcy7Hn7jlIpTf258SOBrAzFIInCpaWYEZy1so+7wWOBPAzE4iUAT5FZqyYo0FrgvePXQKsNM5t7EsHxhxp4acc4VmdicwnsAdB2845xaZ2cNAjnNuLPA6gcPHXAIXZa7yL3HZhbjPjwNJwIfB6+JrnXMDfAtdRiHuc0QJcZ/HA2eb2WKgCLjPORe2R7sh7vMfgFfN7F4CF45vCOd/2JnZKAJlnhK87vFXIB7AOfcygesg/YFcYB9wY5m3Gcb/v0REpBxE4qkhERE5DioCEZEopyIQEYlyKgIRkSinIhARiXIRd/uoyG/MrC7wfXCxPoHbKX+7v7x7cOya8thOJvCFc65deXyeSEVTEUjECt4/3wnAzB4C9jjnnvAz0+GCgx3ab8MhHL5cys/FBcfJEikznRqSaFLVzFaZWTyAmdX4bdnMfjSzZ81srpktNLPuwXWqB8eHnxEc7/5II5welZndZ2Yzg+PG/1fwtczg+PojgYVA78OW083s8WCOBWZ2ZfDn+prZZDMbCywux/8vEuV0RCDRZD/wI3A+gaEYrgI+ds4VBJ+2ruac62RmfYA3CIxV9ACBIUhuMrNawAwz+845t/dYGzOzswmM89OdwNjxY4OfvTb4+vXOuenBU0slly8lcCTTkcCYOTPNbFLwY7sA7YLjCImUCx0RSLR5jf97JP9G4M0S742C/x0PvkbwL/6zgfvNbC6BEkkEMkLc1tnBrznAbKA1/zcA3JrgWPIcYbkXMMo5V+Sc+xWYCHQLvjdDJSDlTUcEElWcc1ODp2b6ArHOuZLTAR4+3ooj8C/5S51zS09gcwb83Tk34l9eDBwBHH5EccwjjONcTyRkOiKQaDQSeI9/PRoA+O1cfC8CIzruJDDY2dDfZrALjuIaqvHATWaWFPzZRmaWFsLPTQauNLNYM0slMHXhjOPYrshx0RGBRKN3gb8RPBVUwgEzm0NgpMebgq89AjwDzA9OdrMKuCCUjTjnvgkOizwt2CN7gGsJ3MZamk+AUwnMz+uAPznnNplZ61C2K3K8NPqoRB0zuwwY6Jz7XYnXfgT+GObzFYicEB0RSFQxs+eB8wiM5y4i6IhARCTq6WKxiEiUUxGIiEQ5FYGISJRTEYiIRDkVgYhIlFMRiIhEuf8PrjENg3izjUgAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# Get hypothesis testing interpretation so we can directly plot it\n", + "fpr_list, fnr_list = composed_mech.plot_fDP()\n", + "\n", + "plt.figure(figsize = (6,6))\n", + "plt.plot(fpr_list,fnr_list)\n", + "plt.xlabel('Type I error')\n", + "plt.ylabel('Type II error')\n", + "plt.show()\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 2: Tight composition of Gaussian mechanism" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Mechanism name is \" Compose:{GM1: 3, GM2: 5} \"\n", + "Parameters are: {'GM1:sigma': 5.0, 'GM2:sigma': 8.0}\n", + "Generic composition: epsilon(delta) = 2.1309868424169824 , at delta = 1e-06\n", + "Gaussian composition: epsilon(delta) = 1.9842739198015718 , at delta = 1e-06\n", + "Generic composition: epsilon(delta) = 1.6484258167240666 , at delta = 0.0001\n", + "Gaussian composition: epsilon(delta) = 1.4867384204500516 , at delta = 0.0001\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.8/site-packages/scipy/optimize/optimize.py:2522: RuntimeWarning: invalid value encountered in double_scalars\n", + " w = xb - ((xb - xc) * tmp2 - (xb - xa) * tmp1) / denom\n" + ] + } + ], + "source": [ + "from autodp.transformer_zoo import ComposeGaussian\n", + "from autodp.mechanism_zoo import GaussianMechanism\n", + "\n", + "sigma1 = 5.0\n", + "sigma2 = 8.0\n", + "\n", + "gm1 = GaussianMechanism(sigma1,name='GM1')\n", + "gm2 = GaussianMechanism(sigma2,name='GM2')\n", + "\n", + "# run gm1 for 3 rounds\n", + "# run gm2 for 5 times\n", + "\n", + "# compose them with the transformation: compose and\n", + "rdp_compose = Composition()\n", + "rdp_composed_mech = rdp_compose([gm1, gm2], [3, 5])\n", + "\n", + "compose = ComposeGaussian()\n", + "composed_mech = compose([gm1, gm2], [3, 5])\n", + "\n", + "\n", + "# Query for eps given delta\n", + "delta1 = 1e-6\n", + "eps1 = composed_mech.get_approxDP(delta1)\n", + "eps1b = rdp_composed_mech.get_approxDP(delta1)\n", + "\n", + "delta2 = 1e-4\n", + "eps2 = composed_mech.get_approxDP(delta2)\n", + "eps2b = rdp_composed_mech.get_approxDP(delta2)\n", + "\n", + "# Get name of the composed object, a structured description of the mechanism generated automatically\n", + "print('Mechanism name is \\\"', composed_mech.name,'\\\"')\n", + "print('Parameters are: ',composed_mech.params)\n", + "\n", + "print('Generic composition: epsilon(delta) = ', eps1b, ', at delta = ', delta1)\n", + "print('Gaussian composition: epsilon(delta) = ', eps1, ', at delta = ', delta1)\n", + "\n", + "print('Generic composition: epsilon(delta) = ', eps2b, ', at delta = ', delta2)\n", + "print('Gaussian composition: epsilon(delta) = ', eps2, ', at delta = ', delta2)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAFzCAYAAAAzNA41AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABKtElEQVR4nO3dd3hUxdfA8e+kJwRCSegBQu8EEnovIh1BpChKEelgw94Q/akoYkGkS5UiIF2K0jsESKFID10IoZf0ef+4IW+AkCzJbjabPZ/nyUP2lrnn0k7unZkzSmuNEEII++Vg7QCEEEJYlyQCIYSwc5IIhBDCzkkiEEIIOyeJQAgh7JwkAiGEsHNO1g7gaXl7e+sSJUpYOwwhhLAp+/btu6q19klpn80lghIlShAUFGTtMIQQwqYopc48aZ+8GhJCCDsniUAIIeycJAIhhLBzNtdHIER2FBsby/nz54mKirJ2KMLGubm5UbRoUZydnU0+RxKBEFnA+fPnyZkzJyVKlEApZe1whI3SWhMZGcn58+fx8/Mz+Tx5NSREFhAVFUW+fPkkCYgMUUqRL1++p36ylEQgRBYhSUCYQ3r+HkkiEEIIOyeJQAiR5PLly7z44ouULFmSgIAA6taty5IlSyx6zaCgIIYPH27Ra2QVye9106ZN7NixI2nfxIkTmTVrllXislhnsVLqN6AdcEVrXTmF/Qr4CWgD3AN6a633WyoeIUTqtNY899xz9OrVi7lz5wJw5swZli9fbtHrBgYGEhgYaNFrZBXJ73XTpk14enpSr149AAYOHGi1uCw5amgG8AvwpBTXGiiT+FUbmJD4qxB27fMVhzh88ZZZ26xYOBefta+U6jEbNmzAxcXlof+QihcvzrBhwwgPD+fll1/m7t27APzyyy/Uq1ePTZs2MWbMGFauXAnA0KFDCQwMpHfv3rz//vssX74cJycnWrZsyZgxY1i4cCGff/45jo6OeHl5sWXLlofa2LNnD6+//jpRUVG4u7szffp0ypUrx4wZM1i+fDn37t3j5MmTdOrUiW+//faJ97JmzRo+/PBD4uPj8fb2Zv369Vy7do2+ffty6tQpPDw8mDx5MlWrVmXkyJGcPn2aU6dOcfbsWX744Qd27drF6tWrKVKkCCtWrMDZ2ZkSJUrQtWtXVq9ejbu7O3PnzqV06dKEh4fTt29frl69io+PD9OnT6dYsWKp3usvv/zCxIkTcXR0ZM6cOYwbN47169fj6enJiBEjCA4OZuDAgdy7d49SpUrx22+/kSdPHpo0aULt2rXZuHEjN27cYNq0aTRs2DDDfz8s9mpIa70FuJbKIR2BWdqwC8itlCpkqXiuXznPgXVzLNW8EDbv0KFD1KhRI8V9+fPn5++//2b//v0sWLAgzVc5kZGRLFmyhEOHDhEaGsrHH38MwKhRo1i7di0hISEpPmmUL1+erVu3cuDAAUaNGsWHH36YtC84OJgFCxYQFhbGggULOHfuXIrXjoiI4LXXXmPx4sWEhISwcOFCAD777DOqV69OaGgoX331Fa+88krSOSdPnmTDhg0sX76cnj170rRpU8LCwnB3d2fVqlVJx3l5eREWFsbQoUN54403ABg2bBi9evUiNDSUl156Ken3JrV7LVGiBAMHDuTNN98kODj4sf/MX3nlFUaPHk1oaChVqlTh888/T9oXFxfHnj17+PHHHx/anhHWnEdQBEj+J3k+cdulRw9USvUH+gMUK1YsXRc7suoX6p2ZwJXKjchfOH1tCJEZ0vrJPbMMGTKEbdu24eLiwj///MPQoUMJDg7G0dGRY8eOpXqul5cXbm5uvPrqq7Rr14527doBUL9+fXr37k3Xrl3p3LnzY+fdvHmTXr16cfz4cZRSxMbGJu1r3rw5Xl5eAFSsWJEzZ87g6+v7WBu7du2iUaNGSePo8+bNC8C2bdtYvHgxAM2aNSMyMpJbt4wnr9atW+Ps7EyVKlWIj4+nVatWAFSpUoXw8PCktnv06JH065tvvgnAzp07+fPPPwF4+eWXeffdd0261ye5efMmN27coHHjxgD06tWLF154IWn/g7YCAgIeii0jbKKzWGs9WWsdqLUO9PFJsYpqmnxrPwfAv9sWmzEyIbKPSpUqsX///3fTjR8/nvXr1xMREcEPP/xAgQIFCAkJISgoiJiYGACcnJxISEhIOufB+HUnJyf27NlDly5dWLlyZdJ/rBMnTuTLL7/k3LlzBAQEEBkZ+VAMn3zyCU2bNuXgwYOsWLHiofHwrq6uSd87OjoSFxdntnt/0LaDgwPOzs5JQzAdHBweuk7yoZlpDdNM614zGqs5fw+smQguAMnTedHEbRbhW6E2V1U+nE78balLCGHTmjVrRlRUFBMmTEjadu/ePcD4KbVQoUI4ODgwe/Zs4uPjAaMP4fDhw0RHR3Pjxg3Wr18PwJ07d7h58yZt2rThhx9+ICQkBDBewdSuXZtRo0bh4+Pz2OudmzdvUqRIEQBmzJiRrvuoU6cOW7Zs4fTp0wBcu2a8oW7YsCG///47YHTUent7kytXrqdqe8GCBUm/1q1bF4B69eoxf/58AH7//fek1zxp3WvOnDm5ffv2Y9fw8vIiT548bN26FYDZs2cnPR1YijVfDS0Hhiql5mN0Et/UWj/2WshslOJKwUZUubiGcxE38PXJbbFLCWGLlFIsXbqUN998k2+//RYfHx9y5MjB6NGjqVGjBs8//zyzZs2iVatW5MiRAwBfX1+6du1K5cqV8fPzo3r16gDcvn2bjh07EhUVhdaasWPHAvDOO+9w/PhxtNY0b96catWqsXnz5qQY3n33XXr16sWXX35J27Zt03UfPj4+TJ48mc6dO5OQkJDUvzFy5Ej69u1L1apV8fDwYObMmU/d9vXr16latSqurq7MmzcPgHHjxtGnTx++++67pM5iU+61ffv2dOnShWXLljFu3LiHrjNz5sykzuKSJUsmtWkpSmttmYaVmgc0AbyBy8BngDOA1npi4vDRX4BWGMNH+2it01xxJjAwUKd3YZqIvX/is6oPy6tNpEOnHulqQwhLOHLkCBUqVLB2GCIVDxbF8vb2tnYoaUrp75NSap/WOsVxuhZ7ItBap/o/rTYy0BBLXT8lPlVbErPKmfh/VwOSCIQQAuyt+qirJ1fy1aRqxB5OXLlD6fye1o5ICJEBtWvXJjo6+qFts2fPpkqVKma/lrlG6GRF9pUIgFxV25Jr40dM372b0u2bWzscIUQG7N6929ohZAs2MXzUnHJVMTqg7h38C0v1jwghhC2xu0RAXj9u5vCjyr3dHDh3w9rRCCGE1dlfIgDcK7WhrsNh5m89ZO1QhBDC6uwyEbhU6YSzikcdWc7lW7JGrBDCvtllIqBoILFeJXnOYSu/7z5r7WiEyDKssR6BNevwW0Ly+5kxYwYXL15M2tevXz8OHz5srdCeyO5GDQGgFM41XqTuxi/5atdehjQthauTo7WjEsKqrLEeQVxcnFXr8FtC8vuZMWMGlStXpnDhwgBMnTrVWmGlyj4TAUDVrrDxSxpHbWRVaGM61yhq7YiEMKx+H/4LM2+bBatA629SPSS19Qji4+N5//332bRpE9HR0QwZMoQBAwawadMmRo4cibe3NwcPHiQgIIA5c+aglGLfvn289dZb3LlzB29vb2bMmEGhQoVo0qQJ/v7+bNu2jR49enD79u2kOvwnTpxg4MCBRERE4OjoyMKFCylVqlSK8Y4ePZo5c+bg4OBA69at+eabb1Kt41+9enW2bt3K3bt3mTVrFl9//TVhYWF069aNL7/8kvDwcFq1akVAQAD79++nUqVKzJo1Cw8PD9avX8+IESOIi4ujZs2aTJgwAVdX1xTXXBg5ciSenp5JM5Ffeukl3N3d2blzJ61bt2bMmDEEBgYyb948vvrqK7TWtG3bltGjRwPg6enJ66+/zsqVK3F3d2fZsmUUKFDAfH8XUmCfr4YA8hRHF69Pd9ftTN92WoaSCruX2noE06ZNw8vLi71797J3716mTJmSVNTtwIED/Pjjjxw+fJhTp06xfft2YmNjGTZsGIsWLWLfvn307duXjz76KKm9mJgYgoKCePvttx+6zksvvcSQIUMICQlhx44dFCqU8hIlq1evZtmyZezevZuQkJCk0s+p1fF3cXEhKCiIgQMH0rFjR8aPH8/BgweZMWNGUmXQo0ePMnjwYI4cOUKuXLn49ddfiYqKonfv3klrIcTFxTFhwoQnrrnwQJcuXQgMDOT3338nODgYd3f3pH0XL17kvffeY8OGDQQHB7N3716WLl0KwN27d6lTpw4hISE0atSIKVOmmPLHlyH2+0QAqGo9KHpmKE6X9rHndEVql8xn7ZCESPMn98ySfD2C4sWLExoayqJFiwCjSujx48dxcXGhVq1aFC1qPFH7+/sTHh5O7ty5OXjwIM888wwA8fHxD/2n3q1bt8eud/v2bS5cuECnTp0AcHNze2Js//zzD3369MHDwwMw1hxIq45/hw4dAGONgUqVKiXFU7JkSc6dO0fu3Lnx9fWlfv36APTs2ZOff/6ZZ555Bj8/P8qWLZvU7vjx4xk6dGiKay6YYu/evTRp0oQHZfVfeukltmzZwnPPPYeLi0tSWwEBAfz9t+UrJtvvEwFAxY5oJzdedNvBr5tOWjsaIawqtfUItNaMGzeO4OBggoODOX36NC1btgRSXidAa02lSpWSjg8LC2PdunVJxz2oXpqZkq85kDzm5GsOPLrGQGprDjxpzYWMSr4egrnXXXgS+04EbrlQ5dvR3nEnO49d5OCFm9aOSAirSW09gmeffZYJEyYkrRh27NixpPWLU1KuXDkiIiLYuXMnALGxsRw6lPq8nZw5c1K0aNGkVyTR0dFJ13/UM888w/Tp05P2X7t2zSx1/M+ePZsU89y5c2nQoAHlypUjPDycEydOPNTuk9ZcePSeUlpzoFatWmzevJmrV68SHx/PvHnzLL7mQGrsOxEA+PfALe4WHV33M2GzPBUI+/VgPYLNmzfj5+dHrVq16NWrF6NHj6Zfv35UrFiRGjVqULlyZQYMGJDqT6ouLi4sWrSI9957j2rVquHv78+OHTvSjGH27Nn8/PPPVK1alXr16vHff/+leFyrVq3o0KEDgYGB+Pv7M2bMGMCo4//OO+9QtWpVgoOD+fTTT5/q96BcuXKMHz+eChUqcP36dQYNGoSbmxvTp0/nhRdeoEqVKjg4ODBw4EBu375Nu3btqFq1Kg0aNEhacyG53r17M3DgQPz9/bl//37S9kKFCvHNN9/QtGlTqlWrRkBAAB07dnyqWM3JYusRWEpG1iNIUUIC/OzP2fh8NLk6gvVvN8HPO/MfW4V9k/UIrC88PJx27dpx8OBBa4eSYU+7HoE8ETg4QEAvit3eTznHS0ySpwIhhJ2RRABQ/WVwcOLjQntYvP88F2/cT/scIYTFhYWF4e/v/9BX7dq1LXKtEiVKZIungfSw6+GjSTzzQ/l21D25Flfa8OumE3z5nPkXthAiNVrrVEep2KMqVaoQHBxs7TBsSnpe98sTwQOBfXGIvsGnJY+zYO85LshTgchEbm5uREZGysRGkSFaayIjI1Odg5ESeSJ4wK8R5C1Fx7i1fEQlxm88wVed5KlAZI6iRYty/vx5IiIirB2KsHFubm5JE/xMJYngAaUgsA+u6z7m9cox/Lj3HIMal8I3r4e1IxN2wNnZGT8/P2uHIeyUvBpKzv8lcHKnr/M6HJRi/MYT1o5ICCEsThJBch55oVo3PP5dTL+AXCzcd54zkU+ePSmEENmBJIJH1RoAcVEMzrUdZ0fF2L+PWTsiIYSwKEkEjypQEfwa4xk6g1fr+rIs+CKHLkoNIiFE9iWJICV1BsGtCwwu9C9e7s58t/aotSMSQgiLkUSQkjItIU8JcuyfwqAmpdh0NIJdpyKtHZUQQliEJIKUODgafQXndtGnxHUK5HLl2zX/ymQfIUS2JIngSar3BNdcuO4ZzxstyrL/7A3WHrps7aiEEMLsJBE8iVsuCOgNh5fyQqkESuf3ZPSaf4mJS7B2ZEIIYVaSCFJTZxAoB5z2TODDNuU5ffUuc3efsXZUQghhVpIIUpOrMFR5AfbPomkxZ+qVysdP649z836stSMTQgizkUSQlnrDIPYeKug3PmpbgRv3Y/lVSk8IIbIRSQRpKVAJSjWH3ZOo5ONK5+pFmb49nHPXUl5UWwghbI0kAlPUHw53r0DIPEY8WxZHB8XXq49YOyohhDALSQSm8GsMhWvA9p8o5OnMoCal+CvsP3aelElmQgjbJ4nAFEpBw7fh+mk4vJT+jUpSJLc7n684RHyCTDITQtg2SQSmKtcGfMrD1u9xc1R82KYC//53m/l7z1o7MiGEyBBJBKZycIAGb8KVw3B8LW2qFKSWX16+X3dMhpMKIWyaJIKnUfl5yF0Mtn6PAj5rX5Eb92L4QdYsEELYMEkET8PRGeq/Duf3wuktVCrsxYu1izFrZzhHLt2ydnRCCJEukgieln9P8CwIW74DYETLcni5O/PpsoNSnVQIYZMkETwtZzdo8AaEb4Xw7eT2cOG9VuXZG36dpcEXrB2dEEI8NUkE6RHQG3Lkh82jAega6Eu1ol589de/3I6SjmMhhG2RRJAezu5GX8HpzXB2Fw4OilEdK3P1TrQsdi+EsDmSCNIrsC/k8El6Kqjmm5sXaxVj5o5wDl6Qxe6FELZDEkF6uXgYlUlPboBzewB499ny5M3hwkdLD8qMYyGEzZBEkBE1+4GHN2z8HwBeHs583LYiIeduMG+PzDgWQtgGSQQZ4ZLDmG18ahOEbwego39h6pXKx+g1/xJxO9q68QkhhAkkEWRUYF/wLGA8FWiNUoovnqtMdGwCX646bO3ohBAiTRZNBEqpVkqpo0qpE0qp91PYX0wptVEpdUApFaqUamPJeCzCxcOoTHpmuzGKCCjl48ngpqVYFnyRTUevWDlAIYRIncUSgVLKERgPtAYqAj2UUhUfOexj4A+tdXWgO/CrpeKxqBq9IFcR2GA8FQAMalKKUj45+HjpQe7FxFk5QCGEeDJLPhHUAk5orU9prWOA+UDHR47RQK7E772AixaMx3Kc3aDRCDi/B47/DYCrkyNfd67K+ev3+fGf41YOUAghnsySiaAIcC7Z5/OJ25IbCfRUSp0H/gKGWTAey/LvCXlKwIZRkJAAQC2/vPSo5cu0badlboEQIsuydmdxD2CG1roo0AaYrZR6LCalVH+lVJBSKigiIiLTgzSJkws0+RD+C4PDS5I2v9+qAnlzuPDe4lDi4hOsGKAQQqTMkongAuCb7HPRxG3JvQr8AaC13gm4Ad6PNqS1nqy1DtRaB/r4+FgoXDOo0gXyVzT6CuKNmkNeHs580bEShy7eYsrW01YOUAghHmfJRLAXKKOU8lNKuWB0Bi9/5JizQHMApVQFjESQRX/kN4GDIzT7BK6dhODfkza3qlyI1pUL8sM/xzgVcceKAQohxOMslgi01nHAUGAtcARjdNAhpdQopVSHxMPeBl5TSoUA84De2taL+pdrDUVrwabREHs/afPnHSvh5uTA+4vDSJDyE0KILMSifQRa67+01mW11qW01v9L3Pap1np54veHtdb1tdbVtNb+Wut1lownUygFzT+F2xdhz5SkzflzuvFJu4rsCb/G77vPWDFAIYR4mLU7i7Mnv4ZQugVs/R7uX0/a3CWgKA3LePP16n85d+2eFQMUQoj/J4nAUlp8DlE3YevYpE1KKb55vioOSvHe4lBZ2lIIkSVIIrCUgpWhWnfYPQlu/P90iiK53fmobQV2nIxkrlQoFUJkAZIILKnpR8avG796aHP3mr40KO3NV6uOcP66vCISQliXJAJLyu0LtQdAyDz472DSZuMVURUA3l0UKqOIhBBWJYnA0hq+BW5e8PenD20umseDj9tVZMfJSObIKCIhhBVJIrA09zzQ+F04uR5O/PPQru41fWlc1oev//qX01fvWilAIYS9k0SQGWq+Bnn8YN0nkBCftFkpxejnq+LsqBixMETWORZCWIUkgszg5AItRsKVw3BgzkO7Cnq5MapjZfaduc6UraesE58Qwq5JIsgsFTuCbx1jScvoh+sNdfQvTKtKBRm77hhHLt2yUoBCCHsliSCzKAXP/g/uXIbtPz2yS/FV5yrkcnfmzQXBRMfFP6ERIYQwP0kEmaloIFTuAjt+fmiSGUDeHC5826UK//53m7HrjlkpQCGEPZJEkNlajDR+/WfkY7ualS9Aj1rFmLz1FLtPRWZqWEII+yWJILPl9oV6w+HgIji7+7HdH7etQLG8Hrz1Rwi3omKtEKAQwt5IIrCG+q9DzkKw9oOk9Y0fyOHqxI/d/PnvVhSfLj34hAaEEMJ8JBFYg6un8Yrowj4I++Ox3dWL5WF4szIsDb7IsuBHV/cUQgjzkkRgLVW6QpEA+PsziL792O4hTUsRUDwPHy85KIXphBAWJYnAWhwcoPW3cOc/YwGbRzg5OvBDV3808OaCYJl1LISwGEkE1lQ0EKq9CDvHQ+TJx3YXy+fBqI6V2Bt+nfEbT1ghQCGEPZBEYG0tPgNHV1j7YYq7O9coynP+hflp/XH2nbmWycEJIeyBJAJry1nQqE56bA0cW5fiIV88V5nCud0YPi+Ym/dlSKkQwrwkEWQFtQdCvtKw5j2Ii35sd043Z37qXp3/bkXx0ZIwWetYCGFWkgiyAicXo+P42imj/EQKahTLw1vPlGVl6CX+CDqX4jFCCJEekgiyitLNoUJ72PI93Eh5UftBjUvRoLQ3ny0/xPHLjw85FUKI9JBEkJU8+7Xx6xM6jh0cFGO7VcPT1Ymhcw8QFStVSoUQGSeJICvJ7QuN3oYjKx5b1vKB/Dnd+L6rP0cv32bUysOZHKAQIjuSRJDV1BsOeUvCX+9CbFSKhzQu68OAxiWZu/ssK0IuZnKAQojsRhJBVuPkCm3GwLWTT+w4BhjRshwBxfPwwZ9hsvC9ECJDJBFkRaWbQ8XnYMsYYyRRCpwdHRjXozpOjoohv++X/gIhRLpJIsiqWn0Njs7GK6InzBsonNud71+oxuFLt/hylfQXCCHSRxJBVpWrMDT9EE78DUeWP/Gw5hUK0L9RSebsOsty6S8QQqSDJIKsrNYAKFAFVr+fYqnqB955thyBxfPwweJQTkbcycQAhRDZgSSCrMzRCdr9ALcvwYb/PfEwZ0cHxr1YHVdnRwbP2c/9GOkvEEKYThJBVudbEwL7wp5JcPHAEw8r5OXOj938OXblNh8vPSj1iIQQJpNEYAuafwo5fGDFG5Dw5J/2G5X1YXizMizef575e6UekRDCNJIIbIF7bmMU0aVg2DM51UOHNy9DwzLefLbsEKHnb2RGdEIIGyeJwFZU6gylW8D6L+DGk3/ad3RQ/NS9Ot6eLgyas58b92IyMUghhC2SRGArlIK2YwENq95+4twCgLw5XPi1ZwBXbkfxxoJgEmS9YyFEKiQR2JI8xaHpR3B8LRxakuqh/r65+bR9JTYdjeCn9cczKUAhhC2SRGBrag+EQv6w+j24fz3VQ3vWLsbzNYry0/rjbPj3cubEJ4SwOZIIbI2jE3T4Ge5FwrqPUz1UKcX/OlWmYqFcvDE/mDORUpxOCPE4SQS2qFA1qDcUDsyBU5tSPdTN2ZFJLweglGLA7H3ci4nLnBiFEDZDEoGtavKBsW7BitchJvWf9H3zevBTd2Mxm/cWh8lkMyHEQyQR2Cpnd+gwDq6Hw8av0jy8Sbn8jGhZjhUhF5m69bTl4xNC2AxJBLasRAOj/MSuX+F8UJqHD25SitaVC/L16iNsO341EwIUQtgCSQS2rsXnkLMQLBsKcdGpHqqUYswL1Sid35Nh8/Zz7tq9TApSCJGVSSKwdW65oN2PEHEEtnyX5uE5XJ2Y9HIg8Qma12YFSeexEEISQbZQtiVU6wFbx8KlkDQP9/POwbgXa3Ds8m1GLAyRzmMh7Jwkguzi2a8ghzcsHQJxadcXalzWh/daleevsP8Yv/FEJgQohMiqJBFkFx55jUVsLofBth9MOqV/o5J09C/M938f4+/DMvNYCHsliSA7Kd8WqrwAW76FS6FpHq6UYvTzValc2Is35h/g6H9PXg5TCJF9WTQRKKVaKaWOKqVOKKXef8IxXZVSh5VSh5RScy0Zj11o/S145IOlg016ReTm7MiUVwLxcHWi36y9XL8rZauFsDcWSwRKKUdgPNAaqAj0UEpVfOSYMsAHQH2tdSXgDUvFYzc88hqjiC6HwdYxJp1S0MuNyS8HcPlWNIN/309sfIJlYxRCZCmWfCKoBZzQWp/SWscA84GOjxzzGjBea30dQGt9xYLx2I/ybYxRRFvGwMVgk06pXiwP33Suws5TkXy2/JCMJBLCjlgyERQBki+ldT5xW3JlgbJKqe1KqV1KqVYpNaSU6q+UClJKBUVERFgo3Gym1dfgmR+WDkpzotkDnWsUZWDjUszdfZaZO8ItG58QIsuwdmexE1AGaAL0AKYopXI/epDWerLWOlBrHejj45O5Edoq9zxGLaIrh2Hj/0w+7d1ny9GiQgFGrTzMlmOSdIWwB6kmAqWUo1JqYzrbvgD4JvtcNHFbcueB5VrrWK31aeAYRmIQ5lDmGajRC7b/DGd3mXSKg4Pix+7+lC2QkyFz93Piyh0LBymEsLZUE4HWOh5IUEp5paPtvUAZpZSfUsoF6A4sf+SYpRhPAyilvDFeFZ1Kx7XEkzz7P8hdDJYMhGjT/lP3dHViaq9AXJ0c6DtjL9dkJJEQ2Zopr4buAGFKqWlKqZ8ffKV1ktY6DhgKrAWOAH9orQ8ppUYppTokHrYWiFRKHQY2Au9orSPTdysiRa454bkJRrnqvz8x+bSieTyY/Eog/92KYsDsIKLj4i0XoxDCqlRao0OUUr1S2q61nmmRiNIQGBiog4LSLrksHrH2I9j5C7y0yHhlZKLlIRcZPu8AnWsU4fsXqqGUsmCQQghLUUrt01oHprTPKa2TtdYzE1/tlE3cdFRrHWvOAEUmaPYJnNwAy4bA4F3GfAMTdKhWmNMRd/nhn2OU9M7B0GbShSNEdpPmqyGlVBPgOMbksF+BY0qpRpYNS5idsxt0ngz3rhnLWz7FPIHhzUvTqXoRxqw7xrLgR/v7hRC2zpQ+gu+BllrrxlrrRsCzgGlVzUTWUrAKNPsYjiyHkPkmn6aU4pvnq1CrRF7eWRTKvjPXLBikECKzmZIInLXWRx980FofA5wtF5KwqHrDoFg9+OsduH7G5NNcnRyZ9HIARXK789qsfZyJvGvBIIUQmcmURLBPKTVVKdUk8WsKIL21tsrBETpNBKVgyQBIMH00UJ4cLvzWuyYJWtNnuhSoEyK7MCURDAQOA8MTvw4DgywZlLCwPMWhzRg4uxO2jX2qU/28czDllUDO37hP/9lBRMXKsFIhbF2aM4uBEK31WK1158SvH7TWphWvEVlX1a5Q+XnY9A1c2PdUp9YskZfvX6jG3vDrjFgYQkKCFKgTwpaZMrP4qFKqWCbFIzKLUtB2LHgWhMWvmTzr+IH21QrzXqvyrAy9xLdrj6Z9ghAiyzLl1VAe4JBSar1SavmDL0sHJjKBe27oPAmunYI17z316QMbl6RnnWJM3HyS2TvDzR6eECJzpDmhDDC9LoGwPSUaQMO3jUVsSjWHyp1NPlUpxcj2lbh0I4rPlh+ioJc7z1QsYMFghRCWYEofwSSt9eZHvzIpPpEZmrwPRWvCijfgxtmnOtXJ0YFxL1anShEvhs3bz4Gz1y0ToxDCYqSPQICjM3SeAjrB6C+Ij3uq0z1cnJjWuyYFcrnx6swgTkVI6WohbIn0EQhDXj9oNxbO7YIt3z716d6erszsUwsF9Jq+h4jbMrBMCFshfQTi/1XtCic3wpbvwK+R0X/wFEp452Ba75r0mLyLPjP2ML9/XTxdTfkrJoSwpjSfCBL7A8IxSk1sxlhwZr+F4xLW0uY7yFvSeEV07+lrCvn75ubXl2pw5NJtBs3ZR0xcggWCFEKYkynVR18DFgGTEjcVwVhZTGRHrp7Q5Te4dxWWDn6qKqUPNC2fn687V2Hr8au8s0gmnAmR1ZnSRzAEqA/cAtBaHwfyWzIoYWWFqsEzX8Cx1bB7Yrqa6BroyzvPlmNZ8EX+99cR0loASQhhPaa8wI3WWsc8WJlKKeUEyL/q7K72ADi9BdZ9Ar61oUiNp25icJNSRNyOZtq20/jkdGVg41IWCFQIkVGmPBFsVkp9CLgrpZ4BFgIrLBuWsDqloOMvkLMgLOoDUTfT0YTi03YVaVe1EN+s/pc/gs5ZIFAhREaZkgjeByKAMGAA8BfwsSWDElmER16jv+DGOVg+LF39BQ4OirFd/WlYxpv3F4fy9+HLFghUCJERpowaStBaT9Fav6C17pL4vbwashe+taD5p3B4Geydmq4mXJwcmNgzgCpFczNk7n52n4o0c5BCiIww5YlA2Lt6w6FMS1j7IVw8kK4mcrg6Mb13TXzzuNNvZhAHLzz9qyYhhGVIIhBpc3CATpMgR35Y2Dtd/QUAeXO4MPvV2uR0c6L39D2cvirLXQqRFUgiEKbxyAsvTIeb52HZkHT1FwAUzu3O7H610Rp6Tt3NpZv3zRyoEOJpPTERKKVWJK8t9OhXZgYpsgjfWtBiJBxZAbsmpLuZUj6ezOxbi5v3Y+k5dTeRd6QukRDWpJ7U76uUapzaidYqRR0YGKiDgoKscWkBxpPAgp5wbA30/guK1U53U7tORdLrtz2UKeDJ3NfqkMvN2YyBCiGSU0rt01oHprjP1gYASSLIAu7fgMlNIC4aBmwBT590N7Xh38v0n7WPGsXyMLNvLdxdHM0WphDi/6WWCFJ7NRSmlAp90pflwhVZnntu6DoL7kXC4lchIT7dTTUrX4Cx3fzZe+YaA6VInRBWkVqJiXaZFoWwPYWqQtsxxkSzTV9Ds/TPMexQrTB3o+P44M8wXp9/gHE9quPkKOMYhMgsT0wEWuszmRmIsEE1XoFzu431C4oEQLnW6W6qR61i3I2O48tVR3h3cShjulTDwUGZMVghxJPIj10iY9qMgYJV4c8BcO1Uhprq17Akb7Yoy5/7L/Dp8oNSsVSITCKJQGSMszt0m20UqVvwCsTcy1Bzw5uXZkCjkszZdZavV/8ryUCITGBSIlBKuSulylk6GGGj8pSA56fC5YOw8s10TzYDo2Lp+63L80rd4kzecoof/zluvjiFECkyZYWy9kAwsCbxs79MKBOPKfMMNPkAQufDnikZakopxcj2lXghoCg/rT/OhE0nzRSkECIlpixMMxKoBWwC0FoHK6X8LBiTsFWN3jGK0q39AApWhuL10t2Ug4Pim+erEh2XwOg1/+LsqOjXsKQZgxVCPGDKq6FYrfWjVcbkxa14nIMDdJ4EuYvDH73g1sUMNefooBjbtRqtKxfky1VHmL0z3DxxCiEeYkoiOKSUehFwVEqVUUqNA3ZYOC5hq9y8oPvvEHMXFrxszD7OACdHB37qXp0WFfLzybJDzN9z1kyBCiEeMCURDAMqAdHAPIxF7N+wYEzC1uWvAJ0mwIUgWPV2hjqPwVjYZvxLNWhc1ocPloSxaN95MwUqhADTVii7p7X+CGgONNVaf6S1jrJ8aMKmVewIDUfAgdkQNC3Dzbk6OTLp5QAalPbmnUUhLDkgyUAIczFl1FBNpVQYEAqEKaVClFIBlg9N2LymH0KZZ2H1e3Am428T3ZwdmfxyIHX88vH2HyEsC75ghiCFEKa8GpoGDNZal9BalwCGANMtGpXIHhwcofNkY57BgpfhxrkMN+nu4si03oHULJGXNxcEsyIkYx3SQgjTEkG81nrrgw9a621AnOVCEtmKe27oPg/iY2D+ixmeeQzg4eLEb71rElg8L28sCGZlqCQDITLClESwWSk1SSnVRCnVWCn1K7BJKVVDKVXD0gGKbMCnLDw/Df4Ly9Ayl8nlcHViep+aBBTLw+vzg1kVeskMgQphn0yZUFYt8dfPHtleHWM+QTOzRiSyp7ItocVn8M9IKFAJGo3IcJMPkkHv6XsYPv8AGk27qoUzHqsQdsaURNBCa53+lUeEeKD+G3D5EGz4whhiWr5thps0kkEt+k7fy+vzg0nQxvoGQgjTmfJq6LhS6julVAWLRyOyN6WgwzgoXAMWvwb/HTRLs54PXhMVz8Mb8w/IaCIhnpIpiaAacAyYppTapZTqr5TKZeG4RHbl7A7d54JbLpjXA+5EmKXZHK5OTO9dM2k00Z/7ZZ6BEKZKbc1iJwCt9W2t9RStdT3gPYy+gktKqZlKqdKZFKfITnIVMpLB3SuwoGeGy1A88KDPoE7JfLy9MISFQRkfriqEPUjtiWAPgFLKUSnVQSm1FPgR+B4oCawA/rJ0gCKbKlIDnpsA53bBitfNMpIIjKGl03rVpEFpb95dHMo8qU0kRJpM6Sw+DmwERmutdybbvkgp1cgyYQm7ULkzXD0Om74C77LQ8C2zNOvu4siUVwIZOGcfH/wZRmx8Aq/ULWGWtoXIjlJ7IsivlHoL+A04DNRVSr314AtAaz08tcaVUq2UUkeVUieUUu+nctzzSimtlApMz00IG9b4XajcBdZ/DkdWmK1ZN2ejNlGLCgX4dNkhpm7N2HrKQmRnqSUCR8Az2a85H/lKlVLKERgPtAYqAj2UUhVTOC4n8Dqw+2mDF9mAUtDxFygSaIwkurDfbE27OjkyoWcN2lYpxJerjjB+4wmztS1EdpLaq6FLWutRGWi7FnBCa30KQCk1H+iI8XSR3BfAaOCdDFxL2DJnd+gxD6Y0h3nd4bUN4FXUPE07OvBTd3+cHRXfrT3K/Zh43m5ZFqWUWdoXIjtI7Ykgo/9SigDJh22cT9z2/xcwSlT4aq1XpdZQ4pDVIKVUUESEeYYbiizGMz+89AfE3oe53SD6ttmadnJ04Puu/nSv6csvG0/wv1VH0GbqnBYiO0gtETS35IWVUg7AWODttI7VWk/WWgdqrQN9fHwsGZawpvwV4IXpcOUILOwD8earbejooPiqUxV61yvB1G2n+XjpQRISJBkIAakkAq31tQy2fQHwTfa5aOK2B3IClTEK2IUDdYDl0mFs50q3gLZj4MTfsPpdsw0rBXBwUHzWviKDmpTi991nGbEwhLj4BLO1L4StMmX4aHrtBcoopfwwEkB34MUHO7XWNwHvB5+VUpuAEVrrIAvGJGxBYF+4dhp2/Ax5/aDeMLM1rZTivVbl8XR14ru1R7kXE89PPfxxdXI02zWEsDWmlJhIF611HDAUWAscAf7QWh9SSo1SSnWw1HVFNtHic2O5y3WfwOFlZm9+SNPSfNa+ImsO/cdrs/ZxP0bqKgr7pWyt0ywwMFAHBclDg12IvQ8zO8B/ofDKcihW2+yX+CPoHO8vDiWgeB6m9a5JLjdns19DiKxAKbVPa53iq3eLPREIkWEPhpXmKmwMK71q/nkAXQN9+eXFGgSfu0GPybu4esc8dY+EsCWSCETWlsMbXlpkTDz7/XmzVStNrk2VQkx5JZCTEXfoOmknF27cN/s1hMjKJBGIrC9fKXjxD7h9GWa2hxvmLyTXpFx+Zr9am4jb0bwwYQcnI+6Y/RpCZFWSCIRtKBpoTDi7dRGmtoCLB8x+iZol8jK/fx1i4hN4YeJODl64afZrCJEVSSIQtsOvEby6DhxdYXob+Nf8VdArFfZi4cB6uDs70n3yLnaejDT7NYTIaiQRCNuSvzz0+wd8ysH8F2HbD2addAbg552DxYPqUcjLjV7T97D20H9mbV+IrEYSgbA9OQtA77+gUif4ZyQsGQCxUWa9REEvNxYOrEulwrkYNGcfC/bKAjci+5JEIGyTiwd0+Q2afgyhC2BGG7h1yayXyO3hwu/9atOwjA/vLQ5j/MYTUqxOZEuSCITtUgoavwPd5sCVf2FyEzhv3smGHi5OTHklkOf8C/Pd2qN8vuKwFKsT2Y4kAmH7KrSHfn+DU2IncvA8szbv4uTA2K7+vNrAjxk7wnl9QTDRcVKSQmQfkghE9lCgEvTfZJShWDoQ1nxg1jLWDg6Kj9tW4P3W5VkRcpG+M/ZyOyrWbO0LYU2SCET24ZEXev4JtQfBrl9hTie4a77hn0opBjYuxfcvVGP3qWt0m7SLK7fN20kthDVIIhDZi6MztP4GnpsAZ3cb/QaXQsx6iecDijK1VyDhkXd5fsIOTsksZGHjJBGI7Mn/Rei7BnQ8TGsJIQvM2nyTcvmZ378O96LjeX7CDvaduW7W9oXITJIIRPZVpAb03wxFAmFJf1j9PsSb771+1aK5+XNwPbzcnXlxyi7WycQzYaMkEYjszdMHXllq9BvsnmCsb3D7stmaL57PmIVcvlAuBs7Zx6yd4WZrW4jMIolAZH8P+g06TzWK1U1ubPQfmEk+T1fmvVabZuUL8OmyQ3z91xGZayBsiiQCYT+qvmDUKXJyM2Yi755ktjpFHi5OTHo5gFfqFmfSllMMn3+AqFiZayBsgyQCYV8KVjbmG5R+Bla/C4v7QcxdszTt6KD4vEMlPmxTnpWhl3h52m6u340xS9tCWJIkAmF/3HND97nQ7BM49CdMaQYRx8zStFKK/o1K8cuL1Qk5f5PnJ+zgTKR5Eo0QliKJQNgnBwdoNMKYgHY3AqY0hYN/mq35dlULM7dfba7fi6HTrzK8VGRtkgiEfSvVFAZshfwVYVEfWP0exJnndU5gibz8Obg+udyc6DFlFytDL5qlXSHMTRKBEF5FoPcqqDMYdk+E6a3Mti6yn3cO/hxcn2pFvRg694CUshZZkiQCIQCcXKDV19B1Flw9DhMbwtE1Zmk6bw4X5vSrTcfEUtbvLAolJi7BLG0LYQ6SCIRIrmJHY1RRbl+Y1w3WfWKW2ciuTo782M2f4c3LsGjfeRlRJLIUSQRCPCpfKXj1HwjsCzt+NtY4uHk+w80qpXjrmbL82M2fA2dv0FkK1oksQhKBEClxdoN2PxjLYV45AhMbwNHVZmn6uepFmPtabW7ej6XTrzvYceKqWdoVIr0kEQiRmsrPw4DN4OUL87rDmg/NMqoosERelg6uT/6crrzy2x7m7jZP57QQ6SGJQIi05CtllKaoNQB2jYffWsK1Uxlutlg+DxYPrkf90t58uCSMz1ccIi5eOpFF5pNEIIQpnFyhzbfQdbaRBCY2grBFGW42l5sz03oF0qd+CaZvD6fvzCBu3pclMEXmkkQgxNOo2AEGboMCFWHxq7BsSIZrFTk5OvBZ+0p83bkKO05cpdOv2zl9VcpSiMwjiUCIp5W7GPT+Cxq+DQd+h0mNzbIcZo9axZjTrzbX78bQ8ZdtbD0eYYZghUibJAIh0sPRCZp/Cq8sg5g7MLUF7Pw1w2Wt65TMx/KhDSjk5U6v3/bw27bTMhNZWJwkAiEyomRjGLgdSjWHtR/A713gzpUMNemb1+hEblGhAKNWHubdRaFEx8naBsJyJBEIkVE58kGPedBmDIRvg1/rwrG1GWrS09WJiT0DGN6sNAv3nafbpF1cvhVlpoCFeJgkAiHMQSmo9ZpRnsKzAMztCqtGQOz9dDfp4KB4q2U5JvaswbHLt2k/bhsHzko5a2F+kgiEMKf8FeC1DUYl071TYHITuBSaoSZbVS7En4Pr4ersQLdJu1iwVyafCfOSRCCEuTm7GZVMe/4J92/A1Oaw/SdISP97/vIFc7F8SANql8zLe4vD+GTpQalgKsxGEoEQllK6OQzaAWVawt+fwswOGVrnIE8OF6b3rkn/RiWZvesML03dxZXb0m8gMk4SgRCWlCMfdJsDHcfDpWCYUB9C5qd7mKmTowMftqnAzz2qE3bhJu3HbZNlMEWGSSIQwtKUguo9jRnJ+SvCkgHwxytwNzLdTXaoVpglg+vj6uRI98k7mbPrjMw3EOkmiUCIzJLXD/r8Bc0/M0paT6gLx9alu7kKhXKxYmgD6pf25uOlB3lnUShRsTLfQDw9SQRCZCYHR2j4FvTfCB7eMPcFWD4Mom+nqzkvD2em9aqZtPJZl4k7OHftnpmDFtmdJAIhrKFgFSMZNHgTDsyBCfWMyWjp4OhgrHw2rVcgZyPv0W7cNjb+m7HZzcK+SCIQwlqcXKHFSOizBhycYEZbWP0exKTvJ/rmFQqwYlgDCud2p8+MvYxdd5T4BOk3EGmTRCCEtRWrbXQk1xoAuycay2Ke3Z2uporny8GSwfXoElCUnzecoPf0PUTeiTZzwCK7kUQgRFbgksNY+KbXCoiPhd+ehbUfpatEhZuzI991qcrXnauw+/Q12v68jX1nrlkgaJFdSCIQIivxawSDd0BgH9j5S7qfDpRS9KhVjD8H1cPZSdFt0i6mSUlr8QSSCITIalxzQrsfjLUO4mKMp4M1H6ar76ByES9WDmtI0/L5+WLlYQbN2S9LYYrHSCIQIqsq2STx6aAv7BoPE+tD+PanbsbL3ZnJLwfwUZsK/HPkMu3HbePghZvmj1fYLIsmAqVUK6XUUaXUCaXU+ynsf0spdVgpFaqUWq+UKm7JeISwOa45od1Yo+9AJ8CMNrDq7aeed6CU4rVGJVkwoA6x8Ql0/nUHs3aGy6siAVgwESilHIHxQGugItBDKVXxkcMOAIFa66rAIuBbS8UjhE3za2QUsKszGPZOg/F14Pg/T91MQPG8rBrekAZlvPl02SGGzN3PrSh5VWTvLPlEUAs4obU+pbWOAeYDHZMfoLXeqLV+8OJzF1DUgvEIYdtcchjlrV9dZ3z/+/Pw5wC493QjgvLmcGHqK4F82KY8aw9dpu3PWwk5d8MyMQubYMlEUAQ4l+zz+cRtT/IqsNqC8QiRPfjWgoFbodG7cHAR/FITwhY9VUVTBwdF/0al+GNAXRISoMvEHUzdekpeFdmpLNFZrJTqCQQC3z1hf3+lVJBSKigiIiJzgxMiK3JyhWYfQf/NkLsYLH4V5naDG+fSPjeZgOJ5WDW8AU3L5efLVUd4dWaQTECzQ5ZMBBcA32SfiyZue4hSqgXwEdBBa53i30Ct9WStdaDWOtDHx8ciwQphkwpWhn7/wLNfQ/hWGF8bdk18qtXQcnu4MOnlAEa2r8i241dp/dNWdpy8asGgRVZjyUSwFyijlPJTSrkA3YHlyQ9QSlUHJmEkAamSJUR6ODhC3cEweBcUrwtr3oOpLZ5qrWSlFL3r+7FkSD083Zx4aepuxqw9Smy8LIdpDyyWCLTWccBQYC1wBPhDa31IKTVKKdUh8bDvAE9goVIqWCm1/AnNCSHSkqc4vLQInp8GN8/B5Caw7mOIuWtyE5UKe7FyWAO61CjKLxtP0HXSTs5GSlnr7E7ZWudQYGCgDgoKsnYYQmRt964Z6yQfmA1exaDtGCj77FM1sSLkIh8uCUNr+F+nynT0T22sh8jqlFL7tNaBKe3LEp3FQggz88gLHX+BPqvB2R3mdoUFL8PNx7rpnqh9tcL8Nbwh5Qrm5PX5wby5IFjmHGRTkgiEyM6K1zNKXDf7BI6vg/G1YOevEB9n0um+eT1Y0L8Ob7Qow7LgC7T5aStB4VLJNLuRRCBEdufkAo1GGJ3JxerC2g9gShM4t9e00x0deKNFWRYOrItS0HXSTsauk47k7EQSgRD2Iq8fvLQQus6Cu5EwrQUsH27yzOSA4nn5a3hDOlU3Fr3pMnEnp6+a3hEtsi5JBELYE6WgYkcYugfqDTPWSx4XAPtmQkLaP+HndHPm+67VGP9iDcKv3qXNT1v5ffcZmZFs4yQRCGGPXHNCyy+N/oP8FWDFcOMJ4eIBk05vW7UQa99oRGCJPHy05CB9Z+zlyu0oCwctLEUSgRD2rEBF6L0KOk02ylNMbgor3jDpdVFBLzdm9qnFyPYV2XEykmd/2MJfYZcsH7MwO0kEQtg7paBaNxgWZJS53j8LxtWAvVPTLFXh4GDMSF41vAG+eT0Y/Pt+3ph/gJv3ZJipLZFEIIQwuHlBq69g0HYoUNlYAGdyYzizM81TS+fPyeJB9XijRRlWhF7i2R+3sOmoVI2xFZIIhBAPy1/BWBGty3TjFdH0VrC4X5qT0ZwTh5kuHVwfTzcnek/fywd/hnIn2rQ5C8J6pMSEEOLJYu7Cth9g+89GcbsGb0G9ocZs5VRExcbzw9/HmLz1FIW93Pm2S1Xql/bOpKBFSqTEhBAifVxyQLOPjeGmpVvAxi+N2cmHlqS6EI6bsyMftKnAooF1cXFy4KWpu/loSZg8HWRRkgiEEGnLUwK6zTZeGbnmgoW9YXobuBic6mkPJqH1a+DH3D1nefaHLWw7LmsdZDWSCIQQpvNrBAO2QLsf4eoxo9T10sFw68nDRt1dHPm4XUUWDjCeDnpO280Hf4ZKAbssRPoIhBDpc/8GbB1jrIjm6AIN3oC6Q8HF44mnPOg7mLL1FAVyufFVpyo0LZ8/00K2Z6n1EUgiEEJkzLVT8PdncGQ55CwMzT+Fqt3A4ckvHILP3eDdRSEcu3yH5/wL82n7SuTN4ZKJQdsf6SwWQlhO3pJG/0Gf1ZCzICwdaFQ3PbX5iaf4++ZmxbAGvN68DKvCLtFi7GaWBV+QmkVWIolACGEexetBv/XQeaox/2BWB/i9K1w5kuLhrk6OvPlMWVYMa4BvHndenx9M3xl7uXDjfiYHLuTVkBDC/GKjYPdE2DoWYm6D/0vQ9EPIVTjFw+MTNDN2hPP9uqMAjGhZjl71SuDooDIz6mxN+giEENZxNxK2fGfULXJwgjqDjE5lN68UDz937R4fLz3I5mMRVCvqxVedq1CpcMrHiqcjiUAIYV3XTsOGL+HgInDPAw1HQM1+4Oz22KFaa1aEXmLUikNcvxfLqw38eKNFGTxcnKwQePYhiUAIkTVcDIb1n8PJDeDlC03eh6rdwfHx/+Rv3ovlmzVHmLfnHEVyu/N5h0q0qFgg82POJmTUkBAiayjsDy8vgVeWQw4fWDYEJtSDw8sfK1nh5eHM152rsmhgXTxdneg3K4j+s4KkM9kC5IlACGEdWsORFbDhC2OWcuEaRl2jUs2MNRKSiY1PYOrW0/y8/jgAr7coQ9/6frg4yc+yppJXQ0KIrCs+DkLnw6bRcPMsFG9gJITidR879Pz1e3y+4jB/H75M6fyejOpYiXqlpKqpKSQRCCGyvrho2DfTGGV094pR7bTpR1CkxmOH/nP4Mp+vPMS5a/fpUK0wH7WtQIFcj3c8i/8niUAIYTti7sHeKbDtR7h/Dcq1MTqVC1V76LCo2HgmbDrJhM0ncXZQDG9ehj7yuuiJJBEIIWxP1C1jUtrOXyDqJpRvB00+gIKVHzrsbOQ9Rq08zD9HLlPSJwefta9E47I+Vgo665JEIISwXfdvwK4JsOtXiL4FFTpA4/ceSwgbj15h1IrDnL56lxYV8vNx24qU8M5hnZizIEkEQgjbd/96YkKYYCSE8u2g8bsPvTKKjotn+vZwxq0/Tkx8An0b+DG0aWlyujlbMfCsQRKBECL7SEoIEyH6JpRtDY3fgSIBSYdcuRXF6DVHWbz/PN6errzzbFm6BPjade0iSQRCiOzn/g3YMyWxD+GGMf+g0TtGFdREIeduMGrlYfaduU6FQrn4pG0F6pW2z+GmkgiEENlX9G3YO81ICHcjoFg9aPQ2lGoOSqG1ZmXoJb5Z/S8XbtynRYX8vN+6AqXze1o78kwliUAIkf3F3IP9s2DHz3DrAhSsCg3fMjqXHRyJijX6D8ZvPMH92Hh61PLl9eZl8cnpau3IM4UkAiGE/YiLgdAFsP1HiDwBeUtB/eFQrQc4uXL1TjQ/rz/O3N1ncXVyYEDjUvRr6Jftq5tKIhBC2J+EePh3JWz7AS4eAM8CUHsgBPYF99ycirjD6DX/svbQZXxyuvJ68zJ0q+mLs2P2nJAmiUAIYb+0htObYftPRvlrF08I6G0khdy+7Dtzja//+pegM9fx887B2y3L0qZyIRyy2QgjSQRCCAFwKdToQzj4p/G5UieoNxRdyJ9/jlxhzNqjHL18m0qFc/HOs+VoXNYHpbJHQpBEIIQQyd04Z5Sv2DfTWFO5WD2oO4T4Mq1YFvofY/8+xvnr96lZIg9vtyxHnZL5rB1xhkkiEEKIlETdhANzjMlpN89CnhJQqz8xVV5kwcFb/LLhOJdvRVO/dD7eeqYsAcXzWjvidJNEIIQQqYmPMzqWd02Ac7uMfgT/F4mu3pfZJ1yZsOkkkXdjaFTWhzdblKF6sTzWjvipSSIQQghTXTxgPCEc+hPiY6BkU6JrvMqMq+WYuCWc6/diaVTWh9ebl7apJwRJBEII8bTuXDH6EIJ+g9sXwasY0dV7MS+2CeN2XSfybgz1S+djSNPS1C2ZL8t3KksiEEKI9IqPhaN/wd6pcHoLOLoQV649a9xa8XloHiLuxFC9WG6GNClNs/L5s+ywU0kEQghhDhFHIWg6hMyFqJsk5CvDfu+OfHamCoduOFO2gCcDGpWig3/hLDcxTRKBEEKYU8w9OLQE9s2A83vQDs5cKNicX2/WZX5kKQp6edC7fgm61ypGriyyFoIkAiGEsJTLh41id6Hz4f51ojwKscqhKT9HBhLpUpRuNX3pXa8Evnk9rBqmJAIhhLC0uGijL+HAHDixHtCcdK/MtNt1+Cu+FrUqlKJPfT/qlMxrlY5lSQRCCJGZbl4wKqCGzIOrx4hTzmzWNfgjpi7nvRvQvW4ZOtUoiqdr5lU8lUQghBDWoLUxLyFsITpsEeruFe4pd1bHBbJO1cenakteqF2KqkW9LP6UIIlACCGsLT4OwregwxYTf3g5TjG3uKlz8E9CDcJyNaFErba0rVHSYgvlWC0RKKVaAT8BjsBUrfU3j+x3BWYBAUAk0E1rHZ5am5IIhBA2Ly4aTm4k5uBS9JGVuMbd5r52YZuuyjmfxhQK7ECD6pXIacYRR1ZJBEopR+AY8AxwHtgL9NBaH052zGCgqtZ6oFKqO9BJa90ttXYlEQghspX4WAjfyo0DS1HH1uAVcxmAf3UxwnPXxb1MI0oHNKVIoSIZukxqicCSPRW1gBNa61OJQcwHOgKHkx3TERiZ+P0i4BellNK29r5KCCHSy9EZSjUjd6lmoDUJl0K5uG8Vrkf/psXNRTgFLYAgCHfw5VqtEdRo1dvsIVgyERQBziX7fB6o/aRjtNZxSqmbQD7gavKDlFL9gf4AxYoVs1S8QghhXUrhULgaRQtXg/YfoqPvcO7Qdi4f2oLzxT04e+SyyGVtYrVmrfVkYDIYr4asHI4QQmQK5eqJb41n8a3xrEWvY8liGBcA32SfiyZuS/EYpZQT4IXRaSyEECKTWDIR7AXKKKX8lFIuQHdg+SPHLAd6JX7fBdgg/QNCCJG5LPZqKPGd/1BgLcbw0d+01oeUUqOAIK31cmAaMFspdQK4hpEshBBCZCKL9hForf8C/npk26fJvo8CXrBkDEIIIVKXtQpmCyGEyHSSCIQQws5JIhBCCDsniUAIIeycJAIhhLBzkgiEEMLOSSIQQgg7J4lACCHsnCQCIYSwcza3VKVSKgI4k87TvXmkxLUdkHu2D3LP9iEj91xca+2T0g6bSwQZoZQKetIKPdmV3LN9kHu2D5a6Z3k1JIQQdk4SgRBC2Dl7SwSTrR2AFcg92we5Z/tgkXu2qz4CIYQQj7O3JwIhhBCPyJaJQCnVSil1VCl1Qin1fgr7XZVSCxL371ZKlbBCmGZlwj2/pZQ6rJQKVUqtV0oVt0ac5pTWPSc77nmllFZK2fwIE1PuWSnVNfHP+pBSam5mx2huJvzdLqaU2qiUOpD497uNNeI0F6XUb0qpK0qpg0/Yr5RSPyf+foQqpWpk+KJa62z1hbEs5kmgJOAChAAVHzlmMDAx8fvuwAJrx50J99wU8Ej8fpA93HPicTmBLcAuINDacWfCn3MZ4ACQJ/FzfmvHnQn3PBkYlPh9RSDc2nFn8J4bATWAg0/Y3wZYDSigDrA7o9fMjk8EtYATWutTWusYYD7Q8ZFjOgIzE79fBDRXSqlMjNHc0rxnrfVGrfW9xI+7gKKZHKO5mfLnDPAFMBqIyszgLMSUe34NGK+1vg6gtb6SyTGamyn3rIFcid97ARczMT6z01pvwVjD/Uk6ArO0YReQWylVKCPXzI6JoAhwLtnn84nbUjxGax0H3ATyZUp0lmHKPSf3KsZPFLYszXtOfGT21VqvyszALMiUP+eyQFml1Hal1C6lVKtMi84yTLnnkUBPpdR5jDXSh2VOaFbztP/e02TRxetF1qOU6gkEAo2tHYslKaUcgLFAbyuHktmcMF4PNcF46tuilKqitb5hzaAsrAcwQ2v9vVKqLjBbKVVZa51g7cBsRXZ8IrgA+Cb7XDRxW4rHKKWcMB4nIzMlOssw5Z5RSrUAPgI6aK2jMyk2S0nrnnMClYFNSqlwjHepy228w9iUP+fzwHKtdazW+jRwDCMx2CpT7vlV4A8ArfVOwA2jJk92ZdK/96eRHRPBXqCMUspPKeWC0Rm8/JFjlgO9Er/vAmzQib0wNirNe1ZKVQcmYSQBW39vDGncs9b6ptbaW2tdQmtdAqNfpIPWOsg64ZqFKX+3l2I8DaCU8sZ4VXQqE2M0N1Pu+SzQHEApVQEjEURkapSZaznwSuLooTrATa31pYw0mO1eDWmt45RSQ4G1GCMOftNaH1JKjQKCtNbLgWkYj48nMDplulsv4owz8Z6/AzyBhYn94me11h2sFnQGmXjP2YqJ97wWaKmUOgzEA+9orW32adfEe34bmKKUehOj47i3Lf9gp5Sah5HMvRP7PT4DnAG01hMx+kHaACeAe0CfDF/Thn+/hBBCmEF2fDUkhBDiKUgiEEIIOyeJQAgh7JwkAiGEsHOSCIQQws5lu+GjQjyglMoHrE/8WBBjOOWD8eW1EmvXmOM6JYCVWuvK5mhPiMwmiUBkW4nj5/0BlFIjgTta6zHWjOlRicUO1YNyCI9+TuU8p8Q6WUJkmLwaEvbEXSl1WinlDKCUyvXgs1Jqk1LqJ6VUsFLqoFKqVuIxORLrw+9JrHefUoXTJ1JKvaOU2ptYN/7zxG0lEuvrzwIOAg0f+eyrlPouMY4wpVS3xPOaKKW2KqWWA4fN+Psi7Jw8EQh7ch/YBLTFKMXQHfhTax2bONvaQ2vtr5RqBPyGUavoI4wSJH2VUrmBPUqpf7TWd9O6mFKqJUadn1oYteOXJ7Z9NnF7L631rsRXS8k/P4/xJFMNo2bOXqXUlsRmawCVE+sICWEW8kQg7M1U/n9Kfh9gerJ98yCpHnyuxP/4WwLvK6WCMZKIG1DMxGu1TPw6AOwHyvP/BeDOJNaSJ4XPDYB5Wut4rfVlYDNQM3HfHkkCwtzkiUDYFa319sRXM00AR6118uUAH623ojF+kn9ea300HZdTwNda60kPbTSeAB59okjzCeMpjxPCZPJEIOzRLGAuDz8NADx4F98Ao6LjTYxiZ8MerGCXWMXVVGuBvkopz8Rziyil8ptw3lagm1LKUSnlg7F04Z6nuK4QT0WeCIQ9+h34ksRXQclEKaUOYFR67Ju47QvgRyA0cbGb00A7Uy6itV6XWBZ5Z2IeuQP0xBjGmpolQF2M9Xk18K7W+j+lVHlTrivE05Lqo8LuKKW6AB211i8n27YJGGHj6xUIkS7yRCDsilJqHNAao567EAJ5IhBCCLsnncVCCGHnJBEIIYSdk0QghBB2ThKBEELYOUkEQghh5yQRCCGEnfs/RLKxP29xFIYAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "# Get hypothesis testing interpretation so we can directly plot it\n", + "fpr_list, fnr_list = composed_mech.plot_fDP()\n", + "fpr_list, fnr_list2 = rdp_composed_mech.plot_fDP()\n", + "\n", + "plt.figure(figsize = (6,6))\n", + "plt.plot(fpr_list,fnr_list,label='Gaussian_composition')\n", + "plt.plot(fpr_list,fnr_list2,label='Generic_composition')\n", + "plt.xlabel('Type I error')\n", + "plt.ylabel('Type II error')\n", + "plt.legend()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 3: Compare RDP vs fDP in describing Gaussian mechanism" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "from autodp.mechanism_zoo import GaussianMechanism\n", + "import numpy as np\n", + "\n", + "sigma = 1.0\n", + "\n", + "gm0 = GaussianMechanism(sigma,name='GM0',approxDP_off=True, use_basic_RDP_to_approxDP_conversion=True)\n", + "gm1 = GaussianMechanism(sigma,name='GM1',approxDP_off=True)\n", + "gm1b = GaussianMechanism(sigma,name='GM1b',approxDP_off=True, use_fDP_based_RDP_to_approxDP_conversion=True)\n", + "gm2 = GaussianMechanism(sigma,name='GM2',RDP_off=True)\n", + "gm3 = GaussianMechanism(sigma,name='GM3',RDP_off=True, approxDP_off=True, fdp_off=False)\n", + "\n", + "\n", + "eps = np.sqrt(2)/sigma # Aligning the variance of the laplace mech and gaussian mech\n", + "laplace = PureDP_Mechanism(eps,name='Laplace')\n", + "\n", + "label_list = ['naive_RDP_conversion','BBGHS_RDP_conversion','Our new method',\n", + " 'exact_eps_delta_DP','exact_fdp',r'laplace mech ($b = \\sqrt{2}/\\sigma$)']\n", + "\n", + "\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.8/site-packages/scipy/optimize/optimize.py:2522: RuntimeWarning: invalid value encountered in double_scalars\n", + " w = xb - ((xb - xc) * tmp2 - (xb - xa) * tmp1) / denom\n", + "/usr/local/lib/python3.8/site-packages/scipy/optimize/optimize.py:2116: RuntimeWarning: invalid value encountered in double_scalars\n", + " tmp2 = (x - v) * (fx - fw)\n", + "/Users/yuxiangw/Documents/github/autodp_experimental/autodp/autodp_core.py:233: RuntimeWarning: divide by zero encountered in log\n", + " fdp = lambda x: 1 - np.exp(fun1(np.log(x)))\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAFzCAYAAAAzNA41AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAACHaUlEQVR4nOzdd1xV9f/A8de5k71BkCGgIEOGuPcq03KkDVuWWpZl076V7d23b+PXtDKbWpktzcq03HuLqAhOlCWyN9x1fn9cuIoyhQson+fjcR/ce86557wvGe97PuP9kWRZRhAEQei4FG0dgCAIgtC2RCIQBEHo4EQiEARB6OBEIhAEQejgRCIQBEHo4EQiEARB6OBUbR1AU3l4eMiBgYFtHYYgCMJlZc+ePTmyLHvWtu+ySwSBgYHs3r27rcMQBEG4rEiSdKqufaJpSBAEoYMTiUAQBKGDE4lAEAShg7vs+ggE4XKi1+tJS0ujoqKirUMROggbGxv8/PxQq9WNfo9IBIJgRWlpaTg6OhIYGIgkSW0djnCFk2WZ3Nxc0tLSCAoKavT7RNOQIFhRRUUF7u7uIgkIrUKSJNzd3Zt8ByoSgSBYmUgCQmu6lH9vIhEIgiB0cCIRCIJg8dlnn7Fw4cIWO9+0adMICgoiNjaWmJgY1qxZY9k3fPhwunfvTnR0NGFhYTz44IMUFBRY9iuVSmJjY+nRowc33XQTZWVlLRZXa7rnnntITExs6zDqZbVEIEnSV5IknZUk6WAd+yVJkj6UJOmYJEkJkiTFWSsWQRAaZ9asWdx5550tes63336b+Ph43n//fWbNmlVj3/fff09CQgIJCQlotVomTpxo2Wdra0t8fDwHDx5Eo9Hw2WeftWhcLclgMNS574svviAiIqIVo2k6a44a+gb4GKjr68VYIKTq0Q/4tOqnIFyRXv7jEIkZRS16zojOTrw4PrLO/SkpKYwdO5bBgwezdetWfH19+f333/nuu+/4/PPP0el0dOvWjUWLFmFnZ8dLL72Eg4MD48aN484772Tnzp2W84wfP54DBw6wZ88e5syZQ0lJCR4eHnzzzTf4+Pg0GOuAAQNIT0+vdZ9Go+Gtt96iW7du7N+/n5iYmBr7hwwZQkJCQp3nXrhwIe+88w6SJBEdHc2iRYtISUlhxowZ5OTk4Onpyddff01AQADTpk3DycmJ3bt3c+bMGd566y1uvPFGbrnlFqZOncp1110HmO9mxo0bx6RJk5g7dy7r16+nsrKS2bNnc99997F+/Xqef/55XF1dSUpKYt++fdx8882kpaVhNBp5/vnnmTJlCsOHD+edd96hd+/eLF68mDfeeANZlrnuuuv43//+B4CDgwOPPPIIf/75J7a2tvz+++906tSpwd9pS7HaHYEsyxuBvHoOmQgslM22Ay6SJDX8r+kS7du4jKVv3ovRYLTWJQShXTp69CizZ8/m0KFDuLi48OuvvzJ58mR27drF/v37CQ8P58svv6zxnrCwMHQ6HSdPngRgyZIlTJkyBb1ez0MPPcQvv/zCnj17mDFjBs8++2yj4li5ciXXX399nfuVSiUxMTEkJSXV2G4wGPj777+Jioqq9X2HDh3itddeY+3atezfv58PPvgAgIceeoi77rqLhIQEbr/9dh5++GHLezIzM9m8eTN//vknc+fOBWDKlCn89NNPAOh0OtasWcN1113Hl19+ibOzM7t27WLXrl0sWLDA8nvZu3cvH3zwAUeOHGHlypV07tyZ/fv3c/DgQcaMGVMjzoyMDJ566inWrl1LfHw8u3btYtmyZQCUlpbSv39/9u/fz9ChQ1mwYEGjfqctpS3nEfgCqee9TqvalnnhgZIk3QvcCxAQEHBJF9u/6EcMuVpSEuPpGt3rks4hCM1R3zd3a6puowfo1asXKSkpHDx4kOeee46CggJKSkq45pprLnrfzTffzJIlS5g7dy5LlixhyZIlJCcnc/DgQa6++moAjEZjg3cDTzzxBM888wxpaWls27at3mNlWbY8Ly8vt8Q9ZMgQ7r777lrfs3btWm666SY8PDwAcHNzA2Dbtm389ttvAEydOpUnn3zS8p7rr78ehUJBREQEWVlZAIwdO5ZHHnmEyspKVq5cydChQ7G1teWff/4hISGBX375BYDCwkKOHj2KRqOhb9++lvH6UVFRPP744zz11FOMGzeOIUOG1Ihz165dDB8+HE9PcwHQ22+/nY0bN3L99dej0WgYN24cYP5v9O+//9b7e2ppl8WEMlmWPwc+B+jdu7fcwOG10mm9KFafIHnjOpEIhA5Fq9VaniuVSsrLy5k2bRrLli0jJiaGb775hvXr11/0vilTpnDTTTcxefJkJEkiJCSEAwcOEBkZ2eAf9PO9/fbb3HjjjXz00UfMmDGDPXv21Hqc0WjkwIEDhIeHA+f6CKzh/N9JdfKxsbFh+PDhrFq1iiVLlnDLLbdY9n/00UcXJcv169djb29veR0aGsrevXtZsWIFzz33HKNGjeKFF15oVDxqtdoy7FOpVNbb52ANbTlqKB3wP++1X9U2q7DrYkBpNJGfcthalxCEy0ZxcTE+Pj7o9Xq+//77Wo/p2rUrSqWSV199lSlTpgDQvXt3srOzLYlAr9dz6NChRl3zwQcfxGQysWrVqov26fV6nn76afz9/YmOjm7SZxk5ciQ///wzubm5AOTlmVukBw4cyI8//giYO6Uv/IZemylTpvD111+zadMmS9PONddcw6effoperwfgyJEjlJaWXvTejIwM7OzsuOOOO3jiiSfYu3dvjf19+/Zlw4YN5OTkYDQaWbx4McOGDWvSZ7WWtrwjWA48KEnSj5g7iQtlWb6oWailaPyCcF2XRMHZXGtdQhAuG6+++ir9+vXD09OTfv36UVxcXOtxU6ZM4YknnrC0iWs0Gn755RcefvhhCgsLMRgMPProo0RGNtzsJUkSzz33HG+99Zbl2/Xtt9+OVqulsrKSq666it9//73JnyUyMpJnn32WYcOGoVQq6dmzJ9988w0fffQR06dP5+2337Z0Fjdk9OjRTJ06lYkTJ6LRaADz8M+UlBTi4uKQZRlPT09L2/75Dhw4wBNPPIFCoUCtVvPpp5/W2O/j48Obb77JiBEjLJ3F54+SakvS+W1yLXpiSVoMDAc8gCzgRUANIMvyZ5L5PuhjYAxQBkyXZbnBFWd69+4tX8rCNMs3LCfvveXkas9w/4LvsXNybvI5BKGpDh8+bGnqEITWUtu/O0mS9siy3Lu24612RyDL8q0N7JeB2da6/oXcPELIcIwB3RnSDx8ipN/A1rq0IAhCu3ZZdBa3BCdHb/xSP+GoVxCpiQdEIhCEFjR79my2bNlSY9sjjzzC9OnTW+waubm5jBo16qLta9aswd3dvcWu0xF1mERgb6um2AEcKytJTTzQ1uEIwhVl3rx5Vr+Gu7u71UYRdXQdptaQnUZJkWssqMPIOZ1CeXHLzvAUBEG4XHWcRKBWobNxxaQNxcO/CyX59U16FgRB6Dg6TNOQrUZJsc12rtq0juA//0AbENjWIQmCILQLHeaOQKNSUGZnnk2oP5OFySRqDgmCIEAHSgQA5TYeHIi8h83/bGbejFsoKyps65AEweqq6/rHxMQQFxfH1q1bAXNFUVtbW8u+gQMHkpycbHnfypUr6du3L2FhYcTGxjJlyhROnz4NmCtzVtfeqebg4ACAyWTi4YcfpkePHkRFRdGnTx/LhLTaBAYGEhUVRXR0NMOGDePUqVMXxR4ZGUlMTAzvvvsuJpMJMJd4cHZ2JjY2lvDwcF5++eWW+YW1gYED23YUY4dpGgIosXeg1M4bp7JTRAwdidGgb+uQBMHqzq/Zs2rVKp5++mk2bNgAmMtIVO+bP38+b7zxBt9++y0HDx7koYceYvny5ZaJScuXLyclJaXBwo9LliwhIyODhIQEFAoFaWlpNWry1GbdunV4eHjw4osv8tprr1mqb54f+9mzZ7ntttsoKiqy/NEfMmQIf/75J6WlpcTGxjJ+/Hji4trn0iYGgwGVqvY/udXJua10qERgUrsRlvgazi59iJjRcqswCUKj/D0XzrTw0GXvKBj7ZqMPLyoqwtXVtcF9//vf/3jmmWdqzE6dMGFCo66RmZmJj48PCoW5wcHPz6/R8Q0YMIAPP/yw1n1eXl58/vnn9OnTh5deeqnGPnt7e3r16sWxY8dqTQRGo5GnnnqKlStXolAomDlzJg899BBr1qzhP//5DwaDgT59+vDpp5+i1WoJDAzkrrvu4o8//kCv1/Pzzz8TGhpKcHAw8fHxuLi4ABASEsLmzZtRKBTMmjXLcsf0/vvvM2jQIF566SWOHz/OiRMnCAgI4LnnnmP69OnodDpMJhO//vorISEhODg4UFJSgizLPPnkk/z999+WkhxTpkxh/fr1vPTSS3h4eHDw4EF69erFd99912LrYXeoRIDCnRxHsKvqI8hLS8VDdBoLV7jqcs4VFRVkZmaydu1ay77jx48TGxtLcXExZWVl7NixAzDX+P/Pf/5T73mfeOIJXnvttYu233zzzQwePJhNmzYxatQo7rjjDnr27NmoWBtasyA4OBij0cjZs2drbM/NzWX79u08//zztb7v888/JyUlhfj4eFQqFXl5eVRUVDBt2jTWrFlDaGgod955J59++imPPvooAB4eHuzdu5dPPvmEd955hy+++IKJEyeydOlSpk+fzo4dO+jSpQudOnXitttu47HHHmPw4MGcPn2aa665hsOHzQUuExMT2bx5M7a2tjz00EM88sgj3H777eh0OozGmn2Vv/32G/Hx8ezfv5+cnBz69OnD0KFDAdi3bx+HDh2ic+fODBo0iC1btjB48OBG/V4b0qESgaTy5ozPVZRJwVQs/43Ni7/lgS9+wNbRqa1DEzqCJnxzb0nnN69s27aNO++8k4MHzSvInt80tGTJEu69915WrlxZ4/3VM3rLysq49957LQmiurx0teo+Aj8/P5KTk1m7di1r165l1KhR/Pzzz7XOCq42YsQI8vLycHBw4NVXX230Z9u0aRM9e/ZEoVAwd+7cOovfrV69mlmzZlmaZtzc3Ni/fz9BQUGEhoYCcNdddzFv3jxLIpg8eTJgXh+gel2DKVOm8MorrzB9+nR+/PFHS1XW1atX11iXuKioiJKSEsB8J2VrawuY73hef/110tLSmDx5MiEhITXi3Lx5M7feeitKpZJOnToxbNgwdu3ahZOTE3379rXcXcXGxpKSktJiiaBDdRaj8aPMRgl6Gb8w8z+YtMO1LqksCFekAQMGkJOTQ3Z29kX7JkyYwMaNGwFzRc/qMsrVM3rvvfdeyx+3hmi1WsaOHcvbb7/NM888U2u1zvOtW7eOU6dOERsby4svvljncSdOnECpVOLl5QWY+wj27dvHnj17LloPubmq1yw4f32AAQMGcOzYMbKzs1m2bJklWZhMJrZv3058fDzx8fGkp6dbEuP5/SO33XYby5cvx9bWlmuvvbbG3Vlj47kwppbQoRKBQuNNjvYfog8swNO7MyqNVpSbEDqUpKQkjEZjrbV5Nm/eTNeuXQF48sknef311y3NGwBlZWWNusbevXvJyMgAzH8gExIS6NKlS4PvU6lUvP/++yxcuNCypsD5srOzmTVrFg8++GCT28avvvpq5s+fb/njmZeXR/fu3UlJSeHYsWMALFq0qMH1ASRJYtKkScyZM4fw8HDL73H06NF89NFHluPqKoVx4sQJgoODefjhh5k4ceJF6zAPGTKEJUuWYDQayc7OZuPGjfTt27dJn/VSdKimITtbDRXmJI2cm0vn0DDSDolEIFzZzl/yUZZlvv32W5RKJXCuj0CWZTQaDV988QVgXnbxgw8+4M4776SoqAgPDw8CAgIaNUTz7NmzzJw5k8rKSsC8IMuDDz7YqFh9fHy49dZbmTdvHs8//7wldr1ej0qlYurUqcyZM6fJv4N77rmHI0eOEB0djVqtZubMmTz44IN8/fXX3HTTTZbO4sbcVUyZMoU+ffrwzTffWLZ9+OGHzJ49m+joaAwGA0OHDuWzzz676L0//fQTixYtQq1W4+3tzTPPPFNj/6RJk9i2bRsxMTFIksRbb72Ft7f3Res4tzSrrUdgLZe6HgHAc8sOUL7zEWJT7mDADSHkkM6Wn74T/QSC1Yj1CIS20NT1CDpU05CdRkWlnR6loRxTfh5+kVGA6CcQBKFj61BNQ7ZqJcWO5fRM+BjXoQ/i2vVaVBotaYkHCekr1icQBGvq16+fpbmo2qJFi4iKimqxa6xatYqnnnqqxragoCCWLl3aYte4EnWoRGCnUSLJthTal+KSdQaVWk3n0DDRYSwIraB6joI1XXPNNZb1kIXG62BNQ0ow2pEUcgc7ssyjGPwjosg+dVKsTyAIQofVwe4IVBiNDugVuWiKzIWrIoaNIqhnb7QN1EIRBEG4UnW4OwK9wYliaSVBR8wzBZ08POkU3A2FQtnG0QmCILSNDpUIbDVKKoyu5DpJUFaBobgYgLTEg2z/bUkbRycI1pGWlsbEiRMJCQmha9euPPLII+h0urYOy2pSUlL44YcfLK+/+eabRs9jqE1z33856FCJwE6josTgTqFjIJsH/pfUzeZJGmmHD7Jr+S/oKyraOEJBaFmyLDN58mSuv/56jh49ypEjRygpKeHZZ59t0nkuLI7Wnl2YCISGdbBEoKRQ78WxTvm45R7ClGpeLCPuuok88MVi1DY2bRyhILSstWvXYmNjw/Tp0wFzjZr33nuPr776irKysou+7Y4bN47169cD5iJyjz/+ODExMWzbtq3GeYcPH85TTz1F3759CQ0NZdOmTYA5YTzxxBP06dOH6Oho5s+fD8Ds2bNZvnw5YJ49O2PGDAC++uqrWpOSg4MDTzzxBJGRkVx11VXs3LmT4cOHExwcbDlPXdeaO3cumzZtIjY2lvfeew+AjIwMxowZQ0hICE8++aTlOosXLyYqKooePXrUGHb69ddfExoaSt++fdmyZcsl/vYvHx2qs9hWo6TA5IHerZBuqYuxS50E3IzGxratQxM6gP/t/B9JeS1bKiDMLYyn+j5V5/5Dhw7Rq1evGtucnJwICAiw1NipS2lpKf369ePdd9+tdb/BYGDnzp2sWLGCl19+mdWrV/Pll1/i7OzMrl27qKysZNCgQYwePZohQ4awadMmJkyYQHp6OpmZmYC5eugtt9xS67VHjhzJ22+/zaRJk3juuef4999/SUxM5K677mLChAl1XuvNN9/knXfe4c8//wTMTTvx8fHs27cPrVZL9+7deeihh1AqlTz11FPs2bMHV1dXRo8ezbJly+jXrx8vvvgie/bswdnZmREjRjS6jPblqkMlAnuNCj0aXGTI81Zjl3jUsm//vytITTzIuEeerOcMgtBxKJVKbrjhhjr3n1+mOSUlBYB//vmHhIQEyzKWhYWFHD16lCFDhvD++++TmJhIREQE+fn5ZGZmsm3btloXotFoNIwZMwYw1z3SarWo1WqioqIavJZGo7nofKNGjcLZ2RmAiIgITp06RW5uLsOHD8fT0xOA22+/3VJ99fztU6ZM4ciRI0363V1uOlQisNWYRwY5myRO+owiSXEVQRWVqGy0lBUVkrxtE6Puvh9bB8c2jlS4EtX3zd1aIiIiLlpbuKioiNOnT9OtWzcSEhIsawADVJzXT2ZjY2MpTleb2so0y7LMRx99VOukroKCAlauXMnQoUPJy8vjp59+wsHBAUfHi/9/U6vVlgqjCoXCci2FQtHgtaqbtmqL9cJ4BbMO10cA4CiryHU+TuCplZQfPQGYJ5Yhy6QfPtSWIQpCi6peUGbhQvPSrEajkccff5xp06ZhZ2dHYGAg8fHxmEwmUlNT2blzZ7Oud8011/Dpp5+i15vXAz9y5AilpaUA9O/fn/fff5+hQ4cyZMgQ3nnnHYYMGdLi13J0dKS4akRgffr27cuGDRvIycnBaDSyePFihg0bRr9+/diwYQO5ubmWZSqvdB0qEaiVCtRKCXu0HPY5TuDpVRiPJQPg3a07KrVGlJsQriiSJLF06VJ+/vlnQkJCCA0NxcbGhjfeeAOAQYMGERQUREREBA8//HCzF36/5557iIiIIC4ujh49enDfffdZvn0PGTIEg8FAt27diIuLIy8vr1mJoK5rRUdHo1QqiYmJsXQW18bHx4c333yTESNGEBMTQ69evZg4cSI+Pj689NJLDBgwgEGDBnWI6rEdqgw1QPRLqxjp+192KbJZMM8O2+tvJuiFJwD4+dVnKC8p4c7/1b54tiA0lShDLbQFUYa6AXYaFWrZniKVgvi4R9me6mvZ51ddd6ik4dtKQRCEK0XHSwRaJSrZvAhNkEsa/idWIld1lvmHi34CQRA6no6XCDRKJNkbABv/XNzTd6FPSwPAu1uo6CcQBKHD6XiJQK2i1GheoPu0cymltl7k7jYv0K3SaPAR6xMIgtDBdLhEYKtRcsbYDa3JxHHbfHb2fY7E3fmW/WEDh+LbPdzSXCQIgnCl61ATysDcNJRhsMXfBCeNGdxasBIXWQbM09yjrxrTtgEKgiC0sg55R1CmM9JFYcspXSGBQRpUh3fVOEY2mSjJz2ujCAVBEFpXh0sE9hoVZToDXWw8SEWPHBJGlrETJacyLccse/tVlr75chtGKQjt37Jly0hMTGzrMAB46aWXeOeddxp9zDfffENGRsYlXWvatGkEBQURExNDaGgod955J2lVA04AAgMDiYqKIjo6mtGjR3PmzJlLuk5r6nCJwK76jsDRH4Mkcdbdk4ToBzi5+lwHcfRVY+k9YXIbRikI7V97SgRN1ZxEAPD222+zf/9+kpOT6dmzJyNHjqyx2M+6detISEigd+/ellnc7VmH6yOw1SipNJgIcOsOudspCcgj7tBS3Hx7A6MB6Nqrb9sGKVyRzrzxBpWHW7YMtTY8DO9nnqn3mO+++44PP/wQnU5Hv379mDFjBjNnzmTnzp0YjUb69u3LkiVLCAwMZOLEieTn56PX63nttdeYOHEiAAsXLuSdd95BkiSio6O5//77Wb58ORs2bOC1117j119/pWvXrhdd+/jx48yePZvs7Gzs7OxYsGABYWFhTJs2DRsbG3bv3k1RURH/93//x7hx4zh06BDTp09Hp9NhMpn49ddfCQkJqfVzvf7663z77bd4eXnh7+9vKbdd1zWr/fLLL+zevZvbb78dW1tbtm3bxttvv80ff/xBeXk5AwcOZP78+Zaid/WRJInHHnuMpUuX8vfff1t+X9WGDh1aa3XV9qbDJYLqwnOd3GPgKJwuSmRwsCOV+2r2E+SmnaaipATfsIi2CFMQWsThw4dZsmQJW7ZsQa1W88ADD5CcnMyECRN47rnnKC8v54477qBHjx4YDAaWLl2Kk5MTOTk59O/fnwkTJpCYmMhrr73G1q1b8fDwIC8vDzc3NyZMmMC4ceO48cYb67z+vffey2effUZISAg7duzggQceYO3atYB5JbGdO3dy/PhxRowYwbFjx/jss8945JFHuP3229HpdHWujLZnzx5+/PFH4uPjMRgMxMXFWRJBfdcEuPHGG/n4449555136N3bXHHhwQcf5IUXXgBg6tSp/Pnnn4wfP77Rv+e4uDiSkpIuSgR//vknUVFRjT5PW+lwicBWY/7IWtce2JtMnCo4waDo4RxdeQifnEJsPcw1y9d8+SmV5WVMffODtgxXuII09M3dGtasWcOePXvo06cPAOXl5Xh5efHCCy/Qp08fbGxsLN9YZVnmmWeeYePGjSgUCtLT08nKymLt2rXcdNNNeHh4AODm5taoa5eUlLB161Zuuukmy7bKykrL85tvvhmFQkFISAjBwcEkJSUxYMAAXn/9ddLS0pg8eXKddwObNm1i0qRJ2NnZATBhwoRGXbMu69at46233qKsrIy8vDwiIyOblAgurNk2YsQIlEol0dHRvPbaa40+T1vpcInATm2+IyhXOtHFKHO6LBN9l0iOdQ2l6/r9dLtxKGCuO7Tt18VUlJZgY+/QliELwiWTZZm77rqL//73vzW2Z2ZmUlJSgl6vp6KiAnt7e77//nuys7PZs2cParWawMDAGusTNJXJZMLFxYX4+Pha91/Y9CJJErfddhv9+vXjr7/+4tprr2X+/PmMHDmyxa5Zm4qKCh544AF2796Nv78/L730UpM/9759+xg1apTl9bp16yyJ83LQ4TqL7bXmRFBaaaSLwo4UfRGBV8cyaPtzOKXttRznH2muO5Qm6g4Jl7FRo0bxyy+/cPbsWQDy8vI4deoU9913H6+++iq33367Za3ewsJCvLy8UKvVrFu3jlOnTgEwcuRIfv75Z3Jzcy3nABqs++/k5ERQUJClnr8sy+zfv9+y/+eff8ZkMnH8+HFOnDhB9+7dOXHiBMHBwTz88MNMnDiRhISEWs89dOhQli1bRnl5OcXFxfzxxx+Numa182Ov/qPv4eFBSUnJRQv51EeWZT788EMyMzMtK6pdjjpcIqhuGirXm4eQZsoGZActziF+lO/eYznOp1t3lGo1aaLchHAZi4iI4LXXXmP06NFER0dz9dVX8+2336JWq7ntttuYO3cuu3btYu3atdx+++3s3r2bqKgoFi5caOlgjYyM5Nlnn2XYsGHExMQwZ84cAG655RbefvttevbsyfHjx2u9/vfff8+XX35JTEwMkZGR/P7775Z9AQEB9O3bl7Fjx/LZZ59hY2PDTz/9RI8ePYiNjeXgwYPceeedtZ43Li6OKVOmEBMTw9ixYy1NXw1ds9q0adOYNWsWsbGxaLVaZs6cSY8ePbjmmmtqnKsuTzzxhGX46K5du1i3bl2tS2ReLjrcegS7UvK46bNtLLq7LwXJz/JM7jZ+H/czlR//S/K+Qq77/j+o7W0A+Onlp0U/gdAsYj2C2k2bNq3Bjmbh0on1CBpgW9VHUKYzEuhm/saTkrkbY5cwstxjyd5+7g7ALyKKsyknqCgtaZNYBUEQWkPH6yyuGj5arjMS4B0HR7/mdPZBBl83FsWbQ9D0ewxGmW8N/SOj2PbLD6QnHaJrr35tGbYgtGuzZ89my5YtNbY98sgjTJ8+vdbjv/nmm0adNzc3t0YnbLU1a9bg7u7e5Diboqmf6XLWAROB+SOX6Yw4d4rC1WgkpfAEGk93tMFBlO/ZA8wEzvUTpCYeFIlAEOoxb948q5zX3d29SSOAWpK1PlN71OGahuy01U1DBrBzI8Aoc7rMXAukoMdothTFYDKYJ7GoNBo6h4SReaRlZ4MKgiC0J1ZNBJIkjZEkKVmSpGOSJM2tZX+AJEnrJEnaJ0lSgiRJ11ozHjg3j6BMZwRJoovSnlP6IgCUQd3QSVoK9x+2HD/2oce5+cX2XytEEAThUlktEUiSpATmAWOBCOBWSZIurNfwHPCTLMs9MS8I8Im14qmmUirQKBXmRAAE2nhwFiNl+jIiJ/ehz963Me3bbjne0c0DpUpt7bAEQRDajDXvCPoCx2RZPiHLsg74EZh4wTEy4FT13Bm49HKATWCrUVKuMwAQ5BQIwIm8ZNSdvNCGhFB8QQfR5h8Xsvfv5a0RmiAIQquzZiLwBVLPe51Wte18LwF3SJKUBqwAHrJiPBbVpagBuntGA5Ccvg2A3NgJrGYclQXnZkxmnThGXnrqxScShA6sMWWok5KSiI2NrXXSWWPWEBBaR1t3Ft8KfCPLsh9wLbBIkqSLYpIk6V5JknZLkrQ7Ozu72Re11Sgp05sTga9fP+xNJpLOmMtLuMZ2xyX/KIU7zpWbmPz0y1x1z+xmX1cQriSNSQTLli3jxhtvZN++fbWWqRbaB2smgnTA/7zXflXbznc38BOALMvbABvgokpNsix/Lstyb1mWe3t6ejY7MHuNivKqOwKFVwTddXqSi8zfVoKv60OPkz9i2nOueai6ONblNgtbaH+WvruXw1vNq+EZjSaWvruX5B3mUWt6nZGl7+7l6O4sACrLDSx9dy/H95nrBJWX6Fj67l5OJuQAUFrYcFVNMK9H0LdvX2JjY7nvvvvYsWMH0dHRVFRUUFpaSmRkJAcPHqSkpIRRo0YRFxdHVFRUjdIMCxcuJDo6mpiYGKZOncrWrVtZvnw5TzzxBLGxsbWWmFixYgXvv/8+n376KSNGjADMawiEhoYyePBgkpOTLccOHz6cRx55hNjYWHr06MHOnTub+qsVmsGa8wh2ASGSJAVhTgC3ALddcMxpYBTwjSRJ4ZgTQfO/8jfAVqOktNLcR4Dalu4KW36vzMMkm1DY2GDXpw/5W/fgXXW8LMv89MrT+ISEMfS2adYOTxBaTFuuR3Dttdcya9YsHBwc+M9//lPvGgIAZWVlxMfHs3HjRmbMmMHBgwdb69fU4VktEciybJAk6UFgFaAEvpJl+ZAkSa8Au2VZXg48DiyQJOkxzB3H0+RW+Nptp1GSV3puWbkwez8W60+TVpxGgFMAZ0NHs0PpivfhU7iGd7HcEZw+EG/t0IQr3KTH4yzPlUpFjddqjbLGa62tqsZrWwdNjdf2ztoGr9eW6xFcqK41BKrdeuutgLmyaFFREQUFBbi4uFzStYSmserMYlmWV2DuBD5/2wvnPU8EBlkzhtrYaZSk5Z9b+ai7eyScOU3S2f0EOAUQMDySs2sWUL5nEK7hXQDwC49ix29LxPoEwmWlLdcjaKra1icQWkdbdxa3CVv1uT4CgG5+/VHKMklVI4c8+4QTUrEP0+5z/QT+EVHIson0pMtzsW6hY2rL9QguVNcaAtWWLFkCwObNm3F2dsbZ2bl5H15otA5Xawiqh48aLK+13jEE6fUk55oXoZEkCduBg0nfnoyP3oBSrcIntDtKlYrUxANicXvhsnH+egQmkwm1Ws3EiRMt6xEYjUYGDhxoWY9g/PjxREVF0bt371rXI1AqlfTs2ZNvvvmGW265hZkzZ/Lhhx/yyy+/NDgq6Pw1BLy8vC6q+29jY0PPnj3R6/V89dVXVvudCBfrcOsRAPz378N8syWF5NfGmjeYTMz9rDu7HF1YM9W8iH3CgpVs2qPh2gmOBF1r/ge75KW56CsrueO/7zXr+kLHIdYjaJzhw4fXWExeaB6xHkEj2KlVVBpMGE1VSVChIEzrzllTBfkV+QB0G9+HyKSv0R7abHmfX0QUZ08ep7KstC3CFgRBsIqOmQg051UgrdLd2Xxbm5xvHtts5+1OUJCK8nWrLceIfgJBqN3s2bOJjY2t8fj6668b/f7169eLu4E21CH7COy15o9dUmnA0cZcUK67dy84nkBy5m76+/QHwGbY1Rz8ZjXO+47g0TO0Rj9BcFzD65oKQkfRkWr3X4k65B2Bi535j39hud6yza1zL7wMBpLOnFvAXtNvEElhd3B0RTwAao2WATfdTkBkdKvGKwiCYE0d8o7AxdacCArKziUCOkUSptOTVHjCssk1vAtDc1/FobAMuBmAftff1JqhCoIgWF0HvSPQAFBQdm52MfYedJfVnNTlUWk8V8PFe3hPKuLjMeSY67vIskxu2mkKz2a1asyCIAjW0kETQS13BECYvS9G4EjeEcs222EjORY0gcQl5sllBr2ORU89TPw/f7VavIIgCNbUsRNBec1EEF21NkHC2XjLNruI7uR49yLrgHk9ArVGy/g5T9PzmnGtE6wgCFcESZIa/WhtHTIR2KqVaFQK8s9vGgK8fcwdxvsztlq2KRQKrul6DP+tCzCVmucPdO3VDydPr1aNWRCEy1dCQgKJiYnIstyoR2vrkIlAkiRcbNUUXtA0hE8MMZU69uccqLHZ+apRyDodxRs3AaCvrCBhzSqyTl5cg10QBOFCSUlJ7XqGeYdMBACudpqL7gjwDCNGZyJDV0h22bllEex69+JI9F2s/+NcB/Harz4lacuG1gpXEITLWHsv5dNhE4GznfqizmKUKmIdAwHYn73fsllSKnEM9EGRdgxjSQlqrQ3e3bqTlljzzkEQ2isHh0svnd6c97aFlJQUevTo0eBx5eXlDBs2DKPRXIl4/fr1TJ06tcXjyc7OxsvL3JScmprKiBEjiIiIIDIykg8++KDW98yaNYstW7bUug9Ap9MxdOhQDAZDncc0RYdNBC626hoTyqqFd+6DWpbZn7Wvxvb+t0TR7cgvlKxdC4B/ZBRZJ45TWVbWKvEKgtCyvvrqKyZPnoxSaS45s3//fnr27Nni19myZQuDBw8GQKVS8e6775KYmMj27duZN29eres+b9++nf79+9d5To1Gw6hRoyylu5urwyaCWpuGAI1vHyIqdezP3FFju21sLCofH7JWrAPOqzuUfKhV4hWElnD99dfTq1cvIiMj+fzzzwHzN+iwsDBuv/12wsPDufHGGymr4wtObe+Hi9c0hovXSq7+5l2t+rrTpk0jNDSU22+/ndWrVzNo0CBCQkJqrFtc17lquy6A0Whk5syZREZGMnr0aMrLyy/6LN9//z0TJ060vI6Pjyc9PZ1+/foRHBzM+vXrm/jbNSsoKOChhx6yvNbr9ajV5pGKPj4+xMWZV5lzdHQkPDyc9PSaS7kfPnyY0NBQlEol+/fvZ+jQoURERKBQKJAkiRdeMK/tdf311/P9999fUowXaWwvdnt59OrVS24Jb/yVKIc+u+LiHTnH5Lc+CJDjvo2RdQZdjV0bnv5WnjdzlVySkSPrKsrl926bKK9f9GWLxCNcmRITE9s6BFmWZdne3l6WZVnOzc2VZVmWy8rK5MjISDknJ0c+efKkDMibN2+WZVmWp0+fLr/99tsXvbeu9x88eFAOCQmRs7OzLcckJibK48aNk3U68/9D999/v/ztt9/WiOnkyZOyUqmUExISZKPRKMfFxcnTp0+XTSaTvGzZMnnixImyLMt1nqu2655/3n379smyLMs33XSTvGjRohrXrqyslDt16lRjW8+ePeUXX3xRlmVZXrVqlTx48OCLfo+DBw+WY2JiLnr8+++/NY57//335YSEBFmn08nLli27+D9IVZz+/v5yYWFhje3vvvuu/OWXX8rl5eVy9+7d5R07dsiyLMvPPfec/J///Ec2mUyyLMuywWCQPTw8aj13bf/uMC8RXOvf1Q5ZYgLMfQSVBhPlOiO2VdVIAXANIsaoZKFsJCkviSjPKMuubqOjqXh1PmUbi7GfcoPoJxCabMnLcxs8JjiuL33GT7YcHznsKnoMv4qyokL+eK/mkpNTXnyzSdf/8MMPWbp0KWBurz569Cje3t74+/szaJB51dg77riDDz/8kP/85z+Nev+uXbsuWtP4hx9+qHWt5AsFBQURFWX+fywyMpJRo0YhSRJRUVGkpKQAda+7XFhYWOdaykFBQcTGxgLQq1cvy7mq5eTk1FgPWa/Xk5OTwzPPPANAbGwsOVXVBM63adOmOn6zNY0fP54ff/yR3Nxcy+/1fCUlJdxwww28//77ODk51di3atUqvv76a1avXk1cXBx9+5oXwoqOjmblypWWeQZKpRKNRkNxcTGOjo6NiqsuHTYRuFaXmSjXYauxPbdDoSDGLQxIZX/2/hqJwGdYDGVvnKB81QqYcgP+ET3YsfRnKsvK0FYtyC0I7dX69etZvXo127Ztw87OjuHDh1vWJG7MesH1vf9Cch1rJV9Iq9VanisUCstrhUJh6Qit61wfffRRo86rVCovahqytbWtEXtSUhLdunVDozH/Xdi7dy8xMTEXnXfIkCG1Ls/5zjvvcNVVV1leBwcHc+zYMUJCQiyJqpper+eGG27g9ttvZ/LkyTX2lZWVUVBQQOfOnVm4cKElSVbHVN2sVK2yshIbG5s6fw+N1WETwfmF53ycbWvs6+TbF+9TJ9mftZc7Iu6wbJckCYcx13L0ty24ns7CLyKK7b8tIT35EME9RVlqoWFN/QZ//vF2Ts5Nfv/5CgsLcXV1xc7OjqSkJLZv327Zd/r0abZt28aAAQP44YcfLJ2bjXn/yJEjmTRpEnPmzMHd3Z28vDxGjRrFxIkTeeyxx/Dy8iIvL4/i4mK6dOnS5LjrOldt1z3/rqA+rq6uGI1GKioqsLGxIT4+npMnT1JZWYler+fll1/mvfcuXomwsXcEAJ06dbqor0WWZe6++27Cw8OZM2fORe9Zt24dI0aMAMDd3Z21VYNTjhw5wm+//cbWrecmu+bm5uLh4WHpf2iODttZXF14rrYOYzrHEVNRyf6zey/aZep3FQk9ZpH44xY6h4ahtrGlqGphcEFoz8aMGYPBYCA8PJy5c+fWGJXSvXt35s2bR3h4OPn5+dx///2Nfv/5axrHxMQwZ86cGmslR0dHc/XVV5OZmXlJcdd1rtqu2xSjR49m82bzCoT79+9n8uTJDBw4kL59+/Lwww/XO2qnMSZPnky/fv1qbNuyZQuLFi1i7dq1lgV8VqxYYdn/999/M2bMGABuvfVWSkpK6NGjB/feey+LFy/G3d3dcuy6deu47rrrmhWjRV2dB+310VKdxYkZhXKXp/6UVyRkXLyzIFVe+K6v3OObHnJWadZFu3fc/JB8bPJNsizLskGvu2i/IFRrL53F9Tl58qQcGRnZ1mG0uj179sh33HFHW4dRQ8+ePS2d4g2ZNGmSnJycXOu+pnYWd+A7gtoLzwHg5EucZG7z35O156Ld3a7rje7QASqPHkWpav5tmSAIrS8uLo4RI0ZcNKy1Le3du7dRTT06nY7rr7+e0NDQFrluh00ErvU1DUkS3b1icZRh55mdF+12GjeONL9h7P52CwVZZ1j8/BOk7L+4GUkQLgeBgYEcPHiwrcNoEzNmzLBMKLucaDQa7rzzzhY7X4dNBDZqJVqV4uLCc1VUvnH0KitnZ8b2i/e5uVHUbTBpJ8uxc3BEUkjtvpaIIAhCXTpsIgBz81CtdwQAvr3oW1HB6ZI0zpSeuWj3iAleRO/5EN2ePdzy8lsExfaycrSCIAjW0aETgaud5uLCc9X8etO33LxkZW3NQ65XDUfp4kLB0mUA6HWVGA11nEsQBKEd69CJwNlWXXtnMYCtKyGu3XBFyY4L6g4BSBoNxSOnsqpgICk7dzNv+hROH0ywcsSCIAgtr0MnAhc7dc0F7C+gCBhA7/JydmburLUPwOuqgTgWnYL9ScgypIpyE0ItRP+R0Jou5d9bh04E9TYNAQQMoF9pCWfKzpBWnHbRbr8RMfQ2bcG04je8u4WQdkgkAqEmGxsbcnNzRTIQWoUsy+Tm5ja57ESHLTEB5xankWW59gWjA/rTt6oeyY4zO/B38q+xW5IkXKfczOnX38V90AAO7tmErrwMja2oOySY+fn5kZaWRnZ2dsMHC0ILsLGxwc/Pr0nv6dCJwNVOg85oolxvxE5Ty6/CxZ9A2054Smp2Zu7kxtAbLzrEfsy1bF/tgMuxJGSTifSkRIJ69m6F6IXLgVqtJigoqK3DEIR6deimofMLz9VFCuhP3/JKdp6pvZ9A4+JEnOsJghL+RaFUknq4Y07MEQTh8tWxE0FVmYk65xKAuZ+gOJ/cilyOFxyv9ZAed47EMe80Hs6uop9AEITLTgdPBOYyE3XNLgYgoD/9y839BFsyal9M2qZHJLqogcj5Ws6cOIquXKxjLAjC5aODJ4J6Cs9V84rAR2VPN6U9m9Jrr0UuSRKVAyZQZNsb2WQiI/mwNcIVBEGwio6dCGzrKTxXTaEE/74MKdexJ2sPpfrSWg+LnjaCwYkL6ePmg2dgsDXCFQRBsIqOnQjsGu4sBiCgP0Ny0zCYDGzPvLgIHYCNmxPe48fguXkHGp0oNSEIwuWjQycCG7USG7Wi3tnFAAQMILaiEnuFls3pm+s8zPmWW9nX/Q6Wv/45uoryOo8TBEFoTzp0IoBGzC4G8O2NWmXDQLUrm9I21TlL1LZbMJKTkZQz20kXo4cEQbhMdPhEUG/huWpqGwjoz+CifLLKsjhacLTOQ8dO7ceQpNO4pme1cKSCIAjW0eETQUOF5yyChjE46wQAm9JqHz0E4DxsOG4+vqT8uLKlQhQEQbCqDp8IGtU0BBA0DC+jkTBb7zqHkQJICgVpgyazXunIiRVbWzBSQRAE6+jwicDFrhFNQwA+MaB1ZoisJf5sPEW6oroPHR6OUXeQ7D8WtWCkgiAI1tHhE4GzrYaCMl3DZYKVKggcxJDs0xhlI1vSa59lDBDUpxeSJJGRdABd2sXlqwVBENqTDp8IXO3U6I0yZTpjwwcHDSM65xRuGhfWnF5T52EaG1s6dQkmy8WdnR+tasFoBUEQWl6HTwSNKjxXLXgYSmCkQxc2pm2k0lhZ56EBsXGUakwcS5fQ5+a1ULSCIAgtr8MnAnd7LQC5JY1IBJ5hYO/FVWWVlBvK2Zaxrc5D/SOiAJmgpHkU/rSkhaIVBEFoeSIROJjrDeWW1v3t3kKSIGgofVP346h2ZPWp1XUe2rl7OAqlkpLwEHK/+wF9mZhpLAhC+9ThE4GHg/mOIKe4EXcEAEFDUZdkMdyzJ+vT1qM31T7iSGNjS6euIeS6urC524PEz/+npUIWBEFoUVZNBJIkjZEkKVmSpGOSJM2t45ibJUlKlCTpkCRJP1gzntpYEkFj7ggAgocDMAo7CisL2ZO1p85D/SOiyM3JxI0zmNb9iWwwNDdcQRCEFme1RCBJkhKYB4wFIoBbJUmKuOCYEOBpYJAsy5HAo9aKpy62GiX2GmXj7whcu4BHKAMzj2Crsq23eSigRwydgrrRZ6IfTkc2U7RiRQtFLQiC0HKseUfQFzgmy/IJWZZ1wI/AxAuOmQnMk2U5H0CW5bNWjKdO7g5ackoaeUcA0O1qbE9tY7BPf9aeXotJNtV6WJeoWG57/V18r5+EKjSc+O+2YDI0YpiqIAhCK7JmIvAFUs97nVa17XyhQKgkSVskSdouSdKY2k4kSdK9kiTtliRpd3Z2dosH6uGgaVxncbWQq8BYySitD9nl2SRkJ9R7uCzLlI+/l4Nu13B08dpmRisIgtCy2rqzWAWEAMOBW4EFkiS5XHiQLMufy7LcW5bl3p6eni0ehLuDtvFNQwBdBoHajmF5WWgUGv4++XedhyasXsm8u28h9JbB9Mv4DvVvnzY8i1kQBKEV1ZsIJElSSpK07hLPnQ74n/far2rb+dKA5bIs62VZPgkcwZwYWpWHg7ZpdwQqLQQNxeHEOob5DWVlykoMpto7gj0CuhA1cjQm2Ui3aeOpPHyY4vXrWyZwQRCEFlBvIpBl2QiYJElyvoRz7wJCJEkKkiRJA9wCLL/gmGWY7waQJMkDc1PRiUu4VrN4OGjIK9VhNDXhm3q3qyA/hWs94siryGNn5s5aD+scGs7wO2di5+SM8/hxnAkfx9+LTmMy1d6vIAiC0Noa0zRUAhyQJOlLSZI+rH409CZZlg3Ag8Aq4DDwkyzLhyRJekWSpAlVh60CciVJSgTWAU/Ispx7aR/l0nk4aDHJjSwzUS3kagCGFBfgqHbkr5N/1Xmo0WAg+3QKklqN04ihUJRH/r/rmxm1IAhCy2hMIvgNeB7YCOw579EgWZZXyLIcKstyV1mWX6/a9oIsy8urnsuyLM+RZTlCluUoWZZ/vLSP0TyWuQRNGTnkGgjuIWiPr+OqLlex5vQaKgwVtR669efv+W7uI+grKug56xp6Fa2k8LOPRV+BIAjtQoOJQJblb4HFnEsAP1Rtu2JYykw0pt7Q+UKuhpTNXOs/ilJ9KRvSNtR6mF94D0xGI+lHDqPQaPCc/QDFx9I49fO/zQ1dEASh2RpMBJIkDQeOYp4c9glwRJKkodYNq3Vd0h0BmPsJjJX0KS/Dw9aDFSdqnzDm2z3cvHJZonlBe6frriOh96NsXJmPySjmFQiC0LYa0zT0LjBaluVhsiwPBa4B3rNuWK3Lo+qOILu4iYkgcDBoHFAeWcWYwDFsSt9EYWXhRYdpbO3wDg4hNfEgAJJKRf8RrkTu+4SSVWK9AkEQ2lZjEoFaluXk6heyLB8B1NYLqfU526pRKSRyS5vYNKTSQrdRkLyC6wLHojfp+edU7cXl/CKjOHPsCPoKcz9CtzuuwdXPmewPPkTWN2KpTEEQBCtpTCLYI0nSF5IkDa96LAB2Wzuw1iRJEu4OGnKaekcAEDYOSrKIrKwk2DmYZceW1XqYf0QUJqOBjCNJ5msqFLg9/CgJ2oHs/kjUIBIEoe00JhHMAhKBh6seicD91gyqLZgnlTXxjgDMHcaSEil5BZNDJpOQncDxguMXHVbdT5Ba1U8A4DRqOJVeweRu3oWprKw54QuCIFyyBmcWA/tlWf4/WZYnVz3ek2X5Er46t29NLjxXzdYVAgdB0l9cF3wdKklV613BuX6Cc4lAoVAw/v5wuiT+St7CRc2IXhAE4dI1ZmZxsiRJAa0UT5vxcNA0ffhotbBxkJOMR2kBQ/2Gsvz48loXrLmwnwDAvlcvHEaM4NT3f1KS1vIF9QRBEBrSmKYhV+CQJElrJElaXv2wdmCtzcNBS3ZJ5aVN8uo+1vwz+S8mhUwiryKPjWkbLzos5qqxTH3zfVQaTY3tdjMfYkf4w2z/SKxiJghC61M14pjnrR5FO+DhoEFnMFFSacDRpomDolwCwDsaklYweMBsPGw9WHZ0GaMCRtU4zNmrU61vd48Lp7fTP9j8sxjd7CFoAq74GzBBENqRxvQRzJdlecOFj1aKr9W421dPKrvU5qHrIHUHqrJ8JnSdwKb0TWSXXdzUkxK/hx3Lfr5oe88nbkUj6Tn77v9d2vUFQRAukegjqOLheImzi6t1vxaQIXkF13e7HqNs5Pfjv1902OlDCexd8TvGC9YvVnfywvau+9iYGcLJv3ZcWgyCIAiXQPQRVPGw1Bu6xETgHQWuQZC4jCDnIPp69+Xn5J8xmmqWkOh/wy3c99m3KFUXt8p5T7sNg50rmT8sRRZlqgVBaCWNSQTPA+OAVzCXm6h+XFGq6w1lX2rTkCRB5PVwYgOU5TGl+xQySjPYlL6pxmEaG1sUCmWtp9A6OzB+oh2ue36n6K+6y1oLgiC0pMZUH90ApGAuNbEB84Ize60cV6tzs2/mHQFA5CSQjXD4D0YEjMDL1osfky+urL37j99Y+Unt5ZpcJk5AGxnJoc+WU5lXdOmxCIIgNFJjqo/OBH4B5ldt8sW8stgVRa1U4GKnvvQ+AjCPHKpqHlIr1NwYeiNb0reQWpRa47CyokKStmxAX3nx+gWSQoH2/rns87+Vne9dcS1wgiC0Q41pGpoNDAKKAGRZPgp4WTOotuLhoL30SWVQ1Tw0ydw8VJrLDaE3oJSULEleUuMw/4gojAYDmUeTaz2N71W9GWC3B/ff30F3+vSlxyMIgtAIjUkElbIsW/46SpKkAq7IpbXc7TXNuyMAcz+BbISkP/Gy82JkwEiWHltaY/Wyzt0jLqo7dKGop+5CqVKS+cb/mhePIAhCAxqTCDZIkvQMYCtJ0tXAz8Af1g2rbXg4NvOOAMzNQ27BcGgpALeG3UqRroi/T/5tOURrZ0en4G6kHqo7Eag7eaGd8RBrywaT9OP65sUkCIJQj8YkgrlANnAAuA9YATxnzaDaioe9huzm3hFIEkRcDyc3QmkuvTv1pptLNxYdXlSjfIV/RBRnjiXX2k9QzW/6LdhRStHi7zBVXnF1/gRBaCcaM2rIJMvyAlmWb5Jl+caq51dk05CHg5biCgMV+mYuH2kZPbQcSZK4M+JOjuYfZXvmdsshDfUTAKjsbbjuvnAck7eQu+CL5sUkCIJQh8bcEXQY7lVzCfIuZV2C83lHgXsIHDCXkrgu+Drcbdz5NvFbyyGWfoJDCfWeymHQIBzGXsu+P5LI3ne0eXEJgiDUQiSC83hWlZk4eykrlZ1PkiB6CpzaAgWpaJQabg27lS3pWziWfwyo6icI6mpZx7g+TrPncCJgLPGfrLi06qiCIAj1EIngPJ2cqhJBUd3t9o0WdaP5Z9VdwZTuU7BR2rDo8LkFaMKHjMQnpHuDp3Lu5su1fQrw3fQZxStXNj82QRCE89RZhlqSpD+oZ5ioLMsTrBJRG+rkZANAVnPvCADcgsC/PyQsgcGP4WLjwsRuE1l6dCkP9XwID1sP4saOb/Tp/GfcTMrKXzn9v/cJjuuHXSe35scoCIJA/XcE71CzttCFjyuOu70GhdRCdwQA0TdDdhKcMQ8TvSP8DvQmPYuTFlsOMRmNlOTlNngqSaXC/bmX2BZ8P+v/Kxa7FwSh5dSZCGpbg+BKXo8AQKVU4OGgJaulEkHkJFCo4MBPAAQ6BzIqYBSLkxZToisB4OdXn+WvD99u1Omc4qKI9MnHc+NXlO7c2TIxCoLQ4dWZCCRJOiBJUkJdj9YMsjV1crIhq6iFxuzbuUHIaDjwC1SVo74n+h6KdcWWshNx100k7rqJjT5l/2en4Oqm5MwLL2KsaKGEJQhCh1Zf09A4YHw9jytSJ6cWvCMAc/NQcaZ5ghkQ6R7JoM6DWJi4kApDBSF9BhDSZ0CjT6ews6PTSy9yUNWLDa8ubbk4BUHosOprGjpV36M1g2xNXk42zR8+er7QMaB1hv3nylHPjJ5JXkUevx79FYCsk8dJTz7c6FM6Dh6MIrg7JQmJlB1sePipIAhCfcTw0Qt4O9mQV6qj0tDM2cXV1LYQdQMk/g4VhQD06tSLOK84vj74NXqjnn8//5jNi79t4EQ1jX15POEFGzjz7HPIumZOgBMEoUMTieAC1XMJslvyrqDnHWAoh4O/WTbNjJ5JVlkWf5z4A//IKDKPJqHXNf6aKlcXvF9+mbzT+ex6c0nDbxAEQahDoxKBJEm2kiQ1PPPpCuBVPZegpTqMATrHgWc47PvOsmlQ50FEukfyecLn+ISFm+sOHam77lBtHEeOIHPw3exPcaR4/6GWi1cQhA6lMSuUjQfigZVVr2OvxMXrq3VyNCeCFptLAOaSEz3vgPTdcPZw1SaJB2IfIL0knb3q40iSgtTEpg/GGvX0tfQ/OZ/sF57FJJqIBEG4BI25I3gJ6AsUAMiyHA8EWS2iNlbdNNSiI4fAXHtIoapxVzDEdwjRntF8ceQbPIOCSGtE3aEL2XX2IPD5J6lITubYO6JCqSAITdeYRKCXZbnwgm1XbOUzVzsNaqXUMmUmzufgaR5BtP9HMOoB813Bg7EPklWWRYm3usn9BNUcR44gf9xD/HsqlNOrxEQzQRCapjGJ4JAkSbcBSkmSQiRJ+gjYauW42oxCIeHlaENWoRUma/WcCmU5cORc4bj+Pv3p1akX64m/pH6CanFz7yAsZw2V7zyPsaS0pSIWBKEDaEwieAiIBCqBxZgXsX/UijG1OS8nLVnFVkgE3a4CB2/Yc26oaPVdwVH7syBB2uG6l6+sj62HEwOevgFDWiqZb77VUhELgtABNGaFsjJZlp8FRgEjZFl+VpblK7q2QSfHFiwzcT6lCuLuhGOrIT/Fsrm3d2/6dBlAvrOB1COXPvrHrndvbO66n39SIzj07eoWCFgQhI6gMaOG+kiSdABIAA5IkrRfkqRe1g+t7bR4mYnz9brLPIpoT80JZI/1eox/47JIG9288tL+D8/ESSqi9NvP0WdlNetcgiB0DI1pGvoSeECW5UBZlgOB2cDXVo2qjXk52VBcYaBMZ2j5kzv7mTuN9y0Cw7nhnmFuYYyKGMsPSYvJLMm85NOr7GwY/8JVOOUfI+OpucgmU0tELQjCFawxicAoy/Km6heyLG8GrPAXsv2oXqDmrDWahwB6z4DSbEj6s8bm2bGziTvkyILvXm7W6bXBQXg9/TRJ6fZsfeOXZp1LEIQrX2MSwQZJkuZLkjRckqRhkiR9AqyXJClOkqQ4awfYFqw2l6Ba11HgEgC7v6qx2dfRlzC9LydPJZKcd2mjh6q53HgDpd36k7HvFGXx8c06lyAIV7bGJIIYIBR4EfPksnCgJ+ZVyt6xWmRtyLsll6ysjUIBvaZDyibIPlJj192vz+NItJF3d7/brIXqFQoF1716HbH5K8mY8zjGwgunggiCIJg1JhFcJcvyiDoeI60eYRvwcrJCmYkL9ZwKCjXs/rLGZhcbF+6PuZ9tGdvYkNa8heC0Hq74/d+7lOaXseHphZhEf4EgCLVoTCI4KknS25IkhVs9mnbCyUaFjVphvaYhMM80jpwE+76HymLLZqPBgLxwF8PTAnl719vojM2rH2QbG0vZTXNI1odw6oufmxu1IAhXoMY2DR0BvpQkabskSfdKkuRk5bjalCRJLbtkZV36zQJdMcT/YNmkVKkAiCrz53Txab4//H3zL/PkZIYr/qHyo9cpP3BpE9YEQbhy1bdmsQpAluViWZYXyLI8EHgKc19BpiRJ30qS1K2V4mx15kllVp4359cL/PrAjvlwXrONf0QUZaczGe49hPkJ88kpz2nWZRQqJSFvPo/Sw4OEp9+nNKN55xME4cpS3x3BTgBJkpSSJE2QJGkZ8D7mTuJg4A9ghbUDbCteTtqWXbKyLv1mQd5x82zjKv6RURj1eu50nUSlsZL397zf7MuoXF1x++//sdfnJja8/IuYXyAIgkWj+giAicD/ZFnuKcvy/8mynCXL8i9UrVFwJTI3DVU0a+ROo0RMBEcf2PGpZZNvWCRIEvpT2dwZcSe/H/+dfWf3NftS7v1jGRZTTMCmT8j9/PNmn08QhCtDfYnAS5KkOcBXQCIwQJKkOdUPAFmWH67v5JIkjZEkKVmSpGOSJM2t57gbJEmSJUnqfSkfwho6OWkp0xkpqbTy3DmlGvrcDcfXQrZ57oCNvQNegcGkJR7gvuj78LH34ZVtr6A36Zt9ufDZN+J67TVkffQJqX9savgNgiBc8epLBErA4byfjhc86iVJkhKYB4wFIoBbJUmKqOU4R+ARYEdTg7emTtZYsrIuvaaDUgvbz90V+EdEkXE0CY2sYm7fuRwrOMYPh3+o5ySNI0kSPq+8zLG4e/j790KKj6Q0+5yCIFze6ksEmbIsvyLL8su1PRpx7r7AMVmWT8iyrAN+xNzEdKFXgf8B7aqiaXUiyCwst/7F7D0gZgrsXwwl2cC5foLMY8mM8B/BML9hzIufx5nSM82+nMLOjkH/uY6w08vIfvIxTOWt8BkFQWi36ksEUjPP7Quknvc6rWrbuQuYS1T4y7L8V30nqhqyuluSpN3Z2dnNDKtxOjvbApBZ0Er5acBDYKiAXQuAc/0EqYcOIEkSc/vORZZl3tz5ZotcziOmK3HP30VlcjInnn1dTDYThA6svkQwypoXliRJAfwf8HhDx8qy/Lksy71lWe7t6elpzbAsOjmb6w1ltMYdAYBnKHS/FnYuAF0ZNvYODLrpdvzCewDg5+jHrJhZrDm9hn9P/dsil3QYMgSbWU+wumgA2/8ritMJQkdVZyKQZTmvmedOB/zPe+1Xta2aI9ADcwG7FKA/sLy9dBhrVUo8HbWtd0cAMPBhKM+DePMksv433EJAj2jL7rsi7yLcLZw3drxBYWXL1A7q8tBddLNJxfbnDyhev75FzikIwuWlMcNHL9UuIESSpCBJkjTALcDy6p2yLBfKsuxx3joH24EJsizvtmJMTdLZ2ab17ggAAvqbJ5ht+xhMRmSTibMpJyjKMTeHqRQqXh74MvkV+by7+90WuaRCoWDk23fhEuxN+uP/ofBA86qeCoJw+bFaIpBl2QA8CKwCDgM/ybJ8SJKkVyRJmmCt67YkH2dbMgpaMRFIkvmuID8FDi+nsqyMRXMf4dD6c5PNwt3DmRY5jaXHlrItY1uLXFZha4vfvI85HjieX98/SGlG6/TDCILQPljzjgBZllfIshwqy3JXWZZfr9r2gizLy2s5dnh7uhsA8HGxIbOwFSaVnS/sOnDrCpvfw8benon/eY6okaNrHDIrZhaBToG8tPUlSvWlLXJZtY8PkfdcS6czu8h+cg4mXfOK3QmCcPmwaiK43Pm62FKmM1JU3ooLsimUMPgxyNwPx9bQrXc/HNzcaxxio7Lh1UGvcqbsDG/vervFLh04tg+DZw2mfPduUp9/RYwkEoQOQiSCevhUDSFNb83mIYDoKeDkBxvforK0lPhVf5GTeqrGIbFesUyLnMavR39lY9rGFru08/hxONz3MGsye7DlVTGSSBA6ApEI6uHj0oqTys6n0sDgRyF1B3LKVtZ8/RlHd2696LDZsbPp5tKNl7a+1GKjiAB8H7kPb6cyVH99S9GKK7auoCAIVUQiqEf1pLKMwjaY9NzzDnDohM2ej/HqYq47dCGNUsMbg98gvyKf17a/1mJ9GQqFgjHv3o5PiCsZT82lYOuuFjmvIAjtk0gE9fB01KJSSGS2dtMQgNoWBjwIJ9bjH+BJRnISBv3FRefC3cO5P/Z+Vqas5M8Tf7bY5RVaLf7zPiYndBQ/fZnJ2d1iWKkgXKlEIqiHUmFeqaxVh5Cer/cMsHXFr2IvBr2OM8dq/2N8d4+7ifOK4/Udr5NalFrrMZdC6eJC2EsP4VFylLy5j2JopfIegiC0LpEIGtDZxaZtmoYAtA4w8CH8CjaABKm1NA8BKBVK3hzyJgpJwdxNc1ukXHU1j6hgxj43GnKzSLn3fspzWq4vQhCE9kEkggb4ONu2fmfx+frei42jM16OUq39BNV8HHx4YcALJOQk8Gn8p3Uedylso6Lw/eB9dquH8fvTf2KsaIXS3IIgtBqRCBrQ2cWWM4UVmEytOKnsfFpHGPQIfqo0MpISa+0nqDYmcAyTuk3iiwNftNis42qOQ4fSfXgwnkdXk/n0XGSjsUXPLwhC2xGJoAGdXWzQG2VyStrwW3Cfmfi7gsFgqLOfoNrcvnMJdg5m7qa5ZJe1bJt+zweuI3bqEIr/XsmJl98SE84E4QohEkEDfNpyCGk1jR2+V9+FWmGgKLH+5SXt1Ha8O/xdyg3lPLnxSQymlp0V7X73DGymPcDq9Cg2v7i4Rc8tCELbEImgAT7OVZPK2mrkUBXbwffxYK8UIgp/hwbmC3R16cpz/Z9jd9ZuPon/pMVj6fLkbLo5n8Xhj0/I/errFj+/IAitSySCBvi6tFGZiQupbVEMewJOb4Mjqxo8fELXCUwOmcyCAwvYkLqhRUOpLl3tNbwvWW+9xfEvf23R8wuC0LpEImiAi50aG7WCzLZsGqqS4zWc7073I/2318HUcGft032fJtwtnKc3Pc2polMNHt8UklKJ71v/I3/oVFbudOboNy03mU0QhNYlEkEDJEmic1sPIa1i7+6F2iMAU94pSPipweNtVDa8P+J9lAolj657lDJ9WYvGI2k09HnnESL0u9G/NZeilQ3fqQiC0P6IRNAInV1syWjNJSvrYOvgyJQ3P8e/WxCsewMMDY9k6uzQmbeGvsWJwhM8v+X5Fl9bQeNkz7CPHsIuJpqUuS9wZPHaFj2/IAjWJxJBI/g4t2GZiQspFOiGPIOp4DTs+rJRbxnQeQCPxj3KP6f+4fOEz1s+JHt7/D+fz4nYu1i3ppy8f9e3+DUEQbAekQgawcfFluySSnSGth83n3oogY9fnUeG63DY+BaU5zfqfdMipzEueBwfx3/MmlNrWjwupaMjo/87hd7Ff3P28Uco2bS5xa8hCIJ1iETQCJ2dbZBlyCpq++Yhjy5ByLJMmutIKC+Aje806n2SJPHSwJeI9ojm6c1Pk5SX1OKx2fm4Ezv/NTRduxL/7Mck/1T/nAdBENoHkQgaoXPVENL20Dxk6+CIZ0AgqafPQtxU2DEfco836r1apZYPRn6Ak8aJh9Y+1OIzj8FcsdT/iy9ICZnArmXJFG9o2aGrgiC0PJEIGsHXtZ3MJajiHxFFxpEkjEPmglIDq19s9Hs9bD34aORHFFYWMnvN7BYfSQSgdndj0iuj6Vu5lvQHH6J43boWv4YgCC1HJIJGqJ5UlpbfPhKBX2QUBl0lZ7IKYchjcPgPSNnS6PeHu4fz9tC3Sc5P5smNT2JsxJyEprL39aDrV5+iCQvn3w+3cfCbf1v8GoIgtAyRCBrBRq3Ey1FLal7Lf3u+FH7hPUCSzOsTDHjQvND9qqcbNcms2jD/YTzd92k2pG3gzZ1vtviwUgClszM+n8yn3D2I1MV/UfiHmHQmCO2RSASN5Odq227uCCz9BIcSzEtaXv0yZO6HvQubdJ5bwm7hroi7+DH5R746+JV1YvV04eZ3xxHRKZ+MJ58k+8dfrHIdQRAunUgEjeTnakdaQfu4IwDwi+hh7icw6KHHDdBlMKx5BcrymnSeOb3nMDZoLO/vfZ9lx5ZZJVa1syP+n89HMeRqfl8psfMtkQwEoT0RiaCR/N1sySyowGBs+7kEYO4wNugqOXPsKEgSjP0fVBTAutebdB6FpOD1Qa/T36c/L219iY1pG60Sr8LGhuD3/kcnuyKkJZ+S/eFHVmmOEgSh6UQiaCQ/VzsMJpkz7WAuAYB/RDRjHngM186+5g3ePaDPTNj9FWQmNOlcaqWa90e8T3e37jy+/nH2ZO2xQsSgtrdh3IdT8R3Tn5xPPuHA8x9jMoiVzgShrYlE0Eh+ru1r5JCNgwORw0Zh5+R8buOIp8HWFVb8B5q4epi92p5PRn2Ct703D655kEO5h1o4YjNJpcLntdfgtgfZlBPJljnzkHU6q1xLEITGEYmgkfxd7YD2kwgAivNy2P/vCnM/AZiTwNWvQOoO2Leoyedzt3VnwegFOGmcmPXvLI4XNG6iWlNJkkT35+6nf2AWbv8uIHXW/RhLSq1yLUEQGiYSQSP5uNggSZCW3346jM8cO8LqLz4h68Sxcxtjb4eAgfDvC1DS9JnD3vbeLBi9AJVCxcx/ZnK66HQLRnyOQqGg19xb8X3tZYp27WXF7C8pPnXGKtcSBKF+IhE0klalpJOjDal57eeOoEt0T+7+YAE+IWHnNkoSjHsPdKXwz3OXdN4ApwAWXL0Ag8nAjFUzSC1KbaGIL+Zyw2TsXniXdE03Dj78MrqUFKtdSxCE2olE0AT+brbt6o5AY2OLi7cPkiTV3OEVBoMehoQf4cSl1frp5tqNBaMXUGGsYMY/M0grTmuBiGsXfNNIbp7hjfvZ/aTccitFu/ZZ7VqCIFxMJIIm8HO1a1d9BAAZRw6z8pP3z/UTVBv6BLgGwl9zQH9pI526u3VnwdULKNOXcfequ0kvSW9+wHVwGxBH4OIfKPYI4cfPUkn+TpSkEITWIhJBE/i5mpes1LeTuQQApfn5HNqwmjPHj9XcobY1NxHlHoMNb17y+cPdw/l89OeU6EuYtnKa1foMADSBgYR+8jbuchYVbz1H7ldfi7kGgtAKRCJoAn9XO0wynGkHC9lX8w2PBCAt8cDFO7uOhNg7YMuHkBF/ydeIdI/ky2u+pMJQwfSV0zlReOKSz9UQx4BOTPpkKu4jBpD11lvseOozjJX6ht8oCMIlE4mgCarnEqS2o34COydnPAICzQXoanPNa2DvAcsfBOOl/0ENcwvjq2u+wiAbmLFyBkfyj1zyuRqisLHB973/w3j7Y+wp6s6O2W9iLCqy2vUEoaMTiaAJ/NrhXAIwl5tIT07EaDBcvNPWFa57F84cgC0fNOs6Ia4hfD3ma5SSkukrp7M/e3+zzlcfSaEg6vl7uapXMS47fiZlyi1UihFFgmAVIhE0gY+LDQoJ0tpJOepq/hFRGCoryTpxtPYDwsdDxPWw4X+QldisawU7B/Pt2G9x1joz85+ZbMvY1qzzNaT7zIkEfvUlZcV6fnxhM8eXNX7dBUEQGkckgiZQKxX4OLefctTVqvsJUg/V0TwE5rsCrRMsvQ8MzSvp4Ofox7djvsXP0Y/Za2bzT8o/zTpfQ+z69KHzp/NRqhTkvvEKeYu+E53IgtCCRCJoIt92tC5BtQb7CcDcTzD+AziTABvfbvY1Pe08+fqar4l0j+Q/G/7D4qTFzT5nfdyjgrnlg/H49OpK1uuvs+/J99CXtp9Oe0G4nIlE0ET+rnbtalJZtXr7CaqFj4OYW2HTu5De/AqjzlpnFoxewDD/Ybyx4w0+3PuhVb+pqxwd8Zv3MZq7H2ZbUTTrH/gAfVaW1a4nCB2FSARN5OdqS2ZRBTpD+5lLABAQFUunoG6UFRXUf+CYN8HRG367DypLmn1dG5UN7w1/jxtCbmDBgQU8u/lZ9M0YndQQSaGg6xP3M3Kgic6HlnLyhhsp3bXLatcThI5AJIIm8nO1RZYhs7B9NQ91692PW17+H45uHvUfaOsC138Kecfh99nQAt/gVQoVLw54kdmxs/njxB/ct/o+CisLm33e+oTfNZpuixch2Tuy7P/2suOtX0W/gSBcIpEImsjfzTyEtD0VnzvfRaUmahM8DEa9CInLYOuHLXJdSZKYFTOL/w75L/Fn45n691RSi61XrA5AGxKC33c/YONsT8Ufv5Lx+OOYSkU5a0FoKpEImqg6EZxuZ0NIAXb/8Ruf3ntH/f0E1QY9AhETYfVLcGJ9i8UwLngcn1/9OXkVedz2123sOmPdZhtbTxcmzbuVyDtHULRyFTunPsmZ7Yetek1BuNKIRNBEPk42aFQKTuW1v2+enYK7EX3VWAy6yoYPliSYOA88QuGnuyDx9xaLo7d3b3649gdcbVy59597+eWIdRerVyiVeMycid8XX3LYeRhrP9hI4fLlVr2mIFxJRCJoIoVCwt/VllM57e+OwD8ymqG3TUNrZ9+4N2gd4dYfzVVKf7oTfrkbyvJaJJYApwC+u/Y7+vn04+VtL/PGjjfQm6xbM8hxYH8mPTOIOPU+Mp58itRnX6CysPkd4oJwpROJ4BJ0cbfnVDtsGgIw6HRknzrZ+De4BcE9q2HEc+a7gnn9ILFlvk07aZz4eNTH3BlxJ4uTFjPzn5nklue2yLnr4hLiS/evPsZ91n3sPqBmyZy/KD1svbpIgnAlEIngEgS42XE6t7RdjlLZ8tN3fPf0Y5QW5Df+TUo1DHsC7l1nHlr601RYMhWKmz9GX6VQ8USfJ/jvkP9yMOcgU/6cwsGcg80+b30klQqvRx8l4o7hdMrZR+qtU8j/6ad2+d9LENoDkQguQaC7HaU6I7mlzSvVYA3Ro67BZDSw/9+/m/5m7yiYudY8oujIKpjXF/Z91yJDTMcFj2PR2EUoJSV3/n0nPyVb/w9z95uHMOqLx7GLi+PY/+azfPYiyjKte0ciCJcjqyYCSZLGSJKULEnSMUmS5tayf44kSYmSJCVIkrRGkqQu1oynpXRxN7fBn8ptfx3Grj6+BPXsTcLqvxs3lPRCSjUMmQOzNoNXhHmuwbfjIedYw+9tQLh7OEvGLaGvT19e3f4qz2x+hjK9dZvYVJ6e+H+xAPnGmWSX2XNy6l2U7txp1WsKwuXGaolAkiQlMA8YC0QAt0qSFHHBYfuA3rIsRwO/AG9ZK56WFOBuHkJ6Krd99hP0HDOe0oJ8jmxvRqVOz1CY9pe5PlFmAnw6ENb995KXvazmYuPCJ6M+4YHYB/jrxF/c9tdtHMtvfpKpj6RQ0PfpW5jycAg2SgOn7prGzpe+wVAmahUJAlj3jqAvcEyW5ROyLOuAH4GJ5x8gy/I6WZar/5puB/ysGE+L8XO1RZLabyIIjO6Jq48v+/7+o3knUiig1zR4cJe5TtGGN+HTAXBsTfNOKym4P+Z+5l89n/zKfG7961aWHl1q9aYix7hogn77Ff346ew6E8C2e16h8vhxq15TEC4H1kwEvsD5U0vTqrbV5W7gEhq2W59WpaSzs227bBoC8zfg2GvGkXksmcxjyc0/oWMnuPErmLoUkOC7yebO5ILmzRwe0HkAv4z/hRjPGF7Y+gJzN82lWFfc/HjrobC3J+atJxg91IjXyQ2cnHwDqQt+wGQwWvW6gtCetYvOYkmS7gB6A7XWR5Yk6V5JknZLkrQ7Ozu7dYOrQxd3u3Y7hBQgctgoNLa2bP/1R/LPZLTMSbuOhPu3moeaHv0XPu5jLmndjOYiTztP5l89nwdjH2RVyipu+uMm4s/Gt0y89Qi57Wq6Ll+GdsAQVm3R8tesBejS0q1+XUFoj6yZCNIB//Ne+1Vtq0GSpKuAZ4EJsizXOiVWluXPZVnuLctyb09PT6sE21Rd3O043U6bhgC0dnbEXTuRE3t3sWHRl5btB9b+Q15G2qWfWG1jHmr64E4IuRrWvmYeXXT4j0seXaRUKLkv5j6+GfMNANNWTuPT+E8xmBpRKqMZVJ6edPnkA2J7qul0ZBUnJ0wg98efMJnaV2VZQbA2ayaCXUCIJElBkiRpgFuAGjOVJEnqCczHnATOWjGWFhfgZk9uqY7iCuvOlm2OgTfdzvT3PmPQlKkAlBUV8s/8Dzm2azsA5cVFbP35B04f3I++oonf6l0CYMoimLoM1Haw5A5YOAHOXPocgVivWH4e/zNjgsbwyf5PuPPvO0kpTLnk8zWGQqGg9yMT6Ln4Y2yio9nx9Q5+mfUD5aebkSwF4TIjWbODTpKka4H3ASXwlSzLr0uS9AqwW5bl5ZIkrQaigMyqt5yWZXlCfefs3bu3vHv3bqvF3Fh/H8jk/u/38udDg+nh69zW4TRacV4OSpUaOydnTh2I55fXnwdZRqFU4hXUFd+wSPzCIuncPRw7p0Z+LqMB9nwN616H8gKIm2puPnLsdMlxrkxZyavbXkVn1DGn9xymdJ+CQrJuS6ZsMrH7/eWkb04kPOVXvJ58Apebb0ZStIsWVEFoFkmS9siy3LvWfZfbbMv2kggOZRRy3Yeb+eT2OK6N8mnrcC5ZRWkJmUeSSEs6RHpSImeOH8GoN9/luPn64xsWQf/JU3Dy8Gr4ZGV5sPEd2Pk5KDXmCqcDHwRNI2sfXeBs2Vle2PoCW9K30Ne7Ly8PfBk/R+sPLNOlpXPmhefJ3XOYY31mMeLBgXjEhlj9uoJgTSIRWEFJpYEeL67iqTFh3D+8a1uH02IMOh1nThwl/fAh0pMOkXE0ien/9xn2Lq4cWPsPpw7EM3b2HJQqVd0nyT0Oq1809xs4dIIRz0DsHaCs5z11kGWZ347+xtu738Ykm5jTaw43d7/Z+ncHskzS57+zZYdMr4Mf4T/zVtynTUNSq616XUGwlvoSQdP/zxQAcNCqcLfXtNshpJdKpdHgV9U8BObmkuqmkcrSEkrz8yxJYOUn71NeXEjn7hH4hUXSqWsIKrUa3LvClO/g9A7493n44xHY+jGMeh7CJ5hLYDeSJEncEHoDAzsP5MWtL/L6jtf5++TfvDTwJYKcg1r+F3DedcPvu56g8WfI+e86st/9P/auP0v3u66hy+ha/18ShMuWuCNohsmfbEGrUrL43v5tHUqb2Pj91xzfvcMyCkmpVuPdNRS/8Eh8u0fQuXs4Wls7SPoL1rwCOcnQOc6cEIJHNCkhgPlb+u/Hf+etXW9Raajkvpj7mB45HbXS+t/Sc1esZumvxXhn7qB3f3s8H3sUpaOj1a8rCC1FNA1ZyWNL4tl5Mo8tc0e2dShtqqyokPTkRHNzUnIiZ08ex2Q0IkkKYkaPZdSM+8FkpGzbN9jt+D8oSoMug80JIaDpSTSnPIc3drzBv6f+patzV14Y8AJxneKs8MlqKs8pJO/TTyj6YRGlfj1Q3zSDqHtGoxCdycJlQCQCK3nv3yN8uPYoSa+OQatStnU47Ya+ooKMo0mkJyXi1tmXsEHDqCgpYd49tzLijmnEeeagX/cuRYVFuIUPQBr5LPg1vbllQ+oGXt/xOpmlmdwQcgOPxj2Ki41Ly3+gC5QfOMg/72zgrKIzo6S/8X3uabTB1mumEoSWIBKBlfy2N405P+1n9ZxhdPNyaOtw2rWK0hIOrvuXLtE98QwI5MTOzSx9901sVQZ8bQvwC/DB9+rpePW9DoWy8Um1TF/Gp/s/ZVHiIhw0Djwa9yiTQyZbvTPZqDeQ+s0vVM5/F2NlJYXXP07cY5PQujlZ9bqCcKlEIrCSPafyueHTrXw1rTcjwy59zHxHVJKfx4m9O0k/lED6gZ0UFpkntKmV4BMchG/MAPzCI+ncPcLcAd2Ao/lHeX3H6+zJ2kMP9x483e9poj2jrf0xMOTkcPjNL9hY0pfIjN+Jm3UNjmPHIjWx/0MQrE0kAivJK9UR9+q/PD8ugrsHi6aB5ijOPEX635+Svncj6UUasivNd1j3zvsaRw9P0g4fpLykmG69+9f5R1aWZf488Sfv7XmP7PJsJnSdwGO9HsPD1sPq8Z9etRP9Z2+iO3yYsn7j8L57Kr5DrZ+IBKGxRCKwElmWiX3lX8bH+PDa9VFtHc6VQVcKu7+iYuM8snLL6RLaDYY8zp8rD5BxNIl7530NQMKalajUGnzDInHy9KqRHEr1pXye8DkLExeiUWi4J+oepkZMxUZlY9XQZaOR/J9/5c8VlWA0clXwSbwefRiVu7tVrysIjSESgRVdP28LdholP8zsmENIrUZfAfHfw5b3oeA0BvcwiiKm4zZsBqg0fD3nfvLSzWWwHdw98K2ay+AbFoGHfxckhYJTRad4d/e7rEtdh4+9D4/GPcqYoDFW7z8oy8wl7fNvMf78NbKNPYWT59Dr4fFoHGytel1BqI9IBFY0Z0k8207ksu3pUW0dypXJaIBDS2Hze3D2EDj5Qr9ZmHreSW52vrk0RtWw1ZI883rEWnt7OoeGEzF0JGEDh7Izcydv736bpLwkIt0jebz34/Tx7mP10CtPnGTfWz+whwH0SfuO7vffgNO114r+A6FNiERgRR+vPco7/xwh8ZVrsNOIidpWI8twbDVs+QBSNoHWCXrdBX3vAxd/ZFmmKDuLtKqkkH74EGGDhjHgxlupLCtj6VuvwIAAvixeypnSMwz1G8rDPR+mu1t3q4d++o/NGL58l8qkJHJ730Dnm68jeMIAq19XEM4nEoEV/ZWQyewf9rLi4SFEdBZDB1tFxj7Y+hEcWmZ+HTERBsy+aC5CdXmMvIx0/v74HYbcNg2vsO58s+I90v/cQKZLOZ27h3PH1bMJDbBuH49sNJL/+x8s/6MS26J0BnoexXPOY9iEhlr1uoJQTSQCK0rMKOLaDzfx8W09GRfdua3D6VgKUs2VTvd8C5WF4Nsb+t9vTgz1lJ1IPZTA5l++J+PoYdCbF6ExOWvpGhlHtx698QuPxNXH1ypNOLqiUs5+t4TSrz+jvFIibdhsBt0/FNeIwBa/liCcTyQCKyrTGYh4YRWPXx3KQ6NEqeI2UVkC+xfD9k8h7zg4eEPvGdBrWr1rIhgNBo4k7WH5xkVkJh3GM1+Djc48mc3WyZlp736CnZMz5cVFaO3smzTRrSHGggL2/99P7Mz0o0/8O/hOHIXHrPtQeVh/qKvQMYnqo1Zkp1HR2dmGkzlXVhXSy4rWAfrOhN53m/sRds6H9W+Y11OOmAB97oGAARcVuVOqVIT36Ed4j35klmTyecJ8lu/7C698Lb2VnShUlGGHM2u/nk/WiWPMeH8+ALlpqTh5eqLWXvpwVKWLC3Gv3Ev3lHSKvhxM/uLF7NhtwCmiG0OeuBaVq2uzfiWC0BTijqAF3P7FdkorjSybPaitQxGq5R6HnQsg/gdzs5FnOPS5G6JvBpu6V15LL0nniwNfsOzYMgAmdp3IWFNf7PQqokaMBmDBgzMoycu1rOjmGxaBb/eIxq/oVovKkydZ9X+b4UQS3c+swvXOqTjddic2niIhCC1DNA1Z2XPLDrA8PoP9L44WQwPbG10pHPwVdn0BmfvN6yv3uAF6TzeXxK7jv1dGSQZfHfyKpUeXYpANjA0ay4weMwhxCSElfk/Vim6HOHPsCEaDATCv6FY9l8E/MhpH96Y381QcOULOvE/I3HyAvT0fY1DXs4Q/cANK58tnOVShfRKJwMq+3HySV/9MZM9zV+HuoG3rcITayDJk7IXdX5sTg74MOkVB3J0QfRPY1v7NO7ssm4WJC1mSvIRyQzlDfIcwo8cMenXqhSRJ51Z0S0o0r+iWfJjKslL6TbqZwbfcib6ygkPr19C1Tz8c3RqfGLK2H2L7wt0EbfwArVaB+uYZeN5+E45+ni31GxE6GJEIrGxd8lmmf72LX2YNoHegW1uHIzSkohAO/AJ7F0JmPKhsIHw89LwDAodCLesLFFYW8mPSj3x/+HvyK/OJ8ojirsi7GBUwCpXiXFebyWQk5/QptHb2OHt1Ii3xIEtensukuS8S3LMP2adOcmLf7porutUXanIyOZ9+xqaMbpTbeXFt2EncZ0xD7dWINaQF4TwiEVjZqdxShr29nrdujObm3v5tHY7QFJn7Ye8iOPCTOUE4B0DsrRBzC7gFX3R4uaGc34/9zqLERZwuPo2vgy+3hd3GpJBJOGouXrGseqKbnbMLaq0N+1b9ydqvPgPqWNHNzr7WMM9sP0z6kj+xW/UNKJWcueYRou4agUfUxTEKQm1EIrAyg9FE+AsruXtwMHPHhrV1OMKl0FdA0p/m+kbH1wEyBAw0J4WIiRd1MBtNRtanrmdh4kL2nt2LncqOSSGTuDXsVro4dan3Uheu6JZ14hiyyQSShGdAIL5hkQy9fVqto5J0p0+TMn8xq7N70v3YT4TFOuF2993YRka24C9DuBKJRNAKRr27nm5eDsyfKhY2v+wVpkHCEvOIo9xj5qaj7tdC9BToNuqiyWqHcg7x3eHvWJmyEoPJwCDfQdwWdhuDfQc3qsDduRXdDpGelEhhdhZ3f7AASZLYtPhbDDodI+6aWeM9BUfTKV/2I0U/LibDPoyzYWMZdXMX3EcNEgMWhFqJRNAKZi7czancUv55bFhbhyK0FFmG9L2Q8KO5T6E8D2zdIHISRN0E/v1q9CfklOfw85Gf+Tn5Z7LLs/F18OWm0JuYFDIJN5vG9x3Jsmz5Y77mq88w6HRcM+thABa/8CR2Ts6W5iR3Dy/2f/Y3Rw+WErP7/7AJDUF54ww633ANanvrlt0WLi8iEbSC/644zNdbUzj8yhiUCvGN7Ipj1MOxNea+hKQVYCgHJz/oMdk8HNUnxjIUVW/Us+b0GpYkL2F31m7UCjVXBVzFTd1vonen3pf8jd1oMPDP/A9JTzpE4dksANRaG3xCw/Dt1h2XgmIUf61ki/tUnCqzGD5YwvXWW1G5iQEMgkgEreLHnaeZ+9sBNj05An83u7YOR7CmymJzMjj4CxxfCyaDuWM5crL5bqFTpCUpHC84zk/JP/HHiT8o1hUT6BTI9d2uZ2K3ic1aOa04L8cyZDX98CGyU0+BLKNQKomIGk6nk2dRb1xGpb0TmYNm03tKLJ0GiRXTOjKRCFrBzpN53Dx/G99M78Pw7mJoX4dRlgeH/zDPTUjZBLIJ3LtBxPXm8hbe0SBJVBgq+PfUv/xy5Bf2nt2LUlIyxG8I13e7nqF+Q1ErGl6XuT4VpSVkHDlM+uFDBMX1wS8skpS1q/l1/vto7cbT++AyXLq5YHPdBHwnXY/aXiyS09GIRNAKCsp09P/vGgLd7fnunn54iIllHU9JNiT9YS6PXZ0UXAMhfIL54dsLFApOFp5k6bGlLD+2nNyKXNxs3Lg26FomdJ1AmFtYi3X2FuWc5fDmDYTG9MO0YS07f1tCoq0SSbLD382J4OEjCeg3wLKim3BlE4mglWw+msM9C3fh62LL9/f0x9tZdNZ1WKU5kPQXHF4OJ9abm48cfcyjj8Kug8AhGBQKtmZsZdmxZaxPXY/epKebSzfGBY/juuDr8Lb3btGQ8jLS2P3tz6QeOkh5WTqVavNEOLVai1+PKHzDIvELi6RzaJhIDFcgkQha0Y4Tucz4ZhcejlpmD+9GkKc9QR72uNtrxLC+jqo8H478Y75bOLbGXN5C6wQhV5sTQ7erKFRIrEpZxR/H/yA+Ox6AXp16cW3QtYzuMhoXG5cWDUmXlsbpRYv4N1GBsuwwMmcpUSnQ2tox+6sfkRQKju7ahtbWjoAeMS16baFtiETQyvadzueeb3eTW6qzbHPUqujiYUeguzkxBLrbE+hhT6C7HW4iSXQc+nLzHULSn5C8EspyQKGCLgMhdAyEjiFVrWHFyRX8dfIvThaeRCWp6Ne5H2MDxzIyYGStM5gvlaGsgvw1Gyn//WfS9h9nT/Q0+ihSiLihL7/9uxwnTy8mz30JgG2/Lsatsx++YZE4uIqRSJcbkQjagMFoIqOgghM5JZzILuVUbiknc8tIySklLb8M03m/dkcbFYHu9nRxt6t62FteezlqRZK4UpmMkL4HkldA8t+QnWTe7t4NQkYjd7uKJCcPVqauZeXJlWSUZqBSqBjYeSCju4xmRMAInDQttzxqQVIK+77bjue2RSjTT5Dr15O8HsMZOnUAthFdmT/rLvQV5QC4dPLBNyyCzt0jrLqim9ByRCJoZ3QGE2n5ZaTklpKSU2ZJEqdzS0nNL8d4XpawVSsJcLMjwN2OADdzojD/tMfXxRaNSrTlXjHyU8xNSEdXwclNYKwEtT0EDUXuOpIED3/+yTvIv6f+JbM003yn4NOPUV1GMcJ/RLOGo55PNhop3bqVvT/Fc6SkMwO2v4htSFf0I29AERNMYWm2ZehqeXERYF7Rzbd7BHFjx+MfKYaptkciEVxG9EYT6fnlnMozJ4aU3DJO5ZZxOq+U03llVFStsQugkMDH2dacKKqShX/Vc39XW9HkdDnTlZlHHh1ZZV51reCUebtbV+TgERzwDmW1MZ/V6RtJLU5FQiLGM4aRASMZ4T+CQOfAlgkjr4DSVX9TsGwZm5TXgKRguM1mnMaPw27EKIpLCs7NZ0hOZOjt0wntN4isE8fY+MM3jJx2L+5+AS0Si9A8IhFcIUwmmeySSk7nVScHc7I4lVdGal45OSWVNY631yjxd7PDz9UOfzdb/F3NiaL6ub1WrFR6WZBlyDthTgjH1pgThL4MFCpkv94c8evJWq2KdUXJHM4zNy8FOgUyzG8Yw/yHEesV2+x5CgBn9xwh99+NqP/9kcqMTLb3e4kQxzP0nBiO/dChKDQaS3mM0wf3s/7bBUx++mUc3NzZt+pPEjeubbEV3YSmE4mggyjTGUjNK+d0XhmpeeZEkZZfZtlWrjfWON7NXoOfq23Vw67Gc18XW5Eo2itDJaTuMFdJPbEOMuIBGTQOZAb0Zb27NxuMRezIT8RgMuCodmSg70CG+A5hkO+gZjchybJMwfa9bP85CeeD/+B6eicGV29S+swg7uoAOl8zAEmjqfGew5vXs//fFRet6OYbFlG1qlskTp5e4g7WikQiEJBlmdxSHWn55aTmlZGaX0Zafrn5kVdGWkE5OoOpxntc7dT4utri62KLr4ud5blf1U8XO7X4H7c9KMsz3yWc2AAnN5grpgKldq5s8+vBRjs7NlVkklNZAEC4WziDfQczsPNAYjxjUCsv/W5BNhgo3b6D479vY3t+OHH7/g8nZQkMH48pdhDdJg1AbXtuPo1lRbfD5qU+M44kUVlWCoC7XwB3vTMPSZIoLcjHzslZzGdoQSIRCA0ymWRySipJzS8nvaCctKpEkV71Oj2//KI7CjuNEl8XWzq72FqSRGcXGzo7m7d5O9ugVor/kVtdUYa5s/nkRnOCKDiFCUh2dGdzp2A2qyX2V57FKJuwU9nR17sv/Tv3Z4DPAIKcgy45uetLKyjfsZWSVavYm6ThdKchDNv/Ki5DB6AYeBVuwwegda059NVkMpKbepq0pEPoysroN+lmABY+8SBOXt5c/8RzAGSdOIa7f5cGV3QT6iYSgdBssiyTX6YnPd+cJNILyskoqCC9oIyMggoyCsprzJsAc901L0ctPs7mJOHjbIOPiy2dz/vp4aBFIaq1WlfBaUjZDKe2QMoWyD9JsSSx08GFLR6d2a6SSTWUAOBl50U/73709elLP+9++Dj4XNIl9aUVpK3ajnb3v5SsWcM+/1sos/fmKruNOI4aie3goWi9667JdXjTOrT2DgTH9aGyrJSPZ9yCUqVq9IpuwsVEIhBaRbnOSGZhzQRR/TqjsJyMgvIao54AVAqJTk42dHaxwdvZnBy8nW3wcbahk5MNPs62eDpqRWnvllSUAae2wuntcHobZB0iTaVgm60dO1292KmSyJPNSd3XwZc+3n3o492HXp164evg2+TLyQYDJ//aSf7OBJy2/YQhI5MdfZ7DU5VDv/62OAwfjjY0pM47EYNOx8n9eywrup09eRyT0YgkKfDoEohv1VwG/8ho0QFdD5EIhHZBlmUKyvRkFJaTWZUkMgsryCw031FkFVWQUVhxUV+FUiHh6aDF29kGbydzoqh+3snp3HNbjbKNPtllrrwA0naZE0PqDuT0PRxDz05bLbscXNitVVOIuVnQx86bOO9exHnFEecVR7BLcKNWYasmyzLlh5PYuSQBzdG9uO5djlGhZne/Z4n0yiHsmnDs+/VDYV/3N/1zK7olkp50kIyjyRgqK7n24ScIHzSMwrNnOHVgP6H9B2Fj79Dc384VQyQC4bIhyzJ5pTrOFFVwpipJZBWZf54prOBMUQVZhRUUVxoueq+jjcqSHMwPreWnV9U2TwetmITXEKMBsg6ak0PaLkxpOzlanMZeGy27bW3YY2tPrsL8d8NJZU+MVyyxneKI9Yylh0cP7NSNX49Df/Ys2f9sYtvmcjon/YHLmQTKHX04Ens3vaOM+I3ui7Z793r7LYwGA9kpJ3Dx7oyNgwP7/13B6i8+Yea8r3Dy8OJk/B7y0lPx7R6BV1BXFMqO+YVBJALhilNSaeBMVZI4U1hBVrE5QZwpqiCrqJKzRRWcLa7EYLr437ebvQYvR3OS8HLU4uWkxcux5nNPRy026o75B6NWpbnmchjpe5DTdpGWFc9eKthnoyVea8NxjXmosQKJEEd/Yrz7EOUVS5RHFEHOQY26a5B1Osr27iVl1V72nHanx56PsK3IpaBLX86GXE2/QQ64j+iP2rv+qqyyLJOfmYFbZ3Mz1pqvPiV+1V/AuRXd/KrmM/iEdEet7RhVgkUiEDokk8k8ZPZscQVniyo5U2T+mVVcYUkUWUUV5JToapT1qOZoo8LLUYunoxbPqkTh6ajF06F6mxYPBy1u9pqO14chy+aSGBl7IWMfhRl72J97mANKE/ttNBzQaimpGvrpoNAQ4RRMpHcvIjv1JMI9Aj8HvwZHJ+mzsijdvIXDG09zuNiPftteRiEbOBt5HaUBsQy82h2Hvn1QeTQ8L+Lcim7mWdDZp1MsK7p5BXUlMCaOQTff0QK/mPZLJAJBqIfRZG6OOltsTg7ZVY+zRRVkl1Q9r9pWpjNe9H6FBG72WjwcNJZE4eFofu3hoMXdoWqfgxZXe82VO6TWZIL8k5CxD1NGPCln9nCg4CgHFAYOaTUkazToq/74Oyk0hDsEEOEZRZhPP8I9IghwDECpqP0uzGQ0ojt6lNLt29m7vZisEgd67X4LgFM970DZ2Ze+wz2w69O7wTsGMK/olnkkibSkQ6QnJaKxtbVUWf3z/f/hE9KdXtdd3yK/lvZCJAJBaCGllQbOFleSU1JJTlWCyCkxP7KLK8ku0ZFTta3ygk7vaq52aktycHfQ4mFv/ulmr7Fsc7PX4G6vwclGfXkPr5VlKEyDrIPoMuM5mrmLQ4XHSdQXkajVcEyjtiQHWxSE2HgQ6tKN7p16EerdixC30FrLbpv0eioPH6Zs50627pTR5+UTfvBrABJ7zcbVXUXPgS7Y9uyJpmtXFKr6Z8lXl8aQTSZ+f/d1OoeG03fijegrKvj2idl4d+tuaU66XFd0E4lAEFqZLMuUVBrIKdFZkkZOqTlJ5JZWklu1PbdUR26JjsJyfa3nUSokXO3MCcLNXmNJEK7n/aze7manwcVOc3l0hleWwNnD6M8kcCJzF4m5hzlSfoZkhYkkjYbi8+6afBRautl2optzN7p1iqWrTx+CXbtiqzq37rJsMFCRnEzZrt1s3ilhk3mYgMPLkIFtA14lSHmSHj3U2EbHQLcIHAM6NSrMkrxc1i/6kvTDBynJzwNAa29P59Bwy4punbqGXBYT3UQiEIR2TmcwkV9mTgq5pZXklerIKdGRV/XcvF1Hfqn5Z12JA8yLILnaa3C1U5sThZ3G8trFToOr3XnP7dW42mnaR8e4LENpNnLWIbIy93AkK54jRSc5WpnLMYXMCY0aQ9XdgyRDZ4WGYK07wY5dCHIPJ8inF4FeUbhqXQHQp6VRvGsfOzcX4ZKxD5cDq9ApbNk86H+EZ/1NaIAeVUQPij3D8BsUjm0n93pCkynKziKtqjRGelIieRlpACjVaiY8/gzBPfugKy9DluV2OdFNJAJBuMIYjCbyy/TmJFFaSUGZntxSHXklOvLLdBSUmRNGQZme/DIdeaW6Wvs3qtmoFbjaaXC2VeNiZ04OLnZqnG3NP11s1TjbqnG2U+Niq6n6qcZOo2ydelOlOeizEknN3MXxnIMcK0rhZEUuJ6gkRaWi8rymGidZIlBpTxdbTwIcAwhwC6WLd098HUORD50mcf0p3LITUB/aTk6hij1x/yHqwGd0ts3HGNaLTPeehPfxxC0uDJWPT52fr6yokPTkRNIPHyLu2gk4eXix/9+/Wf3lJ8z8+EucPLwoPHsGpVrTLlZ0E4lAEAQqDUZLYsgv1VNQpiO/6nVBWXXS0FNUXnVM1XOdsfa+DjDPDHeuShJO1cnCVo2Trcr80+bcPicb83bzTzWONqrmd5wbKjHmHiMzcy8pZxM4WXicU2VnSNEXkSIZybqgb8BJBn+FDX4aF/zsfPCTfXE+7YNfZQn2qWdIPQ3xXhPot+s17MuyyPXrw+nAMfTzScE1rAv4d0MdGIiDf+2VUnNOp3B87y76TrwRSZL4++N3Sdy0DudO3viFRbbpim4iEQiCcElkWaZcbyS/TE9hmZ7Ccj2F5eamqYIyPQXl1dvMSaPo/NcVhlqH5Z7PVq20JAdHGxWO5/10slHhaKPCQWt+7VD12lFrfm7erkKrUtT+R1VfQUXuUdLO7OVU9iHSilJILcsiVVdIGjoylApLU1M1TxMEVTjSvVBJUL4G27xQ8g39iE2Yh6q0hJSAqzkRfD0j9r+MXZAfBf69KHX0p0d/d2yCA1H5+qLUai3nO5tygtMH91uaky5c0c03LAL/iCg6BXdr/n+sBrRZIpAkaQzwAaAEvpBl+c0L9muBhUAvIBeYIstySn3nFIlAEC4PsixTpjNWJYVziaS4wkBRhZ6icgPFFedeF1ec/9r8vK6RV+dTKSQcbFTYa8yJwV5rfjholdhrqp9Xb6/aplHgIBchlx6mpOwoBaWnyK7I4GxlLmcMJZwxVZKpkNBVj9iSZVxLIDzDh675QXTO34pPrkSh/fXkuPVmyNa5ACSH3EyheyhD5ZXYBASS6xgKbh4Ex3mj6uxDsa6S9KNJVYnhEIVns/ANi+CWl81DYRPWrMIrMBjvriEt/t+jTRKBJElK4AhwNZAG7AJulWU58bxjHgCiZVmeJUnSLcAkWZan1HdekQgEoePQGUyUVJ5LEMUVBsvr0koDxZUGSioMNZ/rDJRUGimtNG8vqfrZwM1JDXYaJY5qI521Z3BTp2OvOoNKkYtJkU+FooRSqZxChZ48hQzlWjwLdfjkyQQU9sa5ohOd0/+gUwGkdL2fSo0zffeYvwMnRN6DQaslJO87cHMk07E3CnsV3btr0Xh4s/TXP4joN4yr7r4Xk1rN1p+/t1RabW5BvbZKBAOAl2RZvqbq9dMAsiz/97xjVlUds02SJBVwBvCU6wlKJAJBEJpKlmUqDaaq5GCkVGegTFf1vNJAmc5ofq0zUqYzUl71vLxqe5nOSIXeSLnevL9Sb6p6bkAyFOGmysRZdRZ7VS4aVQEqZRGoylAZDDgUq9BU5mBXIuNcPhSbSiUe2etwL5ZJC5yDVldGzMHPANje63G0lbn0PPgN+bYatoV2Acx3RS7lxbj0C+aGZ7++pN9BfYnAmmsR+gKp571OA/rVdYwsywZJkgoBdyDn/IMkSboXuBcgIEAshC0IQtNIkoSNWomNWol7CxckrU4y5TojFQYjFXrz80qDkUqDiQq9+afleWUJhaU3kFWeiU6Xi7Ekh/TBg1GWFKAq2E1lRSlb3ZxRVxhwIQL0mSgqjqExGVCqtQ0HdAkui0VpZVn+HPgczHcEbRyOIAiCxflJpvHCrRbPpbDmFMR0wP+8135V22o9pqppyBlzp7EgCILQSqyZCHYBIZIkBUmSpAFuAZZfcMxy4K6q5zcCa+vrHxAEQRBantWahqra/B8EVmEePvqVLMuHJEl6Bdgty/Jy4EtgkSRJx4A8zMlCEARBaEVW7SOQZXkFsOKCbS+c97wCuMmaMQiCIAj1uwzKFAqCIAjWJBKBIAhCBycSgSAIQgcnEoEgCEIHJxKBIAhCBycSgSAIQgcnEoEgCEIHJxKBIAhCBycSgSAIQgd32S1VKUlSNnDqEt/uwQUlrjsA8Zk7BvGZO4bmfOYusix71rbjsksEzSFJ0u66Fma4UonP3DGIz9wxWOszi6YhQRCEDk4kAkEQhA6uoyWCz9s6gDYgPnPHID5zx2CVz9yh+ggEQRCEi3W0OwJBEAThAldkIpAkaYwkScmSJB2TJGluLfu1kiQtqdq/Q5KkwDYIs0U14jPPkSQpUZKkBEmS1kiS1KUt4mxJDX3m8467QZIkWZKky36ESWM+syRJN1f9tz4kSdIPrR1jS2vEv+0ASZLWSZK0r+rf97VtEWdLkSTpK0mSzkqSdLCO/ZIkSR9W/T4SJEmKa/ZFZVm+oh6Yl8U8DgQDGmA/EHHBMQ8An1U9vwVY0tZxt8JnHgHYVT2/vyN85qrjHIGNwHagd1vH3Qr/nUOAfYBr1Wuvto67FT7z58D9Vc8jgJS2jruZn3koEAccrGP/tcDfgAT0B3Y095pX4h1BX+CYLMsnZFnWAf/f3r2FWFVHcRz//lLDykzQolBjekgMJrIoSbASjKEbzoOCBpY3esuHKCPwIcsgwooigqSyMsqoqBjqwegyKeIwSkqZUEiW2Z2oATND7dfD/z85DqOzxzmXztnrA8LZe/Y5e/3nOLPO+v/3rP0a0N7vmHbgpfz4TWC2JNUwxkobdMy2P7Z9MG92AZNqHGOlFXmfAVYDjwCHahlclRQZ8x3A07Z/B7D9S41jrLQiYzYwNj8+B/ihhvFVnO1NpHu4n0g7sN5JFzBO0gXDOWczJoKJwHd9tvfnfQMeY/sI0AOMr0l01VFkzH0tI32iaGSDjjmXzJNtv1fLwKqoyPs8BZgiaYukLkk31Cy66igy5lXAQkn7SfdIX16b0OpmqD/vg6rqzevD/4+khcCVwHX1jqWaJJ0GPA4srnMotTaSND00i1T1bZJ0qe0/6hlUld0KvGj7MUkzgJcltdr+p96BNYpmrAi+Byb32Z6U9w14jKSRpHLyt5pEVx1Fxoyk64GVwBzbf9cotmoZbMxnA61Ap6RvSHOpHQ2+YFzkfd4PdNg+bHsv8BUpMTSqImNeBrwOYHsrMJrUk6dZFfp5H4pmTATbgIslXSTpdNJicEe/YzqARfnxPOAj51WYBjXomCVdDqwlJYFGnzeGQcZsu8f2BNsttltI6yJzbG+vT7gVUeT/9jukagBJE0hTRV/XMMZKKzLmfcBsAEmXkBLBrzWNsrY6gNvz1UNXAz22fxzOCzbd1JDtI5LuBDaSrjhYZ/sLSQ8C2213AM+Tysc9pEWZBfWLePgKjnkNMAZ4I6+L77M9p25BD1PBMTeVgmPeCLRJ2g0cBVbYbthqt+CY7waelXQXaeF4cSN/sJO0gZTMJ+R1j/uBUQC2nyGtg9wE7AEOAkuGfc4G/n6FEEKogGacGgohhDAEkQhCCKHkIhGEEELJRSIIIYSSi0QQQggl13SXj4bQS9J44MO8eT7pcsre68un5941lThPC/Cu7dZKvF4ItRaJIDStfP38NABJq4ADth+tZ0z95WaH6m2H0H/7JM8bmftkhTBsMTUUyuQMSXsljQKQNLZ3W1KnpCcl7ZS0S9L0fMxZuT98d+53P1CH0xOStELSttw3/oG8ryX3118P7AKu6bc9WdKaHMfnkubn582StFlSB7C7gt+XUHJREYQy+QvoBG4mtWJYALxl+3D+a+szbU+TdC2wjtSraCWpBclSSeOAbkkf2P5zsJNJaiP1+ZlO6h3fkV97X96/yHZXnlrquz2XVMlcRuqZs03SpvyyVwCtuY9QCBURFUEom+c49if5S4AX+nxtA/zXD35s/sXfBtwnaScpiYwGLix4rrb8bwfwKTCVYw3gvs295BlgeyawwfZR2z8DnwBX5a91RxIIlRYVQSgV21vy1MwsYITtvrcD7N9vxaRP8nNtf3kKpxPwsO21x+1MFUD/imLQCmOIx4VQWFQEoYzWA69yfDUA0DsXP5PU0bGH1Oxsee8d7HIX16I2AksljcnPnSjpvALP2wzMlzRC0rmkWxd2D+G8IQxJVAShjF4BHiJPBfVxSNIOUqfHpXnfauAJ4LN8s5u9wC1FTmL7/dwWeWvOIweAhaTLWE/mbWAG6f68Bu61/ZOkqUXOG8JQRffRUDqS5gHttm/rs68TuKfB71cQwimJiiCUiqSngBtJ/dxDCERFEEIIpReLxSGEUHKRCEIIoeQiEYQQQslFIgghhJKLRBBCCCUXiSCEEEruX+icU7EKCExyAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "\n", + "fpr_list, fnr_list = gm0.plot_fDP()\n", + "fpr_list1, fnr_list1 = gm1.plot_fDP()\n", + "fpr_list1b, fnr_list1b = gm1b.plot_fDP()\n", + "fpr_list2, fnr_list2 = gm2.plot_fDP()\n", + "fpr_list3, fnr_list3 = gm3.plot_fDP()\n", + "fpr_list4, fnr_list4 = laplace.plot_fDP()\n", + "\n", + "plt.figure(figsize=(6,6))\n", + "plt.plot(fpr_list,fnr_list)\n", + "plt.plot(fpr_list1,fnr_list1)\n", + "plt.plot(fpr_list1b,fnr_list1b)\n", + "plt.plot(fpr_list2, fnr_list2)\n", + "plt.plot(fpr_list3, fnr_list3,':')\n", + "plt.plot(fpr_list4, fnr_list4,'-.')\n", + "plt.legend(label_list)\n", + "plt.xlabel('Type I error')\n", + "plt.ylabel('Type II error')\n", + "#plt.savefig('rdp2fdp.pdf')\n", + "plt.show()" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[5.298525912188081, 4.728386984943314, 4.728385602505513, 4.3771780956812245, 4.377178095769466, 1.4142111312027956]\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/lib/python3.8/site-packages/scipy/optimize/optimize.py:2522: RuntimeWarning: invalid value encountered in double_scalars\n", + " w = xb - ((xb - xc) * tmp2 - (xb - xa) * tmp1) / denom\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAFSCAYAAADcn4lzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA2tElEQVR4nO3debxvY/n/8dflGI5ZccxJCpEiThRSyPQ1FzJGCQ1C5pRZkiS/SkqRisg8z5EpZEzIlCkiR8gQJ8P798d1f5xldw77rOPstdb+vJ+Px36c/Rm2x718Pmtd677v677ukISZmdnEmqLpBpiZWTc5gJiZWS0OIGZmVosDiJmZ1eIAYmZmtTiAmJlZLVM23YChMttss2n++edvuhlmZp1y0003PSlp1Phe65sAMv/883PjjTc23Qwzs06JiIcm9JqHsMzMrBYHEDMzq8UBxMzManEAMTOzWhxAzMysFgcQMzOrxQHEzMxqcQAxM7Na+mYh4aSYf8/zmm7CoDx4yJpNN8HM+oh7IGZmVosDiJmZ1eIAYmZmtTiAmJlZLQ4gZmZWiwOImZnV4gBiZma1dHodSEQ8CDwHvAq8Iml0sy0yM+sfnQ4gxYqSnmy6EWZm/cZDWGZmVkvXA4iAiyPipojYtunGmJn1k64PYS0v6dGImB24JCLuknRl78USVLYFmG+++Zpqo5nZsNTpHoikR8u/TwBnAEsPeP1oSaMljR41alQTTTQzG7Y6G0AiYvqImLH3O7AqcHuzrTIz6x9dHsKaAzgjIiCP47eSLmy2SWZm/aOzAUTS/cDiTbfDzKxfdXYIy8zMmuUAYmZmtTiAmJlZLQ4gZmZWiwOImZnV0tksLJs08+95XtNNGJQHD1lzUO8bbsdj1gXugZiZWS0OIGZmVosDiJmZ1eIAYmZmtTiAmJlZLQ4gZmZWiwOImZnV4gBiZma1OICYmVktDiBmZlaLA4iZmdXiAGJmZrU4gJiZWS2uxmvWUsOtwvBwOx5zD8TMzGpyADEzs1ocQMzMrBYHEDMzq8UBxMzManEAMTOzWhxAzMysFgcQMzOrxQHEzMxqcQAxM7NaOh1AImJERNwSEec23RYzs37T6QAC7Aj8telGmJn1o84GkIiYF1gT+EXTbTEz60edDSDAEcDuwGsNt8PMrC91MoBExFrAE5Jueov3bRsRN0bEjWPGjBmi1pmZ9YdOBhBgOWCdiHgQOAlYKSKOH/gmSUdLGi1p9KhRo4a6jWZmw1onA4ikb0iaV9L8wMbAZZI2b7hZZmZ9pZMBxMzMmtf5LW0l/QH4Q8PNMDPrO+6BmJlZLQ4gZmZWiwOImZnV4gBiZma1OICYmVktDiBmZlaLA4iZmdXiAGJmZrU4gJiZWS0OIGZmVosDiJmZ1eIAYmZmtTiAmJlZLQ4gZmZWiwOImZnV4gBiZma1OICYmVktDiBmZlaLA4iZmdXiAGJmZrU4gJiZWS0OIGZmVosDiJmZ1eIAYmZmtTiAmJlZLQ4gZmZWiwOImZnV4gBiZma1OICYmVktDiBmZlZLZwNIRIyMiD9FxJ8j4o6I2L/pNpmZ9ZMpm27AJBgLrCTp+YiYCrg6Ii6QdF3TDTMz6wedDSCSBDxfHk5VftRci8zM+ktnh7AAImJERNwKPAFcIun6hptkZtY3Oh1AJL0qaQlgXmDpiFis+npEbBsRN0bEjWPGjGmkjWZmw1WnA0iPpGeAy4HVBzx/tKTRkkaPGjWqkbaZmQ1XnQ0gETEqImYpv08LrALc1WijzMz6SGcn0YG5gF9FxAgyEJ4s6dyG22Rm1jc6G0Ak3QZ8uOl2mJn1q84OYZmZWbMcQMzMrBYHEDMzq8UBxMzManEAMTOzWhxAzMysFgcQMzOrxQHEzMxqcQAxM7NaHEDMzKwWBxAzM6vFAcTMzGpxADEzs1ocQMzMrBYHEDMzq8UBxMzManEAMTOzWhxAzMysFgcQMzOrxQHEzMxqcQAxM7NaHEDMzKwWBxAzM6vFAcTMzGpxADEzs1ocQMzMrBYHEDMzq8UBxMzManEAMTOzWhxAzMysls4GkIh4V0RcHhF3RsQdEbFj020yM+snUzbdgEnwCrCLpJsjYkbgpoi4RNKdTTfMzKwfdLYHIukxSTeX358D/grM02yrzMz6R2cDSFVEzA98GLi+4aaYmfWNzgeQiJgBOA3YSdKzA17bNiJujIgbx4wZ00wDzcyGqU4HkIiYigweJ0g6feDrko6WNFrS6FGjRg19A83MhrHOBpCICOAY4K+SDm+6PWZm/aazAQRYDtgCWCkibi0//9d0o8zM+kVn03glXQ1E0+0wM+tXXe6BmJlZgxxAzMysFgcQMzOrxQHEzMxqcQAxM7NaHEDMzKwWBxAzM6vFAcTMzGpxADEzs1ocQMzMrBYHEDMzq8UBxMzManEAMTOzWhxAzMysFgcQMzOrxQHEzMxqcQAxM7NaHEDMzKyWzm5pa2bWtPn3PK/pJgzKg4esOVn+u+6BmJlZLQ4gZmZWiwOImZnV4gBiZma1OICYmVktDiBmZlaLA4iZmdXiAGJmZrU4gJiZWS0OIGZmVosDiJmZ1eIAYmZmtXQ2gETEsRHxRETc3nRbzMz6UWcDCHAcsHrTjTAz61edDSCSrgSearodZmb9qrMBZDAiYtuIuDEibhwzZkzTzTEzG1aGdQCRdLSk0ZJGjxo1qunmmJkNK8M6gJiZ2eTjAGJmZrV0NoBExInAtcDCEfFIRGzddJvMzPrJlE03oC5JmzTdBjOzftbZHoiZmTXLAcTMzGpxADEzs1ocQMzMrBYHEDMzq8UBxMzManEAMTOzWhxAzMysFgcQMzOrxQHEzMxqcQAxM7NaHEDMzKwWBxAzM6vFAcTMzGpxADEzs1ocQMzMrBYHEDMzq8UBxMzManEAMTOzWhxAzMysFgcQMzOrxQHEzMxqcQAxM7NaHEDMzKwWBxAzM6vFAcTMzGpxADEzs1ocQMzMrBYHEDMzq8UBxMzMaul0AImI1SPi7oi4LyL2bLo9Zmb9pLMBJCJGAEcCawCLAptExKLNtsrMrH90NoAASwP3Sbpf0n+Bk4B1G26TmVnfCElNt6GWiNgAWF3SF8vjLYBlJG1fec+2wLbl4cLA3UPe0AmbDXiy6Ua8zYbbMQ2344Hhd0zD7Xigfcf0bkmjxvfClEPdkqEk6Wjg6KbbMT4RcaOk0U234+003I5puB0PDL9jGm7HA906pi4PYT0KvKvyeN7ynJmZDYEuB5AbgAUj4j0RMTWwMXB2w20yM+sbnR3CkvRKRGwPXASMAI6VdEfDzZoYrRxam0TD7ZiG2/HA8Dum4XY80KFj6uwkupmZNavLQ1hmZtYgBxAzM6vFAcT6UkQsEhHfiIipmm6LWVc5gFjfKWVwFgDeA+wUEZ1NJumJiHdExDQRMX157HPbJjt/yayvRMQUkl6VdB5wC7AU8OUuB5GIeD9wHnAMcGRELCDptYabZX3AWViTQUSEJEXENJLGNt2et0tELAHMCNws6YWGmzNJIuLrwJrAC2Qa+NXA9yW93GjDJlIJHr8kC4veQ9aDmwXYBRirjp7gvXNowHNTODC2S2fvutqsBI+1gB0i4jLygntx0+2qoxIMVwZ+AjwD/DEiTpV0TbOtqyci5gbWB1aV9FL5rNYGtouIn0p6pdkWDk4ZpjqMDBTHl+feCWwg6aVGGzeJKufQp4BpgYMlPTS+wGLN8RDWZBARCwOfB84ABGwaEes326p6yom8JPA1YDXg48BYYJ2IWK7Rxg1SRMR4np4dWLL8/nvgRfIz+8pQtWtSlbvxrwAzR8TB5en5gHdGxLTNtWzSRcQHgYOBPwH/Ac6PiEXK97Ez162I+FT5mbvy3Pi+j53kHsjbqHwxFgWuAvaWdFREvAtYCVg3IqaUdEqjjZxIETEN2f6VgTkkPRgRRwA7ABtFxAhJVzbZxjdTvWONiIWApyX9IyK+R+4h85KkmyOiV8Xg5MYaO0gRMTPwGjCnpHsjYnXgwohYBpgG2ETSi126W4+IWYF5Jf253LDsBfxO0m+B30bEHsAZEbGMpH832tiJE8CGwPMR8R9J3+rKZzIYngOZDCLiJGB5SfOWx3MDawGfBL4u6Z8NNu8tVYatpiwlY6YFvgV8EPiGpDsiYi5gZ+AYSXc12uBBiIidgXXIHuE1wD+AV8i5gkuA/yO3B7insUYOQtk07YfAv4BPAD8HjgWeBs4E7pD01Y4Fj2mA3YF3kMfyDPAb8hi3l/R4ed8JwH6S7m2oqYNWbqxeLb+PBOYmj+khSZuW5zvzGU2IA8gkqlxsP0hWB75F0mMR8XPgo8ASkl6NiHkAJHWiYnAZf14XGAV8D3iW7IUsDxwg6baImLps5tVq5Y7252TbPwQsRgbDHwFzksNZt0n6W2ONHIQyYX482e7TyePYCniVcUHlIuBySbs01MxaImIBYJvy8HBgajKr7CpyiFHAqcAqbbphqQaKynOvT/aXfYtulnR/eXw58FgviHRdZ8YS26oEj3XJE3s74IiI2FPSNsAfgXvKF+rRDgWP0WTQ+BWZ6roV8AHgd8AdwEERMQN5B9864xljnokcunpR0vVkD2Q+csjkGklndCB4zAWcCJwm6VeSnpN0LXmxFbClpDFkcsDqZbiu9SrzGfMBiwMbAPuQQz/bAssB/4+cn/qSpLvaMIcQEXNFxHvfInh8F/gFOYcDgKQVgXkiYt8hbfBk4gBSQ0TM2FvBHBHTAVuSJ/C6ZDrlbBGxgaTtyAvwss219q1FxEIR8YXKUx8ErpF0taT9yTvAnciA8SNyGO75tqZUVuY8ZilPXUOOQW9fXr8LeBxYqLyv8QvSILwD+DvweETM33tS0t1k72rLiFhC0kPAkh0YintfRHxE0msR8QHgp8CuwPbAf4GvkokNXyR7Vo8D58O4z7cpZUj6UWCLga9VgsdewJeAy8nPZpbK27YBpouIGSd/aycvB5CJVO68v0+e0JDDB3MAC5bH1wKPAasCSNpA0tVD3c6J9CJwb0TMUR7fDswUue4DSScBDwALS3qyrXfrkeVJFiu/bw8cHxG9z+pYYNGIODkitiPTQ38PzV+QBkPSnWRW0orAp3tBpAyh3EquY3mqvL3Vw4plOPdKcv0N5HXoMUl3SrqI7GktD3wHmIpM2FiDTItv9JpV2n478D1J+1Wef38ZhiMi1iDXGC1EHsMo8vvW8wzwbnKb7U5zAJlIkp4nM0RmiIjNlAsFfwSsGhEfVy5Eu5m8AM/Y9Bf+rZQL0N+B64A7ImJfSTeQk8xrR8RGEbEUMJr84rdSCewbkqVJtgfWA75NZsXtCDwEfBe4m9xzev3euHRXSLqOXIuzBLB+5IrzVyPio1R25+xAQJyW3PxtsYj4FfldeyoiNiuJGzeSAWZ6YKSk+8i7/TOa7PWW4PEXMnFkj/LclJGlcbZlXI/kZeDakixzK/AwGQABkPQE+V3sdKo1AJL8M8gfYIrK76sDN5FjtguT+fi3kWPSDwBrNt3eQRxPL4li7vLvQmTX/MvASHLY6jTgXGCdpts7iOP5EJkZdi45Xg7Z+ziOXHC3QNNtfJuO86PAr8l5gY3JYdJ1m27XRLR/JNnLeBLYsTz3hXLu/IBcb3QzsGx5bUQL2jwP2cP7XuW56vVgFuBickHqQuT6lSXLa7MBv+2db8Ppx+tABqlkW70WESuSF9wTIrfS3Qn4MXlCX0d2TU+UdEOb0/Qq2WNrAftHxCaS7inHdzUwlaQjyKSA2SQ92cbjiVx5/X5JfwSmI3sa9wAbRsR1km6NiB3IIaxtI2JvdaxcyUCSrivTNruSvZEdJZ3Xxs9nfJSr/+8iF6TOVJI2fgN8BNgc2BTYt3ymaMBE9VAb0PPYrTxXTdMdIemZiDgEWBr4A5klt1ZEPEMuvp2GHLJrZeJJXU7jnQiRC7Z+BGytsnguIlYF9gB+I+m4Bps30SJieXICdnNJN0XErJL+Fbn48a/k3db+zbbyzZW5gF2B+cnx8nXJjJ4NgbmAnysXp80IzCDpsYaaWku8Sf2nMkcVkm7pSvCoiog5ycnyaYBTlEOnRMS0aslCyDJhfjuV4PEm710Q+CY5PPUyeWyLkz2uTSU9OHlbO/QcQN5C5U59WuBSYC9JV0TESmQe/jlkr+MgYMO2X6AqxzMX8F4yQ+wmstv9BTIJYG8yMWA+SZc21thBKBlU3yXHoH8s6Vvl+SXIicwFgcMl3dZYIydC5fOZh5wbmLLrPaaqgUEhIt4HbEYONZ4h6YrGGjdARMwL3AkcKekblec/AbyP7JX8RdKLldc2IzPHPk0W6pwbeK7cmP3PmpGucwAZhIhYm0yhXKX8/JPsjj5HjoNuExGzKyfHWi8iViHv2o8js8UWAX4GjCEn+35X6WE1fhc40HguQouSwXxt8oQ+tDz/KXJ+6jSV1cxdUL5vu5B3vrcB5wy8MeldjCLTyBdSZmK1Splcfq0ExAl+jyLXrHyO7MXfPaSNfBOR1QuWl/TpynMbA/uRKcUfAw6TdNqAIa2vk+fSiZXnhmcl4aYnYdr+Q47L/pEsG7EAmae+dHltZXKscxoqE2pt/iEvqGcBy5THcwEzld8XIrNGlmm6nYM8lq2BA8l1OHOTqZKnkkMHa5LZcjM13c6JPKblyQnkeciV2H8C9gXmqbxnRPl3FnK8fdGm2z2e45iGHPufkywTswlvMokMTF/5vRWTzeRN4iKVx0uRvY7Fy+NNgLsGfsfIJIAfNN3+Ifl/1HQD2vxDDuP8HvhVeVzNulixXGxbn51U2hvlpP4W2S3/WuW1qcieSGeyeUrwuIbMSHqBHH4bCaxApojeWT352/pDFjTtBYQghz6WJHuCN5LDOxeS6wnmr/zdLGTWzyeaPoYJHNc0wNfJO/X736ydleOfsul2D2wXWcNudHk8K/B/5fcpyr/nAHON52/PIIuQtiIYTq6fVq9RaIF/k2msK0TEehq3yvTd5AV3H0lnl3H4Vqq0bUblmpXvkBsQvbcM8aAcY38K+IKks9p+PGXNx5LksMeCZA/x15JeUg69rU9esP7aYFPfUuQuiOsA80fEp4FDyAvPX8n08E0lnUCuxJ6TDPRExEzlfQepRXMGPWW4Zix5cX0XOcf2cIxn18fKUNwswAHxxhXbjVIOP10EzBER80v6F9njgywfA7nB2hwAEbF4RMxW/nZ9SZepRJPhymm8FZUJzGXJC9NdwEnkArovRsTLks5TbmzzbUnPt3GOoKocz9rAjhHxHJlq/DNymGeVyF0Tz1Mu3nr9bxpq7nhVx49L256PiIeAo8gx9lXK+/YE7pJ0JjkG3VqRJdlfIqsAnE/2PrYvn9dYcghr14j4KTlh+xWNq0K7LrCnsq5Xq5TzoZfuPj95B78tOfR7CnBdZOn2ZyAv0uX/xVnkDdkzTbR7fMqxvBwRf+49J6lX12oqcsX/y8C/I0sBbUQG/oH/jVadT28n90Aqysm7GnmHPpI8sdcALiAXAu0aEeuV9z7f+5tmWjs4EfExci5gK3Js/fPlJD2GvICt0rtraqtKz2+FiPhkuZP9CzmkcHB5bQNyUV2rex2QtdSAPckL603kPMdzwGMlhfU1cl5nXrJszncl3dT7e0m/aWPwgDfcsBwO/ENZm+sw8lqzfuS+HpeTGX69nseZwDfb1puqnNsCPhq55qj3Wq9czJ3AEeQq9C/1rgvj+W8MS87CqihDIz8lJ2ZnJitpriLpn+UuaT3gTpV89baq3vWUO8HpybIJO5PDIg+U/PbngNnV3tpW1eP4MvANct7jPeRk8zZktdZR5CLCr0j6S0PNnSgRsSW5CPA+ssrxquTK8u9IuiyyLtnTwLSS/t2VO9mImJ6s4vx9SddGxFTlLn4Wcj5nMeDCMlQ6DbkA90i1eFMyeH2NxweAC8rwXO/5k8l9fpZQblQ27FJ134wDSNFLwy2pe4uSZTE2Uu7A91kycHTi4gQQud3sv4F3kvWT/gWsJ+npyMWP25ILIlu5u9uA4DELObl8vqTHI+I4smLwR8vFaUGyXPuTjTV4kOKN5b43IQPhLZJ+UYZBNiYTN75JzuPc0lxrJ15JK76QTG89uzLHMWf57N6QzhoR80p6pLkWv7mI+Dhwg3L1/ILAA8pN1nrHtRjwZDm2vgoe4CEsACJXXn+njM3+hyxHsHMJHh8G9mdc9d2u+CS5sO5KcgHkjMCsEbEhWW/oly0OHlNUgseO5BDil8nMNyRtRWbAPVSGfO7tSPDozQ/MV9p9Ijn2PzoitpF0LDn08xqwQReCRy/hIiKmjYjpyhzB78hj+lC5yC4L/CYi5qsEzxEALQ8eQfYSvxoRm5PpuQvDuPIqkm6vBMa+Ch7gHsjrIuJCMm1yP3JydgR50V2Qkm3VXOve2sAhjjJPcAS5OOv6iPgRmTQxCviFpAvbPixSht+2Ioc51ibTdS9XWR1fjumItg7BVVUSNP4POAC4gly/sUbp4S4L3EsG9hcG/l0zrX5zlWNal8yIm46sgPwk2ZNaGbiBzDTbUdJ5jTV2EkRWBZiNTDwZA/xW0h3Ntqod+jqARMQo8v/BE5FlC75NrpN4Cng/mWs/Rrl9a2tP5J7I8ioLkuO0D5cJy/dL+nzlPdOqUnqhjcqd34fJgL67pMMit3PdiJzLuUrS+U22cbB6cwDl9yXI2mObkr2pnciKrS+VILIicLCkhxtq7kSLrA93EDk/eDBZpXoLshz7kuRC1YfaPm/Y0xtiG9/5Hll3bSXgj2rRtrpN6tsAElkL6kdkvaEbJP0mIn5CXpxObLZ1gzdgrmAZMuPqUfIu8BDgMnJi9qyB72+7iPg2OXS1SElkeC85d/MicKjGpVS2UrlB2YIM6H+NiIWBZcjJ8b2BjSXdHxFLS/pTRMyh3EOiMyJiW3I4cS5y4eCpZC/+a106j+ANwWMBsiDnFZVzq9fbGgn8V8OxLEkNfRVAKl+C6SW9UDKRPgj8kMy+Wozsdq/chWGRnjJhPiNwT7kgvZfcR/oJcj7nMnIb2k6M0Q64az+EvAgvI+mRcnI/25E5j4+S6bgPkQtSRd6Zv0jWr3o1siLyHsC2ankhzqqIWBp4uIz/zwqcQK5NuTUiziELda7Qhc8J3hA8FiHTirckU3THqpJ1ZW/UV5PolfHas8ucx4LKLTQ/RY6vP0veeUzdYDMHpTJ5uSx51/cZ4MiI+Lykv0laizypzwN+34XgEREfjnFpn71J1j3JNSv3RsTcku7vykVJuYPgdWRW35bAI2TJlemBdSLic8CRZMn5TgSPiJiq/PpNshYU5JDv/cByEbEyuUhwq658TgOCx6/JzeHGkuvBZmi0cS3XVyvRyzj6TmQX+71kIPm0pN9HxLElPe8XankJDHg9GC5Hrh/YQNI1kem5u5WL8NGSfk/Z97uNQ1cDht/mJNd1/Jbc0Oq13oktaZ+IeJm88HZGmTDfnpxIXo1cufwTMjFgY7Insruki9r4+UzAHGQgvBToLZqbitwGYFmyivCOkv7UTPMmTiUzrhc8vkH23I8jP5t/Ndm+tuubIazIktH7Av+RtE15bgsypXUTSZeU53rDXK09oSt3TD8kh3c+K+niyIVZK5Bpx7+W9NNGGzpIEfEe5eLGb5A1u/Yqzwf5He3ceHPkgrpfkovkrigJDuuSa3O+r5amUL+ZMlR1MVkP6qNk1tWXyD2/nyXLesxehhpbe/4MVOZDLyezrJ4gA8lO5XPrzHE0YVj3QAZ8+I+Q3ewlIxcHXVcmzqcCzipZWE/33t/GL03leGYBnpK0Q0Q8TW5Je6OkpyLiCjKAPN1kWwcrsqDjD0oCw4+ByyLiUUlHlmNt3ecwGGWO7WVyG4ArlKvL5yIzlV6NiMOB59v4Pauq3FB9hKxt9T0y8eTv5JqVPcjriIAtVdZ1tP24qiQ9FlnMcmpyFb2DxyAN2x5I5Yu/LFnJ9DlJl0TE3sDsZJHE68uw1TySHm20wYMUmTa5I/AYcDd5Qn+bLOnxGUljuvTFLxP+vyU/o/3I+larADuoRZsLTYzKd291MinjamXpjkXJz2vXLgyT9pR5w33IBZ1LkL3bkyPiB+SaqfuAD6jFlRom1JuNiKlV6lpFxP7AlWVIuzPnUJOG7SR6OYFXIyvPLg3sGxG/knQguaPgVuSOYpAX42rp81aKiA+Rk66HkpPj7wSOUm63eTc5pzNVF774EfGpiPgq8ApZI+kU8nimIgPIGm3/PCak8v//JsrEeUScRWb3/KRjwWMWcr5mRXK/mDmBq8tw6WhgsXJRvr2xRg7OPBq3Cn7xMqSNpP9WEjb2dfCYOMNyCCsipiBXkm8D7CfptPL8VRHxXTKD5BCyPtTr1V478KWZHrhI0uWRK81vAg6MiKWU2+p+UC3dP3s8J+VYMsPlcPLicwtwn3LV/FNUcvC7qvQGf0Yp7QE8ohZuPfsWXibnNw4gj2EjZdHAj5FzIQ9Au8+dyHT9wyJiXzJbcSvgvoj4u6TtlOnUr38/23wsbTOseiCVO9bpyoV0DHmh6tkamEPSK8Aeku4c6jZOjN7xRMRi5Y5pDLBWRKwp6RVJD5In+PvLn7TyLnBAttUmkXtGjyDTc/embM4F/Ky892fqwErfyufzvsgFZv9DucnV45LO7WDwQFlW5S/kZ7S/cp3RJ8gV9eerA/W6yHPkRrIg5/vIxZybA3NHxPbgoFHXsAkglXHnVciFgZBf/J9E7iAIuTva/KVb3vovTDmedchsnlkl3QfsBuwUEVtHLuZakhyDbu1JUAke2wFfI4d1zgE+VYL4muRY+tPkZ9QJ5fP5P3I4cYHqa9Xht9JbbP0Q6Zs4BTgd2D0iDgOOJm/Arm22WW8uciEnksYA15OB4xVJT5PftSMpuwlaTWrBvrpv1w85YXk3eWHqPbcXcAdZWPAOYM2m2zkRx/MecrvWJSvPTUPWG7qUXCi4ftPtHMRxBFnE8ZdkUbotSvtHUPbBJjNgpm26rRN5XEuRG1gtUR7PAYwa8J7eft/vIC+80zTd7prHOj2ZUbYe8JGm2zPINv8M2K7yeAvgb+TNGOT+Hqd29TNpw89wmwNZGjhA0qURMVI5fHBwRFxEzvccL+nGDk2SvUbeKT0IWQJbWVbhQuDCGLcnQeuOZzxjymMi4m9kEJma3KhLEbFLRFyljiw8G49TgPeXdOTNgD9HxDGSroqIKZVZfrMAJwOHqKNlMZRDWa3aMXAQziCrAAC5k2MZjfh5ZHn2pcpLnVtn1BbDZgirGAGs2wse8Hqpj6ckXa+y73fbLrZv4hWyKOIHIvcufzUilouIb0fETBq3J0GrjmfAnMciJVUXMmlhbnL4Q5F7k2xJlsJovcqcx4yR29I+QK7J+TxZwHJL8vOaE6AEj3eQweMAZWUAGzp3AptGbtTVcxRZEeAisiry7mpp4kkXDLceyJnkCb15jNu17ghy3P2Bpho1GOPrRUh6NCJuJsth3BMRj5JzIF+V9GwT7XwrA4LHTuTq3jsi4mHlwsf3AHuUC/Ds5Ba79zXX4sErQW89slzHNGQBzkPJkv9jyzqPlYCz4fW5j1OBgyRd1Uyr+1P5Hj5cUsWPj4gXJZ0o6V8RcS6wELlr4rPRhzsJvl06vZBw4EW3nLAbkusIFiZP8oMkndlMC99aRExLlod+dcDz1QvxOuQk7bvIzJfW38mWNM9tyOyql4ATyWrBX40sc/4eSjXXBps5USJrqR3JuNXX+5KFKg+LLCJ4EPDd6vctImZTR4oKdtmA86WXUNMb4l2NrEH2HeXWwUHOTb3SxuHfLulUAInc0GVZciLsbknPVL4s1S/QFOQF9yW1uC5PuWM9EBhJzg1cocwY6b0+cP/o1tfpgtdXl59BJjRsXe7yZiBLmv9H0vqNNrCG0nP6Hrmh1doaV4DvfGA74DbgnZLu7A11tfkz6heVc+YjZKmcc4EXJB3ecNOGhc7MgURuxnM68FlgB2DPMg/wP4t/lBVc71O791tehAwaJ5EXoc+SaYZULkDV4DFifMfaRsq9VPYjh6hWKPM3zwMbAIrcIrT1qmm3kh4gM8dGAmtGxMzKFeW/BGZTrvW4s7xXbf+MhouIWLzy++4RcWxEfDYi5oPXhx1HKHdEXJ+c++gFFJtEnZgDKXe0V5JVZ/9QuqTbkEM6/7M3cSX7ZYSkV9t2MkeWgTiU7CGdUp6bkcziOXdgeytd8ZmApSRdPuSNnoABPb8plYs0kXR6Oc6dgSki4mJJz0XEZ9r2eYxP5c51VTKT5xVJPy6923WBZSLievJ7+Lkm29qvIiseHxUR15DXh5WBS4BPAotExG/KzUxva4B/kIUgu5rx1zpd6YG8Sq4feD+AchOoqcltNN+gXGx72S9nlxTKtnkV+C7wUEQcWJ57AniqzOO8rhI8ZgHOIveQaIVyUvaCxxeBVaIAUG5p+lOyN7JSea71wQNev3NdA/gOucL/axFxkKSfkAtUVwLWBrZRlpbpyrk0bChTiz9HznfuD3xZ0mFk1ttIYLOIWKh0CJ2qOxl0ogci6cHIQoJ/KBfYR8iqrW+o/jngYvs7Mu/+maFu74SUCfNXyQVzV0fEWHJV+e/JgPiZ3h18ef8UleM5BdhHuctdK2hccbpVyAvqzgMnMpVVW19hPD3FDlgL2IhccDYG+AWApP9XjulDwCsRMa2k1gT24a46NyjpvsjSOBeQNyqfKwH9NTKhZu2IOELOsposWjuJHrngZ2ng3cCNZehqQbKrOh25mvSVqOyfXf7uHWTwOFAtSp0scx77kSt6RwKXS/p2ZDmSQ4HbJO1Q3jtF5eI8A7mr4K5tOZ4Bw1YLkKVUfi5puzLfMXbg+7ooIn5M3mS9D/iKpHsiYiPgRUnnRMQBwEzAN8vdsE1mA757XwBul/Sn8j08BrhZ0i7l9Y8Bd6qDm3d1hlqwHH7gDzlU9Rcy9e54ciOovcjqrfORwz3bjOfvpiD3llip6WMY0K6FyWqzW5MVTT9Gbsjzg/L6R8hd0A4dz9+OppTKaNsPMF/5d1OyaOXHep9D0217m45vnXJcnymPPwbcBXyi8p5Zm25nP/6QN2OXA++pPLcAuWPi0QPeG023d7j+NN6A8XwxFib3kN6s8txyZLbSt8rjRcuJvcd4/n76po9hQHveWwLe6uVxrzbSXMBDZIVTyPTkk4AFB/x96778ZG2rxYDLgLXKc1uS27UuVx4PlyDyufI5HVVuAnrHO6LptvXrD9kjvBSYqjyeovLaguRw7yxNt7Mfflo1B1LmCI4EbpF0QnluCknXlInZYyLiBkkXRcSHybIYb6D2DSW8Ss7X9FJXXyvDPI9FxJpkXZ6jyHLTX9KAORuVs6JNSptuj4gzga0j4jVJvyrzAldFxLJq0VzNhERuJPSaNOG1NZJ+HRG3kjcsU0q6o7zXY+pDZDyfzZRkuZg5yD1WesO9C0i6NyI2VktrxA03rQogkl6MiKOBL0bWSTpH0kvli3B1RFwCrFZSQu8E7mz7l0TjEgCuiIhZJR0KjI2IqcmMqv8AY5Xbav63ybYORkSsRPaSfibph5H7fu9QPocTyuOnG27mWyopxksD90bEksDMEXHSBILIbQP/nA5sBzAcDJgPHE0WFv0XmWm1bkScJunxyJIly0TENpTzqM3XheGiFQGkTHxPB/xDmbUTwFfIBT/nkBvCiMyEebz6xWjjl6RM9m9GznNcV+5aP0FmkSHpUOVWmnOTd7Yjmmzvm4mIeclAN72kh8kSJMtExFhJx0k6KiLeR24GtaWkkxtt8MQZDXyDnHP7/IS+S5Xsvt66IqeEDpFK8NgX+ChZ0+5FssrB+8hU/auA1YA11NFqx13VeACJrC90Enln8e6IGC3pd2U4ZHtyfPPkyJWjm5AT0a0VuWL+eHK8fBSwTkR8rQSRFYArI+Jf5PzBT8gMnlbWSoqIdYE9yT3k54qI08gkhZeA5cvd4bHk1qYLktVPW6+0e2y5OfkCuTXww1FZCFl5bzU1fPeIOHTgMKNNXhGxOrkHyRoRcT5wr6SjIysaXEFmNR5WhoWncIAfOo2m8ZY79VOB75ex5lPJ3samZVx6I+CL5KZK6wF7SzqnsQa/hXK3fjWwm6RTImt3fR34iaS7y3sWI1fCTgmsK+mCNg7DRcSK5IY8m5C1x+YgM8XOJbczXZHcoEfk/M4G6kBV3d7/63J885OBfFvyInSKpOsiYlbgGYASPGYmK+zuI6lre2J0XpkrXJwcOlxK0qfL88tIur7yPgePIdbY6tmImIosc/E3svAe5In8CGWbyTIccjgZPPZS5t63eVvQZ8k5jcUg5z/IdSy7RsT2EbGkpNvJu/XVJV1Q3teq4FEsC/xQ0k1kyZW7yUV165MZcieSw4xn0pHgAa+vMF+b/F79Q9JDwGHkubB+ROxBpofOV+l5nEn2FB08JrMY/4r+O8k6aqtVgsdewI5lLgt4Y+04GxpN90CWJPeLuIsczvkycDBwDTAjuSDwLOBRSS+08U69pzLUMStZsO0iMjjuRhZLnJ5cV3Au2UP5dy8YtumYKnfoRwH/lLRfaWdvVfzi5J7zG0p6otnWTrzI+km/Inu910ZZiFoCxWZk8L9Q0lnl4vRr4EhJVzbX6v4TEduScx2PS7okInYj59/GAM+Tq8w3LDcA1pBG50Ak3RwRR5LVdY8l14B8gMy735ocGplT0j3l/a250A5UmWT9VxmzPZM8hkUlPQVQLsovqKyMbePxVNp0KlnxeClJN0WESq9xDLnrXtvSpQdLZJXgUeVx7651pKQj441lMsZGxC5qcVXn4WJAttUmwO7kHvI7RMQ7JH0vIpYnq1a/QvZ6Hw5vBtWoVpQyKWmuu5FZSwcrS3/TxS/HgEnXi4ELJO3bcLMmWrlT343MjjtZZTvgMi/1ZWD9LkwmV3pU05Lf9/+UlM85gFMl3Ra57fH+5N4lD5e/69x3r6uqIwtljvCDwA3KOlcrAT8Avifp+AF/58+oYa0IIACRCwO3J7OxTpJ0b7MtenMRMUMv0I3ntepw1tXkkMjXh7aFk65kuXyRLJR4LZlfvwGwiaQ/N9m2wagEj3XJFeXTAd8me1Abk+W/byCHFneUdF5jje1TA4LHt4DPkPXFjgR+odyMbCVy2HH3MvdmLdGaEtSSbiFLfy9EdlFbK7Iw4rkR8eEyrPMG1eEs4OPkcFDnSHqULPT4LfIzeYrsebQ+eMDrE+arA3sDO5LDb6eTiQ3fJVOU/0gGRAePBlSCx5bk/j4fJ4PHQuRmZNNLuowM+K3ZB8fSkPRAJmbyO3Knt39P7N8NlbLO40TgOEk/HM/r1Tuq17vYbTyWflAmY28la499nQzm+wFf891sO0TEKOAq4D5Ja5XndgYWIZNRzlMpl+9hq3aZ7AGkMoywGrkBzz+Aa8aXElkZ+vmfBV1NK5lIU5HZYn+W9KPIWkrTkSti/6nc8az6N73jmZ7cL/vvQ97wPhVZJv9hZZmLWYETgD0l3Rq5gPC9wApq6SLOfhO5UPgE4BhJ3y3PfQuYhSyi+lKDzbMJmOxDWJVhhG+Ti7GWBL4cb77z3gHRsp0Elf4LPEbWTZqBvJP9GXAhcGBErNd7/4DjuQKYecgb3YcqQ4rfJBdBQg693Q8sFxErk4sEt3LwGHrVdVy938tN5g3kMNWny1ocJB0E7Ofg0V5DNQfyIfJkngKYl1wH8UpEzAZvuNjOTK77uKhNGT4R8a6IOKU8vJK8e72XXB17FrA68DDZ5e7tDd47nlOArysXENrkN0f591LKanKy53gtucbj52RWmffFHmJR2QIZxs1/VP69GfgSWUz1q+W556tBx9plsg5hRda1urHcUWxArtLeVNKjkftNvwc4VllxdxZyRfrekq6ebI2qKSJuB+6StEFEzAQsL+n8ykK0bcnqrttVeh5nkyvoW3c8w1EZqrqYrM31UXJ48UtkcH+WLJMzu6RHPCc1tAbMDW5Pnvt3AjdJunXAexcDxkj655A31CbK5A4gJwHXk8UFTye/LDtFVqY9GviqpEujIyt+I+Ia4G+SPjfg+RXIzJFdJF1cnlsHeFbSH4a8oX2kMsf2EbK21Qhynm1JslzJmeSCWQFbtqln248iYk9gBeDHZIbfjyX9dALvdW2rlntbV6KP567ueGBZSWMiYgfgxxFxFrkR1M6SLoV2rviNiNnJMt83adwmVf8E1oyICyWtXhanrQkcQOaoX9z7fyDp7Iaa3lcq6zz2AS4AlgB+LemIiHg3uZPgfcAHHDyGXrxxhflU5BDjusDXyKyrn0bESGBqSc9W/9bBo/3eljmQiJg1Iubp3QmWRYEA1wErR8QXJd0iaTlgO7IK7Xnlb0cAtCl4FFuQwx+LRsS0EXEG8FdJswLTRMQZJbXwZrK44LkeFhl6ZahwY7I68C3kTnVXl17taGCxciHyHNQQGxA8ViOvN2PJtOrlJK1X3voVYJkm2miTZpIDSERMR+6HPWW5I18cOD0ivkzWsvoCuSBoXgBJj1fTXduW012yq5D0feA2sp7V5WRK6DfLayuSe5ecL+l+5SLIVta26gMvk/MbB5DrPDYq368lybmQB8CfTRMqweObZFmcWcny+Q+QQ4tExFfIm7U7mmmlTYpJHsJS1hY6ltxP4evkavIbyPIX3ye3N52dzL5qWy/jDSJiITId9xLgLEmHRMQuZNZVb4/23hDVkhHxsSbba6Cs0vwX8i52R0n3lzm2I4FtesHdmhERnwTWAj5VPqv/AKcBG0XEZmTZkvUk/cNzHt0zSZPoAzIrViFLLN9PrtJ+PCLeCexLbje5k6QL34Y2TxaR5UlOIAPgH1QqAJfX9iCzRk4DrijrQawlImIOckx9GeDP5ILVneXyJI0ra6M2L9mL1coMM5AJD69Jei68wryTageQSvbLPMqaSUTEUmTRusfJSqf3Rm4QM5OkZ9o6R1DSci8kV8EeU3l+TWBsyRTbiRwW+Q1waRuPo59FrvYfDbyD3D/mhoab1HcGzHmMLOn58wJHAD+XdFF5bQdgWo1bcd7K64K9tUntgawJ7EXWsXkK+BGZubQFOS59glpeVRdev4M9RqUOT3luZ2AnshbPpcp92vcGzpAXBZq9wYDRiN3I2mOPk1V0NwPmIxd03l8er9PCxBmbSLUn0SM3d/kO2eOYidzo5RDgHrLY4KyM26yn7Z4GRpYeVM/zZLrhhcCyZXX5gQ4eZuM1Bby+zmNlssTPPuTQ4lXAb4HngBnIis6PxPi3r7UOmagPcEBJgVnJoLEQOfa8L7mQ6xByi9q9JP3t7Wnm2y8i5ispx0uVOY0HgVUjYmoASUeXCdgXyON6Z2ONNWupiFgrIuZWVl+Yjyzn81lyPvQKsnzM18j95/eUtL+kh8qcR1duMG0CJiqAlDmPT0bE58migo+RE+RfkHQueSf/TuBdkp5721v7NoksyX4uuc7jwMh9vo8gN7PZrbxORHycLAL5Y3Vw/2+zIbAi8JeImEu5m+P2ZCr/CpLWJLer/jSwYW/NF7Qvfd/qGVQab2XCfBmynPmdwPJk7vbHgEfLnfsi5Lagd06uBk+qiFgUOA7YR9KZEXEAMLekCyJiLbL8xcdL93o2YH9JlzTXYrP2krRLRLwE3BARH5H0WES8Rg4JzwEsR9a4+42DxvAz6En0yP0VeiU7bouITcmhnbnJXcT+Bpwo6ZQJ/1eaVYbgjgLWljRPee5ucmHTbORWuoeVFMN5gRck/d1ZImbjVG4op5L0cnnuULLi9tIliHyPLC46M3m+/d3rPIafiQkgqwLnA3tI+n7kfh6fBRYGXgJ+Kumptl9sy8r5o8h0zynIsgr7kducXgPsIOnkptpn1mYDsq1mBZ6upO4eRs59LCXpyYj4IPC4shaeg8cwNFFpvJEVZr8DHCTpxDKmuTFws6S/TqY2TrKIeC9ZZO/VMmw1EjgGWEnSXJX37QfcLqmTe5ibTU4D1nnsRc5/3gY8IOnw8vyh5DzIApIeH/h3NrxM7CT62WRNm10jYktJr0o6oeXBYyFy06dlgT0iYlvlDmfbAJdFxOnlfQsDmwKeLDcbj0rw2JjcJG5H4I/AhyJi//Ke3YEDqVxbHDyGr1oLCUtP5BDgU2QXtZVfkDJhfgI5YX5ORGwOzAhcq9wbeySZr74EuWZlL0kXNNZgs5Yra6WuIM+pw8t84YfIjMZ9JD1Yea/Lkwxzk1LKZJSkMW9ze95WZbHjlZJ6i5xuAx4lV8neLmnzEkSOIyfQz2yqrWZtNL7hpzJMtTlZkv2BkpxyMbCrpD830U5rRu1qvG0PHgCSro6INSPifrKEwqmSDigpx7dFxF6SDo6IzcpCqFYnAJgNpQHFD9chtwi+UNLuEfEMcGFE7EquLh9F7gRpfWSybmnbFhGxMlnTaurKOO7WwCzKfT/MbAIi4lvkgsE7yPVfO5FzH/uT8yCnA1+U9F9PmPeXvqhFI+n3wDpknS4i4n1kMsBfmmyXWRtFxMcj4vre78AnJa0MPAlMD3yDDCT7kufRwmQPxBPmfaYveiA9EbE6ebf0ALCLWrw/iVmTIuIyciuDNSK3B94aWFXSehHxS3Lx8BbkttXfAUZJ2rq5FlsTJnlHwi6RdGFErE3uT+LgYTZAbwhK0koRcVFE/EHSJ8u8Ya8S9Q3kvMddZc5wzwGFVq1P9FUPpMoT5mbjN2DB4IXkjeaBwFeBacgtqtdX7jpafa/PqT7TtwHEzCZsQGA4m6wNtxWwAnCWa1sZOICY2QQMCCJXA/dL+lx57EWC5gBiZhM2IIi0fvGwDS0HEDN7UwOCiIet7HUOIGZmVktfLCQ0M7O3nwOImZnV4gBiZma1OICYmVktDiBmZlaLA4iZmdXiAGJmZrU4gJiZWS0OIGZmVsv/B8+uj7cyiWhlAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "\n", + "delta = 1e-5\n", + "\n", + "\n", + "\n", + "eps0 = gm0.approxDP(delta)\n", + "eps1 = gm1.approxDP(delta)\n", + "eps1b = gm1b.approxDP(delta)\n", + "eps3 = gm3.approxDP(delta)\n", + "eps2 = gm2.approxDP(delta)\n", + "\n", + "eps4 = laplace.approxDP(delta)\n", + "\n", + "epsilons = [eps0,eps1,eps1b,eps2,eps3,eps4]\n", + "\n", + "print(epsilons)\n", + "\n", + "plt.bar(label_list,epsilons)\n", + "plt.xticks(rotation=45, ha=\"right\")\n", + "plt.show()\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 4: Privacy amplification by sampling" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism\n", + "from autodp.transformer_zoo import Composition, AmplificationBySampling\n", + "\n", + "sigma1 = 5.0\n", + "sigma2 = 8.0\n", + "\n", + "gm1 = ExactGaussianMechanism(sigma1,name='GM1')\n", + "gm2 = ExactGaussianMechanism(sigma2,name='GM2')\n", + "SVT = PureDP_Mechanism(eps=0.1,name='SVT')\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [], + "source": [ + "# create transformations: amplification by sampling\n", + "poisson_sample = AmplificationBySampling(PoissonSampling=True)\n", + "subsample = AmplificationBySampling(PoissonSampling=False)\n", + "# compose them with the transformation: compose.\n", + "compose = Composition()\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [], + "source": [ + "# Now let's construct three different randomized algorithms that compose over these calls\n", + "# run the first mechanism for 30 rounds\n", + "# run the second mechanism for 50 times\n", + "# run the third mechanism for 10 times\n", + "coeffs = [30,50,10]\n", + "\n", + "# when sampling, let's say the prob is the following\n", + "prob = 0.1\n", + "\n", + "composed_mech = compose([gm1, gm2, SVT], coeffs)\n", + "\n", + "composed_mech = compose([gm1, gm2, SVT], coeffs)\n", + "\n", + "composed_poissonsampled_mech = compose([poisson_sample(gm1,prob),\n", + " poisson_sample(gm2,prob),\n", + " poisson_sample(SVT,prob)],\n", + " coeffs)\n", + "\n", + "composed_poissonsampled_mech1 = compose([poisson_sample(gm1,prob,improved_bound_flag=True),\n", + " poisson_sample(gm2,prob,improved_bound_flag=True),\n", + " poisson_sample(SVT,prob,improved_bound_flag=True)],\n", + " coeffs)\n", + "\n", + "\n", + "\n", + "# Now let's do subsampling. First we need to use replace-one version of the base mechanisms.\n", + "gm1.replace_one = True\n", + "gm2.replace_one = True\n", + "SVT.replace_one = True\n", + "\n", + "composed_subsampled_mech = compose([subsample(gm1,prob),\n", + " subsample(gm2,prob),\n", + " subsample(SVT,prob)],\n", + " coeffs)\n", + "\n", + "composed_subsampled_mech1 = compose([subsample(gm1,prob,improved_bound_flag=True),\n", + " subsample(gm2,prob,improved_bound_flag=True),\n", + " subsample(SVT,prob,improved_bound_flag=True)],\n", + " coeffs)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Mechanism name is \" Compose:{GM1: 30, GM2: 50, SVT: 10} \"\n", + "Parameters are: {'GM1:sigma': 5.0, 'GM2:sigma': 8.0, 'SVT:eps': 0.1}\n", + "epsilon(delta) = 7.942374861004205 , at delta = 1e-06\n", + "epsilon(delta) = 6.457460671942038 , at delta = 0.0001\n", + "---------------------------------------------------\n", + "Mechanism name is \" Compose:{PoissonSample:GM1: 30, PoissonSample:GM2: 50, PoissonSample:SVT: 10} \"\n", + "Parameters are: {'PoissonSample:GM1:sigma': 5.0, 'PoissonSample:GM1:PoissonSample': 0.1, 'PoissonSample:GM2:sigma': 8.0, 'PoissonSample:GM2:PoissonSample': 0.1, 'PoissonSample:SVT:eps': 0.1, 'PoissonSample:SVT:PoissonSample': 0.1}\n", + "epsilon(delta) = 0.8616498207977252 , at delta = 1e-06\n", + "epsilon(delta) = 0.645403895438424 , at delta = 0.0001\n", + "------- If qualified for the improved bounds --------\n", + "epsilon(delta) = 0.6689293610040936 , at delta = 1e-06\n", + "epsilon(delta) = 0.48950668976148426 , at delta = 0.0001\n", + "---------------------------------------------------\n", + "Mechanism name is \" Compose:{Subsample:GM1: 30, Subsample:GM2: 50, Subsample:SVT: 10} \"\n", + "Parameters are: {'Subsample:GM1:sigma': 5.0, 'Subsample:GM1:PoissonSample': 0.1, 'Subsample:GM1:Subsample': 0.1, 'Subsample:GM2:sigma': 8.0, 'Subsample:GM2:PoissonSample': 0.1, 'Subsample:GM2:Subsample': 0.1, 'Subsample:SVT:eps': 0.1, 'Subsample:SVT:PoissonSample': 0.1, 'Subsample:SVT:Subsample': 0.1}\n", + "epsilon(delta) = 3.161674646617909 , at delta = 1e-06\n", + "epsilon(delta) = 2.2643681643522973 , at delta = 0.0001\n", + "------- If qualified for the improved bounds --------\n", + "epsilon(delta) = 1.373662073623351 , at delta = 1e-06\n", + "epsilon(delta) = 1.0419345516946525 , at delta = 0.0001\n" + ] + } + ], + "source": [ + "# Get name of the composed object, a structured description of the mechanism generated automatically\n", + "# Query for eps given delta\n", + "delta1 = 1e-6\n", + "eps1 = composed_mech.get_approxDP(delta1)\n", + "\n", + "delta2 = 1e-4\n", + "eps2 = composed_mech.get_approxDP(delta2)\n", + "\n", + "print('Mechanism name is \\\"', composed_mech.name,'\\\"')\n", + "print('Parameters are: ',composed_mech.params)\n", + "print('epsilon(delta) = ', eps1, ', at delta = ', delta1)\n", + "print('epsilon(delta) = ', eps2, ', at delta = ', delta2)\n", + "\n", + "\n", + "eps1a = composed_poissonsampled_mech.get_approxDP(delta1)\n", + "eps2a = composed_poissonsampled_mech.get_approxDP(delta2)\n", + "\n", + "eps1aa = composed_poissonsampled_mech1.get_approxDP(delta1)\n", + "eps2aa = composed_poissonsampled_mech1.get_approxDP(delta2)\n", + "\n", + "# Get name of the composed object, a structured description of the mechanism generated automatically\n", + "print('---------------------------------------------------')\n", + "print('Mechanism name is \\\"', composed_poissonsampled_mech.name,'\\\"')\n", + "print('Parameters are: ',composed_poissonsampled_mech.params)\n", + "print('epsilon(delta) = ', eps1a, ', at delta = ', delta1)\n", + "print('epsilon(delta) = ', eps2a, ', at delta = ', delta2)\n", + "print('------- If qualified for the improved bounds --------')\n", + "print('epsilon(delta) = ', eps1aa, ', at delta = ', delta1)\n", + "print('epsilon(delta) = ', eps2aa, ', at delta = ', delta2)\n", + "\n", + "\n", + "eps1b = composed_subsampled_mech.get_approxDP(delta1)\n", + "eps2b = composed_subsampled_mech.get_approxDP(delta2)\n", + "\n", + "eps1bb = composed_subsampled_mech1.get_approxDP(delta1)\n", + "eps2bb = composed_subsampled_mech1.get_approxDP(delta2)\n", + "\n", + "# Get name of the composed object, a structured description of the mechanism generated automatically\n", + "print('---------------------------------------------------')\n", + "print('Mechanism name is \\\"', composed_subsampled_mech.name,'\\\"')\n", + "print('Parameters are: ',composed_subsampled_mech.params)\n", + "print('epsilon(delta) = ', eps1b, ', at delta = ', delta1)\n", + "print('epsilon(delta) = ', eps2b, ', at delta = ', delta2)\n", + "print('------- If qualified for the improved bounds --------')\n", + "print('epsilon(delta) = ', eps1bb, ', at delta = ', delta1)\n", + "print('epsilon(delta) = ', eps2bb, ', at delta = ', delta2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 5: Calibrating noise to privacy requirement" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "GM1 {'sigma': 2.904058395116707} 1.499999748603142\n", + "Laplace {'eps': 1.500001569629506} 1.500001346499671\n" + ] + } + ], + "source": [ + "from autodp.calibrator_zoo import eps_delta_calibrator\n", + "from autodp.mechanism_zoo import ExactGaussianMechanism, PureDP_Mechanism\n", + "\n", + "\"\"\"\n", + "Try calibrating noise to privacy budgets.\n", + "Cases with a single parameter.\n", + "\n", + "\"\"\"\n", + "\n", + "calibrate = eps_delta_calibrator()\n", + "eps = 1.5\n", + "delta = 1e-6\n", + "\n", + "mech1 = calibrate(ExactGaussianMechanism,eps,delta,[0,100],name='GM1')\n", + "\n", + "mech2 = calibrate(PureDP_Mechanism,eps,delta,[0,100],name='Laplace')\n", + "\n", + "print(mech1.name, mech1.params,mech1.get_approxDP(delta))\n", + "\n", + "print(mech2.name, mech2.params, mech2.get_approxDP(delta))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Example 6: Four different ways of implementing NoisyGD (or NoisySGD)" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "2.067735846469756" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 1: Short implementation of noisy gradient descent mechanism as a composition of GMs\n", + "\n", + "class NoisyGD_mech(GaussianMechanism):\n", + " def __init__(self,sigma_list,name='NoisyGD'):\n", + " GaussianMechanism.__init__(self, sigma=np.sqrt(1/np.sum(1/sigma_list**2)),name=name)\n", + " self.params = {'sigma_list':sigma_list}\n", + "\n", + "# The user could log sigma_list and then just declare a NoisyGD_mech object.\n", + "sigma_list = np.array([5.0,3.0,4.0])\n", + "\n", + "mech = NoisyGD_mech(sigma_list)\n", + "mech.get_approxDP(delta=1e-6)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "from autodp import mechanism_zoo, transformer_zoo\n", + "\n", + "\n", + "# ----------------------------------------------------------\n", + "# 2: Implementing NoisySGD from basic building blocks\n", + "\n", + "subsample = transformer_zoo.AmplificationBySampling() # by default this is using poisson sampling\n", + "mech = mechanism_zoo.GaussianMechanism(sigma=5.0)\n", + "prob = 0.01\n", + "\n", + "# Create subsampled Gaussian mechanism\n", + "# Gaussian mechanism qualifies for the tight bound\n", + "SubsampledGaussian_mech = subsample(mech,prob,improved_bound_flag=True)\n", + "\n", + "# Now run this for 1000 iterations\n", + "compose = transformer_zoo.Composition()\n", + "noisysgd_v1 = compose([SubsampledGaussian_mech],[1000])\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "# ----------------------------------------------------------\n", + "# 3: You could also package this together by defining a NoisySGD mechanism then put it in the Mechanism_zoo\n", + "\n", + "from autodp.autodp_core import Mechanism\n", + "\n", + "class NoisySGD_mech(Mechanism):\n", + " def __init__(self,prob,sigma,niter,name='NoisySGD'):\n", + " Mechanism.__init__(self)\n", + " self.name=name\n", + " self.params={'prob':prob,'sigma':sigma,'niter':niter}\n", + " \n", + " # create such a mechanism as in previously\n", + " subsample = transformer_zoo.AmplificationBySampling() # by default this is using poisson sampling\n", + " mech = mechanism_zoo.GaussianMechanism(sigma=sigma)\n", + " prob = 0.01\n", + " # Create subsampled Gaussian mechanism\n", + " SubsampledGaussian_mech = subsample(mech,prob,improved_bound_flag=True)\n", + "\n", + " # Now run this for niter iterations\n", + " compose = transformer_zoo.Composition()\n", + " mech = compose([SubsampledGaussian_mech],[niter])\n", + "\n", + " # Now we get it and let's extract the RDP function and assign it to the current mech being constructed\n", + " rdp_total = mech.RenyiDP\n", + " self.propagate_updates(rdp_total,type_of_update='RDP')\n", + "\n", + "noisysgd_v2 = NoisySGD_mech(prob=prob,sigma=5.0,niter=1000)\n", + " \n" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "epoch = 1 , iteration = 100 , epsilon = 0.08216484696031025 , at delta= 1e-06\n", + "updated prob = 0.0101 , sigma = 5.041251663233121\n", + "epoch = 2 , iteration = 200 , epsilon = 0.11676903640602136 , at delta= 1e-06\n", + "updated prob = 0.010201 , sigma = 5.100462902312742\n", + "epoch = 3 , iteration = 300 , epsilon = 0.14319884487742168 , at delta= 1e-06\n", + "updated prob = 0.01030301 , sigma = 5.174026395200586\n", + "epoch = 4 , iteration = 400 , epsilon = 0.16516102028030705 , at delta= 1e-06\n", + "updated prob = 0.0104060401 , sigma = 5.260190733882066\n", + "epoch = 5 , iteration = 500 , epsilon = 0.18411555290130882 , at delta= 1e-06\n", + "updated prob = 0.010510100501 , sigma = 5.357936087232907\n", + "epoch = 6 , iteration = 600 , epsilon = 0.20082446740179327 , at delta= 1e-06\n", + "updated prob = 0.01061520150601 , sigma = 5.46662426274389\n", + "epoch = 7 , iteration = 700 , epsilon = 0.21575356337708634 , at delta= 1e-06\n", + "updated prob = 0.010721353521070101 , sigma = 5.585850175052086\n", + "epoch = 8 , iteration = 800 , epsilon = 0.22920955229405693 , at delta= 1e-06\n", + "updated prob = 0.010828567056280802 , sigma = 5.715361791737599\n", + "epoch = 9 , iteration = 900 , epsilon = 0.24140799650345754 , at delta= 1e-06\n", + "updated prob = 0.01093685272684361 , sigma = 5.855014072419222\n", + "epoch = 10 , iteration = 1000 , epsilon = 0.2525265414600684 , at delta= 1e-06\n", + "updated prob = 0.011046221254112046 , sigma = 6.004751393032068\n" + ] + } + ], + "source": [ + "# ----------------------------------------------------------\n", + "# 4: Online decision. Hetereogenous sigma decided online\n", + "# (maybe as a function of computed eps)\n", + "# Alternatively if we want to do it via composition, so we can make online decision about\n", + "# the sigma in the sigma_list\n", + "\n", + "delta = 1e-6\n", + "\n", + "compose = transformer_zoo.Composition()\n", + "prob= 0.01\n", + "sigma = 5\n", + "nepochs = 10\n", + "\n", + "online_ngd = NoisySGD_mech(prob=prob,sigma=sigma,niter=100)\n", + "for i in range(nepochs):\n", + " eps = online_ngd.get_approxDP(delta)\n", + " print('epoch = ',i+1, ', iteration = ',100*(i+1), ', epsilon = ',eps, ', at delta=', delta)\n", + " #determine the next prob, sigma (maybe adaptively), for example\n", + " prob = prob*1.01\n", + " sigma = sigma * np.exp(0.1*eps)\n", + " print('updated prob = ',prob, ', sigma = ',sigma)\n", + " # use the above class to descrie mechanisms you run on the fly\n", + " mech_cur = NoisySGD_mech(prob=prob,sigma=sigma,niter=100)\n", + " \n", + " online_ngd = compose([online_ngd, mech_cur],[1,1])\n", + "\n", + "# The above is quite general and can be viewed as a privacy accountant" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0.27105623043762284\n", + "0.27105623043762284\n", + "0.26269378582675573\n" + ] + } + ], + "source": [ + "delta = 1e-6\n", + "\n", + "print(noisysgd_v1.get_approxDP(delta))\n", + "print(noisysgd_v2.get_approxDP(delta))\n", + "print(online_ngd.get_approxDP(delta))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.5" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}