FOOGD

This repository is an official PyTorch implementation of paper: FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detection. NeurIPS 2024 (Poster).

FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detection

Abstract

Federated learning (FL) is a promising machine learning paradigm that collaborates with client models to capture global knowledge. However, deploying FL models in real-world scenarios remains unreliable due to the coexistence of in-distribution data and unexpected out-of-distribution (OOD) data, such as covariate-shift and semantic-shift data. Current FL researches typically address either covariate-shift data through OOD generalization or semantic-shift data via OOD detection, overlooking the simultaneous occurrence of various OOD shifts. In this work, we propose FOOGD, a method that estimates the probability density of each client and obtains reliable global distribution as guidance for the subsequent FL process. Firstly, SM3D in FOOGD estimates score model for arbitrary distributions without prior constraints, and detects semantic-shift data powerfully. Then SAG in FOOGD provides invariant yet diverse knowledge for both local covariate-shift generalization and client performance generalization. In empirical validations, FOOGD significantly enjoys three main advantages: (1) reliably estimating non-normalized decentralized distributions, (2) detecting semantic shift data via score values, and (3) generalizing to covariate-shift data by regularizing feature extractor.

Implementation

This project backbone is contributively implemented by my co-authors, @Pengyang, @Fengyuan and @Jiahe.

dataset

Here are links for the OOD dataset used in project: Textures, Places365, LSUN, LSUN-R, iSUN.

You can download corrupted CIFAR-10 dataset CIFAR-10-C via this link CIFAR-10-C.

Download and unzip required datasets in your own folder /path/to/dataset (don't change folder's name if you don't want to adapt codes)

Noted: CIFAR-10 and CIFAR-100 can be downloaded via torchvision. You don't have to download them by yourself if you want to use them. Specifying where to save them is all you need.