Research-Papers
These are the papers I find interesting, mostly focused around the intersection of security, privacy, and ML. I may also list papers relating to the fundamentals of ML/FL infrastructure, or topics involving AI alignment and fairness. There also might be non-papers in here! I am including whatever helps me grasp the concepts the easiest.
My current focus is the burgeoning field of FL. See OpenMined for a brief overview of the types of FL.
This list will be organized by topic and attack model (if applicable).
Table of Contents
Privacy
Defenses
- IBM (Cloud'22): DeTrust-FL: Privacy-Preserving Federated Learning in Decentralized Trust Setting
PDF
Security
Attacks
Model Poisoning
- (ICML'19): Analyzing Federated Learning through an Adversarial Lens
PDF
Github
- Attack Model: "Single, non-colluding malicious agent where the adversarial objective is to cause the model to mis-classify a set of chosen inputs with high confidence."
Defenses
Model Poisoning
-
Federated Learning based on Defending Against Data Poisoning Attacks in IoT
PDF
- Attack Model: "A group of p<n/2 malicious label-flipping poisoning attackers, where n is the total amount of participants’ clients."
-
(NeurIPS'21): FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective
PDF
Github
- Attack Model: "Clients mitigate model poisoning attacks that have already polluted the global model"
Vertical FL
- Vertical Federated Learning: Challenges, Methodologies and Experiments
PDF
FL Optimization
- Oort: Efficient Federated Learning via Guided Participant Selection
PDF
| OSDI 21 🎓 - (ICML'22): Neural Tangent Kernel Empowered Federated Learning
PDF
- Reduces communication rounds, addresses statistical heterogeneity by transmitting update data that is more expressive than simple model weights/gradients
- Fed-SNN: Federated Learning with Spiking Neural Networks
PDF
Github
- Optimizes for energy efficiency
- Swan: A Neural Engine for Efficient DNN Training on Smartphone SoCs
PDF
- (ICLR 2021): Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms
PDF
Github
FL Systems from big tech companies
Paper
Cross-device
- Apple: Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications |
PDF
,PDF
- Google: Towards Federated Learning at Scale: System Design |
MLSys21
,Github
🎓 - Meta: Papaya: Practical, Private, and Scalable Federated Learning |
MLSys22
🎓
Data Center Architecture
- Yarn:
PDF
- Omega:
PDF
- Tiresias: A GPU Cluster Manager for Distributed Deep Learning |
PDF
- Leap: Effectively Prefetching Remote Memory |
PDF
,Github
(USENIX'20)🎓- Two tricks: Prefetching pages wherever possible
- Using more efficient data paths that allow them to discard the operating system’s irrelevant disk-access features.
Surveys
- A survey on security and privacy of federated learning
URL
- Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
PDF
LLMs
- In AI, is bigger always better?
Nature
- Voyager, An Open-Ended Embodied Agent with Large Language Models
Website
- Vector Database of skills (GPT-4 Generated Code). Keys are descriptions, while the Value is the code of "skills"
- MemGPT: Towards LLMs as Operating Systems
PDF
- LLMs are constrained by limited context windows, hindering their utility in tasks like extended conversations and document analysis
- MemGPT manages different memory tiers to provide the appearance of large memory resources through data movement between fast and slow memory (similar to traditional OS virtual context management)
MLSys
- Hidden Technical Debt in Machine Learning Systems
NeurIPS PDF
Other FL paper lists
- https://github.com/AmberLJC/FLsystem-paper/
- ***https://github.com/innovation-cat/Awesome-Federated-Machine-Learning
- https://github.com/chaoyanghe/Awesome-Federated-Learning
- https://github.com/weimingwill/awesome-federated-learning#resource-allocation
- https://github.com/youngfish42/Awesome-Federated-Learning-on-Graph-and-Tabular-Data#federated-learning-framework