# luckmatters

**Repository Path**: facebookresearch/luckmatters

## Basic Information

- **Project Name**: luckmatters
- **Description**: Understanding Training Dynamics of Deep ReLU Networks
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2023-07-24
- **Last Updated**: 2025-04-13

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

This repo contains the code of the following 6 papers:

## Analysis of Self-supervised learning (`./ssl`) 
**Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning**

Yuandong Tian

[arXiv](https://arxiv.org/abs/2206.01342)

**Understanding Deep Contrastive Learning via Coordinate-wise Optimization**

Yuandong Tian

[NeurIPS'22](https://arxiv.org/abs/2201.12680) Oral

**Understanding self-supervised Learning Dynamics without Contrastive Pairs**

Yuandong Tian, Xinlei Chen, Surya Ganguli

ICML 2021 [link](https://arxiv.org/abs/2102.06810), *Outstanding Paper Honorable Mention* 

**Understanding Self-supervised Learning with Dual Deep Networks**

Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli

arXiv [link](https://arxiv.org/abs/2010.00578) 




## Teacher-student setting in supervised learning 
**Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension** (`./student_specialization`) 

Yuandong Tian

ICML 2020 [link](https://arxiv.org/abs/1909.13458)

**Luck Matters: Luck Matters: Understanding Training Dynamics of Deep ReLU Networks** (`./luckmatter`)

Yuandong Tian, Tina Jiang, Qucheng Gong, Ari Morcos

arxiv [link](https://arxiv.org/abs/1905.13405)