Skip to main content

2023 NeurIPS

NeurIPS 2023 | OpenReview

neurips2023.vizhub.ai

Paper Digest: NeurIPS 2023 Highlights – Paper Digest

Papers​

Computer Vision​

Transformers​

![[Pasted image 20231211100815.png]]

2023-12-10 - Sunday​

Schedule

SMALLER MODELS CAN PACK A PUNCH IN THE ERA OF LARGE LANGUAGE MODELS​

11am - 11:50am | Room 206 - 207

  • smoothquant
  • multiLORA
  • text2sql demo

OPTIMIZING AND REASONING ABOUT LLM INFERENCE: FROM FIRST PRINCIPLES TO SOTA TECHNIQUES​

12-12:50 | Room 206 - 207

  • memory bound
  • kv cache
  • prompt in parallel
    • compute bound
  • decode memory bound
  • ideas
    • decompose attention into two steps, block sparse, read less
    • more lower dim tokens
    • predict multiple tokens at a time
      • use position encoding in decoder step, predict token at step x

CONVERSATIONAL RECOMMENDATIONS: PRESENT AND FUTURE​

1pm | Room 208 - 210

  • user history / profile summarized with LLM to include context in conversation

BUILDING USING LLAMA 2​

2pm - 4pm | Room 208 - 210

  • intro

3-3:50pm

THE FUTURE IS HERE: A DEEP DIVE INTO AUTONOMOUS AGENTS​

4pm-4:50 | Room R06-R09 (level 2)

COMMUNICATION WITHOUT LANGUAGE BARRIERS: RECENT ADVANCES IN TRANSLATION FOUNDATION MODELS​

4-6pm

EMPIRICAL RIGOR IN ML AS A MASSIVELY PARALLELIZABLE CHALLENGE​

5pm - 5:50 | Room R06-R09 (level 2)

Ballroom A - C (level 2) Sun 10 Dec 12:30 p.m. - 2 p.m. CST
Sun 10 Dec 3:30 p.m. - 5 p.m. CST

2023-12-11 - Monday​

Language Models Meet World Models​

9:45 a.m. CST — 12:15 p.m. CST

https://sites.google.com/view/neurips2023law

  • theory of mind
  • simulators
    • next frame prediction

Latent Diffusion Models​

9:45 - 12:15 | Hall E2 https://neurips2023-ldm-tutorial.github.io/

Contributing to an Efficient and Democratized Large Model Era​

9:45 - 12:15

Application Development using Large Language Models​

1:45 p.m. CST — 4:15 p.m | Hall E2

Reconsidering Overfitting in the Age of Overparameterized Models​

1:45 p.m. CST — 4:15 p.m | Hall D2 https://sml.inf.ethz.ch/gml23/neuripstut-blank.html

NextGenAI: The Delusion of Scaling and the Future of Generative AI​

Hall F (level 1) | 5:25 p.m. CST — 6:15 p.m. CST

Welcome Reception​

Hall C & D (level 1) | 6:15 p.m. CST — 8:30 p.m. CS

2023-12-12 - Tuesday​

The Many Faces of Responsible AI by Lora Aroyo​

8:30 a.m. CST — 9:20 a.m. CST

  • data quality
    • sources of disagreements
      • ambiguities
      • context (cultures, differences by country / location)
      • diversity of real world

Oral 1D DL Theory​

Room R06-R09 (level 2) 10 a.m. CST — 10:45 a.m

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization​

Abide by the law and follow the flow: conservation laws for gradient flows​

A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning​

Oral 1B Datasets & Benchmarks​

La Nouvelle Orleans Ballroom A-C (level 2) 10 a.m. CST — 10:45 a.m.

Poster Session 1​

Test of Time Award Talk:Distributed Representations of Words and Phrases and their Compositionality​

Tomas Mikolov · Ilya Sutskever · Kai Chen · Greg Corrado · Jeff Dean Hall F (level 1) 3:05 p.m. CST — 3:25 p.m. CST

Oral 2A Efficient Learning​

Hall C2 (level 1 gate 9 south of food court) 3:40 p.m. CST — 4:40 p.m

Poster Session 2​

Great Hall & Hall B1+B2 (level 1) 5:15 p.m. CST — 7:15 p.m.

2023-12-14 - Thursday​

ML for Systems #star​

  • SSMs #TODO

#TODO mamba 4 rec

Oral 5D Vision​

Room R06-R09 (level 2) Thu 14 Dec 10 a.m. CST — 10:45 a.m. CST

Visual Instruction Tuning (Poster 229)​

  • LLaVA. #star

DataComp: In search of the next generation of multimodal datasets #star​

  • check CLIP embeddings provided #TODO

  • check eval pipeline #TODO

  • baselines #star #TODO

    • clip filtering (threshold on CLIP score)
    • image cluster based filtering (matching images in target dataset)

  • smaller filtered datasets work better than larger noisy ones

  • Leaderboard

Oral 6C Vision​

Room R02-R05 (level 2) Thu 14 Dec 3:20 p.m. CST — 4:20 p.m. CST

Siamese Masked Autoencoders​

Image Captioners Are Scalable Vision Learners Too​

The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation​

Spatial-frequency channels, shape bias, and adversarial robustness​

2023-12-15 - Friday​

Foundation Models for Decision Making — Hall E2 (level 1)​

  • Fri 9:30 a.m. - 10:00 a.m. - Percy Liang ( Invited Talk )
    • memory stream
  • Fri 10:00 a.m. - 10:30 a.m. - Ruslan Salakhutdinov ( Invited Talk )
  • Fri 10:30 a.m. - 11:00 a.m. - Jürgen Schmidhuber ( Invited Talk ) %% - Fri 2:00 p.m. - 2:30 p.m. - Chelsea Finn ( Invited Talk ) %%

Table Representation Learning Workshop — Room 235 - 236​

Workshop on robustness of zero/few-shot learning in foundation models (R0-FoMo) — La Nouvelle Orleans Ballroom A+B (level 2)​

NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day — Room 356​

  • 1:30 - 4:30

NeurIPS Large Language Model Efficiency Challenge:1 LLM + 1GPU + 1Day | NeurIPS 2023 Challenge

Instruction Tuning and Instruction Following — Room 220 - 222​

NeurIPS 2023 Workshop on Diffusion Models — Hall B1 (level 1)​

OPT 2023: Optimization for Machine Learning — Hall D2 (level 1)​

2023-12-16 - Saturday​

Practical Vector Search (Big ANN) Challenge 2023 — Room 356​

4th Workshop on Self-Supervised Learning: Theory and Practice — Room 217 - 219​

  • Schedule | NeurIPS 2023 Workshop: Self-Supervised Learning - Theory and Practice

  • 9:45 - Diane Larlus - Visual representations that transfer

    • stable diffusion for synthetic datasets (imagenet)
      • not close to supervised
    • DINO + NeRF = N3F

  • 10:15 - Alyosha Efros - The Revolution Will not Be Supervised... 8 year later

    • should move away from data augmentation
    • MAE / MIM seems like the current winner (thanks to transformers)
    • visual prompting via image inpainting
    • LVM - large visual model (new paper) #star

  • 11:00 - Abhinav Gupta - Self-supervised Learning: Towards Rich Representations?

  • 11:30 - Chelsea Finn - Updating Language Models without Human Supervision

  • 12:00 - 1:00 - Posters

Third Workshop on Efficient Natural Language and Speech Processing (ENLSP-III): Towards the Future of Large Language Models and their Emerging Descendants — Room 206 - 207​

Machine Learning for Systems — Room 211 - 213​

Workshop on Advancing Neural Network Training (WANT): Computational Efficiency, Scalability, and Resource Optimization — Room 243 - 245​

Synthetic Data Generation with Generative AI — Hall E2 (level 1)​

  • scaling
    • foundational models for X
  • LLMs
  • diffusion models
  • efficiency
  • multi modality
    • CLIP
    • seq 2 seq
  • transformers