2024-09-18 - Pytorch Conference Notes

Pytorch Conference

torch.compile automatically does some recomputation
By default, torch.compile will use the min-cut partitioner, which chooses to recompute certain ops with the objective of minimizing number of tensors saved for backward boundary
It’s primary objective is to reduce runtime however, so it is relatively conservative wrt recomputation, e.g. we only recompute fusible, non-compute intensive ops, and have a heuristic to avoid long fusible chains.

in 2.5 a new checkpoint API will allow setting checkpointing policy 2.4 has a new compile only memory budget API to trade off memory for speed

torch._dynamo.config.activation_memory_budget = 0.5 
out = torch.compile(fn)(inp)