fine-tuning

Fine Tuning / Transfer Learning

Parameter efficient transfer learning

• [2303.15647] Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning

• GitHub - calpt/awesome-adapter-resources: Collection of Tools and Papers related to Adapters (aka Parameter-Efficient Transfer Learning/ Fine-Tuning)

• GitHub - microsoft/LoRA: Code for loralib, an implementation of “LoRA: Low-Rank Adaptation of Large Language Models”

• GitHub - huggingface/peft: 🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

• Parameter-Efficient Fine-Tuning using 🤗 PEFT

• [2310.08659] LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models

• [2305.14314] QLoRA: Efficient Finetuning of Quantized LLMs

• [2307.05695] Stack More Layers Differently: High-Rank Training Through Low-Rank Updates

Papers

• [2304.04947] Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference