CURing

Compressing Models using Activation-aware CUR Decomposition.

---

Prerequirements

lm-evaluation-harness

git clone --depth 1 https://github.com/EleutherAI/lm-evaluation-harness

cd lm-evaluation-harness
pip install -e .

Install Requirements

pip install -r requirements.txt

Run

# Benchmarking original models
reproduce_original.sh

# Benchmarking CURing-decomposited models
reproduce_curing.sh

# Ablation analysis
reproduce_ablation.sh