IQ2 IMatrix GGUF
Collection
Tiny quants • 7 items • Updated • 1
WinterGoddess-1.4x-70B-L2 IQ2-GGUF
Description
IQ2-GGUF quants of sophosympatheia/Aurora-Nights-70B-v1.0
Unlike regular GGUF quants this uses important matrix similar to Quip# to keep the quant from degrading too much even at 2bpw allowing you to run larger models on less powerful machines.
NOTE: Currently you will need experimental branches of Koboldcpp or Ooba for this to work.
- Nexesenex have compiled Windows binaries HERE
- llamacpp_0.2.29 branch of Ooba also works
Models
Regular GGUF Quants: Here
Prompt Format
Unclear
Contact
Kooten on discord
- Downloads last month
- 90
Hardware compatibility
Log In to add your hardware
2-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support