Quantize 123B Mistral-Large-Instruct-2407 to 35 GB with only 4% accuracy degeneration.

by /u/RelationshipWeekly78 in /r/LocalLLaMA

Upvotes: 259

Favorite this post:

Mark as read:

Your rating:

Add this post to a custom list