Hugging Face's logo

z-lab
/
Llama-3.1-8B-Instruct-PARO

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
4-bit precision
paroquant
Model card Files
xet
 Community
liang2kl commited on
Commit
f7abebb
·
verified ·
1 Parent(s): 9b3b9a9

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +4 -5
README.md CHANGED
@@ -2,9 +2,8 @@
2
  library_name: transformers
3
  license: llama3
4
  pipeline_tag: text-generation
5
- base_model: meta-llama/Llama-3.1-8B-Instruct
6
- tags:
7
- - mlx
8
  ---
9
 
10
  # z-lab/Llama-3.1-8B-Instruct-PARO
@@ -32,8 +31,8 @@ z-lab/Llama-3.1-8B-Instruct-PARO is a 4-bit [meta-llama/Llama-3.1-8B-Instruct](h
32
  pip install "paroquant[vllm]"
33
 
34
  # NVIDIA GPU (CUDA 13.0)
35
- pip install "paroquant[vllm]" "vllm==0.17.1" \
36
- --extra-index-url https://wheels.vllm.ai/0.17.1/cu130 \
37
  --extra-index-url https://download.pytorch.org/whl/cu130
38
 
39
  # Apple Silicon
 
2
  library_name: transformers
3
  license: llama3
4
  pipeline_tag: text-generation
5
+ base_model:
6
+ - meta-llama/Llama-3.1-8B-Instruct
 
7
  ---
8
 
9
  # z-lab/Llama-3.1-8B-Instruct-PARO
 
31
  pip install "paroquant[vllm]"
32
 
33
  # NVIDIA GPU (CUDA 13.0)
34
+ pip install "paroquant[vllm]" "vllm==0.19.0" \
35
+ --extra-index-url https://wheels.vllm.ai/2a69949bdadf0e8942b7a1619b229cb475beef20/cu130 \
36
  --extra-index-url https://download.pytorch.org/whl/cu130
37
 
38
  # Apple Silicon