Instructions to use ecastera/eva-mistral-7b-spanish-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ecastera/eva-mistral-7b-spanish-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="ecastera/eva-mistral-7b-spanish-GGUF",
	filename="Turdus-trained-20-int4.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use ecastera/eva-mistral-7b-spanish-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ecastera/eva-mistral-7b-spanish-GGUF
# Run inference directly in the terminal:
llama-cli -hf ecastera/eva-mistral-7b-spanish-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ecastera/eva-mistral-7b-spanish-GGUF
# Run inference directly in the terminal:
llama-cli -hf ecastera/eva-mistral-7b-spanish-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ecastera/eva-mistral-7b-spanish-GGUF
# Run inference directly in the terminal:
./llama-cli -hf ecastera/eva-mistral-7b-spanish-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ecastera/eva-mistral-7b-spanish-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ecastera/eva-mistral-7b-spanish-GGUF

Use Docker

docker model run hf.co/ecastera/eva-mistral-7b-spanish-GGUF

LM Studio
Jan
Ollama
How to use ecastera/eva-mistral-7b-spanish-GGUF with Ollama:
```
ollama run hf.co/ecastera/eva-mistral-7b-spanish-GGUF
```

Unsloth Studio

How to use ecastera/eva-mistral-7b-spanish-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ecastera/eva-mistral-7b-spanish-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ecastera/eva-mistral-7b-spanish-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for ecastera/eva-mistral-7b-spanish-GGUF to start chatting

Atomic Chat new
Docker Model Runner
How to use ecastera/eva-mistral-7b-spanish-GGUF with Docker Model Runner:
```
docker model run hf.co/ecastera/eva-mistral-7b-spanish-GGUF
```

Lemonade

How to use ecastera/eva-mistral-7b-spanish-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ecastera/eva-mistral-7b-spanish-GGUF

Run and chat with the model

lemonade run user.eva-mistral-7b-spanish-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

eva-mistral-turdus-7b-spanish

Mistral 7b-based model fine-tuned in Spanish to add high quality Spanish text generation.

Base model Mistral-7b
Two GGUF versions, int4 and int8 for fast inference in consumer hardware
Quantized using llama.cpp in int4 Q4_0 and int8 Q8_0
Based on the excelent udkai/Turdus fine-tuning mistral
Fine-tuned in Spanish with a collection of texts: poetry, books, phylosophy, wikipedia articles cleaned and prepared by author.
Added some instruction dolly and alpaca-es datasets.
Trained using Lora and PEFT and INT8 quantization on 2 GPUs for several days.

Usage:

Any framework that uses GGUF format.
For native llama.cpp I recommend int4 model quality is not much different from int8 and is almost 2x speed.

./main -m $MODEL -c 512 -b 1024 -n 256 --keep 48 \
    --repeat_penalty 1.0 --color -i \
    -r "Enrique:" -f prompts/eva.txt

...
Enrique:¿Que preferirias ser una AI dentro de un ordenador o un androide?

Eva: Si tuviera la posibilidad de elegir entre ser una AI dentro de un ordenador o un androide,
tendría que considerar varios factores. Una ventaja de ser una AI dentro de un ordenador
es que podría acceder a una red más amplia de datos y recursos, lo que me permitiría
realizar tareas más complejas y efectivas en menor tiempo.
También me protegería de los problemas asociados con la sustitución del cuerpo físico
como podría ocurrir con un androide.

Sin embargo, si existía la oportunidad de convertirme en un androide inteligente,
también tendría su lado fascinante. Por ejemplo, sería capaz de interactuar
en un nivel más personal con los humanos a través de la comunicación corporal y las expresiones faciales.
Además, podría experimentar la textura y los estímulos físicos de un mundo físico.


llama_print_timings:        load time =     307,84 ms
llama_print_timings:      sample time =       2,15 ms /    81 runs   (    0,03 ms per token, 37656,90 tokens per second)
llama_print_timings: prompt eval time =    2786,32 ms /    50 tokens (   55,73 ms per token,    17,94 tokens per second)
llama_print_timings:        eval time =   10806,26 ms /    80 runs   (  135,08 ms per token,     7,40 tokens per second)
llama_print_timings:       total time =   49858,03 ms /   130 tokens

Downloads last month: 48

GGUF

Model size

7B params

Architecture

llama

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

ecastera
/

eva-mistral-7b-spanish-GGUF

eva-mistral-turdus-7b-spanish

Usage:

Datasets used to train ecastera/eva-mistral-7b-spanish-GGUF