Instructions to use mixedbread-ai/mxbai-embed-large-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mixedbread-ai/mxbai-embed-large-v1 with sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("mixedbread-ai/mxbai-embed-large-v1")

sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium."
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Transformers.js

How to use mixedbread-ai/mxbai-embed-large-v1 with Transformers.js:

// npm i @huggingface/transformers
import { pipeline } from '@huggingface/transformers';

// Allocate pipeline
const pipe = await pipeline('feature-extraction', 'mixedbread-ai/mxbai-embed-large-v1');

Transformers

How to use mixedbread-ai/mxbai-embed-large-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("feature-extraction", model="mixedbread-ai/mxbai-embed-large-v1")

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("mixedbread-ai/mxbai-embed-large-v1")
model = AutoModel.from_pretrained("mixedbread-ai/mxbai-embed-large-v1")

llama-cpp-python

How to use mixedbread-ai/mxbai-embed-large-v1 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="mixedbread-ai/mxbai-embed-large-v1",
	filename="gguf/mxbai-embed-large-v1-f16.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use mixedbread-ai/mxbai-embed-large-v1 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf mixedbread-ai/mxbai-embed-large-v1:F16
# Run inference directly in the terminal:
llama-cli -hf mixedbread-ai/mxbai-embed-large-v1:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf mixedbread-ai/mxbai-embed-large-v1:F16
# Run inference directly in the terminal:
llama-cli -hf mixedbread-ai/mxbai-embed-large-v1:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf mixedbread-ai/mxbai-embed-large-v1:F16
# Run inference directly in the terminal:
./llama-cli -hf mixedbread-ai/mxbai-embed-large-v1:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf mixedbread-ai/mxbai-embed-large-v1:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf mixedbread-ai/mxbai-embed-large-v1:F16

Use Docker

docker model run hf.co/mixedbread-ai/mxbai-embed-large-v1:F16

LM Studio
Jan
Ollama
How to use mixedbread-ai/mxbai-embed-large-v1 with Ollama:
```
ollama run hf.co/mixedbread-ai/mxbai-embed-large-v1:F16
```

Unsloth Studio

How to use mixedbread-ai/mxbai-embed-large-v1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mixedbread-ai/mxbai-embed-large-v1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for mixedbread-ai/mxbai-embed-large-v1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for mixedbread-ai/mxbai-embed-large-v1 to start chatting

Atomic Chat new
Docker Model Runner
How to use mixedbread-ai/mxbai-embed-large-v1 with Docker Model Runner:
```
docker model run hf.co/mixedbread-ai/mxbai-embed-large-v1:F16
```

Lemonade

How to use mixedbread-ai/mxbai-embed-large-v1 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull mixedbread-ai/mxbai-embed-large-v1:F16

Run and chat with the model

lemonade run user.mxbai-embed-large-v1-F16

List all available models

lemonade list

juliuslipp commited on Apr 17, 2024

Commit

69b25ae

verified ·

1 Parent(s): 8eb1294

Update README.md

Browse files

Files changed (1) hide show

README.md +2 -4

README.md CHANGED Viewed

@@ -2797,11 +2797,9 @@ Please find more information in our [blog post](https://mixedbread.ai/blog/mxbai
 ## Matryoshka and Binary Quantization
-Embeddings in their commonly used form (float arrays) have a high memory footprint when used at scale. Two approaches to solve this problem are Matryoshka Representation Learning (MRL) and (Binary) Quantization.
-While MRL reduces the number of dimensions of an embedding, binary quantization transforms the value of each dimension from a float32 into a lower precision (int8 or even binary). <b> The model supports both approaches! </b>
-You can also take it one step further, and combine these. This combination of binary quantization and MRL allows you to reduce the memory usage of your embeddings significantly. This leads to much lower costs when using a vector database in particular. You can read more about the technology and its advantages in our [blog post](https://www.mixedbread.ai/blog/binary-mrl).
 ## Community
 Please join our [Discord Community](https://discord.gg/jDfMHzAVfU) and share your feedback and thoughts! We are here to help and also always happy to chat.

 ## Matryoshka and Binary Quantization
+Embeddings in their commonly used form (float arrays) have a high memory footprint when used at scale. Two approaches to solve this problem are Matryoshka Representation Learning (MRL) and (Binary) Quantization. While MRL reduces the number of dimensions of an embedding, binary quantization transforms the value of each dimension from a float32 into a lower precision (int8 or even binary). <b> The model supports both approaches! </b>
+You can also take it one step further, and combine both MRL and quantization. This combination of binary quantization and MRL allows you to reduce the memory usage of your embeddings significantly. This leads to much lower costs when using a vector database in particular. You can read more about the technology and its advantages in our [blog post](https://www.mixedbread.ai/blog/binary-mrl).
 ## Community
 Please join our [Discord Community](https://discord.gg/jDfMHzAVfU) and share your feedback and thoughts! We are here to help and also always happy to chat.