Not able to load locally downloaded model using SentenceTransformer
My simple code to load the model using locally downloaded path:
from sentence_transformers import SentenceTransformer
self.model = SentenceTransformer(self.model_path)
model_path is : models/downloads/mxbai-embed-large-v1
ll models/downloads/mxbai-embed-large-v1
total 7248200
drwxr-xr-x 3 umesh staff 96B Aug 28 15:43 1_Pooling
-rw-r--r-- 1 umesh staff 11K Aug 28 15:43 LICENSE
-rw-r--r-- 1 umesh staff 111K Aug 28 15:43 README.md
-rw-r--r-- 1 umesh staff 677B Aug 28 15:43 config.json
-rw-r--r-- 1 umesh staff 171B Aug 28 15:43 config_sentence_transformers.json
drwxr-xr-x 3 umesh staff 96B Aug 28 15:43 gguf
-rw-r--r-- 1 umesh staff 1.2G Aug 28 15:45 model.onnx
-rw-r--r-- 1 umesh staff 639M Aug 28 15:44 model.safetensors
-rw-r--r-- 1 umesh staff 638M Aug 28 15:46 model_fp16.onnx
-rw-r--r-- 1 umesh staff 321M Aug 28 15:46 model_quantized.onnx
-rw-r--r-- 1 umesh staff 229B Aug 28 15:43 modules.json
-rw-r--r-- 1 umesh staff 639M Aug 28 15:44 mxbai-embed-large-v1-f16.gguf
drwxr-xr-x 5 umesh staff 160B Aug 28 15:43 onnx
-rw-r--r-- 1 umesh staff 53B Aug 28 15:43 sentence_bert_config.json
-rw-r--r-- 1 umesh staff 695B Aug 28 15:43 special_tokens_map.json
-rw-r--r-- 1 umesh staff 695K Aug 28 15:43 tokenizer.json
-rw-r--r-- 1 umesh staff 1.2K Aug 28 15:43 tokenizer_config.json
-rw-r--r-- 1 umesh staff 226K Aug 28 15:43 vocab.txt
Lib versions are as follows :
Python 3.11.9
sentence-transformers 3.0.1
torch 2.4.0
The error which I am getting with below thread dump:
Fatal Python error: Aborted
Current thread 0x00000001f59e2500 (most recent call first):
File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1160 in convert
File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 805 in _apply
File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780 in _apply
File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780 in _apply
File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780 in _apply
File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 780 in _apply
File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1174 in to
File "/Users/umesh/git-repos/genai_search/cxg/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 316 in __init__
File "/Users/umesh/git-repos/genai_search/models/impl/mxbai_embed_large_v1.py", line 13 in __init__
Clueless what might be wrong as I am doing same as mentioned in the README. Can someone point the possible issue?
BTW, I am able to load the bge-large-en-v1.5 model just fine using SentenceTransformer.
Thanks!
Facing same issue, were you able to find the solution?
Try to use SentenceTransformer like this:
# setting device as GPU if available, else CPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
self.model = SentenceTransformer(self.model_path, device=device)
Thanks for reply, I am using machine with no internet access; I think it is trying to download something.
Getting this error.
Cannot find an appropriate cached snapshot folder for the specified revision on the local disk and outgoing traffic has been disabled. To enable repo look-ups and downloads online