--- base_model: unsloth/orpheus-3b-0.1-ft-unsloth-bnb-4bit tags: - transformers - llama - trl - tts - tex-to-speech license: apache-2.0 language: - pl pipeline_tag: text-to-speech datasets: - czyzi0/the-mc-speech-dataset --- # VoxPolska: Next-Gen Polish Voice Generation ![image/png](https://cdn-uploads.huggingface.co/production/uploads/66a8afaf4bbd71186602585e/YvUeYq_EQHSuhknYk6k9N.png) ## 📌 Model Highlights - Context-Aware Voice: Generates speech that captures the nuances and tone of the Polish language. - Showcases advanced proficiency in speech synthesis and Polish language processing. - Converts written Polish text into natural, fluent, and expressive speech. - Advanced Deep Learning: Built using cutting-edge deep learning techniques for optimal performance. - State-of-the-Art Technology: Showcases advanced proficiency in speech synthesis and Polish language processing. ## 🔧 Technical Details - Base Model: Orpheus TTS - LoRA (Low-Rank Adaptation) fine-tuning applied to optimize model performance. - Sample Rate: 24 kHz audio output for high-fidelity sound. - Trained with 24000+ Polish transcript and audio pairs - Merged 16 bit quantization - Audio Decoding: Customized layer-wise processing for audio generation - Repetition Penalty: 1.1 to avoid repetitive phrases - Gradient Checkpointing: Enabled for efficient memory usage ## 🎧 Example Usage (Pipeline) - Here is an example code snippet to run the model on a notebook: ```py !pip install transformers from transformers import pipeline pipe = pipeline("text-to-speech", model="salihfurkaan/VoxPolska-V1-Merged-16bit") ``` ## 🎧 Example Usage (Directly) - Here is an example code to run the model on a notebook: ```py !pip install snac torch transformers import torch import snac from snac import SNAC from transformers import AutoTokenizer, AutoModelForCausalLM import os from IPython.display import display, Audio device = torch.device("cuda" if torch.cuda.is_available() else "cpu") tokenizer = AutoTokenizer.from_pretrained("salihfurkaan/VoxPolska-V1-Merged-16bit") model = AutoModelForCausalLM.from_pretrained("salihfurkaan/VoxPolska-V1-Merged-16bit").to(device) os.environ["HF_TOKEN"] = "your huggingface token here" snac_model = SNAC.from_pretrained("hubertsiuzdak/snac_24khz").to(device) prompts = [ "Cześć, jestem dużym modelem języka sztucznej inteligencji" ] #an example prompt chosen_voice = None prompts_ = [(f"{chosen_voice}: " + p) if chosen_voice else p for p in prompts] all_input_ids = [] for prompt in prompts_: input_ids = tokenizer(prompt, return_tensors="pt").input_ids all_input_ids.append(input_ids) start_token = torch.tensor([[128259]], dtype=torch.int64) # Start of human end_tokens = torch.tensor([[128009, 128260]], dtype=torch.int64) # End of text, End of human all_modified_input_ids = [] for input_ids in all_input_ids: modified_input_ids = torch.cat([start_token, input_ids, end_tokens], dim=1) all_modified_input_ids.append(modified_input_ids) all_padded_tensors = [] all_attention_masks = [] max_length = max([x.shape[1] for x in all_modified_input_ids]) for modified_input_ids in all_modified_input_ids: padding = max_length - modified_input_ids.shape[1] padded_tensor = torch.cat([torch.full((1, padding), 128263, dtype=torch.int64), modified_input_ids], dim=1) attention_mask = torch.cat([torch.zeros((1, padding), dtype=torch.int64), torch.ones((1, modified_input_ids.shape[1]), dtype=torch.int64)], dim=1) all_padded_tensors.append(padded_tensor) all_attention_masks.append(attention_mask) all_padded_tensors = torch.cat(all_padded_tensors, dim=0).to(device) all_attention_masks = torch.cat(all_attention_masks, dim=0).to(device) generated_ids = model.generate( input_ids=all_padded_tensors, attention_mask=all_attention_masks, max_new_tokens=1200, do_sample=True, temperature=0.6, top_p=0.95, repetition_penalty=1.1, num_return_sequences=1, eos_token_id=128258, use_cache=True ) token_to_find = 128257 token_to_remove = 128258 token_indices = (generated_ids == token_to_find).nonzero(as_tuple=True) if len(token_indices[1]) > 0: last_occurrence_idx = token_indices[1][-1].item() cropped_tensor = generated_ids[:, last_occurrence_idx+1:] else: cropped_tensor = generated_ids processed_rows = [] for row in cropped_tensor: masked_row = row[row != token_to_remove] processed_rows.append(masked_row) code_lists = [] for row in processed_rows: row_length = row.size(0) new_length = (row_length // 7) * 7 trimmed_row = row[:new_length] trimmed_row = [t - 128266 for t in trimmed_row] code_lists.append(trimmed_row) def redistribute_codes(code_list): layer_1 = [] layer_2 = [] layer_3 = [] for i in range((len(code_list) + 1) // 7): layer_1.append(code_list[7 * i]) layer_2.append(code_list[7 * i + 1] - 4096) layer_3.append(code_list[7 * i + 2] - (2 * 4096)) layer_3.append(code_list[7 * i + 3] - (3 * 4096)) layer_2.append(code_list[7 * i + 4] - (4 * 4096)) layer_3.append(code_list[7 * i + 5] - (5 * 4096)) layer_3.append(code_list[7 * i + 6] - (6 * 4096)) codes = [ torch.tensor(layer_1).unsqueeze(0).to(device), torch.tensor(layer_2).unsqueeze(0).to(device), torch.tensor(layer_3).unsqueeze(0).to(device) ] audio_hat = snac_model.decode(codes) return audio_hat my_samples = [] for code_list in code_lists: samples = redistribute_codes(code_list) my_samples.append(samples) if len(prompts) != len(my_samples): raise Exception("Number of prompts and samples do not match") else: for i in range(len(my_samples)): print(prompts[i]) samples = my_samples[i] display(Audio(samples.detach().squeeze().to("cpu").numpy(), rate=24000)) del my_samples, samples ``` You can get your huggingface token from [here](https://huggingface.co/settings/tokens) ## 📫 Contact and Support For questions, suggestions, and feedback, please open an issue on HuggingFace. You can also reach the author via: LinkedIn ## Model Misuse Do not use this model for impersonation without consent, misinformation or deception (including fake news or fraudulent calls), or any illegal or harmful activity. By using this model, you agree to follow all applicable laws and ethical guidelines. ## Citation ```none @misc{ title={salihfurkaan/VoxPolska-V1-Merged-16bit}, author={Salih Furkan Erik}, year={2025}, url={https://huggingface.co/salihfurkaan/VoxPolska-V1-Merged-16bit/} } ```