chore: squash history — keep latest version only

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,112 @@
+---
+language: vi
+tags:
+  - ner
+  - phobert
+  - vietnamese
+  - document-ai
+  - cccd
+  - synthetic-data
+license: mit
+base_model: vinai/phobert-base
+---
+
+# VietNerm - Căn cước công dân NER Model
+
+PhoBERT-based Named Entity Recognition model for Vietnamese **Căn cước công dân** documents.
+
+## ⚠️ DISCLAIMER: SYNTHETIC / MOCKUP DATA
+
+> **Model này được train hoàn toàn trên dữ liệu giả lập (synthetic/mockup data), KHÔNG sử dụng dữ liệu cá nhân thật.**
+
+- Tất cả dữ liệu training được **sinh tự động** bằng hệ thống template + generator
+- **Không** sử dụng giấy tờ thật, thông tin cá nhân thật, hoặc dữ liệu thu thập từ người dùng
+- Số định danh (ID, CCCD...) được sinh ngẫu nhiên, thiết kế để **không trùng** với dữ liệu thật
+- Dữ liệu có inject nhiễu OCR (noise) để giả lập điều kiện thực tế
+- Mục đích: **nghiên cứu AI, Document AI, OCR/NER pipeline**
+- **Không** được sử dụng để giả mạo giấy tờ, tạo giấy tờ giả, lừa đảo hoặc gian lận
+
+## Model Description
+
+This model is fine-tuned from [`vinai/phobert-base`](https://huggingface.co/vinai/phobert-base) for token-level NER on Vietnamese administrative/medical documents. It extracts structured fields from OCR text output.
+
+- **Base model**: vinai/phobert-base
+- **Task**: Token Classification (NER)
+- **Language**: Vietnamese (vi)
+- **Document type**: Căn cước công dân
+- **Number of labels**: 13
+- **Training data**: Synthetic/Mockup (not real personal data)
+
+## Labels
+
+- `B-date_of_birth`
+- `B-date_of_expiry`
+- `B-full_name`
+- `B-gender`
+- `B-id_number`
+- `B-nationality`
+- `B-place_of_origin`
+- `B-place_of_residence`
+- `I-full_name`
+- `I-nationality`
+- `I-place_of_origin`
+- `I-place_of_residence`
+
+## Usage
+
+### With VietNerm SDK
+
+```python
+from vietnerm import VietNerm
+
+ner = VietNerm(doc_type="cccd", model_path="ngocthanhdoan/phobert-cccd-ner")
+result = ner.extract("your document text here")
+print(result)
+```
+
+### With Transformers
+
+```python
+from transformers import AutoTokenizer, AutoModelForTokenClassification
+import torch
+
+tokenizer = AutoTokenizer.from_pretrained("ngocthanhdoan/phobert-cccd-ner")
+model = AutoModelForTokenClassification.from_pretrained("ngocthanhdoan/phobert-cccd-ner")
+
+text = "your document text here"
+inputs = tokenizer(text, return_tensors="pt")
+
+with torch.no_grad():
+    outputs = model(**inputs)
+    predictions = torch.argmax(outputs.logits, dim=-1)
+```
+
+## Training
+
+- **Dataset**: Synthetically generated (mockup data) with OCR noise simulation
+- **Data source**: Auto-generated from Jinja2 templates + random generators (no real personal data)
+- **Framework**: HuggingFace Transformers + Trainer API
+- **Optimizer**: AdamW (lr=2e-5)
+- **Epochs**: 5-7 (with early stopping)
+
+## Ethical Use
+
+This model is built for **research and development purposes only**:
+
+- ✅ AI/NLP research
+- ✅ Document AI development
+- ✅ OCR/NER pipeline prototyping
+- ✅ Educational purposes
+- ❌ Forging documents
+- ❌ Creating fake identity papers
+- ❌ Fraud or deception
+
+## About VietNerm
+
+VietNerm is a Document AI Factory for Vietnamese documents. It provides a complete pipeline
+from template-based synthetic data generation to model training and deployment.
+
+- **Repository**: [Devhub-Solutions/VietNerm](https://github.com/Devhub-Solutions/VietNerm)
+- **Training dataset**: [ngocthanhdoan/vietnerm-cccd-dataset](https://huggingface.co/datasets/ngocthanhdoan/vietnerm-cccd-dataset)
+- **SDK**: `pip install vietnerm`
+- **License**: MIT — Copyright (c) 2026 Devhub Solutions

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<mask>": 64000
+}

config.json

@@ -0,0 +1,58 @@
+{
+  "architectures": [
+    "RobertaForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "O",
+    "1": "B-date_of_birth",
+    "2": "B-date_of_expiry",
+    "3": "B-full_name",
+    "4": "B-gender",
+    "5": "B-id_number",
+    "6": "B-nationality",
+    "7": "B-place_of_origin",
+    "8": "B-place_of_residence",
+    "9": "I-full_name",
+    "10": "I-nationality",
+    "11": "I-place_of_origin",
+    "12": "I-place_of_residence"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "B-date_of_birth": 1,
+    "B-date_of_expiry": 2,
+    "B-full_name": 3,
+    "B-gender": 4,
+    "B-id_number": 5,
+    "B-nationality": 6,
+    "B-place_of_origin": 7,
+    "B-place_of_residence": 8,
+    "I-full_name": 9,
+    "I-nationality": 10,
+    "I-place_of_origin": 11,
+    "I-place_of_residence": 12,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 258,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "tokenizer_class": "PhobertTokenizer",
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.3",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 64001
+}

label_map.json

@@ -0,0 +1,32 @@
+{
+  "id2label": {
+    "0": "O",
+    "1": "B-date_of_birth",
+    "2": "B-date_of_expiry",
+    "3": "B-full_name",
+    "4": "B-gender",
+    "5": "B-id_number",
+    "6": "B-nationality",
+    "7": "B-place_of_origin",
+    "8": "B-place_of_residence",
+    "9": "I-full_name",
+    "10": "I-nationality",
+    "11": "I-place_of_origin",
+    "12": "I-place_of_residence"
+  },
+  "label2id": {
+    "O": 0,
+    "B-date_of_birth": 1,
+    "B-date_of_expiry": 2,
+    "B-full_name": 3,
+    "B-gender": 4,
+    "B-id_number": 5,
+    "B-nationality": 6,
+    "B-place_of_origin": 7,
+    "B-place_of_residence": 8,
+    "I-full_name": 9,
+    "I-nationality": 10,
+    "I-place_of_origin": 11,
+    "I-place_of_residence": 12
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0955f8d22ca7c2ceee23423c13259eadc80bd78976685fbd8817d0bc0e09f269
+size 537694636

special_tokens_map.json

@@ -0,0 +1,9 @@
+{
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "mask_token": "<mask>",
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "unk_token": "<unk>"
+}

tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "64000": {
+      "content": "<mask>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "extra_special_tokens": {},
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "PhobertTokenizer",
+  "unk_token": "<unk>"
+}