Function loadVocab

Load vocab from a parsed tokenizer.json object.

Expected structure (HuggingFace tokenizer.json): model.vocab: { "token": id, ... } added_tokens: [{ "content": "...", "id": N, "special": true }, ...] unk_token: "[UNK]", pad_token: "[PAD]", etc.