Developmentbluryar/@bluryar/tokenizers-js

@bluryar/tokenizers-js

基于 [HuggingFace Tokenizers](https://github.com/huggingface/tokenizers) 的JavaScript封装,使用WebAssembly实现高性能的文本标记化功能。

Suggested install command

npm view @bluryar/tokenizers-js

Always inspect the linked repository and skill instructions before running commands. Skills are instructions; permissions and execution still matter.

Compatibility

3 supported