Technology > Machinese > Glossary > tokenization

Tokenization

A process of something into smaller pieces and assigning them a meaning: e.g., splitting a sentence into word tokens.

Connexor Oy, Helsinki Business and Science Park. Finland. info@connexor.com
© Connexor Oy. Powered by ToimiSait