Tokenizer and token counting

A tokenizer is the tool that splits text into the tokens a particular model understands.