What are Tokens?

If you're looking to top up your AI tokens, read this article.

What is a token?

A token is determined by the AI model you use. For example, OpenAI (GPT models) and Anthropic (Claude models) considered a token as around 3-4 characters. To read more on how tokens are calculated you can read the documentation from either AI provider below.

How are tokens calculated?

Tokens are used by both the input (what is being sent to the AI model) and output (what is being received to the AI model).

It is important to note, that the 'token' slider on the AI steps (shown below) only control the output message. Depending on your design, there may be a large amount of information that is sent to the AI model as a part of your prompt

How many tokens do different features use in Voiceflow?

  • Knowledge Base: A question and answer will use anywhere from 800-1500 tokens depending on the size of question and answer. This is because the Knowledge base passes in relevant information from your documentation to the AI model alongside your prompt.
  • Response AI Step (With Memory): A question and answer when using the Response AI step with Memory will use anywhere from 150-2000 tokens. Voiceflow saves the last 3 conversation turns and passes those to your AI model to provide context for your answer.
  • Response AI Step (Without Memory): A question and answer when using the Response AI step without, memory will use roughly 150-300 tokens.

You can turn memory on or off via the setting on the Set AI and Response AI step below.