Optimizing the number of chunks
Configuring via the Knowledge Base settings
Within the Knowledge Base CMS settings you can set the number of chunks to retrieve. Fetching more chunks will provide more context to your RAG solution, using fewer will save on tokens. We've found that most applications work well with a default of 3 chunks, but it's important to experiment for your use case!
Using the KB Api
You can also use the KB API to fetch a number of chunks.
{
"question": "{last_utterance}",
"chunkLimit": 3,
"synthesis": false,
"tags": {
"includeAllTagged": true,
"includeAllNonTagged": true
}
}
For more details checkout the cookbook recipe Advanced Rag Filtering and Tagging
Updated 6 months ago