New evaluation type: options
by Michael HoodDrive measurable agent improvements with objective, repeatable scoring by evaluating conversations against predefined outcome options instead of subjective criteria.
- Define a set of options and the true/expected state for each
- Choose which LLM audits the transcript against these options
- Color-code options for fast visual scanning
- Control which options are included in reporting
How do I use this feature?
- Navigate to the evaluations tab under transcripts
- Create new evaluation (top right)
- From the metric dropdown, select "Options"
- Define a list of pre-determined options
- Provide a general description and true description for each option
- Select a color for each option
- Test on last transcript to see how it works (button at bottom of modal)
- Create evaluation
Tip
If you want to retroactively run evaluations on old transcripts, you can do so by using the 'Bulk run evaluations' feature found in the transcripts tab.













