New evaluation type: options

Drive measurable agent improvements with objective, repeatable scoring by evaluating conversations against predefined outcome options instead of subjective criteria.

1. Define a set of options and the true/expected state for each
2. Choose which LLM audits the transcript against these options
3. Color-code options for fast visual scanning
4. Control which options are included in reporting

How do I use this feature?

1. Navigate to the evaluations tab under transcripts
2. Create new evaluation (top right)
3. From the metric dropdown, select "Options"
4. Define a list of pre-determined options
   1. Provide a general description and true description for each option
   2. Select a color for each option
   3. Test on last transcript to see how it works (button at bottom of modal)
5. Create evaluation

Tip
If you want to retroactively run evaluations on old transcripts, you can do so by using the 'Bulk run evaluations' feature found in the transcripts tab.