Users can now set a Priority which allows them to say "If Label A is true, then Label B should never be true."
This will help with policies that deal with ratings and other use-cases that need to enforce a hierarchy in the response.
Improvements
Policy Testing
Added an F1 score to better assess the accuracy your results. This is especially useful when dealing with imbalanced datasets such as having too many unlabelled things in a dataset vs how many things are labelled.
Overall metrics in the All metrics view are now an average of the label metrics.
When testing policy and within the All metrics view, users can click into any label to get the label focused view
Downloading a CSV shows all policy label and user labels for the view you are currently on.
New filtering within the label focused view allow for easier filtering of True Positive, False Positive, True Negative, or False Negative.
Note: True Negatives are off by default
Hover over metrics to see how they are calculated
Hovering over Status badges will tell you why the badge is what it is
Bug Fixes
Fixed an issue where very large datasets weren't finishing test runs during policy testing
We identified a bug with our "exact" text matching operator -> =("some word or phrase") that would sometimes cause it to return a score of 1.0 for a review whether the word or phrase was actually matched, or not. This issue has been corrected and the scores reported from this operator will now be reported correctly in the web UI.
Did this answer your question?Thanks for the feedbackThere was a problem submitting your feedback. Please try again later.
