Policy Writing 101

Introduction

Effective policy writing in Clavata.ai is crucial for accurate labelling or content. This guide provides best practices for crafting clear, maintainable, and efficient policies using Clavata.ai's syntax and features.

Understanding Policy Structure

A Clavata.ai policy comprises Labels containing one or more Rules. Each Rule uses Conditions and Signals to define what content should be flagged.

Labels

Definition: Organize related Rules under a common category.
Syntax:

LABEL "Label Name": "Risk Level" {
  // Rules go here
}

Rules

Definition: Logical statements that evaluate content against specified Conditions and Signals.
Example:

"dogs" [IN] "park"

This Rule flags instances where "dogs" appear in the context of a "park".

Writing Efficient Rules

Simple Inclusion vs. Logical Operators

Example 1:

"dogs [IN] "park"

Example 2:

ANY("dogs") AND ANY("park")

Both Rules aim to flag content mentioning "dogs" in relation to a "park".

Handling Multiple Contexts

Example 1:

"dogs [IN] "park"
"dogs [IN] "city"
"dogs [IN] "field"

Example 2:

ANY("dogs") AND ANY("park", "city", "field")

Other than readability or organization, there is no difference in the two "dogs" examples. They are both functionally looking for the same thing. Context and personal workflows can dictate how you write different policies.

Policy Pitfalls

Vagueness

The enemy of a good policy is not having the AI understand what you mean. Much like a human, if you were to say "there is a deli down the street", a human might ask:

Which direction?
How far?
How long will it take me?

Think of this same logic when writing policy. You may want to flag content if it contains Vulgarity so you write something simple like the below thinking your bases are covered.

"vulgarity"

You know what the definition of vulgarity is but does the AI? Think of what "vulgarity" might mean to a random human. Things like "damn" or "hell" could be considered vulgarity to some. Consider refining your definition so the model has a better understanding. You could try any of the below:

"vulgarity not acceptable in the presence of children"
"vulgarity" [IN] "pg-13 movies"
"common terms considered vulgar" [THROUGHOUT] "all countries"

Any of the above may work for your platform but be sure to test them out in the testing section of your policies.

Contradictory Statements

Contradictory statement can be just as harmful for AI as it can be for humans when it comes to understanding. Things like below cause your brain to pause and the same thing can happen with AI.

"Shattered glass still whole"
"bloody wound with no blood"
"Drenched but completely dry"

Avoid contradictory statements whenever you can as they will cause confusion both with the AI and with anyone else reading your policy.

Placing { } in the wrong place

An easy mistake to make is placing your braces { } in the wrong locations when using UNLESS statements. These statements allow you to carve out exceptions for your policy. Want to label discussions of sports but not when it comes to Baseball? By adding it to an UNLESS block you can.

LABEL "Sports": {
   "discussion" [ABOUT] "sports
   UNLESS {
     "baseball
  }
}

The above example is considering the UNLESS statement as part of the overall "Sports" label. If someone were to say "basketball is the best" that would come back as true since it is a discussion of sports but isn't about baseball. However, if they were say "baseball is the best" that would come back as false since it is a discussion about sports but baseball was part of the discussion.

LABEL "Sports": {
   "discussion" [ABOUT] "sports
}
   UNLESS {
     "baseball
}
LABEL "Drinking": {
  "drinking beer"
}

The above example works differently than the previous one. Since the { } are outside of the label then the UNLESS is applied to the entire policy. For example, if someone said "let's go drink some beer at the baseball game" this would come back as false. Even though the text would normally be marked as true by the "Drinking" label, since UNLESS {Baseball} is applied to the entire policy, it counts anything mentioning baseball to come back as false.

There is no right or wrong answer for how to use an UNLESS statement. It all depends on what you are trying to achieve.

Best Practices

Clarity: Write Rules that are easy to understand.
Maintainability: Use concise structures to simplify updates.
Testing: Regularly test Policies to ensure accuracy.
Documentation: Comment complex Rules for future reference.

Additional Resources

General Syntax of Policies
Types of Conditions
Creating a Policy

For further assistance, contact our support team at support@clavata.ai.