Enhance Moderation

Bring AI to auto moderate contents and create safer online communities

Enhance moderation is a feature that allows all uploaded text, images, and videos on posts as well as text and images comments to be scanned and moderated for inappropriate, offensive, and unwanted content by using AI.

To this end, we've implemented AI Text moderation to detect and moderate text on posts and comments that contain

• Sexually explicit or adult in certain situations

• Sexually suggestive or mature in certain situations

• Offensive in certain situations.

And AI Image and Video moderation technology to detect and moderate images, and videos on posts as well as images on comments that contain

• Adult Toys

• Air crash

• Alcohol

• Alcoholic Beverages

• Bare-chested Male

• Corpses

• Drinking

• Drug Paraphernalia

• Drug Products

• Drug Use

• Drugs

• Emaciated Bodies

• Explicit Nudity

• Explosions and blasts

• Extremist

• Female Swimwear Or Underwear

• Gambling

• Graphic Female Nudity

• Graphic Male Nudity

• Graphic Violence Or Gore

• Hanging

• Hate Symbols

• Illustrated Explicit Nudity

• Male Swimwear Or Underwear

• Middle Finger

• Nazi Party

• Nudity

• Partial Nudity

• Physical Violence

• Pills

• Revealing Clothes

• Rude Gestures

• Self Injury

• Sexual Activity

• Sexual Situations

• Smoking

• Suggestive

• Tobacco

• Tobacco Products

• Violence

• Visually Disturbing

• Weapon Violence

• Weapons

• White Supremacy

Our enhance moderation solution empowers you to establish a secure online environment for your users, effortlessly eliminating the necessity for any human intervention. By harnessing the power of advanced AI technology that automatically scans and moderates all user-generated content, effectively identifying and addressing any inappropriate or offensive material. This proactive approach enables you to cultivate a welcoming and protected community for your users, without relying on manual oversight.

Enhance Moderation is currently in Private Beta. Please submit your request to Amity Help Center to enable this feature. It will take approximately 5 business days to process your request.

How does Enhance Moderation Work?

The Enhance AI moderation is performed using two main factors - flagConfidence and blockConfidence. When a post or a comment is successfully created, our AI system will automatically scans both the post's text and media, generating a confidence value as a result. If this confidence value falls below flagConfidence, the post/comment seamlessly passes the moderation without any issues. In cases where the confidence value falls between flagConfidence and blockConfidence, the Enhance moderation feature flags the post/comment for review. You have the option to review the flagged content within the Amity Console or listen to Amity real-time flag post event through your webhook on the server side, allowing you to implement appropriate actions based on your moderation policies. Finally, if the confidence value surpasses blockConfidence, the system deletes the post/comment altogether, ensuring a proactive approach to maintaining a safe online environment.

As a default configuration, all categories have an initial flagConfidence value of 40, and an initial blockConfidence value of 80.

Enhance Moderation Description

Name	Data Type	Description
category	String	Name of each moderation's category
flagConfidence	Number	Value of moderation category’s flag confidence
blockConfidence	Number	Value of moderation category’s block confidence
moderationType	String	Value of moderation type for the category. There are 2 possible values, "text" or "media"

Specify API Endpoint

To use the enhanced moderation APIs, specify the appropriate API endpoint on each HTTP request. Each data center has a unique endpoint, so it's essential to adjust it accordingly. By selecting the correct endpoint associated with the right location, you ensure faster response times and optimize the overall performance of your API requests.

Region	API Endpoint
Europe	https://beta-eu.amity.services/
Singapore	https://beta-sg.amity.services/
United States	https://beta-us.amity.services/

Get Moderation Confidence Level

Get moderation confidence level API can be used to retrieve confidence level for each moderation category. The API will return list of moderation category along with the corresponding confidence level.

Retrieve confidence level for each moderation category

GETAPI_ENDPOINT{basePath}/v1/content-moderation/confidences

Header parameters

Response

200

Moderation categories and confidence level JSON

Body

application/json

confidencesarray of CategoryConfidence (object)

Request

JavaScriptCurl

const response = await fetch('API_ENDPOINT{basePath}/v1/content-moderation/confidences', {
    method: 'GET',
    headers: {},
});
const data = await response.json();

Response

{
  "confidences": [
    {
      "category": "Alcohol",
      "flagConfidence": 40,
      "blockConfidence": 80,
      "moderationType": "media"
    },
    {
      "category": "Drugs",
      "flagConfidence": 30,
      "blockConfidence": 80,
      "moderationType": "media"
    }
  ]
}

Update Moderation Confidence Level

Set moderation confidence level API can be used to set confidence level to any moderation category. The API will return list of moderation category along with the corresponding confidence level.

Set confidence level to the moderation category

PUTAPI_ENDPOINT{basePath}/v1/content-moderation/confidences

Header parameters

Body

application/json

confidencesarray of CategoryConfidenceRequest (object)

Response

200

Request has succeeded

Body

application/json

messagestring

Request has succeeded.

Request

JavaScriptCurl

const response = await fetch('API_ENDPOINT{basePath}/v1/content-moderation/confidences', {
    method: 'PUT',
    headers: {
      "Content-Type": "application/json"
    },
    body: JSON.stringify({}),
});
const data = await response.json();

Response

{
  "message": {
    "message": "Success"
  }
}

We recommend adjusting the confidence levels for each category based on your moderation policies and the needs of your community. By doing so, you can ensure that Enhance Moderation provides the optimal level of moderation for your platform.