VST Teams · Video walkthrough · 3 min

Media classification

How VST Teams grades sexual and abusive material: your team sets the severity scale, the model proposes a grade for every file, and a reviewer confirms or overrides it.

Recorded on a development build, using a test set of adult images. Every image stays blurred throughout. The Poland profile on screen is an example set up for this test installation; your team sets its own scale. Captions in ten languages, from the CC menu.

  1. A jurisdiction profile maps the classifier's findings onto your team's scale. You can start from a grading scheme and adjust it label by label, or require a higher score before a label counts. Every change is recorded, and lowering the grade a label maps to needs a reason. The 0–5 scale shown is an example. Your team chooses its own.

  2. The model proposes a severity for each file and shows which detections produced it. In the profile shown, a subject with no face to estimate an age from is graded as a minor. Reviewers filter by severity and certainty, confirm in bulk or one at a time, and override with a single key. Blur and greyscale can be switched on for the whole review.

  3. Export a report for a case or a single folder. For every file it lists the model's proposed severity, its certainty and top reasons, the effective severity after review, who reviewed it, and the file's hashes.

Read the transcript

Part 1 · Severity scales

This is a walkthrough of our media classification workflow, designed for people who have to grade material that may contain child sexual abuse or violence. Classification systems vary greatly by country, so we have to give you a means of adapting what the machine sees to your local classification scheme. Here we can see our severity scale. This is where we set a mapping between things that the machine detects, body parts, nudity, sexual acts, fluids, and our classification levels. You don't have to start from scratch. We have several standard schemes that you can use as your starting point. This is how we can adapt the machine's detections to whatever local law or policy requires. Since we've got so many red flags on a single line, it's probably worthwhile the administrator having a look. It could be that the machine is over- or under-reacting to a particular visual element, and the administrator can change the detection thresholds.

Part 2 · Review auto-classification

When pictures and videos are uploaded into VST, the application will attempt to classify the media using the severity levels that you have configured. To check the results, enter the review workflow. We can select a case or a folder to continue a review task that we began earlier. Now I'm testing with benign but explicit adult images, so I've turned on blur and greyscale. Now I'm going to filter so that I just see the images where the AI is very sure of the severity that it has selected. These all look right to me, so I'm just going to select all and confirm. Now let's assume I want to question the severity class the machine has applied to one of the images. I can double-click the image to open a focused view, and then expand this drawer on the right to see why the image has been assigned a severity class of 3. I can click the X to disagree with any specific finding, or I can simply go down to the bottom and override.

Part 3 · Report

Now I'm going to show you how to create a report, which we can do at the level of a case or of a specific folder. We currently support four formats for the report. I'm going to choose Excel. The spreadsheets make for a really rich export format because you can see why each image received the classification that it did. And you can also see who made the determination. Although I didn't demonstrate it today, we could have classified images using a pre-existing hash set. If you'd like to see a demo of the full application, please get in touch, [email protected].

Questions: [email protected]