Review Settings in Labelo are essential for managing how tasks are evaluated after annotation. This feature ensures that tasks meet the required quality standards before final approval, maintaining consistency and accuracy across your project’s annotations.

You can customize these settings to fit your project’s specific needs, allowing for a thorough and tailored review process. By offering various options for assessing and approving tasks, Review Settings help streamline the evaluation workflow, enhancing overall accuracy and reliability.

Reviewers: These are users who assess and verify the quality of labeled tasks to ensure accuracy and consistency.

Review Instructions

• Provide detailed instructions for reviewers in this field, which supports HTML formatting.
• Activate “Show before labeling” to present a pop-up message to reviewers when they access the review stream. If this option is turned off, reviewers will have to click the “Show instructions” button at the bottom of the review interface to view the instructions.

Distribute Task Labeling

Auto	When this setting is activated, tasks are automatically assigned to the reviewer. This means that manual assignment is unnecessary, as tasks are automatically assigned to all project members, excluding those with the annotator role.
Manual	Tasks must be assigned to the reviewer individually by the project manager, administrator, or owner. Reviewers can access tasks only after they have been manually assigned.

Here’s a detailed guide on how to perform Manual tasks Distribution

1. Open the Project

• Navigate to your Labelo dashboard and select the specific project you wish to manage.

2. Go to Project Settings

• In the project, go to the “Settings” tab found in the upper right corner.

3. Set Task Distribution to Manual

• Under the “Review” section in the settings, find the “Task Distribution” option and set it to “Manual.”

4. Go to the Tasks Section

• Browse through the task list and select the task you want to assign, then click on the “Action” button, which is shown below.

5. Assign Reviewer

• Within the task details view, look for the option “Assign Reviewers“.

6. Choose Reviewer

• A list of all project members, excluding those with the annotator role, will be displayed. Choose the reviewer you want to assign to the task.

7. Confirm Assignment

• After selecting the annotator, click the ‘>‘ arrow to assign them, then click the “Assign” button to finalize. The assigned user will now appear in the Reviewer task list. To remove an assigned reviewer, select the user and click the ‘<‘ arrow. Using this method, you can assign tasks to both single and multiple users.

Review Instructions

• Provide instructions for the reviewer in this field that support HTML formatting.
• Enable “Show before labeling” to display a pop-up message to reviewers when they begin labeling. If this option is disabled, users will need to click the “Show instructions” button at the bottom of the labeling interface to view the instructions.

Labelo’s Review Settings are vital for ensuring the quality and accuracy of your project’s annotations. By customizing these settings, you can streamline the review process, making it more efficient and tailored to your project’s needs.