Annotation Settings in Labelo allow you to control how tasks are labeled and managed within your project. Before diving into project annotation settings in labelo, take a look at what are the annotating Options to Improve the annotation Process in labelo. Which we covered in our previous guide.

Here’s a brief overview of the key settings:

Annotators:  These are the users responsible for labeling or tagging tasks within the project.

Distribute Task Labeling

Auto	When this setting is activated, tasks are automatically assigned to annotators. This means that manual assignment is unnecessary, as tasks are automatically assigned to all project members, excluding those with the reviewer role.
Manual	Tasks must be assigned to annotators individually by the project manager, administrator, or owner. Annotators can access tasks only after they have been manually assigned.

Here’s a detailed guide on how to perform Manual task Distribution:

1. Open the Project

• Navigate to your projects page and select the specific project you wish to manage.

2. Go to Project Settings

Within the project, click on the “Settings” tab located in the upper right corner.

3. Set Task Distribution to Manual

• Under the “Annotation” section in the settings, find the “Task Distribution” option and set it to “Manual.”

4. Go to the Tasks Section

• Browse through the task list and select the task you want to assign then click on the “Action” button which is shown in above.

5. Assign Annotator

• Within the task details view, look for the option “Assign Annotator“.

6. Choose Annotator

• A list of all project members, excluding those with the reviewer role, will be displayed. Choose the annotator you want to assign to the task.

7. Confirm Assignment

• After selecting the annotator, click the ‘>‘ arrow to assign them, then click the “Assign” button to finalize. The task will now appear in the annotator’s task list. To remove an assigned annotator, select the user and click the ‘<‘ arrow.

Labeling Instructions

• Provide instructions for the annotators in this field, which supports HTML formatting.
• Enable the “Show before labeling” option to present a pop-up message to annotators when they start labeling. If this option is turned off, annotators will have to click the “Show instructions” button at the bottom of the labeling interface to access the instructions.

Skip Queue

The Skip Queue is designed to improve efficiency and manage workload by allowing annotators to skip tasks they find difficult or irrelevant. 

This ensures that annotators focus on tasks they can handle effectively and helps maintain high-quality annotations.

Requeue Skipped Tasks Back to the Annotator	When an annotator skips a task, it moves to the end of the queue. They will see it again as they continue working through their tasks. If the annotator leaves and then returns to the labeling stream: Auto Distribution: They might see the skipped task again if it hasn’t been completed by someone else. If it’s still incomplete, they can update and resubmit it. Manual Distribution: They will keep seeing the skipped task until it’s labeled. Skipped tasks remain incomplete and affect the overall project progress shown on the project Dashboard, making it less than 100% if there are skipped tasks.
Requeue skipped tasks to others	When an annotator skips a task, it is taken out of their queue and given to another annotator. Once the annotator has skipped the task and finished their current set of tasks, they won’t be able to return to it. How the task is reassigned depends on the setup: Auto Distribution: The task is automatically reassigned to another annotator. Manual Distribution: Someone needs to manually assign the task to another annotator. If no one is available to take on the task, or if all annotators skip it, the task remains incomplete. This will affect the overall project progress shown on the Dashboard, making it less than 100% if there are skipped tasks.
Ignore skipped	* Auto Distribution: When an annotator skips a task, it is marked as completed and removed from their queue. If the task requires more annotations, it will be assigned to other annotators until the required number is met. * Manual Distribution: Skipped tasks are removed from the original annotator’s queue but remain in the queues of other annotators. In both methods, skipped tasks are considered completed and do not affect the overall project progress. This means the project’s progress can still be 100% if all other tasks are finished.

Annotating Options

Show Skip button	Use this option to control whether the “Skip” button is visible to annotators.
Allow empty annotations	This setting determines if annotators can submit tasks without making any annotations. When enabled, tasks can be submitted even if annotators haven’t added labels or regions, leading to an empty annotation.
Show the Data Manager to annotators	When this option is disabled, annotators can only access the label stream. When enabled, they get access to the Data Manager, allowing them to choose which tasks to complete from a list. However, some information remains hidden from annotators. They can view only a subset of the Data Manager columns and will not see details such as Annotators, Agreement, Reviewers, and others.
Annotators must leave a comment on skip	Annotators must leave a comment on the skip

Prelabeling 

If you are using an ML backend or model, or leveraging Prompts for predictions, you can adjust the settings to automatically pre-label tasks based on these predictions. Configure this feature by selecting the prediction source from the drop-down menu.

Task Sampling

Configure the sequence in which tasks are shown to annotators.

Uncertainty Sampling	When using a machine learning backend with active learning, this option helps improve your model. Active learning continuously trains and refines predictions from your model as new annotations are added. With Uncertainty Sampling enabled, Labelo picks tasks where the model is least confident or most uncertain. This helps reduce the amount of data that needs labeling while enhancing the model’s performance.
Sequential Sampling	Tasks are presented to annotators in the order they appear in the Data Manager.
Uniform Sampling	Tasks are displayed in a random order.

Effectively managing your annotation settings in Labelo can greatly enhance your project’s efficiency and model performance. By selecting the appropriate task distribution and sampling methods, you can streamline workflow and improve the quality of your annotations.

Mastering annotation settings in Labelo can greatly improve your project’s efficiency and model accuracy. By selecting the best task distribution and sampling methods, you ensure smoother workflows and more effective data labeling.

Successfully managing your tasks and streamlining workflows begins with a clear understanding of How to Set Up & Configure Projects in Labelo, as this process enables you to organize your projects efficiently, customize settings to meet specific needs, and ensure your team collaborates effectively toward achieving your goals.