Labeling Quality

Quality has to be at the center of every project in order to reach success. We have designed a flow to make sure all annotations are up to the company standards.

When reviewing, the QAs can flag the wrong tasks. They can also add a comment for the specific task to further explain what's wrong or what was expected on that task.



When the annotators check on their flagged tasks, they can make the necessary changes to fix them. To let the QA know, they can click on "mark as fixed". They can add a comment if needed too.



Finally, the QA can review again to make sure it's now correct. If it is, they can mark it as "closed"; if it's not, they can flag it again.



The task will now look like this if it's correct:



Or it will look like this if the QA still needs to review the correction:





Also, on the "Issues" tab it will appear with the green check icon if it's closed or the round arrow icon if it's fixed by the annotator but not closed by the QA. You will be able to see the comments and the exact time when it was flagged, fixed and closed.