Review evaluation results in Lab Review

Find evaluation items in the queue

Open Lab, then Review. The Pending tab lists requests waiting for a person to check them. Items that come from an evaluation carry one of two reasons: Judge disagreement or Judge failed. Use the Reason filter to show only those reasons, and the Assigned to filter to see your own items or those nobody has taken. Select Start reviewing to open the first pending item.

Check the answer before you see the scores

Open an item to see the Request and the Answer that was evaluated. For an evaluation item, judge scores stay hidden while the item is pending. The page shows a note that scores appear once you have reviewed the answer, so your decision is not influenced by them. If no candidate answer was saved for the result, the Answer card says so. Reject or reassign the item rather than guessing.

Score and decide

The Your review card lists the criteria attached to the item. Evaluation items use their evaluation rubric; other items use the project review rubric. Score each criterion from 1 to 5, where 1 is poor and 5 is right. Scores are optional. Add a note describing what was right or wrong. Then choose one of three decisions:

• Approve keeps the answer as it is. Press A.
• Correct opens a dialog where you write the answer that should have been given. Press C.
• Reject marks the answer as not acceptable. Press R.

Press J and K to move to the next and previous pending item. The shortcuts do not fire while you are typing in a field or dialog.

Save sign-offs to a dataset

Use Save to dataset to add an approved answer or a correction to an existing Lab dataset, or choose New dataset to create one. Rejected items are never saved. If the item already has a saved example, you can keep it, update it to match your new decision, or remove it. When you have decided an evaluation item, Open evaluation takes you to the evaluation run.

Assign, change and reopen

Use the Assigned to card to take an item yourself or assign it to another available member of your organisation. Change decision lets you edit a decision. Reopen returns the item to Pending, with an optional note explaining why. The earlier decision stays in the History card, and an example it saved is removed from its dataset. If someone else decides an item first, your save is refused and you are asked to refresh to see their decision.

If the queue looks empty

Nothing appears under Pending until requests are queued. Choose what gets queued in Settings, Project, Review. If an item says it is not in the queue, it may belong to another project, so choose All projects in the top bar.