How to create a new task

Vivaria supports running agents on tasks that conform to the METR Task Standard.

See the implementation instructions for a guide to implementing a new task, or see the count_odds task for a simple example that conforms to the standard.

Keeping old tasks around

If you've shared a task with other people, we recommend not meaningfully changing the task. Instead, you can create a new task in the same task family or create a new task family altogether. It could be confusing to your collaborators if the definition of a task changes meaningfully like this.