nocaps v0.1

Validation Set

  • Sourced from Open Images validation set (v4).
  • 4500 images, 10 captions per image.
    • 648 in-domain images.
    • 2938 near-domain images.
    • 914 out-domain images.

Download Validation Set Image Info

Test Set

  • Sourced from Open Images test set (v4).
  • 10600 images, 10 captions per image.
    • 1311 in-domain images.
    • 7406 near-domain images.
    • 1883 out-domain images.

Download Test Set Image Info

Annotations Format

To discourage training on the validation set, both validation and test reference captions will be kept private. However, there is no limit to the number of validation set evaluations that can be performed through the evaluation server. Further, to better understand the data, a random selection of validation set examples can be found on the Explore page. The image info files follow the COCO annotations format, and are compatible to read with COCO API. Their format (explained by an example) is as follows:
{
    "licenses": [],
    "info": {
        "url": "http://nocaps.org",
        "date_created": "2018/11/06",
        "version": "0.1",
        "description": "nocaps validation dataset",
        "contributor": "nocaps team",
        "year": 2018
    },
    "images": [
        {
            "id": 0,
            "open_images_id": "0013ea2087020901",
            "height": 1024,
            "width": 732,
            "coco_url": "https://requestor-proxy.figure-eight.com/figure_eight_datasets/open-images/validation/0013ea2087020901.jpg",
            "file_name": "0013ea2087020901.jpg",
            "license": 0,
            "date_captured": "2018-11-06 11:04:33"
        },
    ...
    ]
}

Evaluation Server

To facilitate evaluation and avoid exposing the novel object captions, we have also setup an evaluation server on EvalAI. Submission instructions and leaderboard are available here:

Website design courtesy: Karan Desai