STRIVE · 08/28/2026
Dyn generalization · val_24 · super_clevr_1023
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"brown utility\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 0 669 1 2 669 2 2 659 3 2 655 4 5 654 5 2 651 7 5 652 8 8 654 9 16 654 10 18 653 11 23 653 12 26 654 13 33 655 14 36 654 15 5 678 16 47 656 17 11 676 18 55 656 19 21 679 20 24 678 21 28 679 22 34 680 23 35 678 [OBJ] 1 \"gray jet\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 365 246 1 366 248 2 369 250 3 370 253 4 372 257 5 374 263 6 376 268 7 378 275 8 380 281 9 381 289 10 383 297 11 385 306 12 387 316 13 388 325 14 391 335 15 392 347 16 393 358 17 393 369 18 394 381 19 396 395 20 397 408 21 411 418 22 412 430 23 408 449 [OBJ] 2 \"red cruiser\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 556 655 13 552 657 22 556 655 [OBJ] 3 \"purple minivan\" 23 24 [VIS] 0 false 23 true [IN-FRAME] 0 false 23 true [PX] 0 null null 23 0 644 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/val_24/super_clevr_1023/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"red and dark grey plastic flashlight\" 0 24 [VIS] 0 true 10 false [IN-FRAME] 0 true 10 false [PX] 0 1 677 1 5 689 2 5 677 3 9 689 4 13 701 5 13 685 6 17 697 7 17 685 8 13 666 10 null null [OBJ] 1 \"grey and white baked good\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 1 994 1 null null [OBJ] 2 \"brown cardboard cinnamon coffeecake kit box\" 0 24 [VIS] 0 true 15 false [IN-FRAME] 0 true 15 false [PX] 0 13 525 1 9 517 6 5 525 10 1 517 15 null null [OBJ] 3 \"brown cardboard cinnamon coffeecake kit box\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 142 443 2 126 447 3 115 447 4 103 447 5 91 447 6 72 447 7 185 447 8 173 447 9 162 447 10 150 447 11 138 447 12 123 447 13 111 447 14 95 447 15 83 447 16 68 447 17 185 443 18 173 443 19 158 443 20 146 443 21 130 443 22 115 443 23 99 443 [OBJ] 4 \"red and white leather high-top sneaker\" 0 24 [VIS] 0 false 21 true [IN-FRAME] 0 true [PX] 0 null null 21 1 455 [OBJ] 5 \"red and blue plastic Mario action figure\" 0 24 [VIS] 0 false 22 true [IN-FRAME] 0 true [PX] 0 null null 22 220 458 [OBJ] 6 \"grey and white cardboard hair color box\" 0 24 [VIS] 0 false 22 true [IN-FRAME] 0 true [PX] 0 null null 22 240 458 [OBJ] 7 \"grey and white plush shark\" 0 24 [VIS] 0 false 22 true [IN-FRAME] 0 true [PX] 0 null null 22 263 458 [OBJ] 8 \"dark grey plastic House of Cards DVD case\" 0 24 [VIS] 0 false 22 true [IN-FRAME] 0 true [PX] 0 null null 22 291 458 [OBJ] 9 \"dark grey plastic supplement bottle with orange cap\" 0 24 [VIS] 0 false 22 true [IN-FRAME] 0 true [PX] 0 null null 22 310 458 [OBJ] 10 \"red and blue plastic Mario action figure\" 0 24 [VIS] 0 false 22 true [IN-FRAME] 0 true [PX] 0 null null 22 314 458 [OBJ] 11 \"red and black plastic handheld game console\" 0 24 [VIS] 0 false 22 true [IN-FRAME] 0 true [PX] 0 null null 22 330 458 [OBJ] 12 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 12 false [IN-FRAME] 0 true [PX] 0 365 447 11 357 451 12 null null [OBJ] 13 \"grey and blue running shoe\" 0 24 [VIS] 0 true 5 false 10 true 12 false 13 true [IN-FRAME] 0 true [PX] 0 431 447 5 null null 10 435 447 12 null null 13 431 447 [OBJ] 14 \"dark grey and light grey plastic monster action figure\" 0 24 [VIS] 0 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 435 455 11 null null 12 435 455 [OBJ] 15 \"dark grey plastic House of Cards DVD case\" 0 24 [VIS] 0 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 462 447 11 null null 12 458 451 [OBJ] 16 \"red and white leather high-top sneaker\" 0 24 [VIS] 0 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 466 451 11 null null 12 462 455 [OBJ] 17 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 501 447 11 null null 12 498 451 [OBJ] 18 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 11 false 13 true [IN-FRAME] 0 true [PX] 0 505 451 11 null null 13 501 455 [OBJ] 19 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 11 false 13 true [IN-FRAME] 0 true [PX] 0 548 451 11 null null 13 544 455 [OBJ] 20 \"red and white leather high-top sneaker\" 0 24 [VIS] 0 true 11 false 14 true [IN-FRAME] 0 true [PX] 0 583 447 11 null null 14 576 451 [OBJ] 21 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 11 false 15 true [IN-FRAME] 0 true [PX] 0 583 451 11 null null 15 580 455 [OBJ] 22 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 11 false 16 true [IN-FRAME] 0 true [PX] 0 611 451 11 null null 16 607 455 [OBJ] 23 \"red and white cardboard ink cartridge box\" 0 24 [VIS] 0 true 11 false 17 true [IN-FRAME] 0 true [PX] 0 615 447 11 null null 17 611 451 [OBJ] 24 \"red and white leather high-top sneaker\" 0 24 [VIS] 0 true 11 false 17 true [IN-FRAME] 0 true [PX] 0 615 455 11 null null 17 611 458 [OBJ] 25 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 11 false 18 true [IN-FRAME] 0 true [PX] 0 638 451 11 null null 18 634 455 [OBJ] 26 \"red and white leather high-top sneaker\" 0 24 [VIS] 0 true 11 false 21 true [IN-FRAME] 0 true [PX] 0 642 447 11 null null 21 638 451 [OBJ] 27 \"red and white leather high-top sneaker\" 0 24 [VIS] 0 true 11 false 22 true [IN-FRAME] 0 true [PX] 0 650 451 11 null null 22 646 455 [OBJ] 28 \"red and white leather high-top sneaker\" 0 24 [VIS] 0 true 11 false 23 true [IN-FRAME] 0 true [PX] 0 658 447 11 null null 23 654 451 [OBJ] 29 \"grey and white plastic shark figurine\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 662 451 13 650 451 14 662 455 [OBJ] 30 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 11 false 16 true [IN-FRAME] 0 true [PX] 0 712 451 11 null null 16 705 455 [OBJ] 31 \"dark grey and light grey plastic monster action figure\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 724 451 8 716 455 11 724 451 12 716 455 [OBJ] 32 \"red and yellow wooden toy airplane with pilot\" 0 24 [VIS] 0 true 11 false 15 true [IN-FRAME] 0 true [PX] 0 740 451 11 null null 15 732 455 [OBJ] 33 \"grey and white plastic shark figurine\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 748 455 11 736 455 12 748 455 [OBJ] 34 \"red and white leather high-top sneaker\" 0 24 [VIS] 0 true 11 false 18 true [IN-FRAME] 0 true [PX] 0 755 451 11 null null 18 748 455 [OBJ] 35 \"grey plastic dog bowl with bones\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 791 451 10 783 455 [OBJ] 36 \"dark grey and yellow running shoe\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 912 451 7 904 455 15 896 458 23 888 462 [OBJ] 37 \"dark brown leather boat shoe\" 0 24 [VIS] 0 true 11 false [IN-FRAME] 0 true 11 false [PX] 0 994 458 11 null null [OBJ] 38 \"red and black plastic handheld game console\" 1 24 [VIS] 0 false 1 true [IN-FRAME] 0 false 1 true [PX] 0 null null 1 9 685 2 13 693 3 13 669 4 17 677 5 21 685 6 21 666 7 25 673 8 29 681 9 29 662 10 33 669 11 33 654 12 37 662 13 41 669 14 41 654 15 44 662 16 48 669 17 48 654 18 52 662 19 52 646 20 56 654 21 60 662 22 60 646 23 64 654 [OBJS-END] [INTERACTIONS-START] [INTERACT] 2 \"contact\" 0 3 [IN] 0 38 [OUT] 0 38 [INTERACT] 33 \"contact\" 0 4 [IN] 2 38 [OUT] 2 38 [INTERACT] 39 \"contact\" 0 4 [IN] 3 4 [OUT] 3 4 [INTERACT] 43 \"contact\" 0 4 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 34 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 49 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 50 \"contact\" 0 23 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 51 \"contact\" 0 23 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 52 \"contact\" 0 23 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 53 \"contact\" 0 23 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 54 \"contact\" 0 23 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 5 \"contact\" 0 23 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 6 \"contact\" 0 23 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 7 \"contact\" 0 23 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 8 \"contact\" 0 23 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 9 \"contact\" 0 23 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 10 \"contact\" 0 23 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 11 \"contact\" 0 23 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 12 \"contact\" 0 23 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 13 \"contact\" 0 23 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 14 \"contact\" 0 23 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 15 \"contact\" 0 23 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 16 \"contact\" 0 23 [IN] 22 65535 [OUT] 22 65535 [INTERACT] 17 \"contact\" 0 23 [IN] 23 65535 [OUT] 23 65535 [INTERACT] 18 \"contact\" 0 23 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 19 \"contact\" 0 23 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 20 \"contact\" 0 23 [IN] 26 65535 [OUT] 26 65535 [INTERACT] 21 \"contact\" 0 23 [IN] 27 65535 [OUT] 27 65535 [INTERACT] 22 \"contact\" 0 23 [IN] 28 65535 [OUT] 28 65535 [INTERACT] 23 \"contact\" 0 23 [IN] 29 65535 [OUT] 29 65535 [INTERACT] 24 \"contact\" 0 23 [IN] 30 65535 [OUT] 30 65535 [INTERACT] 25 \"contact\" 0 23 [IN] 31 65535 [OUT] 31 65535 [INTERACT] 26 \"contact\" 0 23 [IN] 32 65535 [OUT] 32 65535 [INTERACT] 27 \"contact\" 0 23 [IN] 33 65535 [OUT] 33 65535 [INTERACT] 28 \"contact\" 0 23 [IN] 34 65535 [OUT] 34 65535 [INTERACT] 29 \"contact\" 0 23 [IN] 35 65535 [OUT] 35 65535 [INTERACT] 30 \"contact\" 0 23 [IN] 36 65535 [OUT] 36 65535 [INTERACT] 31 \"contact\" 0 23 [IN] 37 65535 [OUT] 37 65535 [INTERACT] 32 \"contact\" 0 23 [IN] 38 65535 [OUT] 38 65535 [INTERACT] 44 \"collision\" 6 6 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 45 \"contact\" 7 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 46 \"collision\" 9 9 [IN] 3 4 [OUT] 3 4 [INTERACT] 48 \"collision\" 9 9 [IN] 3 6 [OUT] 3 6 [INTERACT] 35 \"collision\" 10 10 [IN] 3 12 [OUT] 3 12 [INTERACT] 36 \"contact\" 11 12 [IN] 3 12 [OUT] 3 12 [INTERACT] 40 \"collision\" 12 12 [IN] 3 4 [OUT] 3 4 [INTERACT] 41 \"contact\" 13 14 [IN] 3 4 [OUT] 3 4 [INTERACT] 0 \"collision\" 14 14 [IN] 0 38 [OUT] 0 38 [INTERACT] 37 \"collision\" 14 14 [IN] 3 12 [OUT] 3 12 [INTERACT] 42 \"collision\" 16 16 [IN] 3 4 [OUT] 3 4 [INTERACT] 38 \"collision\" 18 18 [IN] 3 12 [OUT] 3 12 [INTERACT] 47 \"collision\" 23 23 [IN] 3 4 [OUT] 3 4 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"brown utility\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 0 669 1 2 669 2 2 659 3 2 655 4 5 654 5 2 651 7 5 652 8 8 654 9 16 654 10 18 653 11 23 653 12 26 654 13 33 655 14 36 654 15 5 678 16 47 656 17 11 676 18 55 656 19 21 679 20 24 678 21 28 679 22 34 680 23 35 678 [OBJ] 1 \"gray jet\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 365 246 1 366 248 2 369 250 3 370 253 4 372 257 5 374 263 6 376 268 7 378 275 8 380 281 9 381 289 10 383 297 11 385 306 12 387 316 13 388 325 14 391 335 15 392 347 16 393 358 17 393 369 18 394 381 19 396 395 20 397 408 21 411 418 22 412 430 23 408 449 [OBJ] 2 \"red cruiser\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 556 655 13 552 657 22 556 655 [OBJ] 3 \"purple minivan\" 23 24 [VIS] 0 false 23 true [IN-FRAME] 0 false 23 true [PX] 0 null null 23 0 644 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "red and dark grey plastic flashlight",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 10,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 10,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 677
},
{
"t": 1,
"x": 5,
"y": 689
},
{
"t": 2,
"x": 5,
"y": 677
},
{
"t": 3,
"x": 9,
"y": 689
},
{
"t": 4,
"x": 13,
"y": 701
},
{
"t": 5,
"x": 13,
"y": 685
},
{
"t": 6,
"x": 17,
"y": 697
},
{
"t": 7,
"x": 17,
"y": 685
},
{
"t": 8,
"x": 13,
"y": 666
},
{
"t": 10,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "grey and white baked good",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "brown cardboard cinnamon coffeecake kit box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 15,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 15,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 13,
"y": 525
},
{
"t": 1,
"x": 9,
"y": 517
},
{
"t": 6,
"x": 5,
"y": 525
},
{
"t": 10,
"x": 1,
"y": 517
},
{
"t": 15,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "brown cardboard cinnamon coffeecake kit box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 142,
"y": 443
},
{
"t": 2,
"x": 126,
"y": 447
},
{
"t": 3,
"x": 115,
"y": 447
},
{
"t": 4,
"x": 103,
"y": 447
},
{
"t": 5,
"x": 91,
"y": 447
},
{
"t": 6,
"x": 72,
"y": 447
},
{
"t": 7,
"x": 185,
"y": 447
},
{
"t": 8,
"x": 173,
"y": 447
},
{
"t": 9,
"x": 162,
"y": 447
},
{
"t": 10,
"x": 150,
"y": 447
},
{
"t": 11,
"x": 138,
"y": 447
},
{
"t": 12,
"x": 123,
"y": 447
},
{
"t": 13,
"x": 111,
"y": 447
},
{
"t": 14,
"x": 95,
"y": 447
},
{
"t": 15,
"x": 83,
"y": 447
},
{
"t": 16,
"x": 68,
"y": 447
},
{
"t": 17,
"x": 185,
"y": 443
},
{
"t": 18,
"x": 173,
"y": 443
},
{
"t": 19,
"x": 158,
"y": 443
},
{
"t": 20,
"x": 146,
"y": 443
},
{
"t": 21,
"x": 130,
"y": 443
},
{
"t": 22,
"x": 115,
"y": 443
},
{
"t": 23,
"x": 99,
"y": 443
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "red and white leather high-top sneaker",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 21,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 21,
"x": 1,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 5,
"type": "red and blue plastic Mario action figure",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 22,
"x": 220,
"y": 458
}
],
"birth": 0,
"death": 24
},
{
"id": 6,
"type": "grey and white cardboard hair color box",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 22,
"x": 240,
"y": 458
}
],
"birth": 0,
"death": 24
},
{
"id": 7,
"type": "grey and white plush shark",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 22,
"x": 263,
"y": 458
}
],
"birth": 0,
"death": 24
},
{
"id": 8,
"type": "dark grey plastic House of Cards DVD case",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 22,
"x": 291,
"y": 458
}
],
"birth": 0,
"death": 24
},
{
"id": 9,
"type": "dark grey plastic supplement bottle with orange cap",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 22,
"x": 310,
"y": 458
}
],
"birth": 0,
"death": 24
},
{
"id": 10,
"type": "red and blue plastic Mario action figure",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 22,
"x": 314,
"y": 458
}
],
"birth": 0,
"death": 24
},
{
"id": 11,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 22,
"x": 330,
"y": 458
}
],
"birth": 0,
"death": 24
},
{
"id": 12,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 12,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 365,
"y": 447
},
{
"t": 11,
"x": 357,
"y": 451
},
{
"t": 12,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 13,
"type": "grey and blue running shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 5,
"v": false
},
{
"t": 10,
"v": true
},
{
"t": 12,
"v": false
},
{
"t": 13,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 431,
"y": 447
},
{
"t": 5,
"x": null,
"y": null
},
{
"t": 10,
"x": 435,
"y": 447
},
{
"t": 12,
"x": null,
"y": null
},
{
"t": 13,
"x": 431,
"y": 447
}
],
"birth": 0,
"death": 24
},
{
"id": 14,
"type": "dark grey and light grey plastic monster action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 435,
"y": 455
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 435,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 15,
"type": "dark grey plastic House of Cards DVD case",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 462,
"y": 447
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 458,
"y": 451
}
],
"birth": 0,
"death": 24
},
{
"id": 16,
"type": "red and white leather high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 466,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 462,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 17,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 501,
"y": 447
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 498,
"y": 451
}
],
"birth": 0,
"death": 24
},
{
"id": 18,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 13,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 505,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 13,
"x": 501,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 19,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 13,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 548,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 13,
"x": 544,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 20,
"type": "red and white leather high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 14,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 583,
"y": 447
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 14,
"x": 576,
"y": 451
}
],
"birth": 0,
"death": 24
},
{
"id": 21,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 15,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 583,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 15,
"x": 580,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 22,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 16,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 611,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 16,
"x": 607,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 23,
"type": "red and white cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 17,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 615,
"y": 447
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 17,
"x": 611,
"y": 451
}
],
"birth": 0,
"death": 24
},
{
"id": 24,
"type": "red and white leather high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 17,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 615,
"y": 455
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 17,
"x": 611,
"y": 458
}
],
"birth": 0,
"death": 24
},
{
"id": 25,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 18,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 638,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 18,
"x": 634,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 26,
"type": "red and white leather high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 21,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 642,
"y": 447
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 21,
"x": 638,
"y": 451
}
],
"birth": 0,
"death": 24
},
{
"id": 27,
"type": "red and white leather high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 650,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 22,
"x": 646,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 28,
"type": "red and white leather high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 23,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 658,
"y": 447
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 23,
"x": 654,
"y": 451
}
],
"birth": 0,
"death": 24
},
{
"id": 29,
"type": "grey and white plastic shark figurine",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 662,
"y": 451
},
{
"t": 13,
"x": 650,
"y": 451
},
{
"t": 14,
"x": 662,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 30,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 16,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 712,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 16,
"x": 705,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 31,
"type": "dark grey and light grey plastic monster action figure",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 724,
"y": 451
},
{
"t": 8,
"x": 716,
"y": 455
},
{
"t": 11,
"x": 724,
"y": 451
},
{
"t": 12,
"x": 716,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 32,
"type": "red and yellow wooden toy airplane with pilot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 15,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 740,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 15,
"x": 732,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 33,
"type": "grey and white plastic shark figurine",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 748,
"y": 455
},
{
"t": 11,
"x": 736,
"y": 455
},
{
"t": 12,
"x": 748,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 34,
"type": "red and white leather high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 18,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 755,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 18,
"x": 748,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 35,
"type": "grey plastic dog bowl with bones",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 791,
"y": 451
},
{
"t": 10,
"x": 783,
"y": 455
}
],
"birth": 0,
"death": 24
},
{
"id": 36,
"type": "dark grey and yellow running shoe",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 912,
"y": 451
},
{
"t": 7,
"x": 904,
"y": 455
},
{
"t": 15,
"x": 896,
"y": 458
},
{
"t": 23,
"x": 888,
"y": 462
}
],
"birth": 0,
"death": 24
},
{
"id": 37,
"type": "dark brown leather boat shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 994,
"y": 458
},
{
"t": 11,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 38,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 1,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 1,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 1,
"x": 9,
"y": 685
},
{
"t": 2,
"x": 13,
"y": 693
},
{
"t": 3,
"x": 13,
"y": 669
},
{
"t": 4,
"x": 17,
"y": 677
},
{
"t": 5,
"x": 21,
"y": 685
},
{
"t": 6,
"x": 21,
"y": 666
},
{
"t": 7,
"x": 25,
"y": 673
},
{
"t": 8,
"x": 29,
"y": 681
},
{
"t": 9,
"x": 29,
"y": 662
},
{
"t": 10,
"x": 33,
"y": 669
},
{
"t": 11,
"x": 33,
"y": 654
},
{
"t": 12,
"x": 37,
"y": 662
},
{
"t": 13,
"x": 41,
"y": 669
},
{
"t": 14,
"x": 41,
"y": 654
},
{
"t": 15,
"x": 44,
"y": 662
},
{
"t": 16,
"x": 48,
"y": 669
},
{
"t": 17,
"x": 48,
"y": 654
},
{
"t": 18,
"x": 52,
"y": 662
},
{
"t": 19,
"x": 52,
"y": 646
},
{
"t": 20,
"x": 56,
"y": 654
},
{
"t": 21,
"x": 60,
"y": 662
},
{
"t": 22,
"x": 60,
"y": 646
},
{
"t": 23,
"x": 64,
"y": 654
}
],
"birth": 1,
"death": 24
}
],
"interactions": [
{
"id": 2,
"inputs": [
0,
38
],
"outputs": [
0,
38
],
"begins_at": 0,
"ends_at": 3,
"type": "contact"
},
{
"id": 33,
"inputs": [
2,
38
],
"outputs": [
2,
38
],
"begins_at": 0,
"ends_at": 4,
"type": "contact"
},
{
"id": 39,
"inputs": [
3,
4
],
"outputs": [
3,
4
],
"begins_at": 0,
"ends_at": 4,
"type": "contact"
},
{
"id": 43,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 4,
"type": "contact"
},
{
"id": 1,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 34,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 49,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 50,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 51,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 52,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 53,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 54,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 4,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 5,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 6,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 7,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 8,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 9,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 10,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 11,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 12,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 13,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 14,
"inputs": [
20,
65535
],
"outputs": [
20,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 15,
"inputs": [
21,
65535
],
"outputs": [
21,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 16,
"inputs": [
22,
65535
],
"outputs": [
22,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 17,
"inputs": [
23,
65535
],
"outputs": [
23,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 18,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 19,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 20,
"inputs": [
26,
65535
],
"outputs": [
26,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 21,
"inputs": [
27,
65535
],
"outputs": [
27,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 22,
"inputs": [
28,
65535
],
"outputs": [
28,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 23,
"inputs": [
29,
65535
],
"outputs": [
29,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 24,
"inputs": [
30,
65535
],
"outputs": [
30,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 25,
"inputs": [
31,
65535
],
"outputs": [
31,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 26,
"inputs": [
32,
65535
],
"outputs": [
32,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 27,
"inputs": [
33,
65535
],
"outputs": [
33,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 28,
"inputs": [
34,
65535
],
"outputs": [
34,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 29,
"inputs": [
35,
65535
],
"outputs": [
35,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 30,
"inputs": [
36,
65535
],
"outputs": [
36,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 31,
"inputs": [
37,
65535
],
"outputs": [
37,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 32,
"inputs": [
38,
65535
],
"outputs": [
38,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 44,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 6,
"ends_at": 6,
"type": "collision"
},
{
"id": 45,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 7,
"ends_at": 23,
"type": "contact"
},
{
"id": 46,
"inputs": [
3,
4
],
"outputs": [
3,
4
],
"begins_at": 9,
"ends_at": 9,
"type": "collision"
},
{
"id": 48,
"inputs": [
3,
6
],
"outputs": [
3,
6
],
"begins_at": 9,
"ends_at": 9,
"type": "collision"
},
{
"id": 35,
"inputs": [
3,
12
],
"outputs": [
3,
12
],
"begins_at": 10,
"ends_at": 10,
"type": "collision"
},
{
"id": 36,
"inputs": [
3,
12
],
"outputs": [
3,
12
],
"begins_at": 11,
"ends_at": 12,
"type": "contact"
},
{
"id": 40,
"inputs": [
3,
4
],
"outputs": [
3,
4
],
"begins_at": 12,
"ends_at": 12,
"type": "collision"
},
{
"id": 41,
"inputs": [
3,
4
],
"outputs": [
3,
4
],
"begins_at": 13,
"ends_at": 14,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
38
],
"outputs": [
0,
38
],
"begins_at": 14,
"ends_at": 14,
"type": "collision"
},
{
"id": 37,
"inputs": [
3,
12
],
"outputs": [
3,
12
],
"begins_at": 14,
"ends_at": 14,
"type": "collision"
},
{
"id": 42,
"inputs": [
3,
4
],
"outputs": [
3,
4
],
"begins_at": 16,
"ends_at": 16,
"type": "collision"
},
{
"id": 38,
"inputs": [
3,
12
],
"outputs": [
3,
12
],
"begins_at": 18,
"ends_at": 18,
"type": "collision"
},
{
"id": 47,
"inputs": [
3,
4
],
"outputs": [
3,
4
],
"begins_at": 23,
"ends_at": 23,
"type": "collision"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "brown utility",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 0,
"y": 669
},
{
"t": 1,
"x": 2,
"y": 669
},
{
"t": 2,
"x": 2,
"y": 659
},
{
"t": 3,
"x": 2,
"y": 655
},
{
"t": 4,
"x": 5,
"y": 654
},
{
"t": 5,
"x": 2,
"y": 651
},
{
"t": 7,
"x": 5,
"y": 652
},
{
"t": 8,
"x": 8,
"y": 654
},
{
"t": 9,
"x": 16,
"y": 654
},
{
"t": 10,
"x": 18,
"y": 653
},
{
"t": 11,
"x": 23,
"y": 653
},
{
"t": 12,
"x": 26,
"y": 654
},
{
"t": 13,
"x": 33,
"y": 655
},
{
"t": 14,
"x": 36,
"y": 654
},
{
"t": 15,
"x": 5,
"y": 678
},
{
"t": 16,
"x": 47,
"y": 656
},
{
"t": 17,
"x": 11,
"y": 676
},
{
"t": 18,
"x": 55,
"y": 656
},
{
"t": 19,
"x": 21,
"y": 679
},
{
"t": 20,
"x": 24,
"y": 678
},
{
"t": 21,
"x": 28,
"y": 679
},
{
"t": 22,
"x": 34,
"y": 680
},
{
"t": 23,
"x": 35,
"y": 678
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "gray jet",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 365,
"y": 246
},
{
"t": 1,
"x": 366,
"y": 248
},
{
"t": 2,
"x": 369,
"y": 250
},
{
"t": 3,
"x": 370,
"y": 253
},
{
"t": 4,
"x": 372,
"y": 257
},
{
"t": 5,
"x": 374,
"y": 263
},
{
"t": 6,
"x": 376,
"y": 268
},
{
"t": 7,
"x": 378,
"y": 275
},
{
"t": 8,
"x": 380,
"y": 281
},
{
"t": 9,
"x": 381,
"y": 289
},
{
"t": 10,
"x": 383,
"y": 297
},
{
"t": 11,
"x": 385,
"y": 306
},
{
"t": 12,
"x": 387,
"y": 316
},
{
"t": 13,
"x": 388,
"y": 325
},
{
"t": 14,
"x": 391,
"y": 335
},
{
"t": 15,
"x": 392,
"y": 347
},
{
"t": 16,
"x": 393,
"y": 358
},
{
"t": 17,
"x": 393,
"y": 369
},
{
"t": 18,
"x": 394,
"y": 381
},
{
"t": 19,
"x": 396,
"y": 395
},
{
"t": 20,
"x": 397,
"y": 408
},
{
"t": 21,
"x": 411,
"y": 418
},
{
"t": 22,
"x": 412,
"y": 430
},
{
"t": 23,
"x": 408,
"y": 449
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "red cruiser",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 556,
"y": 655
},
{
"t": 13,
"x": 552,
"y": 657
},
{
"t": 22,
"x": 556,
"y": 655
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "purple minivan",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 23,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 23,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 23,
"x": 0,
"y": 644
}
],
"birth": 23,
"death": 24
}
],
"interactions": [
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.3656138473164636,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 628.4333218517711,
"avg_image_position_segmentation_acc": 0.020833333333333332,
"avg_image_position_segmentation_acc_small": 0.0,
"avg_image_position_segmentation_acc_medium": 0.0,
"avg_image_position_segmentation_acc_large": 0.08333333333333333,
"avg_image_position_segmentation_acc_first_frame": 0.0,
"avg_image_position_segmentation_visible_acc": 0.020833333333333332,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 0.5729166666666666,
"visibility_true_acc": 0.5833333333333333,
"visibility_false_acc": 0.9565217391304348,
"is_in_frame_acc": 0.6249999999999999,
"is_in_frame_true_acc": 0.8645833333333333,
"is_in_frame_false_acc": 0.0,
"vis_acc_from_px": 0.5729166666666666,
"vis_acc_from_px_true_acc": 0.5833333333333333,
"vis_acc_from_px_false_acc": 0.9565217391304348,
"num_matches": 4,
"num_unmatched_pred": 35,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 4,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 4,
"avg_image_position_segmentation_acc": 4,
"avg_image_position_segmentation_acc_small": 1,
"avg_image_position_segmentation_acc_medium": 2,
"avg_image_position_segmentation_acc_large": 1,
"avg_image_position_segmentation_acc_first_frame": 3,
"avg_image_position_segmentation_visible_acc": 4,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 4,
"visibility_true_acc": 4,
"visibility_false_acc": 1,
"is_in_frame_acc": 4,
"is_in_frame_true_acc": 4,
"is_in_frame_false_acc": 1,
"vis_acc_from_px": 4,
"vis_acc_from_px_true_acc": 4,
"vis_acc_from_px_false_acc": 1
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 37,
"target_id": 0,
"pred_id": 37,
"cost": 1.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 13,
"target_id": 1,
"pred_id": 13,
"cost": 1.4166666666666665,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.9166666666666666,
"weighted_cost": 0.9166666666666666,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 4,
"target_id": 2,
"pred_id": 4,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 7,
"target_id": 3,
"pred_id": 7,
"cost": 1.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
2,
3,
5,
6,
8,
9,
10,
11,
12,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
38
],
"target_type_counts": {
"brown utility": 1,
"gray jet": 1,
"red cruiser": 1,
"purple minivan": 1
},
"pred_type_counts": {
"red and dark grey plastic flashlight": 1,
"grey and white baked good": 1,
"brown cardboard cinnamon coffeecake kit box": 2,
"red and white leather high-top sneaker": 8,
"red and blue plastic Mario action figure": 2,
"grey and white cardboard hair color box": 1,
"grey and white plush shark": 1,
"dark grey plastic House of Cards DVD case": 2,
"dark grey plastic supplement bottle with orange cap": 1,
"red and black plastic handheld game console": 2,
"red and yellow wooden toy airplane with pilot": 9,
"grey and blue running shoe": 1,
"dark grey and light grey plastic monster action figure": 2,
"red and white cardboard ink cartridge box": 1,
"grey and white plastic shark figurine": 2,
"grey plastic dog bowl with bones": 1,
"dark grey and yellow running shoe": 1,
"dark brown leather boat shoe": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 0.5,
"1.0": 0.0,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 36,
"false_negatives": 1
},
"0.5": {
"true_positives": 2,
"false_positives": 37,
"false_negatives": 2
},
"0.75": {
"true_positives": 2,
"false_positives": 37,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 4
}
},
"visibility_precision_by_threshold": {
"0.25": 0.07692307692307693,
"0.5": 0.05128205128205128,
"0.75": 0.05128205128205128,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.75,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.13953488372093026,
"0.5": 0.09302325581395347,
"0.75": 0.09302325581395347,
"1.0": 0.0
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 1,
"false_positives": 2,
"false_negatives": 2
},
"0.75": {
"true_positives": 1,
"false_positives": 2,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 3
}
},
"visibility_precision_by_threshold": {
"0.25": 0.6666666666666666,
"0.5": 0.3333333333333333,
"0.75": 0.3333333333333333,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.6666666666666666,
"0.5": 0.3333333333333333,
"0.75": 0.3333333333333333,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.6666666666666666,
"0.5": 0.3333333333333333,
"0.75": 0.3333333333333333,
"1.0": 0.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 35,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 35,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 36,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.027777777777777776,
"0.5": 0.027777777777777776,
"0.75": 0.027777777777777776,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.05405405405405406,
"0.5": 0.05405405405405406,
"0.75": 0.05405405405405406,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0,
"segmentation_by_motion_group": {
"static": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 3
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 36,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 36,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 36,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 36,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 36,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 37,
"cost": 1.5,
"type_similarity": 0.34544373768731695,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 1278.3756975021245,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.4583333333333333,
"visibility_true_acc": 0.4583333333333333,
"visibility_false_acc": null,
"is_in_frame_acc": 0.4583333333333333,
"is_in_frame_true_acc": 0.4583333333333333,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.4583333333333333,
"vis_acc_from_px_true_acc": 0.4583333333333333,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 13,
"cost": 1.4166666666666665,
"type_similarity": 0.420790944745116,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 120.78905328934908,
"image_position_segmentation_acc": 0.08333333333333333,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.08333333333333333,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.08333333333333333,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.75,
"visibility_true_acc": 0.75,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.75,
"vis_acc_from_px_true_acc": 0.75,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 4,
"cost": 1.0,
"type_similarity": 0.4014247417849204,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 733.8235169436525,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.125,
"visibility_true_acc": 0.125,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.125,
"vis_acc_from_px_true_acc": 0.125,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 7,
"cost": 1.5,
"type_similarity": 0.2947959650485012,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 380.7450196719584,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": 0.0,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": null,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.9583333333333334,
"visibility_true_acc": 1.0,
"visibility_false_acc": 0.9565217391304348,
"is_in_frame_acc": 0.041666666666666664,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": 0.0,
"vis_acc_from_px": 0.9583333333333334,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": 0.9565217391304348
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": 1.0,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 3,
"num_unmatched_pred_interactions": 52,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.05454545454545454,
"0.5": 0.05454545454545454,
"0.75": 0.05454545454545454,
"1.0": 0.05454545454545454
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.10344827586206896,
"0.5": 0.10344827586206896,
"0.75": 0.10344827586206896,
"1.0": 0.10344827586206896
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.05454545454545454,
"0.5": 0.05454545454545454,
"0.75": 0.05454545454545454,
"1.0": 0.05454545454545454
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.10344827586206896,
"0.5": 0.10344827586206896,
"0.75": 0.10344827586206896,
"1.0": 0.10344827586206896
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 52,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.05454545454545454,
"0.5": 0.05454545454545454,
"0.75": 0.05454545454545454,
"1.0": 0.05454545454545454
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.10344827586206896,
"0.5": 0.10344827586206896,
"0.75": 0.10344827586206896,
"1.0": 0.10344827586206896
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 3,
"avg_interaction_input_object_jaccard": 3,
"avg_interaction_output_object_jaccard": 3,
"avg_interaction_type_exact_match_acc": 3
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 31,
"target_id": 0,
"pred_id": 31,
"cost": 0.0,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 49,
"target_id": 1,
"pred_id": 49,
"cost": 0.0,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 52,
"target_id": 2,
"pred_id": 52,
"cost": 0.0,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
50,
51,
53,
54
],
"target_type_counts": {
"contact": 3
},
"pred_type_counts": {
"collision": 10,
"contact": 45
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 31,
"target_id": 0,
"pred_id": 31,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.0,
"temporal_span_iou": 1.0,
"temporal_span_giou": 1.0,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 1.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 1,
"pred_index": 49,
"target_id": 1,
"pred_id": 49,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.0,
"temporal_span_iou": 1.0,
"temporal_span_giou": 1.0,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 1.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 2,
"pred_index": 52,
"target_id": 2,
"pred_id": 52,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.0,
"temporal_span_iou": 1.0,
"temporal_span_giou": 1.0,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 1.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 1.0,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 3,
"num_unmatched_pred_interactions": 38,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.07317073170731707,
"0.5": 0.07317073170731707,
"0.75": 0.07317073170731707,
"1.0": 0.07317073170731707
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.13636363636363635,
"0.5": 0.13636363636363635,
"0.75": 0.13636363636363635,
"1.0": 0.13636363636363635
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.07317073170731707,
"0.5": 0.07317073170731707,
"0.75": 0.07317073170731707,
"1.0": 0.07317073170731707
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.13636363636363635,
"0.5": 0.13636363636363635,
"0.75": 0.13636363636363635,
"1.0": 0.13636363636363635
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 38,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.07317073170731707,
"0.5": 0.07317073170731707,
"0.75": 0.07317073170731707,
"1.0": 0.07317073170731707
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.13636363636363635,
"0.5": 0.13636363636363635,
"0.75": 0.13636363636363635,
"1.0": 0.13636363636363635
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 14,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 3,
"avg_interaction_input_object_jaccard": 3,
"avg_interaction_output_object_jaccard": 3,
"avg_interaction_type_exact_match_acc": 3
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 10,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 10,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": 1.0,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 3,
"num_unmatched_pred_interactions": 42,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.06666666666666667,
"0.5": 0.06666666666666667,
"0.75": 0.06666666666666667,
"1.0": 0.06666666666666667
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.125,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.06666666666666667,
"0.5": 0.06666666666666667,
"0.75": 0.06666666666666667,
"1.0": 0.06666666666666667
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.125,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.06666666666666667,
"0.5": 0.06666666666666667,
"0.75": 0.06666666666666667,
"1.0": 0.06666666666666667
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.125,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"contact": {
"avg_interaction_temporal_span_giou": 3,
"avg_interaction_input_object_jaccard": 3,
"avg_interaction_output_object_jaccard": 3,
"avg_interaction_type_exact_match_acc": 3
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}