STRIVE · 08/28/2026
Dyn generalization · val_24 · super_clevr_1045
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"purple fighter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 12 228 1 19 228 2 24 229 3 31 230 4 40 231 5 51 233 6 57 234 7 13 233 8 16 232 9 24 235 10 33 239 11 41 242 12 49 245 13 57 249 14 66 254 15 74 258 16 83 264 17 91 269 18 101 275 19 109 280 20 118 286 21 127 294 22 137 301 23 146 308 [OBJ] 1 \"yellow scooter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 284 909 [OBJ] 2 \"yellow double\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 455 709 2 451 709 3 454 709 5 451 709 13 447 710 18 562 696 19 444 711 21 441 711 23 437 711 [OBJ] 3 \"purple utility\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 519 635 1 null null [OBJ] 4 \"green double\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 873 723 8 870 723 12 868 723 15 866 723 17 863 723 19 861 723 20 858 722 22 856 722 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/val_24/super_clevr_1045/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"pink and purple soccer cleat\" 0 1 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 1 169 1 null null [OBJ] 1 \"pink and purple soccer cleat\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 13 232 1 13 244 2 13 255 3 13 267 4 13 283 5 13 294 7 13 306 9 13 318 10 17 337 11 13 345 12 13 357 13 13 369 15 13 384 17 13 396 18 13 408 20 13 419 22 13 431 23 13 443 [OBJ] 2 \"dark blue leather boat shoe\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 5 990 1 null null [OBJ] 3 \"dark blue leather boat shoe\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 9 908 1 null null [OBJ] 4 \"dark blue fabric high-top boat shoe\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 13 986 1 null null [OBJ] 5 \"dark blue fabric chukka boot\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 9 994 1 null null [OBJ] 6 \"grey and white plush shark\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 56 353 1 68 357 2 80 361 3 91 369 4 103 373 5 115 376 6 126 380 7 138 384 8 150 392 9 162 396 10 173 400 11 181 404 12 193 408 13 201 412 14 208 416 15 216 419 16 224 423 17 232 427 18 240 431 19 248 435 20 255 439 21 263 443 22 271 447 23 279 451 [OBJ] 7 \"yellow wooden toy roller\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 29 994 1 null null [OBJ] 8 \"wooden toy tableware set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 291 998 1 null null [OBJ] 9 \"yellow wooden toy construction vehicle set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 294 990 1 null null [OBJ] 10 \"yellow wooden toy construction vehicle set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 302 998 1 null null [OBJ] 11 \"wooden toy vegetable cutting set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 306 994 1 null null [OBJ] 12 \"yellow wooden toy roller\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 310 990 1 null null [OBJ] 13 \"wooden toy juicer set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 318 998 1 null null [OBJ] 14 \"wooden toy meat set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 333 998 1 null null [OBJ] 15 \"yellow wooden toy construction vehicle set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 353 994 1 null null [OBJ] 16 \"yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 365 998 1 null null [OBJ] 17 \"yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 369 994 1 null null [OBJ] 18 \"pink and purple soccer cleat\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 37 228 1 37 240 2 37 251 3 41 259 4 44 267 5 48 275 6 52 283 7 56 291 8 60 298 9 68 302 10 76 310 11 83 314 12 91 318 13 99 322 14 107 326 15 115 330 16 123 333 17 130 337 18 138 341 19 146 345 20 154 349 21 162 353 22 169 357 23 177 361 [OBJ] 19 \"green Lego Android robot\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 474 998 1 null null [OBJ] 20 \"pink and black fabric flip-flop\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 501 998 1 null null [OBJ] 21 \"yellow cardboard DVD set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 505 994 1 null null [OBJ] 22 \"yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 517 994 1 null null [OBJ] 23 \"pink and purple wooden toy castle\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 521 994 1 null null [OBJ] 24 \"yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 537 994 1 null null [OBJ] 25 \"yellow cardboard Raisinets box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 552 994 1 null null [OBJ] 26 \"pink and white fabric high-top sneaker\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 564 994 1 null null [OBJ] 27 \"yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 580 994 1 null null [OBJ] 28 \"green and yellow running shoe\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 619 994 1 null null [OBJ] 29 \"yellow plastic plant pot\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 634 994 1 null null [OBJ] 30 \"yellow plastic chocolate powder canister\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 634 998 1 null null [OBJ] 31 \"yellow cardboard DVD set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 638 994 1 null null [OBJ] 32 \"green and yellow running shoe\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 650 994 1 null null [OBJ] 33 \"yellow plastic plant pot\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 654 990 1 null null [OBJ] 34 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 662 994 1 null null [OBJ] 35 \"yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 669 990 1 null null [OBJ] 36 \"yellow plastic chocolate powder canister\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 685 990 1 null null [OBJ] 37 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 697 990 1 null null [OBJ] 38 \"dark grey and light grey plastic monster action figure\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 697 994 1 null null [OBJ] 39 \"yellow and blue plastic soccer cleat\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 701 998 1 null null [OBJ] 40 \"yellow hoodie dollhouse boy\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 716 994 1 null null [OBJ] 41 \"yellow plastic Game Boy cartridge\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 720 998 1 null null [OBJ] 42 \"yellow hoodie dollhouse boy\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 728 990 1 null null [OBJ] 43 \"yellow plastic chocolate powder canister\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 732 994 1 null null [OBJ] 44 \"yellow and blue plastic Spongebob toy set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 751 994 1 null null [OBJ] 45 \"white plastic creatine jar\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 771 994 1 null null [OBJ] 46 \"green and blue plastic turtle action figure\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 787 990 1 null null [OBJ] 47 \"yellow and blue plastic Spongebob toy set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 810 990 1 null null [OBJ] 48 \"green and blue plastic turtle action figure\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 826 990 1 null null [OBJ] 49 \"yellow and blue plastic Spongebob toy set\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 837 990 1 null null [OBJ] 50 \"yellow hoodie dollhouse boy\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 857 990 1 null null [OBJ] 51 \"white cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 873 990 1 null null [OBJ] 52 \"yellow plastic Game Boy cartridge\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 888 990 1 null null [OBJ] 53 \"green and blue plastic turtle action figure\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 912 990 1 null null [OBJ] 54 \"yellow plastic Game Boy cartridge\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 923 990 1 null null [OBJ] 55 \"black plastic gaming mouse\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 935 990 1 null null [OBJ] 56 \"black and white athletic cleat\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 947 990 1 null null [OBJ] 57 \"yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 970 990 1 null null [OBJ] 58 \"white cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 990 990 1 null null [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 10 [IN] 0 1 [OUT] 0 1 [INTERACT] 1 \"contact\" 0 10 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 2 \"contact\" 0 10 [IN] 1 18 [OUT] 1 18 [INTERACT] 3 \"contact\" 0 10 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 31 \"contact\" 0 10 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 32 \"contact\" 0 10 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 33 \"contact\" 0 10 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 34 \"contact\" 0 10 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 35 \"contact\" 0 10 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 36 \"contact\" 0 10 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 37 \"contact\" 0 10 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 38 \"contact\" 0 10 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 10 \"contact\" 0 10 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 11 \"contact\" 0 10 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 12 \"contact\" 0 10 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 13 \"contact\" 0 10 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 14 \"contact\" 0 10 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 15 \"contact\" 0 10 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 16 \"contact\" 0 10 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 17 \"contact\" 0 10 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 18 \"contact\" 0 10 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 19 \"contact\" 0 10 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 20 \"contact\" 0 10 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 21 \"contact\" 0 10 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 22 \"contact\" 0 10 [IN] 22 65535 [OUT] 22 65535 [INTERACT] 23 \"contact\" 0 10 [IN] 23 65535 [OUT] 23 65535 [INTERACT] 24 \"contact\" 0 10 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 25 \"contact\" 0 10 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 26 \"contact\" 0 10 [IN] 26 65535 [OUT] 26 65535 [INTERACT] 27 \"contact\" 0 10 [IN] 27 65535 [OUT] 27 65535 [INTERACT] 28 \"contact\" 0 10 [IN] 28 65535 [OUT] 28 65535 [INTERACT] 29 \"contact\" 0 10 [IN] 29 65535 [OUT] 29 65535 [INTERACT] 30 \"contact\" 0 10 [IN] 30 65535 [OUT] 30 65535 [INTERACT] 4 \"contact\" 0 10 [IN] 10 11 [OUT] 10 11 [INTERACT] 5 \"contact\" 0 10 [IN] 10 18 [OUT] 10 18 [INTERACT] 6 \"contact\" 0 10 [IN] 10 22 [OUT] 10 22 [INTERACT] 7 \"contact\" 0 10 [IN] 10 25 [OUT] 10 25 [INTERACT] 8 \"contact\" 0 10 [IN] 10 56 [OUT] 10 56 [INTERACT] 9 \"contact\" 0 10 [IN] 10 65535 [OUT] 10 65535 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"purple fighter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 12 228 1 19 228 2 24 229 3 31 230 4 40 231 5 51 233 6 57 234 7 13 233 8 16 232 9 24 235 10 33 239 11 41 242 12 49 245 13 57 249 14 66 254 15 74 258 16 83 264 17 91 269 18 101 275 19 109 280 20 118 286 21 127 294 22 137 301 23 146 308 [OBJ] 1 \"yellow scooter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 284 909 [OBJ] 2 \"yellow double\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 455 709 2 451 709 3 454 709 5 451 709 13 447 710 18 562 696 19 444 711 21 441 711 23 437 711 [OBJ] 3 \"purple utility\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 519 635 1 null null [OBJ] 4 \"green double\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 873 723 8 870 723 12 868 723 15 866 723 17 863 723 19 861 723 20 858 722 22 856 722 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "pink and purple soccer cleat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 169
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 1
},
{
"id": 1,
"type": "pink and purple soccer cleat",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 13,
"y": 232
},
{
"t": 1,
"x": 13,
"y": 244
},
{
"t": 2,
"x": 13,
"y": 255
},
{
"t": 3,
"x": 13,
"y": 267
},
{
"t": 4,
"x": 13,
"y": 283
},
{
"t": 5,
"x": 13,
"y": 294
},
{
"t": 7,
"x": 13,
"y": 306
},
{
"t": 9,
"x": 13,
"y": 318
},
{
"t": 10,
"x": 17,
"y": 337
},
{
"t": 11,
"x": 13,
"y": 345
},
{
"t": 12,
"x": 13,
"y": 357
},
{
"t": 13,
"x": 13,
"y": 369
},
{
"t": 15,
"x": 13,
"y": 384
},
{
"t": 17,
"x": 13,
"y": 396
},
{
"t": 18,
"x": 13,
"y": 408
},
{
"t": 20,
"x": 13,
"y": 419
},
{
"t": 22,
"x": 13,
"y": 431
},
{
"t": 23,
"x": 13,
"y": 443
}
],
"birth": 0,
"death": 10
},
{
"id": 2,
"type": "dark blue leather boat shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 5,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 3,
"type": "dark blue leather boat shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 9,
"y": 908
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 4,
"type": "dark blue fabric high-top boat shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 13,
"y": 986
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 5,
"type": "dark blue fabric chukka boot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 9,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 6,
"type": "grey and white plush shark",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 56,
"y": 353
},
{
"t": 1,
"x": 68,
"y": 357
},
{
"t": 2,
"x": 80,
"y": 361
},
{
"t": 3,
"x": 91,
"y": 369
},
{
"t": 4,
"x": 103,
"y": 373
},
{
"t": 5,
"x": 115,
"y": 376
},
{
"t": 6,
"x": 126,
"y": 380
},
{
"t": 7,
"x": 138,
"y": 384
},
{
"t": 8,
"x": 150,
"y": 392
},
{
"t": 9,
"x": 162,
"y": 396
},
{
"t": 10,
"x": 173,
"y": 400
},
{
"t": 11,
"x": 181,
"y": 404
},
{
"t": 12,
"x": 193,
"y": 408
},
{
"t": 13,
"x": 201,
"y": 412
},
{
"t": 14,
"x": 208,
"y": 416
},
{
"t": 15,
"x": 216,
"y": 419
},
{
"t": 16,
"x": 224,
"y": 423
},
{
"t": 17,
"x": 232,
"y": 427
},
{
"t": 18,
"x": 240,
"y": 431
},
{
"t": 19,
"x": 248,
"y": 435
},
{
"t": 20,
"x": 255,
"y": 439
},
{
"t": 21,
"x": 263,
"y": 443
},
{
"t": 22,
"x": 271,
"y": 447
},
{
"t": 23,
"x": 279,
"y": 451
}
],
"birth": 0,
"death": 10
},
{
"id": 7,
"type": "yellow wooden toy roller",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 29,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 8,
"type": "wooden toy tableware set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 291,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 9,
"type": "yellow wooden toy construction vehicle set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 294,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 10,
"type": "yellow wooden toy construction vehicle set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 302,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 11,
"type": "wooden toy vegetable cutting set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 306,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 12,
"type": "yellow wooden toy roller",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 310,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 13,
"type": "wooden toy juicer set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 318,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 14,
"type": "wooden toy meat set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 333,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 15,
"type": "yellow wooden toy construction vehicle set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 353,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 16,
"type": "yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 365,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 17,
"type": "yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 369,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 18,
"type": "pink and purple soccer cleat",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 37,
"y": 228
},
{
"t": 1,
"x": 37,
"y": 240
},
{
"t": 2,
"x": 37,
"y": 251
},
{
"t": 3,
"x": 41,
"y": 259
},
{
"t": 4,
"x": 44,
"y": 267
},
{
"t": 5,
"x": 48,
"y": 275
},
{
"t": 6,
"x": 52,
"y": 283
},
{
"t": 7,
"x": 56,
"y": 291
},
{
"t": 8,
"x": 60,
"y": 298
},
{
"t": 9,
"x": 68,
"y": 302
},
{
"t": 10,
"x": 76,
"y": 310
},
{
"t": 11,
"x": 83,
"y": 314
},
{
"t": 12,
"x": 91,
"y": 318
},
{
"t": 13,
"x": 99,
"y": 322
},
{
"t": 14,
"x": 107,
"y": 326
},
{
"t": 15,
"x": 115,
"y": 330
},
{
"t": 16,
"x": 123,
"y": 333
},
{
"t": 17,
"x": 130,
"y": 337
},
{
"t": 18,
"x": 138,
"y": 341
},
{
"t": 19,
"x": 146,
"y": 345
},
{
"t": 20,
"x": 154,
"y": 349
},
{
"t": 21,
"x": 162,
"y": 353
},
{
"t": 22,
"x": 169,
"y": 357
},
{
"t": 23,
"x": 177,
"y": 361
}
],
"birth": 0,
"death": 10
},
{
"id": 19,
"type": "green Lego Android robot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 474,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 20,
"type": "pink and black fabric flip-flop",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 501,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 21,
"type": "yellow cardboard DVD set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 505,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 22,
"type": "yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 517,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 23,
"type": "pink and purple wooden toy castle",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 521,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 24,
"type": "yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 537,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 25,
"type": "yellow cardboard Raisinets box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 552,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 26,
"type": "pink and white fabric high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 564,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 27,
"type": "yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 580,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 28,
"type": "green and yellow running shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 619,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 29,
"type": "yellow plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 634,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 30,
"type": "yellow plastic chocolate powder canister",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 634,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 31,
"type": "yellow cardboard DVD set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 638,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 32,
"type": "green and yellow running shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 650,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 33,
"type": "yellow plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 654,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 34,
"type": "green and yellow cardboard whey protein box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 662,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 35,
"type": "yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 669,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 36,
"type": "yellow plastic chocolate powder canister",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 685,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 37,
"type": "green and yellow cardboard whey protein box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 697,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 38,
"type": "dark grey and light grey plastic monster action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 697,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 39,
"type": "yellow and blue plastic soccer cleat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 701,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 40,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 716,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 41,
"type": "yellow plastic Game Boy cartridge",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 720,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 42,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 728,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 43,
"type": "yellow plastic chocolate powder canister",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 732,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 44,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 751,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 45,
"type": "white plastic creatine jar",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 771,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 46,
"type": "green and blue plastic turtle action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 787,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 47,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 810,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 48,
"type": "green and blue plastic turtle action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 826,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 49,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 837,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 50,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 857,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 51,
"type": "white cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 873,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 52,
"type": "yellow plastic Game Boy cartridge",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 888,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 53,
"type": "green and blue plastic turtle action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 912,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 54,
"type": "yellow plastic Game Boy cartridge",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 923,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 55,
"type": "black plastic gaming mouse",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 935,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 56,
"type": "black and white athletic cleat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 947,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 57,
"type": "yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 970,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 58,
"type": "white cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 990,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
}
],
"interactions": [
{
"id": 0,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 1,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 2,
"inputs": [
1,
18
],
"outputs": [
1,
18
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 3,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 31,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 32,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 33,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 34,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 35,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 36,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 37,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 38,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 10,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 11,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 12,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 13,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 14,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 15,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 16,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 17,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 18,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 19,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 20,
"inputs": [
20,
65535
],
"outputs": [
20,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 21,
"inputs": [
21,
65535
],
"outputs": [
21,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 22,
"inputs": [
22,
65535
],
"outputs": [
22,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 23,
"inputs": [
23,
65535
],
"outputs": [
23,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 24,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 25,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 26,
"inputs": [
26,
65535
],
"outputs": [
26,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 27,
"inputs": [
27,
65535
],
"outputs": [
27,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 28,
"inputs": [
28,
65535
],
"outputs": [
28,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 29,
"inputs": [
29,
65535
],
"outputs": [
29,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 30,
"inputs": [
30,
65535
],
"outputs": [
30,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 4,
"inputs": [
10,
11
],
"outputs": [
10,
11
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 5,
"inputs": [
10,
18
],
"outputs": [
10,
18
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 6,
"inputs": [
10,
22
],
"outputs": [
10,
22
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 7,
"inputs": [
10,
25
],
"outputs": [
10,
25
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 8,
"inputs": [
10,
56
],
"outputs": [
10,
56
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 9,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "purple fighter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 12,
"y": 228
},
{
"t": 1,
"x": 19,
"y": 228
},
{
"t": 2,
"x": 24,
"y": 229
},
{
"t": 3,
"x": 31,
"y": 230
},
{
"t": 4,
"x": 40,
"y": 231
},
{
"t": 5,
"x": 51,
"y": 233
},
{
"t": 6,
"x": 57,
"y": 234
},
{
"t": 7,
"x": 13,
"y": 233
},
{
"t": 8,
"x": 16,
"y": 232
},
{
"t": 9,
"x": 24,
"y": 235
},
{
"t": 10,
"x": 33,
"y": 239
},
{
"t": 11,
"x": 41,
"y": 242
},
{
"t": 12,
"x": 49,
"y": 245
},
{
"t": 13,
"x": 57,
"y": 249
},
{
"t": 14,
"x": 66,
"y": 254
},
{
"t": 15,
"x": 74,
"y": 258
},
{
"t": 16,
"x": 83,
"y": 264
},
{
"t": 17,
"x": 91,
"y": 269
},
{
"t": 18,
"x": 101,
"y": 275
},
{
"t": 19,
"x": 109,
"y": 280
},
{
"t": 20,
"x": 118,
"y": 286
},
{
"t": 21,
"x": 127,
"y": 294
},
{
"t": 22,
"x": 137,
"y": 301
},
{
"t": 23,
"x": 146,
"y": 308
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "yellow scooter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 284,
"y": 909
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "yellow double",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 455,
"y": 709
},
{
"t": 2,
"x": 451,
"y": 709
},
{
"t": 3,
"x": 454,
"y": 709
},
{
"t": 5,
"x": 451,
"y": 709
},
{
"t": 13,
"x": 447,
"y": 710
},
{
"t": 18,
"x": 562,
"y": 696
},
{
"t": 19,
"x": 444,
"y": 711
},
{
"t": 21,
"x": 441,
"y": 711
},
{
"t": 23,
"x": 437,
"y": 711
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "purple utility",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 519,
"y": 635
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "green double",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 873,
"y": 723
},
{
"t": 8,
"x": 870,
"y": 723
},
{
"t": 12,
"x": 868,
"y": 723
},
{
"t": 15,
"x": 866,
"y": 723
},
{
"t": 17,
"x": 863,
"y": 723
},
{
"t": 19,
"x": 861,
"y": 723
},
{
"t": 20,
"x": 858,
"y": 722
},
{
"t": 22,
"x": 856,
"y": 722
}
],
"birth": 0,
"death": 24
}
],
"interactions": [
{
"id": 0,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.43463347506180716,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 547.0129472725957,
"avg_image_position_segmentation_acc": 0.0,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.0,
"avg_image_position_segmentation_acc_large": 0.0,
"avg_image_position_segmentation_acc_first_frame": 0.0,
"avg_image_position_segmentation_visible_acc": 0.0,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 0.42499999999999993,
"visibility_true_acc": 0.42499999999999993,
"visibility_false_acc": 1.0,
"is_in_frame_acc": 0.23333333333333334,
"is_in_frame_true_acc": 0.23333333333333334,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.42499999999999993,
"vis_acc_from_px_true_acc": 0.42499999999999993,
"vis_acc_from_px_false_acc": 1.0,
"num_matches": 5,
"num_unmatched_pred": 54,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 5,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 5,
"avg_image_position_segmentation_acc": 4,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 2,
"avg_image_position_segmentation_acc_large": 2,
"avg_image_position_segmentation_acc_first_frame": 4,
"avg_image_position_segmentation_visible_acc": 4,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 5,
"visibility_true_acc": 5,
"visibility_false_acc": 1,
"is_in_frame_acc": 5,
"is_in_frame_true_acc": 5,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 5,
"vis_acc_from_px_true_acc": 5,
"vis_acc_from_px_false_acc": 1
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 23,
"target_id": 0,
"pred_id": 23,
"cost": 1.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 9,
"target_id": 1,
"pred_id": 9,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 6,
"target_id": 2,
"pred_id": 6,
"cost": 1.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 0,
"target_id": 3,
"pred_id": 0,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": true
}
]
},
{
"target_index": 4,
"pred_index": 28,
"target_id": 4,
"pred_id": 28,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
1,
2,
3,
4,
5,
7,
8,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
24,
25,
26,
27,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58
],
"target_type_counts": {
"purple fighter": 1,
"yellow scooter": 1,
"yellow double": 1,
"purple utility": 1,
"green double": 1
},
"pred_type_counts": {
"pink and purple soccer cleat": 3,
"dark blue leather boat shoe": 2,
"dark blue fabric high-top boat shoe": 1,
"dark blue fabric chukka boot": 1,
"grey and white plush shark": 1,
"yellow wooden toy roller": 2,
"wooden toy tableware set": 1,
"yellow wooden toy construction vehicle set": 3,
"wooden toy vegetable cutting set": 1,
"wooden toy juicer set": 1,
"wooden toy meat set": 1,
"yellow cardboard ink cartridge box": 7,
"green Lego Android robot": 1,
"pink and black fabric flip-flop": 1,
"yellow cardboard DVD set": 2,
"pink and purple wooden toy castle": 1,
"yellow cardboard Raisinets box": 1,
"pink and white fabric high-top sneaker": 1,
"green and yellow running shoe": 2,
"yellow plastic plant pot": 2,
"yellow plastic chocolate powder canister": 3,
"green and yellow cardboard whey protein box": 2,
"dark grey and light grey plastic monster action figure": 1,
"yellow and blue plastic soccer cleat": 1,
"yellow hoodie dollhouse boy": 3,
"yellow plastic Game Boy cartridge": 3,
"yellow and blue plastic Spongebob toy set": 3,
"white plastic creatine jar": 1,
"green and blue plastic turtle action figure": 3,
"white cardboard ink cartridge box": 2,
"black plastic gaming mouse": 1,
"black and white athletic cleat": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 3,
"false_positives": 2,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 0.6,
"1.0": 0.0,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 57,
"false_negatives": 3
},
"0.5": {
"true_positives": 2,
"false_positives": 57,
"false_negatives": 3
},
"0.75": {
"true_positives": 2,
"false_positives": 57,
"false_negatives": 3
},
"1.0": {
"true_positives": 2,
"false_positives": 57,
"false_negatives": 3
}
},
"visibility_precision_by_threshold": {
"0.25": 0.03389830508474576,
"0.5": 0.03389830508474576,
"0.75": 0.03389830508474576,
"1.0": 0.03389830508474576
},
"visibility_recall_by_threshold": {
"0.25": 0.4,
"0.5": 0.4,
"0.75": 0.4,
"1.0": 0.4
},
"visibility_f1_by_threshold": {
"0.25": 0.0625,
"0.5": 0.0625,
"0.75": 0.0625,
"1.0": 0.0625
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
},
"0.5": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
},
"0.75": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
},
"1.0": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
},
"visibility_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
},
"visibility_f1_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 59,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 59,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 59,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 59,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 59,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0,
"segmentation_by_motion_group": {
"static": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 3
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 55,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 23,
"cost": 1.5,
"type_similarity": 0.3618645741818161,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 981.5516629077657,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.041666666666666664,
"visibility_true_acc": 0.041666666666666664,
"visibility_false_acc": null,
"is_in_frame_acc": 0.041666666666666664,
"is_in_frame_true_acc": 0.041666666666666664,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.041666666666666664,
"vis_acc_from_px_true_acc": 0.041666666666666664,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 9,
"cost": 1.0,
"type_similarity": 0.5060178215635571,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 78.72490800883793,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.041666666666666664,
"visibility_true_acc": 0.041666666666666664,
"visibility_false_acc": null,
"is_in_frame_acc": 0.041666666666666664,
"is_in_frame_true_acc": 0.041666666666666664,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.041666666666666664,
"vis_acc_from_px_true_acc": 0.041666666666666664,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 6,
"cost": 1.5,
"type_similarity": 0.3160203927076827,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 459.6034833972039,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.0,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 0,
"cost": 1.0,
"type_similarity": 0.46078359985320205,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 799.1555841761979,
"image_position_segmentation_acc": null,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": null,
"image_position_segmentation_visible_acc": null,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": 1.0,
"is_in_frame_acc": 0.041666666666666664,
"is_in_frame_true_acc": 0.041666666666666664,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": 1.0
},
{
"target_id": 4,
"pred_id": 28,
"cost": 1.0,
"type_similarity": 0.5284809870027776,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 416.0290978729733,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.0,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.041666666666666664,
"visibility_true_acc": 0.041666666666666664,
"visibility_false_acc": null,
"is_in_frame_acc": 0.041666666666666664,
"is_in_frame_true_acc": 0.041666666666666664,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.041666666666666664,
"vis_acc_from_px_true_acc": 0.041666666666666664,
"vis_acc_from_px_false_acc": null
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": 0.4583333333333333,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 35,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.10256410256410256,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.18604651162790695,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.10256410256410256,
"0.5": 0.10256410256410256,
"0.75": 0.10256410256410256,
"1.0": 0.10256410256410256
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.18604651162790695,
"0.5": 0.18604651162790695,
"0.75": 0.18604651162790695,
"1.0": 0.18604651162790695
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.10256410256410256,
"0.5": 0.10256410256410256,
"0.75": 0.10256410256410256,
"1.0": 0.10256410256410256
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.18604651162790695,
"0.5": 0.18604651162790695,
"0.75": 0.18604651162790695,
"1.0": 0.18604651162790695
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 38,
"target_id": 0,
"pred_id": 38,
"cost": 0.27083333333333337,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.27083333333333337,
"weighted_cost": 0.27083333333333337,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 35,
"target_id": 1,
"pred_id": 35,
"cost": 0.27083333333333337,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.27083333333333337,
"weighted_cost": 0.27083333333333337,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 1,
"target_id": 2,
"pred_id": 1,
"cost": 0.27083333333333337,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.27083333333333337,
"weighted_cost": 0.27083333333333337,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 28,
"target_id": 3,
"pred_id": 28,
"cost": 0.27083333333333337,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.27083333333333337,
"weighted_cost": 0.27083333333333337,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
29,
30,
31,
32,
33,
34,
36,
37
],
"target_type_counts": {
"contact": 4
},
"pred_type_counts": {
"contact": 39
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 38,
"target_id": 0,
"pred_id": 38,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.27083333333333337,
"temporal_span_iou": 0.4583333333333333,
"temporal_span_giou": 0.4583333333333333,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 1,
"pred_index": 35,
"target_id": 1,
"pred_id": 35,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.27083333333333337,
"temporal_span_iou": 0.4583333333333333,
"temporal_span_giou": 0.4583333333333333,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 2,
"pred_index": 1,
"target_id": 2,
"pred_id": 1,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.27083333333333337,
"temporal_span_iou": 0.4583333333333333,
"temporal_span_giou": 0.4583333333333333,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 3,
"pred_index": 28,
"target_id": 3,
"pred_id": 28,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.27083333333333337,
"temporal_span_iou": 0.4583333333333333,
"temporal_span_giou": 0.4583333333333333,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0.4583333333333333,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 28,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 32,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 32,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 32,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.125,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.2222222222222222,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.125,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.2222222222222222,
"0.5": 0.2222222222222222,
"0.75": 0.2222222222222222,
"1.0": 0.2222222222222222
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 28,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.125,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.2222222222222222,
"0.5": 0.2222222222222222,
"0.75": 0.2222222222222222,
"1.0": 0.2222222222222222
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 7,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"contact": {
"avg_interaction_temporal_span_giou": 0.4583333333333333,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 35,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.10256410256410256,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.18604651162790695,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.10256410256410256,
"0.5": 0.10256410256410256,
"0.75": 0.10256410256410256,
"1.0": 0.10256410256410256
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.18604651162790695,
"0.5": 0.18604651162790695,
"0.75": 0.18604651162790695,
"1.0": 0.18604651162790695
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 35,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.10256410256410256,
"0.5": 0.10256410256410256,
"0.75": 0.10256410256410256,
"1.0": 0.10256410256410256
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.18604651162790695,
"0.5": 0.18604651162790695,
"0.75": 0.18604651162790695,
"1.0": 0.18604651162790695
}
}
},
"interaction_supports_by_type": {
"contact": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}