STRIVE · 08/28/2026
Dyn generalization · test_24 · super_clevr_1100
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"brown fighter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 358 405 3 361 406 7 363 407 9 366 408 13 368 409 16 370 410 19 373 411 22 376 413 [OBJ] 1 \"green scooter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 432 613 [OBJ] 2 \"green mountain\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 544 802 1 538 798 2 532 793 3 523 789 4 518 783 5 512 779 6 505 774 7 499 771 8 494 766 9 488 759 10 481 757 11 476 755 12 471 750 13 465 744 14 459 742 15 453 736 16 448 733 17 442 718 18 437 726 19 432 711 20 426 706 21 422 706 22 416 701 23 411 696 [OBJ] 3 \"red wagon\" 6 24 [VIS] 0 false 6 true [IN-FRAME] 0 false 6 true [PX] 0 null null 6 511 999 7 515 996 8 506 993 9 515 990 10 502 988 11 516 984 12 510 981 13 520 978 14 518 976 15 507 973 16 523 970 17 522 968 18 522 965 19 515 963 20 512 959 22 513 954 23 515 952 [OBJ] 4 \"yellow scooter\" 7 24 [VIS] 0 false 7 true [IN-FRAME] 0 false 7 true [PX] 0 null null 7 998 735 8 994 732 9 991 720 10 987 716 11 982 709 12 973 710 13 964 711 14 955 711 15 945 713 16 983 727 17 980 725 18 973 727 19 965 726 20 955 728 21 945 729 22 936 729 23 926 731 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_24/super_clevr_1100/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"green wooden toy train\" 0 1 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 1 458 3 null null [OBJ] 1 \"yellow plastic toy school bus\" 0 8 [VIS] 0 true 7 false [IN-FRAME] 0 true 7 false [PX] 0 1 525 7 null null [OBJ] 2 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 10 false [IN-FRAME] 0 true 10 false [PX] 0 1 576 5 1 587 10 null null [OBJ] 3 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 1 701 1 null null [OBJ] 4 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 5 958 1 null null [OBJ] 5 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 9 986 1 5 994 3 null null [OBJ] 6 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 13 974 1 9 982 3 null null [OBJ] 7 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 17 986 1 null null [OBJ] 8 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 5 false [IN-FRAME] 0 true 5 false [PX] 0 21 986 3 9 994 5 null null [OBJ] 9 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 29 994 1 21 998 3 null null [OBJ] 10 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 37 994 3 null null [OBJ] 11 \"dark grey and orange athletic shoe\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 41 994 3 null null [OBJ] 12 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 56 994 3 null null [OBJ] 13 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 68 994 3 null null [OBJ] 14 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 80 994 3 null null [OBJ] 15 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 99 994 3 null null [OBJ] 16 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 119 994 3 null null [OBJ] 17 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 138 994 3 null null [OBJ] 18 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 158 994 3 null null [OBJ] 19 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 169 994 3 null null [OBJ] 20 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 189 994 3 null null [OBJ] 21 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 208 994 3 null null [OBJ] 22 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 220 994 3 null null [OBJ] 23 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 236 994 3 null null [OBJ] 24 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 251 994 3 null null [OBJ] 25 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 263 994 3 null null [OBJ] 26 \"yellow plastic toy school bus\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 345 408 7 357 412 11 365 416 [OBJ] 27 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 298 994 3 null null [OBJ] 28 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 314 994 3 null null [OBJ] 29 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 326 994 3 null null [OBJ] 30 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 337 994 3 null null [OBJ] 31 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 349 994 3 null null [OBJ] 32 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 361 994 3 null null [OBJ] 33 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 373 994 3 null null [OBJ] 34 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 384 994 3 null null [OBJ] 35 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 396 994 3 null null [OBJ] 36 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 408 994 3 null null [OBJ] 37 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 423 994 3 null null [OBJ] 38 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 435 994 3 null null [OBJ] 39 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 455 994 3 null null [OBJ] 40 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 466 994 3 null null [OBJ] 41 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 478 994 3 null null [OBJ] 42 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 490 994 3 null null [OBJ] 43 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 505 994 3 null null [OBJ] 44 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 513 998 3 null null [OBJ] 45 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 525 994 3 null null [OBJ] 46 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 533 998 3 null null [OBJ] 47 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 544 994 3 null null [OBJ] 48 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 564 994 3 null null [OBJ] 49 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 572 990 3 null null [OBJ] 50 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 587 990 3 null null [OBJ] 51 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 599 986 3 null null [OBJ] 52 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 615 986 3 null null [OBJ] 53 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 626 986 3 null null [OBJ] 54 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 638 982 3 null null [OBJ] 55 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 654 982 3 null null [OBJ] 56 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 666 982 3 null null [OBJ] 57 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 677 982 3 null null [OBJ] 58 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 693 982 3 null null [OBJ] 59 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 705 982 3 null null [OBJ] 60 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 716 982 3 null null [OBJ] 61 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 728 982 3 null null [OBJ] 62 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 740 982 3 null null [OBJ] 63 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 755 982 3 null null [OBJ] 64 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 767 982 3 null null [OBJ] 65 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 783 982 3 null null [OBJ] 66 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 798 982 3 null null [OBJ] 67 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 814 982 3 null null [OBJ] 68 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 826 982 3 null null [OBJ] 69 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 841 982 3 null null [OBJ] 70 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 857 982 3 null null [OBJ] 71 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 869 982 3 null null [OBJ] 72 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 884 982 3 null null [OBJ] 73 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 900 982 3 null null [OBJ] 74 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 912 982 3 null null [OBJ] 75 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 923 982 3 null null [OBJ] 76 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 939 982 3 null null [OBJ] 77 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 951 982 3 null null [OBJ] 78 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 970 982 3 null null [OBJ] 79 \"green metallic lime squeezer\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true 3 false [PX] 0 990 978 2 994 955 3 994 970 4 994 982 5 998 962 6 998 974 7 998 986 8 998 998 9 994 990 10 994 978 11 994 994 12 994 982 13 994 994 14 990 986 16 990 998 17 986 990 19 982 998 21 978 990 23 974 998 [OBJ] 80 \"green wooden toy train\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 759 1 994 748 2 994 736 3 994 724 4 990 744 5 990 728 6 986 744 7 986 728 8 982 740 9 978 748 10 978 732 11 974 740 12 970 748 13 966 755 14 966 740 15 962 748 16 958 755 17 958 740 18 955 748 19 951 755 20 951 744 21 947 751 22 943 759 23 943 748 [OBJS-END] [INTERACTIONS-START] [INTERACT] 4 \"contact\" 0 1 [IN] 1 3 [OUT] 1 3 [INTERACT] 12 \"contact\" 0 3 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 0 \"contact\" 0 10 [IN] 0 1 [OUT] 0 1 [INTERACT] 3 \"contact\" 0 10 [IN] 1 2 [OUT] 1 2 [INTERACT] 8 \"contact\" 0 10 [IN] 1 6 [OUT] 1 6 [INTERACT] 2 \"contact\" 0 10 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 23 \"contact\" 0 10 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 24 \"contact\" 0 10 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 25 \"contact\" 0 10 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 26 \"contact\" 0 10 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 27 \"contact\" 0 10 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 28 \"contact\" 0 10 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 29 \"contact\" 0 10 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 13 \"contact\" 0 10 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 14 \"contact\" 0 10 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 15 \"contact\" 0 10 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 16 \"contact\" 0 10 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 17 \"contact\" 0 10 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 18 \"contact\" 0 10 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 19 \"contact\" 0 10 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 20 \"contact\" 0 10 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 21 \"contact\" 0 10 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 22 \"contact\" 0 10 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 5 \"contact\" 0 10 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 9 \"contact\" 0 10 [IN] 10 21 [OUT] 10 21 [INTERACT] 6 \"contact\" 0 10 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 7 \"contact\" 0 10 [IN] 10 22 [OUT] 10 22 [INTERACT] 1 \"collision\" 2 2 [IN] 0 1 [OUT] 0 1 [INTERACT] 10 \"collision\" 5 5 [IN] 10 21 [OUT] 10 21 [INTERACT] 11 \"contact\" 6 10 [IN] 10 21 [OUT] 10 21 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"brown fighter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 358 405 3 361 406 7 363 407 9 366 408 13 368 409 16 370 410 19 373 411 22 376 413 [OBJ] 1 \"green scooter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 432 613 [OBJ] 2 \"green mountain\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 544 802 1 538 798 2 532 793 3 523 789 4 518 783 5 512 779 6 505 774 7 499 771 8 494 766 9 488 759 10 481 757 11 476 755 12 471 750 13 465 744 14 459 742 15 453 736 16 448 733 17 442 718 18 437 726 19 432 711 20 426 706 21 422 706 22 416 701 23 411 696 [OBJ] 3 \"red wagon\" 6 24 [VIS] 0 false 6 true [IN-FRAME] 0 false 6 true [PX] 0 null null 6 511 999 7 515 996 8 506 993 9 515 990 10 502 988 11 516 984 12 510 981 13 520 978 14 518 976 15 507 973 16 523 970 17 522 968 18 522 965 19 515 963 20 512 959 22 513 954 23 515 952 [OBJ] 4 \"yellow scooter\" 7 24 [VIS] 0 false 7 true [IN-FRAME] 0 false 7 true [PX] 0 null null 7 998 735 8 994 732 9 991 720 10 987 716 11 982 709 12 973 710 13 964 711 14 955 711 15 945 713 16 983 727 17 980 725 18 973 727 19 965 726 20 955 728 21 945 729 22 936 729 23 926 731 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 458
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 1
},
{
"id": 1,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 7,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 7,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 525
},
{
"t": 7,
"x": null,
"y": null
}
],
"birth": 0,
"death": 8
},
{
"id": 2,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 10,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 10,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 576
},
{
"t": 5,
"x": 1,
"y": 587
},
{
"t": 10,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 3,
"type": "green and yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 701
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 4,
"type": "green and yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 5,
"y": 958
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 5,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 9,
"y": 986
},
{
"t": 1,
"x": 5,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 6,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 13,
"y": 974
},
{
"t": 1,
"x": 9,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 7,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 17,
"y": 986
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 8,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 5,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 5,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 21,
"y": 986
},
{
"t": 3,
"x": 9,
"y": 994
},
{
"t": 5,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 9,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 29,
"y": 994
},
{
"t": 1,
"x": 21,
"y": 998
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 10,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 37,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 11,
"type": "dark grey and orange athletic shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 41,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 12,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 56,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 13,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 68,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 14,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 80,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 15,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 99,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 16,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 119,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 17,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 138,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 18,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 158,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 19,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 169,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 20,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 189,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 21,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 208,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 22,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 220,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 23,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 236,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 24,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 251,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 25,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 263,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 26,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 345,
"y": 408
},
{
"t": 7,
"x": 357,
"y": 412
},
{
"t": 11,
"x": 365,
"y": 416
}
],
"birth": 0,
"death": 10
},
{
"id": 27,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 298,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 28,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 314,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 29,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 326,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 30,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 337,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 31,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 349,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 32,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 361,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 33,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 373,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 34,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 384,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 35,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 396,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 36,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 408,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 37,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 423,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 38,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 435,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 39,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 455,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 40,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 466,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 41,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 478,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 42,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 490,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 43,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 505,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 44,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 513,
"y": 998
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 45,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 525,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 46,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 533,
"y": 998
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 47,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 544,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 48,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 564,
"y": 994
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 49,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 572,
"y": 990
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 50,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 587,
"y": 990
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 51,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 599,
"y": 986
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 52,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 615,
"y": 986
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 53,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 626,
"y": 986
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 54,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 638,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 55,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 654,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 56,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 666,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 57,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 677,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 58,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 693,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 59,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 705,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 60,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 716,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 61,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 728,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 62,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 740,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 63,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 755,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 64,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 767,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 65,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 783,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 66,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 798,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 67,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 814,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 68,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 826,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 69,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 841,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 70,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 857,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 71,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 869,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 72,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 884,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 73,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 900,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 74,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 912,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 75,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 923,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 76,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 939,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 77,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 951,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 78,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 970,
"y": 982
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 10
},
{
"id": 79,
"type": "green metallic lime squeezer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 990,
"y": 978
},
{
"t": 2,
"x": 994,
"y": 955
},
{
"t": 3,
"x": 994,
"y": 970
},
{
"t": 4,
"x": 994,
"y": 982
},
{
"t": 5,
"x": 998,
"y": 962
},
{
"t": 6,
"x": 998,
"y": 974
},
{
"t": 7,
"x": 998,
"y": 986
},
{
"t": 8,
"x": 998,
"y": 998
},
{
"t": 9,
"x": 994,
"y": 990
},
{
"t": 10,
"x": 994,
"y": 978
},
{
"t": 11,
"x": 994,
"y": 994
},
{
"t": 12,
"x": 994,
"y": 982
},
{
"t": 13,
"x": 994,
"y": 994
},
{
"t": 14,
"x": 990,
"y": 986
},
{
"t": 16,
"x": 990,
"y": 998
},
{
"t": 17,
"x": 986,
"y": 990
},
{
"t": 19,
"x": 982,
"y": 998
},
{
"t": 21,
"x": 978,
"y": 990
},
{
"t": 23,
"x": 974,
"y": 998
}
],
"birth": 0,
"death": 10
},
{
"id": 80,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 994,
"y": 759
},
{
"t": 1,
"x": 994,
"y": 748
},
{
"t": 2,
"x": 994,
"y": 736
},
{
"t": 3,
"x": 994,
"y": 724
},
{
"t": 4,
"x": 990,
"y": 744
},
{
"t": 5,
"x": 990,
"y": 728
},
{
"t": 6,
"x": 986,
"y": 744
},
{
"t": 7,
"x": 986,
"y": 728
},
{
"t": 8,
"x": 982,
"y": 740
},
{
"t": 9,
"x": 978,
"y": 748
},
{
"t": 10,
"x": 978,
"y": 732
},
{
"t": 11,
"x": 974,
"y": 740
},
{
"t": 12,
"x": 970,
"y": 748
},
{
"t": 13,
"x": 966,
"y": 755
},
{
"t": 14,
"x": 966,
"y": 740
},
{
"t": 15,
"x": 962,
"y": 748
},
{
"t": 16,
"x": 958,
"y": 755
},
{
"t": 17,
"x": 958,
"y": 740
},
{
"t": 18,
"x": 955,
"y": 748
},
{
"t": 19,
"x": 951,
"y": 755
},
{
"t": 20,
"x": 951,
"y": 744
},
{
"t": 21,
"x": 947,
"y": 751
},
{
"t": 22,
"x": 943,
"y": 759
},
{
"t": 23,
"x": 943,
"y": 748
}
],
"birth": 0,
"death": 10
}
],
"interactions": [
{
"id": 4,
"inputs": [
1,
3
],
"outputs": [
1,
3
],
"begins_at": 0,
"ends_at": 1,
"type": "contact"
},
{
"id": 12,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 3,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 3,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 8,
"inputs": [
1,
6
],
"outputs": [
1,
6
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 2,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 23,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 24,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 25,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 26,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 27,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 28,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 29,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 13,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 14,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 15,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 16,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 17,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 18,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 19,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 20,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 21,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 22,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 5,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 9,
"inputs": [
10,
21
],
"outputs": [
10,
21
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 6,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 7,
"inputs": [
10,
22
],
"outputs": [
10,
22
],
"begins_at": 0,
"ends_at": 10,
"type": "contact"
},
{
"id": 1,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 2,
"ends_at": 2,
"type": "collision"
},
{
"id": 10,
"inputs": [
10,
21
],
"outputs": [
10,
21
],
"begins_at": 5,
"ends_at": 5,
"type": "collision"
},
{
"id": 11,
"inputs": [
10,
21
],
"outputs": [
10,
21
],
"begins_at": 6,
"ends_at": 10,
"type": "contact"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "brown fighter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 358,
"y": 405
},
{
"t": 3,
"x": 361,
"y": 406
},
{
"t": 7,
"x": 363,
"y": 407
},
{
"t": 9,
"x": 366,
"y": 408
},
{
"t": 13,
"x": 368,
"y": 409
},
{
"t": 16,
"x": 370,
"y": 410
},
{
"t": 19,
"x": 373,
"y": 411
},
{
"t": 22,
"x": 376,
"y": 413
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "green scooter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 432,
"y": 613
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "green mountain",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 544,
"y": 802
},
{
"t": 1,
"x": 538,
"y": 798
},
{
"t": 2,
"x": 532,
"y": 793
},
{
"t": 3,
"x": 523,
"y": 789
},
{
"t": 4,
"x": 518,
"y": 783
},
{
"t": 5,
"x": 512,
"y": 779
},
{
"t": 6,
"x": 505,
"y": 774
},
{
"t": 7,
"x": 499,
"y": 771
},
{
"t": 8,
"x": 494,
"y": 766
},
{
"t": 9,
"x": 488,
"y": 759
},
{
"t": 10,
"x": 481,
"y": 757
},
{
"t": 11,
"x": 476,
"y": 755
},
{
"t": 12,
"x": 471,
"y": 750
},
{
"t": 13,
"x": 465,
"y": 744
},
{
"t": 14,
"x": 459,
"y": 742
},
{
"t": 15,
"x": 453,
"y": 736
},
{
"t": 16,
"x": 448,
"y": 733
},
{
"t": 17,
"x": 442,
"y": 718
},
{
"t": 18,
"x": 437,
"y": 726
},
{
"t": 19,
"x": 432,
"y": 711
},
{
"t": 20,
"x": 426,
"y": 706
},
{
"t": 21,
"x": 422,
"y": 706
},
{
"t": 22,
"x": 416,
"y": 701
},
{
"t": 23,
"x": 411,
"y": 696
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "red wagon",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 6,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 6,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 6,
"x": 511,
"y": 999
},
{
"t": 7,
"x": 515,
"y": 996
},
{
"t": 8,
"x": 506,
"y": 993
},
{
"t": 9,
"x": 515,
"y": 990
},
{
"t": 10,
"x": 502,
"y": 988
},
{
"t": 11,
"x": 516,
"y": 984
},
{
"t": 12,
"x": 510,
"y": 981
},
{
"t": 13,
"x": 520,
"y": 978
},
{
"t": 14,
"x": 518,
"y": 976
},
{
"t": 15,
"x": 507,
"y": 973
},
{
"t": 16,
"x": 523,
"y": 970
},
{
"t": 17,
"x": 522,
"y": 968
},
{
"t": 18,
"x": 522,
"y": 965
},
{
"t": 19,
"x": 515,
"y": 963
},
{
"t": 20,
"x": 512,
"y": 959
},
{
"t": 22,
"x": 513,
"y": 954
},
{
"t": 23,
"x": 515,
"y": 952
}
],
"birth": 6,
"death": 24
},
{
"id": 4,
"type": "yellow scooter",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 7,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 7,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 7,
"x": 998,
"y": 735
},
{
"t": 8,
"x": 994,
"y": 732
},
{
"t": 9,
"x": 991,
"y": 720
},
{
"t": 10,
"x": 987,
"y": 716
},
{
"t": 11,
"x": 982,
"y": 709
},
{
"t": 12,
"x": 973,
"y": 710
},
{
"t": 13,
"x": 964,
"y": 711
},
{
"t": 14,
"x": 955,
"y": 711
},
{
"t": 15,
"x": 945,
"y": 713
},
{
"t": 16,
"x": 983,
"y": 727
},
{
"t": 17,
"x": 980,
"y": 725
},
{
"t": 18,
"x": 973,
"y": 727
},
{
"t": 19,
"x": 965,
"y": 726
},
{
"t": 20,
"x": 955,
"y": 728
},
{
"t": 21,
"x": 945,
"y": 729
},
{
"t": 22,
"x": 936,
"y": 729
},
{
"t": 23,
"x": 926,
"y": 731
}
],
"birth": 7,
"death": 24
}
],
"interactions": [
{
"id": 0,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.33771276026051056,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 495.229876260239,
"avg_image_position_segmentation_acc": 0.2,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.2,
"avg_image_position_segmentation_acc_large": null,
"avg_image_position_segmentation_acc_first_frame": 0.3333333333333333,
"avg_image_position_segmentation_visible_acc": 0.2,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 0.4583333333333333,
"visibility_true_acc": 0.46944444444444444,
"visibility_false_acc": 0.0,
"is_in_frame_acc": 0.4583333333333333,
"is_in_frame_true_acc": 0.46944444444444444,
"is_in_frame_false_acc": 0.0,
"vis_acc_from_px": 0.4583333333333333,
"vis_acc_from_px_true_acc": 0.46944444444444444,
"vis_acc_from_px_false_acc": 0.0,
"num_matches": 5,
"num_unmatched_pred": 76,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 5,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 4,
"avg_image_position_segmentation_acc": 5,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 5,
"avg_image_position_segmentation_acc_large": 0,
"avg_image_position_segmentation_acc_first_frame": 3,
"avg_image_position_segmentation_visible_acc": 5,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 5,
"visibility_true_acc": 5,
"visibility_false_acc": 2,
"is_in_frame_acc": 5,
"is_in_frame_true_acc": 5,
"is_in_frame_false_acc": 2,
"vis_acc_from_px": 5,
"vis_acc_from_px_true_acc": 5,
"vis_acc_from_px_false_acc": 2
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 26,
"target_id": 0,
"pred_id": 26,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 0,
"target_id": 1,
"pred_id": 0,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 80,
"target_id": 2,
"pred_id": 80,
"cost": 1.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 2,
"target_id": 3,
"pred_id": 2,
"cost": 2.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 1,
"target_id": 4,
"pred_id": 1,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
50,
51,
52,
53,
54,
55,
56,
57,
58,
59,
60,
61,
62,
63,
64,
65,
66,
67,
68,
69,
70,
71,
72,
73,
74,
75,
76,
77,
78,
79
],
"target_type_counts": {
"brown fighter": 1,
"green scooter": 1,
"green mountain": 1,
"red wagon": 1,
"yellow scooter": 1
},
"pred_type_counts": {
"green wooden toy train": 2,
"yellow plastic toy school bus": 15,
"green and yellow cardboard ink cartridge box": 2,
"green metallic lime squeezer": 61,
"dark grey and orange athletic shoe": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 3,
"false_positives": 2,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 0.6,
"1.0": 0.0,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 79,
"false_negatives": 3
},
"0.5": {
"true_positives": 2,
"false_positives": 79,
"false_negatives": 3
},
"0.75": {
"true_positives": 2,
"false_positives": 79,
"false_negatives": 3
},
"1.0": {
"true_positives": 2,
"false_positives": 79,
"false_negatives": 3
}
},
"visibility_precision_by_threshold": {
"0.25": 0.024691358024691357,
"0.5": 0.024691358024691357,
"0.75": 0.024691358024691357,
"1.0": 0.024691358024691357
},
"visibility_recall_by_threshold": {
"0.25": 0.4,
"0.5": 0.4,
"0.75": 0.4,
"1.0": 0.4
},
"visibility_f1_by_threshold": {
"0.25": 0.046511627906976744,
"0.5": 0.046511627906976744,
"0.75": 0.046511627906976744,
"1.0": 0.046511627906976744
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 3
},
"0.5": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 3
},
"0.75": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 3
},
"1.0": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 3
}
},
"visibility_precision_by_threshold": {
"0.25": 0.25,
"0.5": 0.25,
"0.75": 0.25,
"1.0": 0.25
},
"visibility_recall_by_threshold": {
"0.25": 0.25,
"0.5": 0.25,
"0.75": 0.25,
"1.0": 0.25
},
"visibility_f1_by_threshold": {
"0.25": 0.25,
"0.5": 0.25,
"0.75": 0.25,
"1.0": 0.25
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 0.012987012987012988,
"0.5": 0.012987012987012988,
"0.75": 0.012987012987012988,
"1.0": 0.012987012987012988
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 0.025641025641025647,
"0.5": 0.025641025641025647,
"0.75": 0.025641025641025647,
"1.0": 0.025641025641025647
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 80,
"false_negatives": 4
},
"0.5": {
"true_positives": 1,
"false_positives": 80,
"false_negatives": 4
},
"0.75": {
"true_positives": 1,
"false_positives": 80,
"false_negatives": 4
},
"1.0": {
"true_positives": 1,
"false_positives": 80,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.012345679012345678,
"0.5": 0.012345679012345678,
"0.75": 0.012345679012345678,
"1.0": 0.012345679012345678
},
"segmentation_recall_by_threshold": {
"0.25": 0.2,
"0.5": 0.2,
"0.75": 0.2,
"1.0": 0.2
},
"segmentation_f1_by_threshold": {
"0.25": 0.023255813953488372,
"0.5": 0.023255813953488372,
"0.75": 0.023255813953488372,
"1.0": 0.023255813953488372
},
"segmentation_frame_average_precision": 0.012345679012345678,
"segmentation_frame_average_recall": 0.2,
"segmentation_frame_average_f1": 0.023255813953488372,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 80,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.012345679012345678,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.024390243902439022,
"segmentation_by_motion_group": {
"static": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.012987012987012988,
"0.5": 0.012987012987012988,
"0.75": 0.012987012987012988,
"1.0": 0.012987012987012988
},
"segmentation_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.025641025641025647,
"0.5": 0.025641025641025647,
"0.75": 0.025641025641025647,
"1.0": 0.025641025641025647
},
"segmentation_frame_average_precision": 0.012987012987012988,
"segmentation_frame_average_recall": 1.0,
"segmentation_frame_average_f1": 0.025641025641025647,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 76,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.012987012987012988,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.025641025641025647
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 26,
"cost": 1.0,
"type_similarity": 0.05586884067861058,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 12.59922048477413,
"image_position_segmentation_acc": 1.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 1.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 1.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 0,
"cost": 1.0,
"type_similarity": 0.46799395568152485,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 570.9385221072405,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.125,
"visibility_true_acc": 0.125,
"visibility_false_acc": null,
"is_in_frame_acc": 0.125,
"is_in_frame_true_acc": 0.125,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.125,
"vis_acc_from_px_true_acc": 0.125,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 80,
"cost": 1.5,
"type_similarity": 0.37597116173046463,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 636.1537492988733,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 2,
"cost": 2.0,
"type_similarity": 0.36665560170466255,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 761.2280131500681,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": null,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.16666666666666666,
"visibility_true_acc": 0.2222222222222222,
"visibility_false_acc": 0.0,
"is_in_frame_acc": 0.16666666666666666,
"is_in_frame_true_acc": 0.2222222222222222,
"is_in_frame_false_acc": 0.0,
"vis_acc_from_px": 0.16666666666666666,
"vis_acc_from_px_true_acc": 0.2222222222222222,
"vis_acc_from_px_false_acc": 0.0
},
{
"target_id": 4,
"pred_id": 1,
"cost": 1.0,
"type_similarity": 0.4220742415072903,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": null,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": null,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.0,
"visibility_true_acc": 0.0,
"visibility_false_acc": 0.0,
"is_in_frame_acc": 0.0,
"is_in_frame_true_acc": 0.0,
"is_in_frame_false_acc": 0.0,
"vis_acc_from_px": 0.0,
"vis_acc_from_px_true_acc": 0.0,
"vis_acc_from_px_false_acc": 0.0
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": 0.4583333333333333,
"avg_interaction_input_object_jaccard": 0.6666666666666666,
"avg_interaction_output_object_jaccard": 0.6666666666666666,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 26,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 26,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.13333333333333333,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.23529411764705882,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 26,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 28,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 28,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 28,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.13333333333333333,
"0.5": 0.06666666666666667,
"0.75": 0.06666666666666667,
"1.0": 0.06666666666666667
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.23529411764705882,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 26,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 28,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 28,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 28,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.13333333333333333,
"0.5": 0.06666666666666667,
"0.75": 0.06666666666666667,
"1.0": 0.06666666666666667
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.23529411764705882,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 2,
"target_id": 0,
"pred_id": 2,
"cost": 0.27083333333333337,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.27083333333333337,
"weighted_cost": 0.27083333333333337,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 6,
"target_id": 1,
"pred_id": 6,
"cost": 0.27083333333333337,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.27083333333333337,
"weighted_cost": 0.27083333333333337,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 3,
"target_id": 2,
"pred_id": 3,
"cost": 0.27083333333333337,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.27083333333333337,
"weighted_cost": 0.27083333333333337,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 5,
"target_id": 3,
"pred_id": 5,
"cost": 0.27083333333333337,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.27083333333333337,
"weighted_cost": 0.27083333333333337,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
4,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29
],
"target_type_counts": {
"contact": 4
},
"pred_type_counts": {
"contact": 28,
"collision": 2
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 2,
"target_id": 0,
"pred_id": 2,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.27083333333333337,
"temporal_span_iou": 0.4583333333333333,
"temporal_span_giou": 0.4583333333333333,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 1,
"pred_index": 6,
"target_id": 1,
"pred_id": 6,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.27083333333333337,
"temporal_span_iou": 0.4583333333333333,
"temporal_span_giou": 0.4583333333333333,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 2,
"pred_index": 3,
"target_id": 2,
"pred_id": 3,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.27083333333333337,
"temporal_span_iou": 0.4583333333333333,
"temporal_span_giou": 0.4583333333333333,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 3,
"pred_index": 5,
"target_id": 3,
"pred_id": 5,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.27083333333333337,
"temporal_span_iou": 0.4583333333333333,
"temporal_span_giou": 0.4583333333333333,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0.4583333333333333,
"avg_interaction_input_object_jaccard": 0.6666666666666666,
"avg_interaction_output_object_jaccard": 0.6666666666666666,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 18,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.18181818181818182,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.3076923076923077,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 20,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 20,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 20,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.18181818181818182,
"0.5": 0.09090909090909091,
"0.75": 0.09090909090909091,
"1.0": 0.09090909090909091
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.3076923076923077,
"0.5": 0.16666666666666669,
"0.75": 0.16666666666666669,
"1.0": 0.16666666666666669
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 20,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 20,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 20,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.18181818181818182,
"0.5": 0.09090909090909091,
"0.75": 0.09090909090909091,
"1.0": 0.09090909090909091
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.3076923076923077,
"0.5": 0.16666666666666669,
"0.75": 0.16666666666666669,
"1.0": 0.16666666666666669
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 8,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 2,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": 0.4583333333333333,
"avg_interaction_input_object_jaccard": 0.6666666666666666,
"avg_interaction_output_object_jaccard": 0.6666666666666666,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 24,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 28,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 28,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 28,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.14285714285714285,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.25,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 26,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 26,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 26,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.14285714285714285,
"0.5": 0.07142857142857142,
"0.75": 0.07142857142857142,
"1.0": 0.07142857142857142
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.25,
"0.5": 0.13333333333333333,
"0.75": 0.13333333333333333,
"1.0": 0.13333333333333333
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 26,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 26,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 26,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.14285714285714285,
"0.5": 0.07142857142857142,
"0.75": 0.07142857142857142,
"1.0": 0.07142857142857142
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.25,
"0.5": 0.13333333333333333,
"0.75": 0.13333333333333333,
"1.0": 0.13333333333333333
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"contact": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}