STRIVE · 08/28/2026
Dyn generalization · test_120 · super_clevr_1163
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"purple minivan\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 100 982 1 116 976 2 100 970 3 117 964 4 118 958 5 133 952 6 121 947 7 121 942 8 126 936 9 139 931 10 133 927 11 135 922 12 141 918 13 133 914 14 128 909 15 127 905 16 130 901 17 128 898 18 127 892 19 126 884 20 125 879 21 123 872 22 121 871 23 120 860 24 118 860 25 116 858 26 115 857 27 111 880 29 110 867 30 109 870 31 108 867 32 108 853 33 108 847 34 108 852 35 106 839 36 105 836 37 103 826 38 101 820 39 101 822 40 98 813 41 97 806 42 95 801 43 94 800 44 92 798 45 90 796 46 90 792 47 87 789 48 90 785 49 90 782 50 91 779 51 91 776 52 91 772 53 71 773 54 93 766 55 73 767 56 76 763 57 61 764 58 74 757 59 59 758 60 77 751 61 62 752 62 61 750 63 60 747 65 59 743 66 59 740 68 59 734 70 56 731 71 60 727 72 56 726 73 57 724 74 58 721 75 59 719 76 55 718 77 56 715 78 58 713 79 58 710 80 59 707 81 56 706 82 58 704 83 62 699 84 54 701 86 53 697 87 60 693 88 51 694 89 51 692 90 53 691 91 50 689 92 62 685 93 60 682 94 51 683 96 51 680 98 53 678 99 52 676 101 51 672 103 51 669 105 51 667 107 51 664 109 51 661 110 51 659 112 51 657 113 60 655 114 58 652 116 58 650 118 59 648 [OBJ] 1 \"purple dirtbike\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 233 772 2 250 743 13 243 782 17 233 769 18 237 765 19 241 760 20 244 756 21 251 752 22 255 748 23 258 746 24 262 742 25 266 742 26 270 738 27 273 736 28 276 735 29 279 734 30 283 735 31 280 705 32 281 704 33 283 703 34 284 702 36 287 701 38 290 701 40 291 696 41 293 698 42 292 696 43 297 700 45 298 701 46 301 703 47 317 717 48 319 718 49 320 719 50 321 717 51 327 716 52 330 713 53 329 710 54 333 706 55 334 704 56 336 702 57 338 702 58 339 700 60 342 697 61 343 694 63 319 682 64 335 684 65 322 679 66 324 678 67 326 677 68 328 677 69 353 681 70 356 679 71 358 678 72 359 676 74 361 674 75 362 672 77 364 670 78 366 669 79 368 667 81 370 664 82 372 663 84 375 659 86 378 657 87 380 655 88 382 654 89 383 653 90 385 652 91 387 651 92 388 650 93 390 648 95 392 646 96 394 645 97 396 644 98 394 641 100 398 639 101 394 640 102 399 635 104 401 634 105 402 633 106 405 632 107 406 631 109 412 630 110 405 630 111 407 629 112 409 628 113 411 627 114 412 626 116 421 624 117 414 624 118 416 623 119 418 622 [OBJ] 2 \"red utility\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 703 670 3 738 703 5 758 650 6 770 703 7 758 650 24 770 703 25 758 650 26 738 703 27 758 650 28 703 670 29 747 703 49 770 703 [OBJ] 3 \"green airliner\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 752 196 1 748 195 2 744 195 3 741 195 4 737 195 5 733 195 6 729 194 8 726 195 10 722 195 11 719 195 12 708 194 13 705 194 14 701 194 15 698 194 16 695 194 17 691 193 18 687 193 19 683 193 20 679 193 21 675 193 22 670 192 23 668 192 24 664 192 25 660 192 26 656 192 27 685 193 28 654 192 29 651 192 30 648 192 31 645 192 32 633 191 33 629 191 34 625 191 35 623 191 36 616 190 37 613 190 38 608 190 39 605 190 40 601 190 41 595 189 42 591 189 43 587 189 44 583 189 45 579 188 46 582 189 47 579 189 48 576 189 49 561 186 50 558 188 51 555 188 52 552 188 54 541 186 55 537 186 56 532 186 57 528 186 58 523 185 59 519 185 60 514 185 61 544 188 62 504 184 63 535 186 64 506 185 65 526 186 66 522 186 67 483 184 69 480 184 70 476 184 71 498 185 72 494 185 73 489 185 74 484 185 75 479 184 76 475 184 77 470 184 78 466 184 79 459 183 80 455 183 81 450 183 82 413 181 83 441 183 84 408 181 85 405 181 86 390 180 87 421 182 88 416 182 89 411 182 90 405 181 91 401 181 92 396 181 93 391 181 94 384 180 95 379 180 96 374 180 97 340 178 98 337 178 99 333 178 100 317 177 101 348 179 102 343 179 103 299 176 104 294 176 105 326 178 106 321 178 107 421 153 108 309 177 109 304 177 110 298 177 111 266 175 112 261 175 113 244 174 114 240 174 115 380 150 116 374 150 117 219 173 118 363 149 119 358 149 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 18 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 4 \"contact\" 0 51 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 8 \"contact\" 0 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 0 \"collision\" 17 17 [IN] 0 1 [OUT] 0 1 [INTERACT] 2 \"collision\" 22 22 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 3 \"contact\" 23 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 5 \"collision\" 58 58 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 6 \"collision\" 60 60 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 7 \"contact\" 61 119 [IN] 1 65535 [OUT] 1 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_full/super_clevr_1163/frames.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"pink and purple plastic package of overnight pads\" 0 24 [VIS] 0 true 13 false [IN-FRAME] 0 true 13 false [PX] 0 1 994 2 5 986 5 5 974 7 5 962 9 5 951 11 1 943 13 null null [OBJ] 1 \"pink and purple plastic package of overnight pads\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 13 1 1 17 5 2 25 9 3 33 13 4 41 17 5 48 21 6 56 25 7 64 29 8 72 33 9 80 37 10 87 41 11 95 44 12 103 48 13 111 52 14 119 56 15 126 60 16 134 64 17 142 68 18 150 72 19 158 76 20 166 80 21 173 83 22 181 87 23 189 91 [OBJ] 2 \"pink and purple plastic package of overnight pads\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 130 970 2 138 962 3 142 955 4 150 947 5 158 939 6 162 931 7 166 923 8 169 916 9 173 908 11 177 892 13 177 880 15 173 865 16 169 853 17 166 845 18 162 837 19 158 830 20 154 822 21 150 814 22 146 806 23 142 798 [OBJ] 3 \"black and pink running shoe\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 236 783 3 240 791 7 244 783 11 255 779 13 267 779 14 275 783 15 283 779 16 287 771 18 294 767 20 302 759 21 310 755 23 318 751 [OBJ] 4 \"red and white striped dishtowel\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 630 580 1 null null [OBJ] 5 \"red and green wooden bird rattle\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 689 603 1 null null [OBJ] 6 \"red and green cardboard ink cartridge box\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 705 580 1 null null [OBJ] 7 \"red and green cardboard ink cartridge box\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 712 591 1 null null [OBJ] 8 \"red and black plastic handheld game console\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 716 587 1 null null [OBJ] 9 \"red and black plastic handheld game console\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 716 599 1 null null [OBJ] 10 \"olive green and brown leather flip-flop\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 724 583 1 null null [OBJ] 11 \"red and green cardboard ink cartridge box\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 724 591 1 null null [OBJ] 12 \"red and black plastic handheld game console\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 732 591 1 null null [OBJ] 13 \"red and black plastic handheld game console\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 736 595 1 null null [OBJ] 14 \"red and green cardboard ink cartridge box\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 740 583 1 null null [OBJ] 15 \"red and black plastic handheld game console\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 744 595 1 null null [OBJ] 16 \"red and green wooden bird rattle\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 748 583 1 null null [OBJ] 17 \"red and green cardboard ink cartridge box\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 751 587 1 null null [OBJ] 18 \"red and black plastic handheld game console\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 755 591 1 null null [OBJ] 19 \"olive green and brown leather flip-flop\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 763 587 1 null null [OBJ] 20 \"red and black plastic handheld game console\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 767 595 1 null null [OBJ] 21 \"red scissors\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 787 708 1 771 712 2 787 708 10 775 712 12 763 712 13 775 708 16 763 712 18 775 708 19 759 712 21 771 708 22 755 712 [OBJ] 22 \"dark grey and purple plastic Shredder action figure\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 603 3 998 615 5 994 623 9 990 630 13 994 638 17 990 646 21 994 654 23 986 658 [OBJ] 23 \"dark green plastic triceratops figurine\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 779 3 994 791 5 994 779 6 990 791 10 986 783 11 982 791 14 978 783 15 974 791 19 966 794 21 958 791 22 958 802 23 955 794 [OBJ] 24 \"dark grey and purple plastic Shredder action figure\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 798 3 994 806 5 994 794 6 990 802 10 986 794 11 982 802 15 978 794 16 974 802 20 970 794 21 966 802 25 962 794 [OBJ] 25 \"dark green plastic triceratops figurine\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 810 2 994 818 6 994 806 8 990 814 12 990 802 13 986 810 17 986 798 18 982 806 22 978 798 [OBJ] 26 \"dark blue fabric chukka boot\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 849 4 994 857 9 994 845 10 990 853 15 986 845 16 982 853 21 978 845 22 974 853 [OBJ] 27 \"dark green plastic triceratops figurine\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 873 1 994 880 3 994 869 6 990 876 8 990 861 11 986 869 14 986 857 17 982 865 20 982 853 23 978 861 [OBJ] 28 \"dark green plastic triceratops figurine\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 919 4 994 912 7 994 896 10 990 904 13 990 892 16 986 884 19 982 892 22 982 880 [OBJ] 29 \"purple and beige wooden dollhouse lamp\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 986 3 990 990 6 982 986 10 970 986 13 962 982 16 955 978 19 943 978 22 935 974 23 927 978 [OBJ] 30 \"black and white fabric stuffed orca\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 994 2 990 990 6 982 986 9 974 982 12 966 978 16 958 974 19 947 970 22 939 966 23 931 962 [OBJ] 31 \"dark teal wooden toy train\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 998 4 990 994 8 982 990 12 974 986 16 966 982 20 955 978 23 947 974 [OBJ] 32 \"green and yellow cardboard whey protein box\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 998 998 1 null null [OBJ] 33 \"dark teal wooden toy train\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 998 999 1 null null [OBJ] 34 \"purple and black cardboard Nintendo 3DS game box\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 999 990 1 null null [OBJ] 35 \"dark blue fabric high-top boat shoe\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 999 994 3 998 986 7 994 978 11 990 970 15 986 962 19 982 955 22 978 947 [OBJ] 36 \"green and yellow cardboard ink cartridge box\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 999 998 3 994 990 7 990 982 10 986 974 14 978 966 18 974 958 21 970 951 23 966 943 [OBJ] 37 \"green and white plastic portable speaker\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 853 173 1 845 169 2 837 166 4 826 166 6 814 166 8 802 166 10 791 166 12 779 166 14 767 166 16 755 166 18 744 166 20 732 166 22 720 166 23 708 169 [OBJ] 38 \"black and white fabric stuffed orca\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 998 1 994 990 5 990 982 9 986 974 13 982 966 17 978 958 20 974 951 23 970 943 [OBJ] 39 \"dark grey cardboard sound samples box\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 994 1 994 986 5 990 978 9 986 970 12 978 966 16 974 958 19 966 955 23 962 947 [OBJ] 40 \"dark red and black plastic curling wand\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 998 4 994 990 7 990 982 11 986 974 15 982 966 18 978 958 22 974 951 [OBJ] 41 \"dark green plastic triceratops figurine\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 999 994 2 994 986 6 990 978 10 986 970 14 978 966 17 974 958 20 970 951 23 966 943 [OBJ] 42 \"green ink cartridge cardboard box\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 999 990 4 998 978 7 994 970 10 990 962 13 990 951 16 986 943 19 982 935 22 978 927 [OBJ] 43 \"dark brown leather boat shoe\" 1 24 [VIS] 0 false 1 true [IN-FRAME] 0 false 1 true [PX] 0 null null 1 41 455 2 64 455 3 87 455 4 111 455 5 134 455 6 158 455 7 181 455 8 193 451 9 205 451 10 216 451 11 228 451 12 240 451 13 251 451 14 263 451 15 271 447 16 283 451 18 294 447 20 306 447 22 318 447 23 326 451 [OBJ] 44 \"dark grey fabric boat shoe\" 2 24 [VIS] 0 false 2 true [IN-FRAME] 0 false 2 true [PX] 0 null null 2 998 998 5 990 994 7 986 986 10 978 982 12 974 974 15 966 970 18 962 962 20 955 958 23 951 951 [OBJ] 45 \"dark green plastic triceratops figurine\" 12 24 [VIS] 0 false 12 true [IN-FRAME] 0 false 12 true [PX] 0 null null 12 1 580 14 1 568 16 1 556 18 1 544 20 5 560 22 5 548 23 5 537 [OBJ] 46 \"dark grey cardboard sound samples box\" 15 24 [VIS] 0 false 15 true [IN-FRAME] 0 false 15 true [PX] 0 null null 15 1 587 16 1 576 17 1 564 18 5 583 19 5 572 20 5 560 21 9 580 22 9 568 23 9 556 [OBJ] 47 \"dark brown leather loafer\" 18 24 [VIS] 0 false 18 true [IN-FRAME] 0 false 18 true [PX] 0 null null 18 1 591 20 1 576 22 5 587 23 5 576 [OBJ] 48 \"black plastic gaming mouse\" 22 24 [VIS] 0 false 22 true [IN-FRAME] 0 false 22 true [PX] 0 null null 22 1 576 [OBJS-END] [INTERACTIONS-START] [INTERACT] 14 \"contact\" 0 1 [IN] 12 17 [OUT] 12 17 [INTERACT] 28 \"contact\" 0 1 [IN] 3 4 [OUT] 3 4 [INTERACT] 12 \"contact\" 0 14 [IN] 12 15 [OUT] 12 15 [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 31 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 38 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 39 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 40 \"contact\" 0 23 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 41 \"contact\" 0 23 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 42 \"contact\" 0 23 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 43 \"contact\" 0 23 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 44 \"contact\" 0 23 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 11 \"contact\" 0 23 [IN] 12 14 [OUT] 12 14 [INTERACT] 22 \"contact\" 0 23 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 23 \"contact\" 0 23 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 24 \"contact\" 0 23 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 25 \"contact\" 0 23 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 26 \"contact\" 0 23 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 27 \"contact\" 0 23 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 29 \"contact\" 0 23 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 30 \"contact\" 0 23 [IN] 22 65535 [OUT] 22 65535 [INTERACT] 32 \"contact\" 0 23 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 33 \"contact\" 0 23 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 34 \"contact\" 0 23 [IN] 26 65535 [OUT] 26 65535 [INTERACT] 35 \"contact\" 0 23 [IN] 27 65535 [OUT] 27 65535 [INTERACT] 36 \"contact\" 0 23 [IN] 28 65535 [OUT] 28 65535 [INTERACT] 37 \"contact\" 0 23 [IN] 29 65535 [OUT] 29 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 15 \"contact\" 0 23 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 16 \"contact\" 0 23 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 17 \"contact\" 0 23 [IN] 14 15 [OUT] 14 15 [INTERACT] 18 \"contact\" 0 23 [IN] 14 17 [OUT] 14 17 [INTERACT] 19 \"contact\" 0 23 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 20 \"contact\" 0 23 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 21 \"contact\" 0 23 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 9 \"collision\" 2 2 [IN] 12 17 [OUT] 12 17 [INTERACT] 10 \"contact\" 3 23 [IN] 12 17 [OUT] 12 17 [INTERACT] 5 \"collision\" 5 5 [IN] 12 15 [OUT] 12 15 [INTERACT] 6 \"contact\" 6 7 [IN] 12 15 [OUT] 12 15 [INTERACT] 7 \"collision\" 9 9 [IN] 12 15 [OUT] 12 15 [INTERACT] 8 \"collision\" 11 11 [IN] 12 15 [OUT] 12 15 [INTERACT] 13 \"collision\" 11 11 [IN] 12 17 [OUT] 12 17 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"purple minivan\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 100 982 1 116 976 2 100 970 3 117 964 4 118 958 5 133 952 6 121 947 7 121 942 8 126 936 9 139 931 10 133 927 11 135 922 12 141 918 13 133 914 14 128 909 15 127 905 16 130 901 17 128 898 18 127 892 19 126 884 20 125 879 21 123 872 22 121 871 23 120 860 24 118 860 25 116 858 26 115 857 27 111 880 29 110 867 30 109 870 31 108 867 32 108 853 33 108 847 34 108 852 35 106 839 36 105 836 37 103 826 38 101 820 39 101 822 40 98 813 41 97 806 42 95 801 43 94 800 44 92 798 45 90 796 46 90 792 47 87 789 48 90 785 49 90 782 50 91 779 51 91 776 52 91 772 53 71 773 54 93 766 55 73 767 56 76 763 57 61 764 58 74 757 59 59 758 60 77 751 61 62 752 62 61 750 63 60 747 65 59 743 66 59 740 68 59 734 70 56 731 71 60 727 72 56 726 73 57 724 74 58 721 75 59 719 76 55 718 77 56 715 78 58 713 79 58 710 80 59 707 81 56 706 82 58 704 83 62 699 84 54 701 86 53 697 87 60 693 88 51 694 89 51 692 90 53 691 91 50 689 92 62 685 93 60 682 94 51 683 96 51 680 98 53 678 99 52 676 101 51 672 103 51 669 105 51 667 107 51 664 109 51 661 110 51 659 112 51 657 113 60 655 114 58 652 116 58 650 118 59 648 [OBJ] 1 \"purple dirtbike\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 233 772 2 250 743 13 243 782 17 233 769 18 237 765 19 241 760 20 244 756 21 251 752 22 255 748 23 258 746 24 262 742 25 266 742 26 270 738 27 273 736 28 276 735 29 279 734 30 283 735 31 280 705 32 281 704 33 283 703 34 284 702 36 287 701 38 290 701 40 291 696 41 293 698 42 292 696 43 297 700 45 298 701 46 301 703 47 317 717 48 319 718 49 320 719 50 321 717 51 327 716 52 330 713 53 329 710 54 333 706 55 334 704 56 336 702 57 338 702 58 339 700 60 342 697 61 343 694 63 319 682 64 335 684 65 322 679 66 324 678 67 326 677 68 328 677 69 353 681 70 356 679 71 358 678 72 359 676 74 361 674 75 362 672 77 364 670 78 366 669 79 368 667 81 370 664 82 372 663 84 375 659 86 378 657 87 380 655 88 382 654 89 383 653 90 385 652 91 387 651 92 388 650 93 390 648 95 392 646 96 394 645 97 396 644 98 394 641 100 398 639 101 394 640 102 399 635 104 401 634 105 402 633 106 405 632 107 406 631 109 412 630 110 405 630 111 407 629 112 409 628 113 411 627 114 412 626 116 421 624 117 414 624 118 416 623 119 418 622 [OBJ] 2 \"red utility\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 703 670 3 738 703 5 758 650 6 770 703 7 758 650 24 770 703 25 758 650 26 738 703 27 758 650 28 703 670 29 747 703 49 770 703 [OBJ] 3 \"green airliner\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 752 196 1 748 195 2 744 195 3 741 195 4 737 195 5 733 195 6 729 194 8 726 195 10 722 195 11 719 195 12 708 194 13 705 194 14 701 194 15 698 194 16 695 194 17 691 193 18 687 193 19 683 193 20 679 193 21 675 193 22 670 192 23 668 192 24 664 192 25 660 192 26 656 192 27 685 193 28 654 192 29 651 192 30 648 192 31 645 192 32 633 191 33 629 191 34 625 191 35 623 191 36 616 190 37 613 190 38 608 190 39 605 190 40 601 190 41 595 189 42 591 189 43 587 189 44 583 189 45 579 188 46 582 189 47 579 189 48 576 189 49 561 186 50 558 188 51 555 188 52 552 188 54 541 186 55 537 186 56 532 186 57 528 186 58 523 185 59 519 185 60 514 185 61 544 188 62 504 184 63 535 186 64 506 185 65 526 186 66 522 186 67 483 184 69 480 184 70 476 184 71 498 185 72 494 185 73 489 185 74 484 185 75 479 184 76 475 184 77 470 184 78 466 184 79 459 183 80 455 183 81 450 183 82 413 181 83 441 183 84 408 181 85 405 181 86 390 180 87 421 182 88 416 182 89 411 182 90 405 181 91 401 181 92 396 181 93 391 181 94 384 180 95 379 180 96 374 180 97 340 178 98 337 178 99 333 178 100 317 177 101 348 179 102 343 179 103 299 176 104 294 176 105 326 178 106 321 178 107 421 153 108 309 177 109 304 177 110 298 177 111 266 175 112 261 175 113 244 174 114 240 174 115 380 150 116 374 150 117 219 173 118 363 149 119 358 149 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 18 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 4 \"contact\" 0 51 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 8 \"contact\" 0 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 0 \"collision\" 17 17 [IN] 0 1 [OUT] 0 1 [INTERACT] 2 \"collision\" 22 22 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 3 \"contact\" 23 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 5 \"collision\" 58 58 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 6 \"collision\" 60 60 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 7 \"contact\" 61 119 [IN] 1 65535 [OUT] 1 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "pink and purple plastic package of overnight pads",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 13,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 13,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 994
},
{
"t": 2,
"x": 5,
"y": 986
},
{
"t": 5,
"x": 5,
"y": 974
},
{
"t": 7,
"x": 5,
"y": 962
},
{
"t": 9,
"x": 5,
"y": 951
},
{
"t": 11,
"x": 1,
"y": 943
},
{
"t": 13,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "pink and purple plastic package of overnight pads",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 13,
"y": 1
},
{
"t": 1,
"x": 17,
"y": 5
},
{
"t": 2,
"x": 25,
"y": 9
},
{
"t": 3,
"x": 33,
"y": 13
},
{
"t": 4,
"x": 41,
"y": 17
},
{
"t": 5,
"x": 48,
"y": 21
},
{
"t": 6,
"x": 56,
"y": 25
},
{
"t": 7,
"x": 64,
"y": 29
},
{
"t": 8,
"x": 72,
"y": 33
},
{
"t": 9,
"x": 80,
"y": 37
},
{
"t": 10,
"x": 87,
"y": 41
},
{
"t": 11,
"x": 95,
"y": 44
},
{
"t": 12,
"x": 103,
"y": 48
},
{
"t": 13,
"x": 111,
"y": 52
},
{
"t": 14,
"x": 119,
"y": 56
},
{
"t": 15,
"x": 126,
"y": 60
},
{
"t": 16,
"x": 134,
"y": 64
},
{
"t": 17,
"x": 142,
"y": 68
},
{
"t": 18,
"x": 150,
"y": 72
},
{
"t": 19,
"x": 158,
"y": 76
},
{
"t": 20,
"x": 166,
"y": 80
},
{
"t": 21,
"x": 173,
"y": 83
},
{
"t": 22,
"x": 181,
"y": 87
},
{
"t": 23,
"x": 189,
"y": 91
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "pink and purple plastic package of overnight pads",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 130,
"y": 970
},
{
"t": 2,
"x": 138,
"y": 962
},
{
"t": 3,
"x": 142,
"y": 955
},
{
"t": 4,
"x": 150,
"y": 947
},
{
"t": 5,
"x": 158,
"y": 939
},
{
"t": 6,
"x": 162,
"y": 931
},
{
"t": 7,
"x": 166,
"y": 923
},
{
"t": 8,
"x": 169,
"y": 916
},
{
"t": 9,
"x": 173,
"y": 908
},
{
"t": 11,
"x": 177,
"y": 892
},
{
"t": 13,
"x": 177,
"y": 880
},
{
"t": 15,
"x": 173,
"y": 865
},
{
"t": 16,
"x": 169,
"y": 853
},
{
"t": 17,
"x": 166,
"y": 845
},
{
"t": 18,
"x": 162,
"y": 837
},
{
"t": 19,
"x": 158,
"y": 830
},
{
"t": 20,
"x": 154,
"y": 822
},
{
"t": 21,
"x": 150,
"y": 814
},
{
"t": 22,
"x": 146,
"y": 806
},
{
"t": 23,
"x": 142,
"y": 798
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "black and pink running shoe",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 236,
"y": 783
},
{
"t": 3,
"x": 240,
"y": 791
},
{
"t": 7,
"x": 244,
"y": 783
},
{
"t": 11,
"x": 255,
"y": 779
},
{
"t": 13,
"x": 267,
"y": 779
},
{
"t": 14,
"x": 275,
"y": 783
},
{
"t": 15,
"x": 283,
"y": 779
},
{
"t": 16,
"x": 287,
"y": 771
},
{
"t": 18,
"x": 294,
"y": 767
},
{
"t": 20,
"x": 302,
"y": 759
},
{
"t": 21,
"x": 310,
"y": 755
},
{
"t": 23,
"x": 318,
"y": 751
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "red and white striped dishtowel",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 630,
"y": 580
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 5,
"type": "red and green wooden bird rattle",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 689,
"y": 603
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 6,
"type": "red and green cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 705,
"y": 580
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 7,
"type": "red and green cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 712,
"y": 591
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 8,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 716,
"y": 587
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 9,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 716,
"y": 599
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 10,
"type": "olive green and brown leather flip-flop",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 724,
"y": 583
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 11,
"type": "red and green cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 724,
"y": 591
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 12,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 732,
"y": 591
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 13,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 736,
"y": 595
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 14,
"type": "red and green cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 740,
"y": 583
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 15,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 744,
"y": 595
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 16,
"type": "red and green wooden bird rattle",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 748,
"y": 583
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 17,
"type": "red and green cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 751,
"y": 587
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 18,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 755,
"y": 591
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 19,
"type": "olive green and brown leather flip-flop",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 763,
"y": 587
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 20,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 767,
"y": 595
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 21,
"type": "red scissors",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 787,
"y": 708
},
{
"t": 1,
"x": 771,
"y": 712
},
{
"t": 2,
"x": 787,
"y": 708
},
{
"t": 10,
"x": 775,
"y": 712
},
{
"t": 12,
"x": 763,
"y": 712
},
{
"t": 13,
"x": 775,
"y": 708
},
{
"t": 16,
"x": 763,
"y": 712
},
{
"t": 18,
"x": 775,
"y": 708
},
{
"t": 19,
"x": 759,
"y": 712
},
{
"t": 21,
"x": 771,
"y": 708
},
{
"t": 22,
"x": 755,
"y": 712
}
],
"birth": 0,
"death": 24
},
{
"id": 22,
"type": "dark grey and purple plastic Shredder action figure",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 603
},
{
"t": 3,
"x": 998,
"y": 615
},
{
"t": 5,
"x": 994,
"y": 623
},
{
"t": 9,
"x": 990,
"y": 630
},
{
"t": 13,
"x": 994,
"y": 638
},
{
"t": 17,
"x": 990,
"y": 646
},
{
"t": 21,
"x": 994,
"y": 654
},
{
"t": 23,
"x": 986,
"y": 658
}
],
"birth": 0,
"death": 24
},
{
"id": 23,
"type": "dark green plastic triceratops figurine",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 779
},
{
"t": 3,
"x": 994,
"y": 791
},
{
"t": 5,
"x": 994,
"y": 779
},
{
"t": 6,
"x": 990,
"y": 791
},
{
"t": 10,
"x": 986,
"y": 783
},
{
"t": 11,
"x": 982,
"y": 791
},
{
"t": 14,
"x": 978,
"y": 783
},
{
"t": 15,
"x": 974,
"y": 791
},
{
"t": 19,
"x": 966,
"y": 794
},
{
"t": 21,
"x": 958,
"y": 791
},
{
"t": 22,
"x": 958,
"y": 802
},
{
"t": 23,
"x": 955,
"y": 794
}
],
"birth": 0,
"death": 24
},
{
"id": 24,
"type": "dark grey and purple plastic Shredder action figure",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 798
},
{
"t": 3,
"x": 994,
"y": 806
},
{
"t": 5,
"x": 994,
"y": 794
},
{
"t": 6,
"x": 990,
"y": 802
},
{
"t": 10,
"x": 986,
"y": 794
},
{
"t": 11,
"x": 982,
"y": 802
},
{
"t": 15,
"x": 978,
"y": 794
},
{
"t": 16,
"x": 974,
"y": 802
},
{
"t": 20,
"x": 970,
"y": 794
},
{
"t": 21,
"x": 966,
"y": 802
},
{
"t": 25,
"x": 962,
"y": 794
}
],
"birth": 0,
"death": 24
},
{
"id": 25,
"type": "dark green plastic triceratops figurine",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 810
},
{
"t": 2,
"x": 994,
"y": 818
},
{
"t": 6,
"x": 994,
"y": 806
},
{
"t": 8,
"x": 990,
"y": 814
},
{
"t": 12,
"x": 990,
"y": 802
},
{
"t": 13,
"x": 986,
"y": 810
},
{
"t": 17,
"x": 986,
"y": 798
},
{
"t": 18,
"x": 982,
"y": 806
},
{
"t": 22,
"x": 978,
"y": 798
}
],
"birth": 0,
"death": 24
},
{
"id": 26,
"type": "dark blue fabric chukka boot",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 849
},
{
"t": 4,
"x": 994,
"y": 857
},
{
"t": 9,
"x": 994,
"y": 845
},
{
"t": 10,
"x": 990,
"y": 853
},
{
"t": 15,
"x": 986,
"y": 845
},
{
"t": 16,
"x": 982,
"y": 853
},
{
"t": 21,
"x": 978,
"y": 845
},
{
"t": 22,
"x": 974,
"y": 853
}
],
"birth": 0,
"death": 24
},
{
"id": 27,
"type": "dark green plastic triceratops figurine",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 873
},
{
"t": 1,
"x": 994,
"y": 880
},
{
"t": 3,
"x": 994,
"y": 869
},
{
"t": 6,
"x": 990,
"y": 876
},
{
"t": 8,
"x": 990,
"y": 861
},
{
"t": 11,
"x": 986,
"y": 869
},
{
"t": 14,
"x": 986,
"y": 857
},
{
"t": 17,
"x": 982,
"y": 865
},
{
"t": 20,
"x": 982,
"y": 853
},
{
"t": 23,
"x": 978,
"y": 861
}
],
"birth": 0,
"death": 24
},
{
"id": 28,
"type": "dark green plastic triceratops figurine",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 919
},
{
"t": 4,
"x": 994,
"y": 912
},
{
"t": 7,
"x": 994,
"y": 896
},
{
"t": 10,
"x": 990,
"y": 904
},
{
"t": 13,
"x": 990,
"y": 892
},
{
"t": 16,
"x": 986,
"y": 884
},
{
"t": 19,
"x": 982,
"y": 892
},
{
"t": 22,
"x": 982,
"y": 880
}
],
"birth": 0,
"death": 24
},
{
"id": 29,
"type": "purple and beige wooden dollhouse lamp",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 986
},
{
"t": 3,
"x": 990,
"y": 990
},
{
"t": 6,
"x": 982,
"y": 986
},
{
"t": 10,
"x": 970,
"y": 986
},
{
"t": 13,
"x": 962,
"y": 982
},
{
"t": 16,
"x": 955,
"y": 978
},
{
"t": 19,
"x": 943,
"y": 978
},
{
"t": 22,
"x": 935,
"y": 974
},
{
"t": 23,
"x": 927,
"y": 978
}
],
"birth": 0,
"death": 24
},
{
"id": 30,
"type": "black and white fabric stuffed orca",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 994
},
{
"t": 2,
"x": 990,
"y": 990
},
{
"t": 6,
"x": 982,
"y": 986
},
{
"t": 9,
"x": 974,
"y": 982
},
{
"t": 12,
"x": 966,
"y": 978
},
{
"t": 16,
"x": 958,
"y": 974
},
{
"t": 19,
"x": 947,
"y": 970
},
{
"t": 22,
"x": 939,
"y": 966
},
{
"t": 23,
"x": 931,
"y": 962
}
],
"birth": 0,
"death": 24
},
{
"id": 31,
"type": "dark teal wooden toy train",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 998
},
{
"t": 4,
"x": 990,
"y": 994
},
{
"t": 8,
"x": 982,
"y": 990
},
{
"t": 12,
"x": 974,
"y": 986
},
{
"t": 16,
"x": 966,
"y": 982
},
{
"t": 20,
"x": 955,
"y": 978
},
{
"t": 23,
"x": 947,
"y": 974
}
],
"birth": 0,
"death": 24
},
{
"id": 32,
"type": "green and yellow cardboard whey protein box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 998
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 33,
"type": "dark teal wooden toy train",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 999
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 34,
"type": "purple and black cardboard Nintendo 3DS game box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 999,
"y": 990
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 35,
"type": "dark blue fabric high-top boat shoe",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 999,
"y": 994
},
{
"t": 3,
"x": 998,
"y": 986
},
{
"t": 7,
"x": 994,
"y": 978
},
{
"t": 11,
"x": 990,
"y": 970
},
{
"t": 15,
"x": 986,
"y": 962
},
{
"t": 19,
"x": 982,
"y": 955
},
{
"t": 22,
"x": 978,
"y": 947
}
],
"birth": 0,
"death": 24
},
{
"id": 36,
"type": "green and yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 999,
"y": 998
},
{
"t": 3,
"x": 994,
"y": 990
},
{
"t": 7,
"x": 990,
"y": 982
},
{
"t": 10,
"x": 986,
"y": 974
},
{
"t": 14,
"x": 978,
"y": 966
},
{
"t": 18,
"x": 974,
"y": 958
},
{
"t": 21,
"x": 970,
"y": 951
},
{
"t": 23,
"x": 966,
"y": 943
}
],
"birth": 0,
"death": 24
},
{
"id": 37,
"type": "green and white plastic portable speaker",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 853,
"y": 173
},
{
"t": 1,
"x": 845,
"y": 169
},
{
"t": 2,
"x": 837,
"y": 166
},
{
"t": 4,
"x": 826,
"y": 166
},
{
"t": 6,
"x": 814,
"y": 166
},
{
"t": 8,
"x": 802,
"y": 166
},
{
"t": 10,
"x": 791,
"y": 166
},
{
"t": 12,
"x": 779,
"y": 166
},
{
"t": 14,
"x": 767,
"y": 166
},
{
"t": 16,
"x": 755,
"y": 166
},
{
"t": 18,
"x": 744,
"y": 166
},
{
"t": 20,
"x": 732,
"y": 166
},
{
"t": 22,
"x": 720,
"y": 166
},
{
"t": 23,
"x": 708,
"y": 169
}
],
"birth": 0,
"death": 24
},
{
"id": 38,
"type": "black and white fabric stuffed orca",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 998
},
{
"t": 1,
"x": 994,
"y": 990
},
{
"t": 5,
"x": 990,
"y": 982
},
{
"t": 9,
"x": 986,
"y": 974
},
{
"t": 13,
"x": 982,
"y": 966
},
{
"t": 17,
"x": 978,
"y": 958
},
{
"t": 20,
"x": 974,
"y": 951
},
{
"t": 23,
"x": 970,
"y": 943
}
],
"birth": 0,
"death": 24
},
{
"id": 39,
"type": "dark grey cardboard sound samples box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 994
},
{
"t": 1,
"x": 994,
"y": 986
},
{
"t": 5,
"x": 990,
"y": 978
},
{
"t": 9,
"x": 986,
"y": 970
},
{
"t": 12,
"x": 978,
"y": 966
},
{
"t": 16,
"x": 974,
"y": 958
},
{
"t": 19,
"x": 966,
"y": 955
},
{
"t": 23,
"x": 962,
"y": 947
}
],
"birth": 0,
"death": 24
},
{
"id": 40,
"type": "dark red and black plastic curling wand",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 998
},
{
"t": 4,
"x": 994,
"y": 990
},
{
"t": 7,
"x": 990,
"y": 982
},
{
"t": 11,
"x": 986,
"y": 974
},
{
"t": 15,
"x": 982,
"y": 966
},
{
"t": 18,
"x": 978,
"y": 958
},
{
"t": 22,
"x": 974,
"y": 951
}
],
"birth": 0,
"death": 24
},
{
"id": 41,
"type": "dark green plastic triceratops figurine",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 999,
"y": 994
},
{
"t": 2,
"x": 994,
"y": 986
},
{
"t": 6,
"x": 990,
"y": 978
},
{
"t": 10,
"x": 986,
"y": 970
},
{
"t": 14,
"x": 978,
"y": 966
},
{
"t": 17,
"x": 974,
"y": 958
},
{
"t": 20,
"x": 970,
"y": 951
},
{
"t": 23,
"x": 966,
"y": 943
}
],
"birth": 0,
"death": 24
},
{
"id": 42,
"type": "green ink cartridge cardboard box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 999,
"y": 990
},
{
"t": 4,
"x": 998,
"y": 978
},
{
"t": 7,
"x": 994,
"y": 970
},
{
"t": 10,
"x": 990,
"y": 962
},
{
"t": 13,
"x": 990,
"y": 951
},
{
"t": 16,
"x": 986,
"y": 943
},
{
"t": 19,
"x": 982,
"y": 935
},
{
"t": 22,
"x": 978,
"y": 927
}
],
"birth": 0,
"death": 24
},
{
"id": 43,
"type": "dark brown leather boat shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 1,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 1,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 1,
"x": 41,
"y": 455
},
{
"t": 2,
"x": 64,
"y": 455
},
{
"t": 3,
"x": 87,
"y": 455
},
{
"t": 4,
"x": 111,
"y": 455
},
{
"t": 5,
"x": 134,
"y": 455
},
{
"t": 6,
"x": 158,
"y": 455
},
{
"t": 7,
"x": 181,
"y": 455
},
{
"t": 8,
"x": 193,
"y": 451
},
{
"t": 9,
"x": 205,
"y": 451
},
{
"t": 10,
"x": 216,
"y": 451
},
{
"t": 11,
"x": 228,
"y": 451
},
{
"t": 12,
"x": 240,
"y": 451
},
{
"t": 13,
"x": 251,
"y": 451
},
{
"t": 14,
"x": 263,
"y": 451
},
{
"t": 15,
"x": 271,
"y": 447
},
{
"t": 16,
"x": 283,
"y": 451
},
{
"t": 18,
"x": 294,
"y": 447
},
{
"t": 20,
"x": 306,
"y": 447
},
{
"t": 22,
"x": 318,
"y": 447
},
{
"t": 23,
"x": 326,
"y": 451
}
],
"birth": 1,
"death": 24
},
{
"id": 44,
"type": "dark grey fabric boat shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 2,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 2,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 2,
"x": 998,
"y": 998
},
{
"t": 5,
"x": 990,
"y": 994
},
{
"t": 7,
"x": 986,
"y": 986
},
{
"t": 10,
"x": 978,
"y": 982
},
{
"t": 12,
"x": 974,
"y": 974
},
{
"t": 15,
"x": 966,
"y": 970
},
{
"t": 18,
"x": 962,
"y": 962
},
{
"t": 20,
"x": 955,
"y": 958
},
{
"t": 23,
"x": 951,
"y": 951
}
],
"birth": 2,
"death": 24
},
{
"id": 45,
"type": "dark green plastic triceratops figurine",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 12,
"x": 1,
"y": 580
},
{
"t": 14,
"x": 1,
"y": 568
},
{
"t": 16,
"x": 1,
"y": 556
},
{
"t": 18,
"x": 1,
"y": 544
},
{
"t": 20,
"x": 5,
"y": 560
},
{
"t": 22,
"x": 5,
"y": 548
},
{
"t": 23,
"x": 5,
"y": 537
}
],
"birth": 12,
"death": 24
},
{
"id": 46,
"type": "dark grey cardboard sound samples box",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 15,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 15,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 15,
"x": 1,
"y": 587
},
{
"t": 16,
"x": 1,
"y": 576
},
{
"t": 17,
"x": 1,
"y": 564
},
{
"t": 18,
"x": 5,
"y": 583
},
{
"t": 19,
"x": 5,
"y": 572
},
{
"t": 20,
"x": 5,
"y": 560
},
{
"t": 21,
"x": 9,
"y": 580
},
{
"t": 22,
"x": 9,
"y": 568
},
{
"t": 23,
"x": 9,
"y": 556
}
],
"birth": 15,
"death": 24
},
{
"id": 47,
"type": "dark brown leather loafer",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 18,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 18,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 18,
"x": 1,
"y": 591
},
{
"t": 20,
"x": 1,
"y": 576
},
{
"t": 22,
"x": 5,
"y": 587
},
{
"t": 23,
"x": 5,
"y": 576
}
],
"birth": 18,
"death": 24
},
{
"id": 48,
"type": "black plastic gaming mouse",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 22,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 22,
"x": 1,
"y": 576
}
],
"birth": 22,
"death": 24
}
],
"interactions": [
{
"id": 14,
"inputs": [
12,
17
],
"outputs": [
12,
17
],
"begins_at": 0,
"ends_at": 1,
"type": "contact"
},
{
"id": 28,
"inputs": [
3,
4
],
"outputs": [
3,
4
],
"begins_at": 0,
"ends_at": 1,
"type": "contact"
},
{
"id": 12,
"inputs": [
12,
15
],
"outputs": [
12,
15
],
"begins_at": 0,
"ends_at": 14,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 31,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 38,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 39,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 40,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 41,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 42,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 43,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 44,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 11,
"inputs": [
12,
14
],
"outputs": [
12,
14
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 22,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 23,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 24,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 25,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 26,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 27,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 29,
"inputs": [
21,
65535
],
"outputs": [
21,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 30,
"inputs": [
22,
65535
],
"outputs": [
22,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 32,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 33,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 34,
"inputs": [
26,
65535
],
"outputs": [
26,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 35,
"inputs": [
27,
65535
],
"outputs": [
27,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 36,
"inputs": [
28,
65535
],
"outputs": [
28,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 37,
"inputs": [
29,
65535
],
"outputs": [
29,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 4,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 15,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 16,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 17,
"inputs": [
14,
15
],
"outputs": [
14,
15
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 18,
"inputs": [
14,
17
],
"outputs": [
14,
17
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 19,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 20,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 21,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 9,
"inputs": [
12,
17
],
"outputs": [
12,
17
],
"begins_at": 2,
"ends_at": 2,
"type": "collision"
},
{
"id": 10,
"inputs": [
12,
17
],
"outputs": [
12,
17
],
"begins_at": 3,
"ends_at": 23,
"type": "contact"
},
{
"id": 5,
"inputs": [
12,
15
],
"outputs": [
12,
15
],
"begins_at": 5,
"ends_at": 5,
"type": "collision"
},
{
"id": 6,
"inputs": [
12,
15
],
"outputs": [
12,
15
],
"begins_at": 6,
"ends_at": 7,
"type": "contact"
},
{
"id": 7,
"inputs": [
12,
15
],
"outputs": [
12,
15
],
"begins_at": 9,
"ends_at": 9,
"type": "collision"
},
{
"id": 8,
"inputs": [
12,
15
],
"outputs": [
12,
15
],
"begins_at": 11,
"ends_at": 11,
"type": "collision"
},
{
"id": 13,
"inputs": [
12,
17
],
"outputs": [
12,
17
],
"begins_at": 11,
"ends_at": 11,
"type": "collision"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "purple minivan",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 100,
"y": 982
},
{
"t": 1,
"x": 116,
"y": 976
},
{
"t": 2,
"x": 100,
"y": 970
},
{
"t": 3,
"x": 117,
"y": 964
},
{
"t": 4,
"x": 118,
"y": 958
},
{
"t": 5,
"x": 133,
"y": 952
},
{
"t": 6,
"x": 121,
"y": 947
},
{
"t": 7,
"x": 121,
"y": 942
},
{
"t": 8,
"x": 126,
"y": 936
},
{
"t": 9,
"x": 139,
"y": 931
},
{
"t": 10,
"x": 133,
"y": 927
},
{
"t": 11,
"x": 135,
"y": 922
},
{
"t": 12,
"x": 141,
"y": 918
},
{
"t": 13,
"x": 133,
"y": 914
},
{
"t": 14,
"x": 128,
"y": 909
},
{
"t": 15,
"x": 127,
"y": 905
},
{
"t": 16,
"x": 130,
"y": 901
},
{
"t": 17,
"x": 128,
"y": 898
},
{
"t": 18,
"x": 127,
"y": 892
},
{
"t": 19,
"x": 126,
"y": 884
},
{
"t": 20,
"x": 125,
"y": 879
},
{
"t": 21,
"x": 123,
"y": 872
},
{
"t": 22,
"x": 121,
"y": 871
},
{
"t": 23,
"x": 120,
"y": 860
},
{
"t": 24,
"x": 118,
"y": 860
},
{
"t": 25,
"x": 116,
"y": 858
},
{
"t": 26,
"x": 115,
"y": 857
},
{
"t": 27,
"x": 111,
"y": 880
},
{
"t": 29,
"x": 110,
"y": 867
},
{
"t": 30,
"x": 109,
"y": 870
},
{
"t": 31,
"x": 108,
"y": 867
},
{
"t": 32,
"x": 108,
"y": 853
},
{
"t": 33,
"x": 108,
"y": 847
},
{
"t": 34,
"x": 108,
"y": 852
},
{
"t": 35,
"x": 106,
"y": 839
},
{
"t": 36,
"x": 105,
"y": 836
},
{
"t": 37,
"x": 103,
"y": 826
},
{
"t": 38,
"x": 101,
"y": 820
},
{
"t": 39,
"x": 101,
"y": 822
},
{
"t": 40,
"x": 98,
"y": 813
},
{
"t": 41,
"x": 97,
"y": 806
},
{
"t": 42,
"x": 95,
"y": 801
},
{
"t": 43,
"x": 94,
"y": 800
},
{
"t": 44,
"x": 92,
"y": 798
},
{
"t": 45,
"x": 90,
"y": 796
},
{
"t": 46,
"x": 90,
"y": 792
},
{
"t": 47,
"x": 87,
"y": 789
},
{
"t": 48,
"x": 90,
"y": 785
},
{
"t": 49,
"x": 90,
"y": 782
},
{
"t": 50,
"x": 91,
"y": 779
},
{
"t": 51,
"x": 91,
"y": 776
},
{
"t": 52,
"x": 91,
"y": 772
},
{
"t": 53,
"x": 71,
"y": 773
},
{
"t": 54,
"x": 93,
"y": 766
},
{
"t": 55,
"x": 73,
"y": 767
},
{
"t": 56,
"x": 76,
"y": 763
},
{
"t": 57,
"x": 61,
"y": 764
},
{
"t": 58,
"x": 74,
"y": 757
},
{
"t": 59,
"x": 59,
"y": 758
},
{
"t": 60,
"x": 77,
"y": 751
},
{
"t": 61,
"x": 62,
"y": 752
},
{
"t": 62,
"x": 61,
"y": 750
},
{
"t": 63,
"x": 60,
"y": 747
},
{
"t": 65,
"x": 59,
"y": 743
},
{
"t": 66,
"x": 59,
"y": 740
},
{
"t": 68,
"x": 59,
"y": 734
},
{
"t": 70,
"x": 56,
"y": 731
},
{
"t": 71,
"x": 60,
"y": 727
},
{
"t": 72,
"x": 56,
"y": 726
},
{
"t": 73,
"x": 57,
"y": 724
},
{
"t": 74,
"x": 58,
"y": 721
},
{
"t": 75,
"x": 59,
"y": 719
},
{
"t": 76,
"x": 55,
"y": 718
},
{
"t": 77,
"x": 56,
"y": 715
},
{
"t": 78,
"x": 58,
"y": 713
},
{
"t": 79,
"x": 58,
"y": 710
},
{
"t": 80,
"x": 59,
"y": 707
},
{
"t": 81,
"x": 56,
"y": 706
},
{
"t": 82,
"x": 58,
"y": 704
},
{
"t": 83,
"x": 62,
"y": 699
},
{
"t": 84,
"x": 54,
"y": 701
},
{
"t": 86,
"x": 53,
"y": 697
},
{
"t": 87,
"x": 60,
"y": 693
},
{
"t": 88,
"x": 51,
"y": 694
},
{
"t": 89,
"x": 51,
"y": 692
},
{
"t": 90,
"x": 53,
"y": 691
},
{
"t": 91,
"x": 50,
"y": 689
},
{
"t": 92,
"x": 62,
"y": 685
},
{
"t": 93,
"x": 60,
"y": 682
},
{
"t": 94,
"x": 51,
"y": 683
},
{
"t": 96,
"x": 51,
"y": 680
},
{
"t": 98,
"x": 53,
"y": 678
},
{
"t": 99,
"x": 52,
"y": 676
},
{
"t": 101,
"x": 51,
"y": 672
},
{
"t": 103,
"x": 51,
"y": 669
},
{
"t": 105,
"x": 51,
"y": 667
},
{
"t": 107,
"x": 51,
"y": 664
},
{
"t": 109,
"x": 51,
"y": 661
},
{
"t": 110,
"x": 51,
"y": 659
},
{
"t": 112,
"x": 51,
"y": 657
},
{
"t": 113,
"x": 60,
"y": 655
},
{
"t": 114,
"x": 58,
"y": 652
},
{
"t": 116,
"x": 58,
"y": 650
},
{
"t": 118,
"x": 59,
"y": 648
}
],
"birth": 0,
"death": 120
},
{
"id": 1,
"type": "purple dirtbike",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 233,
"y": 772
},
{
"t": 2,
"x": 250,
"y": 743
},
{
"t": 13,
"x": 243,
"y": 782
},
{
"t": 17,
"x": 233,
"y": 769
},
{
"t": 18,
"x": 237,
"y": 765
},
{
"t": 19,
"x": 241,
"y": 760
},
{
"t": 20,
"x": 244,
"y": 756
},
{
"t": 21,
"x": 251,
"y": 752
},
{
"t": 22,
"x": 255,
"y": 748
},
{
"t": 23,
"x": 258,
"y": 746
},
{
"t": 24,
"x": 262,
"y": 742
},
{
"t": 25,
"x": 266,
"y": 742
},
{
"t": 26,
"x": 270,
"y": 738
},
{
"t": 27,
"x": 273,
"y": 736
},
{
"t": 28,
"x": 276,
"y": 735
},
{
"t": 29,
"x": 279,
"y": 734
},
{
"t": 30,
"x": 283,
"y": 735
},
{
"t": 31,
"x": 280,
"y": 705
},
{
"t": 32,
"x": 281,
"y": 704
},
{
"t": 33,
"x": 283,
"y": 703
},
{
"t": 34,
"x": 284,
"y": 702
},
{
"t": 36,
"x": 287,
"y": 701
},
{
"t": 38,
"x": 290,
"y": 701
},
{
"t": 40,
"x": 291,
"y": 696
},
{
"t": 41,
"x": 293,
"y": 698
},
{
"t": 42,
"x": 292,
"y": 696
},
{
"t": 43,
"x": 297,
"y": 700
},
{
"t": 45,
"x": 298,
"y": 701
},
{
"t": 46,
"x": 301,
"y": 703
},
{
"t": 47,
"x": 317,
"y": 717
},
{
"t": 48,
"x": 319,
"y": 718
},
{
"t": 49,
"x": 320,
"y": 719
},
{
"t": 50,
"x": 321,
"y": 717
},
{
"t": 51,
"x": 327,
"y": 716
},
{
"t": 52,
"x": 330,
"y": 713
},
{
"t": 53,
"x": 329,
"y": 710
},
{
"t": 54,
"x": 333,
"y": 706
},
{
"t": 55,
"x": 334,
"y": 704
},
{
"t": 56,
"x": 336,
"y": 702
},
{
"t": 57,
"x": 338,
"y": 702
},
{
"t": 58,
"x": 339,
"y": 700
},
{
"t": 60,
"x": 342,
"y": 697
},
{
"t": 61,
"x": 343,
"y": 694
},
{
"t": 63,
"x": 319,
"y": 682
},
{
"t": 64,
"x": 335,
"y": 684
},
{
"t": 65,
"x": 322,
"y": 679
},
{
"t": 66,
"x": 324,
"y": 678
},
{
"t": 67,
"x": 326,
"y": 677
},
{
"t": 68,
"x": 328,
"y": 677
},
{
"t": 69,
"x": 353,
"y": 681
},
{
"t": 70,
"x": 356,
"y": 679
},
{
"t": 71,
"x": 358,
"y": 678
},
{
"t": 72,
"x": 359,
"y": 676
},
{
"t": 74,
"x": 361,
"y": 674
},
{
"t": 75,
"x": 362,
"y": 672
},
{
"t": 77,
"x": 364,
"y": 670
},
{
"t": 78,
"x": 366,
"y": 669
},
{
"t": 79,
"x": 368,
"y": 667
},
{
"t": 81,
"x": 370,
"y": 664
},
{
"t": 82,
"x": 372,
"y": 663
},
{
"t": 84,
"x": 375,
"y": 659
},
{
"t": 86,
"x": 378,
"y": 657
},
{
"t": 87,
"x": 380,
"y": 655
},
{
"t": 88,
"x": 382,
"y": 654
},
{
"t": 89,
"x": 383,
"y": 653
},
{
"t": 90,
"x": 385,
"y": 652
},
{
"t": 91,
"x": 387,
"y": 651
},
{
"t": 92,
"x": 388,
"y": 650
},
{
"t": 93,
"x": 390,
"y": 648
},
{
"t": 95,
"x": 392,
"y": 646
},
{
"t": 96,
"x": 394,
"y": 645
},
{
"t": 97,
"x": 396,
"y": 644
},
{
"t": 98,
"x": 394,
"y": 641
},
{
"t": 100,
"x": 398,
"y": 639
},
{
"t": 101,
"x": 394,
"y": 640
},
{
"t": 102,
"x": 399,
"y": 635
},
{
"t": 104,
"x": 401,
"y": 634
},
{
"t": 105,
"x": 402,
"y": 633
},
{
"t": 106,
"x": 405,
"y": 632
},
{
"t": 107,
"x": 406,
"y": 631
},
{
"t": 109,
"x": 412,
"y": 630
},
{
"t": 110,
"x": 405,
"y": 630
},
{
"t": 111,
"x": 407,
"y": 629
},
{
"t": 112,
"x": 409,
"y": 628
},
{
"t": 113,
"x": 411,
"y": 627
},
{
"t": 114,
"x": 412,
"y": 626
},
{
"t": 116,
"x": 421,
"y": 624
},
{
"t": 117,
"x": 414,
"y": 624
},
{
"t": 118,
"x": 416,
"y": 623
},
{
"t": 119,
"x": 418,
"y": 622
}
],
"birth": 0,
"death": 120
},
{
"id": 2,
"type": "red utility",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 703,
"y": 670
},
{
"t": 3,
"x": 738,
"y": 703
},
{
"t": 5,
"x": 758,
"y": 650
},
{
"t": 6,
"x": 770,
"y": 703
},
{
"t": 7,
"x": 758,
"y": 650
},
{
"t": 24,
"x": 770,
"y": 703
},
{
"t": 25,
"x": 758,
"y": 650
},
{
"t": 26,
"x": 738,
"y": 703
},
{
"t": 27,
"x": 758,
"y": 650
},
{
"t": 28,
"x": 703,
"y": 670
},
{
"t": 29,
"x": 747,
"y": 703
},
{
"t": 49,
"x": 770,
"y": 703
}
],
"birth": 0,
"death": 120
},
{
"id": 3,
"type": "green airliner",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 752,
"y": 196
},
{
"t": 1,
"x": 748,
"y": 195
},
{
"t": 2,
"x": 744,
"y": 195
},
{
"t": 3,
"x": 741,
"y": 195
},
{
"t": 4,
"x": 737,
"y": 195
},
{
"t": 5,
"x": 733,
"y": 195
},
{
"t": 6,
"x": 729,
"y": 194
},
{
"t": 8,
"x": 726,
"y": 195
},
{
"t": 10,
"x": 722,
"y": 195
},
{
"t": 11,
"x": 719,
"y": 195
},
{
"t": 12,
"x": 708,
"y": 194
},
{
"t": 13,
"x": 705,
"y": 194
},
{
"t": 14,
"x": 701,
"y": 194
},
{
"t": 15,
"x": 698,
"y": 194
},
{
"t": 16,
"x": 695,
"y": 194
},
{
"t": 17,
"x": 691,
"y": 193
},
{
"t": 18,
"x": 687,
"y": 193
},
{
"t": 19,
"x": 683,
"y": 193
},
{
"t": 20,
"x": 679,
"y": 193
},
{
"t": 21,
"x": 675,
"y": 193
},
{
"t": 22,
"x": 670,
"y": 192
},
{
"t": 23,
"x": 668,
"y": 192
},
{
"t": 24,
"x": 664,
"y": 192
},
{
"t": 25,
"x": 660,
"y": 192
},
{
"t": 26,
"x": 656,
"y": 192
},
{
"t": 27,
"x": 685,
"y": 193
},
{
"t": 28,
"x": 654,
"y": 192
},
{
"t": 29,
"x": 651,
"y": 192
},
{
"t": 30,
"x": 648,
"y": 192
},
{
"t": 31,
"x": 645,
"y": 192
},
{
"t": 32,
"x": 633,
"y": 191
},
{
"t": 33,
"x": 629,
"y": 191
},
{
"t": 34,
"x": 625,
"y": 191
},
{
"t": 35,
"x": 623,
"y": 191
},
{
"t": 36,
"x": 616,
"y": 190
},
{
"t": 37,
"x": 613,
"y": 190
},
{
"t": 38,
"x": 608,
"y": 190
},
{
"t": 39,
"x": 605,
"y": 190
},
{
"t": 40,
"x": 601,
"y": 190
},
{
"t": 41,
"x": 595,
"y": 189
},
{
"t": 42,
"x": 591,
"y": 189
},
{
"t": 43,
"x": 587,
"y": 189
},
{
"t": 44,
"x": 583,
"y": 189
},
{
"t": 45,
"x": 579,
"y": 188
},
{
"t": 46,
"x": 582,
"y": 189
},
{
"t": 47,
"x": 579,
"y": 189
},
{
"t": 48,
"x": 576,
"y": 189
},
{
"t": 49,
"x": 561,
"y": 186
},
{
"t": 50,
"x": 558,
"y": 188
},
{
"t": 51,
"x": 555,
"y": 188
},
{
"t": 52,
"x": 552,
"y": 188
},
{
"t": 54,
"x": 541,
"y": 186
},
{
"t": 55,
"x": 537,
"y": 186
},
{
"t": 56,
"x": 532,
"y": 186
},
{
"t": 57,
"x": 528,
"y": 186
},
{
"t": 58,
"x": 523,
"y": 185
},
{
"t": 59,
"x": 519,
"y": 185
},
{
"t": 60,
"x": 514,
"y": 185
},
{
"t": 61,
"x": 544,
"y": 188
},
{
"t": 62,
"x": 504,
"y": 184
},
{
"t": 63,
"x": 535,
"y": 186
},
{
"t": 64,
"x": 506,
"y": 185
},
{
"t": 65,
"x": 526,
"y": 186
},
{
"t": 66,
"x": 522,
"y": 186
},
{
"t": 67,
"x": 483,
"y": 184
},
{
"t": 69,
"x": 480,
"y": 184
},
{
"t": 70,
"x": 476,
"y": 184
},
{
"t": 71,
"x": 498,
"y": 185
},
{
"t": 72,
"x": 494,
"y": 185
},
{
"t": 73,
"x": 489,
"y": 185
},
{
"t": 74,
"x": 484,
"y": 185
},
{
"t": 75,
"x": 479,
"y": 184
},
{
"t": 76,
"x": 475,
"y": 184
},
{
"t": 77,
"x": 470,
"y": 184
},
{
"t": 78,
"x": 466,
"y": 184
},
{
"t": 79,
"x": 459,
"y": 183
},
{
"t": 80,
"x": 455,
"y": 183
},
{
"t": 81,
"x": 450,
"y": 183
},
{
"t": 82,
"x": 413,
"y": 181
},
{
"t": 83,
"x": 441,
"y": 183
},
{
"t": 84,
"x": 408,
"y": 181
},
{
"t": 85,
"x": 405,
"y": 181
},
{
"t": 86,
"x": 390,
"y": 180
},
{
"t": 87,
"x": 421,
"y": 182
},
{
"t": 88,
"x": 416,
"y": 182
},
{
"t": 89,
"x": 411,
"y": 182
},
{
"t": 90,
"x": 405,
"y": 181
},
{
"t": 91,
"x": 401,
"y": 181
},
{
"t": 92,
"x": 396,
"y": 181
},
{
"t": 93,
"x": 391,
"y": 181
},
{
"t": 94,
"x": 384,
"y": 180
},
{
"t": 95,
"x": 379,
"y": 180
},
{
"t": 96,
"x": 374,
"y": 180
},
{
"t": 97,
"x": 340,
"y": 178
},
{
"t": 98,
"x": 337,
"y": 178
},
{
"t": 99,
"x": 333,
"y": 178
},
{
"t": 100,
"x": 317,
"y": 177
},
{
"t": 101,
"x": 348,
"y": 179
},
{
"t": 102,
"x": 343,
"y": 179
},
{
"t": 103,
"x": 299,
"y": 176
},
{
"t": 104,
"x": 294,
"y": 176
},
{
"t": 105,
"x": 326,
"y": 178
},
{
"t": 106,
"x": 321,
"y": 178
},
{
"t": 107,
"x": 421,
"y": 153
},
{
"t": 108,
"x": 309,
"y": 177
},
{
"t": 109,
"x": 304,
"y": 177
},
{
"t": 110,
"x": 298,
"y": 177
},
{
"t": 111,
"x": 266,
"y": 175
},
{
"t": 112,
"x": 261,
"y": 175
},
{
"t": 113,
"x": 244,
"y": 174
},
{
"t": 114,
"x": 240,
"y": 174
},
{
"t": 115,
"x": 380,
"y": 150
},
{
"t": 116,
"x": 374,
"y": 150
},
{
"t": 117,
"x": 219,
"y": 173
},
{
"t": 118,
"x": 363,
"y": 149
},
{
"t": 119,
"x": 358,
"y": 149
}
],
"birth": 0,
"death": 120
}
],
"interactions": [
{
"id": 1,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 18,
"type": "contact"
},
{
"id": 4,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 51,
"type": "contact"
},
{
"id": 8,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 17,
"ends_at": 17,
"type": "collision"
},
{
"id": 2,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 22,
"ends_at": 22,
"type": "collision"
},
{
"id": 3,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 23,
"ends_at": 119,
"type": "contact"
},
{
"id": 5,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 58,
"ends_at": 58,
"type": "collision"
},
{
"id": 6,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 60,
"ends_at": 60,
"type": "collision"
},
{
"id": 7,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 61,
"ends_at": 119,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.42709225863057443,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 310.18830610194516,
"avg_image_position_segmentation_acc": 0.16458333333333333,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.004166666666666667,
"avg_image_position_segmentation_acc_large": 0.325,
"avg_image_position_segmentation_acc_first_frame": 0.5,
"avg_image_position_segmentation_visible_acc": 0.16458333333333333,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null,
"num_matches": 4,
"num_unmatched_pred": 45,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 4,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 4,
"avg_image_position_segmentation_acc": 4,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 2,
"avg_image_position_segmentation_acc_large": 2,
"avg_image_position_segmentation_acc_first_frame": 4,
"avg_image_position_segmentation_visible_acc": 4,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 4,
"visibility_true_acc": 4,
"visibility_false_acc": 0,
"is_in_frame_acc": 4,
"is_in_frame_true_acc": 4,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 4,
"vis_acc_from_px_true_acc": 4,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 2,
"target_id": 0,
"pred_id": 2,
"cost": 0.975,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.475,
"weighted_cost": 0.475,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 22,
"target_id": 1,
"pred_id": 22,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 21,
"target_id": 2,
"pred_id": 21,
"cost": 0.9916666666666667,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.9916666666666667,
"weighted_cost": 0.9916666666666667,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 37,
"target_id": 3,
"pred_id": 37,
"cost": 0.875,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.875,
"weighted_cost": 0.875,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48
],
"target_type_counts": {
"purple minivan": 1,
"purple dirtbike": 1,
"red utility": 1,
"green airliner": 1
},
"pred_type_counts": {
"pink and purple plastic package of overnight pads": 3,
"black and pink running shoe": 1,
"red and white striped dishtowel": 1,
"red and green wooden bird rattle": 2,
"red and green cardboard ink cartridge box": 5,
"red and black plastic handheld game console": 7,
"olive green and brown leather flip-flop": 2,
"red scissors": 1,
"dark grey and purple plastic Shredder action figure": 2,
"dark green plastic triceratops figurine": 6,
"dark blue fabric chukka boot": 1,
"purple and beige wooden dollhouse lamp": 1,
"black and white fabric stuffed orca": 2,
"dark teal wooden toy train": 2,
"green and yellow cardboard whey protein box": 1,
"purple and black cardboard Nintendo 3DS game box": 1,
"dark blue fabric high-top boat shoe": 1,
"green and yellow cardboard ink cartridge box": 1,
"green and white plastic portable speaker": 1,
"dark grey cardboard sound samples box": 2,
"dark red and black plastic curling wand": 1,
"green ink cartridge cardboard box": 1,
"dark brown leather boat shoe": 1,
"dark grey fabric boat shoe": 1,
"dark brown leather loafer": 1,
"black plastic gaming mouse": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 120
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 1,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 1.0,
"1.0": 0.75,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 45,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 45,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 45,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 45,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 0.08163265306122448,
"0.5": 0.08163265306122448,
"0.75": 0.08163265306122448,
"1.0": 0.08163265306122448
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 0.15094339622641506,
"0.5": 0.15094339622641506,
"0.75": 0.15094339622641506,
"1.0": 0.15094339622641506
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 0,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 0,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 0,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
}
},
"dynamic": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 0,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 0,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 0,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 45,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 45,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 45,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 45,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 0.021739130434782608,
"0.5": 0.021739130434782608,
"0.75": 0.021739130434782608,
"1.0": 0.021739130434782608
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 0.042553191489361694,
"0.5": 0.042553191489361694,
"0.75": 0.042553191489361694,
"1.0": 0.042553191489361694
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 48,
"false_negatives": 3
},
"0.5": {
"true_positives": 1,
"false_positives": 48,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 49,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 49,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.02040816326530612,
"0.5": 0.02040816326530612,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.25,
"0.5": 0.25,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.037735849056603765,
"0.5": 0.037735849056603765,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.01020408163265306,
"segmentation_frame_average_recall": 0.125,
"segmentation_frame_average_f1": 0.018867924528301883,
"segmentation_first_frame_counts": {
"true_positives": 2,
"false_positives": 47,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.04081632653061224,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.07843137254901959,
"segmentation_by_motion_group": {
"static": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 1,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 1,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 1,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 1,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
},
"dynamic": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 2
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.25,
"segmentation_frame_average_recall": 0.25,
"segmentation_frame_average_f1": 0.25,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.5,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.6666666666666666
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 45,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.021739130434782608,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.042553191489361694
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 2,
"cost": 0.975,
"type_similarity": 0.3087387009974858,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 107.92579817759147,
"image_position_segmentation_acc": 0.525,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.525,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.525,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 22,
"cost": 1.0,
"type_similarity": 0.4417059053449024,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 844.1072546397219,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 21,
"cost": 0.9916666666666667,
"type_similarity": 0.5529469554538966,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 29.849175766364382,
"image_position_segmentation_acc": 0.008333333333333333,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.008333333333333333,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.008333333333333333,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 37,
"cost": 0.875,
"type_similarity": 0.40497747272601303,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 258.87099582410286,
"image_position_segmentation_acc": 0.125,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.125,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.125,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": -0.25760955094288424,
"avg_interaction_input_object_jaccard": 0.3703703703703704,
"avg_interaction_output_object_jaccard": 0.3703703703703704,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 9,
"num_unmatched_pred_interactions": 36,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 43,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 44,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 44,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 45,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.044444444444444446,
"0.5": 0.022222222222222223,
"0.75": 0.022222222222222223,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0851063829787234,
"0.5": 0.04347826086956522,
"0.75": 0.04347826086956522,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 41,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.08888888888888889,
"0.5": 0.06666666666666667,
"0.75": 0.06666666666666667,
"1.0": 0.06666666666666667
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.163265306122449,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 41,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.08888888888888889,
"0.5": 0.06666666666666667,
"0.75": 0.06666666666666667,
"1.0": 0.06666666666666667
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.163265306122449,
"0.5": 0.125,
"0.75": 0.125,
"1.0": 0.125
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 9,
"avg_interaction_input_object_jaccard": 9,
"avg_interaction_output_object_jaccard": 9,
"avg_interaction_type_exact_match_acc": 9
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 8,
"target_id": 0,
"pred_id": 8,
"cost": 1.3571428571428572,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8571428571428572,
"weighted_cost": 0.8571428571428572,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 31,
"target_id": 1,
"pred_id": 31,
"cost": 0.10416666666666674,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.10416666666666674,
"weighted_cost": 0.10416666666666674,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 13,
"target_id": 2,
"pred_id": 13,
"cost": 1.4166666666666667,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9166666666666667,
"weighted_cost": 0.9166666666666667,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 10,
"target_id": 3,
"pred_id": 10,
"cost": 0.9957264957264957,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.49572649572649574,
"weighted_cost": 0.49572649572649574,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 30,
"target_id": 4,
"pred_id": 30,
"cost": 0.26923076923076916,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.26923076923076916,
"weighted_cost": 0.26923076923076916,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 5,
"pred_index": 7,
"target_id": 5,
"pred_id": 7,
"cost": 1.48,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.98,
"weighted_cost": 0.98,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 6,
"pred_index": 5,
"target_id": 6,
"pred_id": 5,
"cost": 1.4821428571428572,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9821428571428572,
"weighted_cost": 0.9821428571428572,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 7,
"pred_index": 1,
"target_id": 7,
"pred_id": 1,
"cost": 1.1541666666666668,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.6541666666666667,
"weighted_cost": 0.6541666666666667,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 8,
"pred_index": 29,
"target_id": 8,
"pred_id": 29,
"cost": 0.4,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.4,
"weighted_cost": 0.4,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
2,
3,
4,
6,
9,
11,
12,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44
],
"target_type_counts": {
"collision": 4,
"contact": 5
},
"pred_type_counts": {
"contact": 40,
"collision": 5
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 8,
"target_id": 0,
"pred_id": 8,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.3571428571428572,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.7142857142857143,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 1,
"pred_index": 31,
"target_id": 1,
"pred_id": 31,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.10416666666666674,
"temporal_span_iou": 0.7916666666666666,
"temporal_span_giou": 0.7916666666666666,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 2,
"pred_index": 13,
"target_id": 2,
"pred_id": 13,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4166666666666667,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.8333333333333334,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 3,
"pred_index": 10,
"target_id": 3,
"pred_id": 10,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.9957264957264957,
"temporal_span_iou": 0.008547008547008548,
"temporal_span_giou": 0.008547008547008548,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 4,
"pred_index": 30,
"target_id": 4,
"pred_id": 30,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.26923076923076916,
"temporal_span_iou": 0.46153846153846156,
"temporal_span_giou": 0.46153846153846156,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 5,
"pred_index": 7,
"target_id": 5,
"pred_id": 7,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.48,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.96,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 6,
"pred_index": 5,
"target_id": 6,
"pred_id": 5,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4821428571428572,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9642857142857143,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 7,
"pred_index": 1,
"target_id": 7,
"pred_id": 1,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.1541666666666668,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.30833333333333335,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 8,
"pred_index": 29,
"target_id": 8,
"pred_id": 29,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.4,
"temporal_span_iou": 0.2,
"temporal_span_giou": 0.2,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": -0.20052503052503054,
"avg_interaction_input_object_jaccard": 0.4166666666666667,
"avg_interaction_output_object_jaccard": 0.4166666666666667,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 8,
"num_unmatched_pred_interactions": 28,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 34,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 35,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 35,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 36,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.05555555555555555,
"0.5": 0.027777777777777776,
"0.75": 0.027777777777777776,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.10526315789473684,
"0.5": 0.05405405405405406,
"0.75": 0.05405405405405406,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 32,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 33,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 33,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 33,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.1111111111111111,
"0.5": 0.08333333333333333,
"0.75": 0.08333333333333333,
"1.0": 0.08333333333333333
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.19999999999999998,
"0.5": 0.15384615384615385,
"0.75": 0.15384615384615385,
"1.0": 0.15384615384615385
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 32,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 33,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 33,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 33,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.1111111111111111,
"0.5": 0.08333333333333333,
"0.75": 0.08333333333333333,
"1.0": 0.08333333333333333
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.19999999999999998,
"0.5": 0.15384615384615385,
"0.75": 0.15384615384615385,
"1.0": 0.15384615384615385
}
},
"object_object": {
"avg_interaction_temporal_span_giou": -0.7142857142857143,
"avg_interaction_input_object_jaccard": 0.0,
"avg_interaction_output_object_jaccard": 0.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 1,
"num_unmatched_pred_interactions": 8,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 8,
"avg_interaction_input_object_jaccard": 8,
"avg_interaction_output_object_jaccard": 8,
"avg_interaction_type_exact_match_acc": 8
},
"object_object": {
"avg_interaction_temporal_span_giou": 1,
"avg_interaction_input_object_jaccard": 1,
"avg_interaction_output_object_jaccard": 1,
"avg_interaction_type_exact_match_acc": 1
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": -0.8679761904761905,
"avg_interaction_input_object_jaccard": 0.0,
"avg_interaction_output_object_jaccard": 0.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 1,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": 0.23068376068376067,
"avg_interaction_input_object_jaccard": 0.6666666666666667,
"avg_interaction_output_object_jaccard": 0.6666666666666667,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 5,
"num_unmatched_pred_interactions": 35,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 38,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 39,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 39,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 40,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.05,
"0.5": 0.025,
"0.75": 0.025,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.09523809523809523,
"0.5": 0.04878048780487806,
"0.75": 0.04878048780487806,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 36,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 37,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 37,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 37,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.1,
"0.5": 0.075,
"0.75": 0.075,
"1.0": 0.075
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.18181818181818182,
"0.5": 0.13953488372093023,
"0.75": 0.13953488372093023,
"1.0": 0.13953488372093023
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 36,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 37,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 37,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 37,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.1,
"0.5": 0.075,
"0.75": 0.075,
"1.0": 0.075
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.18181818181818182,
"0.5": 0.13953488372093023,
"0.75": 0.13953488372093023,
"1.0": 0.13953488372093023
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"contact": {
"avg_interaction_temporal_span_giou": 5,
"avg_interaction_input_object_jaccard": 5,
"avg_interaction_output_object_jaccard": 5,
"avg_interaction_type_exact_match_acc": 5
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}