STRIVE · 08/28/2026
Dyn generalization · test_24 · super_clevr_1184
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"blue chopper\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 25 679 1 33 684 2 42 688 3 55 685 4 70 691 5 82 690 6 94 675 7 100 684 8 116 688 9 128 674 10 137 686 11 149 673 12 161 671 13 173 669 14 183 668 15 194 667 16 186 681 17 213 670 18 225 667 19 235 679 20 248 661 21 239 678 22 269 660 23 279 660 [OBJ] 1 \"brown scooter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 301 873 1 304 869 2 305 867 3 310 859 4 313 855 5 314 853 6 319 846 7 323 842 8 323 840 9 330 831 10 333 827 11 334 825 12 339 819 13 341 816 14 344 811 15 347 807 16 350 802 17 353 798 18 355 795 19 360 789 20 362 785 21 365 781 22 369 776 23 372 772 [OBJ] 2 \"green dirtbike\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 319 721 3 317 723 4 319 722 8 325 716 10 297 735 11 331 709 12 298 734 13 301 732 14 316 726 15 299 732 16 302 731 17 326 716 18 326 718 20 329 716 21 330 715 22 332 714 23 333 713 [OBJ] 3 \"yellow airliner\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 472 182 4 476 181 8 481 180 12 480 181 13 485 180 14 479 181 15 488 180 16 484 180 17 490 179 23 494 178 [OBJ] 4 \"purple utility\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 995 614 2 989 613 3 980 614 4 972 615 5 961 614 6 953 615 7 980 627 8 933 616 9 976 589 10 964 590 11 940 630 12 943 591 13 920 629 14 872 619 15 912 592 16 850 619 17 893 592 18 829 620 19 820 623 20 861 593 21 848 594 22 838 594 23 812 634 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_24/super_clevr_1184/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"light blue fabric backpack with owl and bird pattern\" 0 24 [VIS] 0 true 2 false [IN-FRAME] 0 true 2 false [PX] 0 1 658 2 null null [OBJ] 1 \"light blue plastic conditioner tube\" 0 24 [VIS] 0 true 16 false [IN-FRAME] 0 true 16 false [PX] 0 1 705 1 1 716 2 5 732 3 9 748 4 13 763 5 17 775 6 21 787 7 25 798 8 29 806 9 33 818 10 37 826 11 41 833 12 44 841 13 48 849 14 52 857 15 60 869 16 null null [OBJ] 2 \"light blue plastic conditioner tube\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 1 720 1 9 732 2 17 740 3 25 748 4 33 755 5 41 763 6 48 771 7 56 779 8 64 787 9 72 794 10 80 802 11 87 806 12 95 814 13 103 818 14 111 826 15 119 830 16 126 837 17 134 841 18 142 845 19 150 849 20 158 853 21 166 857 22 173 861 23 181 865 [OBJ] 3 \"light blue plastic conditioner tube\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 9 666 1 17 673 2 21 681 3 29 689 4 33 697 5 41 705 6 44 712 7 52 716 8 60 724 9 68 728 10 76 736 11 83 740 12 91 744 13 99 748 14 107 751 15 115 755 16 123 759 17 130 763 18 138 767 19 146 771 20 154 775 21 162 779 22 169 783 23 177 787 [OBJ] 4 \"dark grey and light grey plastic monster action figure\" 0 24 [VIS] 0 false 19 true [IN-FRAME] 0 true [PX] 0 null null 19 111 720 20 119 724 21 126 732 22 134 736 23 142 744 [OBJ] 5 \"light blue plastic conditioner tube\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 17 650 1 37 654 2 60 658 3 80 662 4 103 666 5 115 669 6 134 673 7 150 677 8 166 681 9 177 681 10 193 685 11 205 689 12 216 689 13 228 693 14 240 693 15 251 697 16 263 697 17 275 701 18 287 701 19 298 705 20 310 705 21 322 705 22 333 708 23 345 708 [OBJ] 6 \"dark green plastic coffee container\" 0 24 [VIS] 0 false 13 true [IN-FRAME] 0 true [PX] 0 null null 13 162 705 14 169 708 15 177 712 16 189 712 17 201 716 18 212 716 19 224 720 20 236 720 21 248 724 22 259 724 23 271 724 [OBJ] 7 \"yellow plastic toy school bus\" 0 24 [VIS] 0 false 13 true [IN-FRAME] 0 true [PX] 0 null null 13 177 720 14 189 720 15 201 724 16 212 724 17 224 724 18 236 728 19 248 728 20 259 728 21 271 728 22 283 728 23 294 728 [OBJ] 8 \"yellow plastic toy school bus\" 0 24 [VIS] 0 false 12 true [IN-FRAME] 0 true [PX] 0 null null 12 212 705 14 228 705 15 240 705 16 251 705 17 263 705 18 275 705 19 287 705 20 298 705 21 310 705 22 322 705 23 333 705 [OBJ] 9 \"yellow plastic toy school bus\" 0 24 [VIS] 0 false 12 true [IN-FRAME] 0 true [PX] 0 null null 12 251 705 14 267 705 15 279 705 16 291 705 17 302 705 18 314 705 19 326 705 20 337 705 21 349 705 22 361 705 23 373 705 [OBJ] 10 \"yellow plastic toy school bus\" 0 24 [VIS] 0 false 12 true [IN-FRAME] 0 true [PX] 0 null null 12 271 701 13 283 701 14 294 701 15 306 701 16 318 701 17 330 701 18 341 701 19 353 701 20 365 701 21 376 701 22 388 701 23 396 697 [OBJ] 11 \"yellow plastic toy school bus\" 0 24 [VIS] 0 false 12 true [IN-FRAME] 0 true [PX] 0 null null 12 314 697 13 322 693 14 333 693 15 345 693 16 353 689 17 365 689 18 376 689 19 384 685 20 396 685 21 408 685 22 416 681 23 427 681 [OBJ] 12 \"yellow plastic toy school bus\" 0 24 [VIS] 0 false 12 true [IN-FRAME] 0 true [PX] 0 null null 12 345 693 14 357 689 16 369 685 18 380 681 19 392 681 21 404 677 22 412 673 23 423 673 [OBJ] 13 \"yellow wooden toy roller\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 314 896 1 318 888 2 326 880 3 330 873 4 333 865 5 337 857 6 345 849 7 349 841 8 353 833 9 361 826 10 365 818 11 369 810 12 373 802 13 376 794 14 380 787 15 384 779 16 388 771 17 388 759 18 392 751 19 392 740 20 396 732 21 396 720 22 396 708 23 396 697 [OBJ] 14 \"green wooden toy train\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 318 736 2 322 728 4 310 732 5 318 724 7 310 728 10 306 720 11 314 712 12 306 716 13 314 708 14 322 705 16 330 697 17 314 701 18 322 697 19 330 693 20 341 689 21 353 685 22 345 681 23 357 681 [OBJ] 15 \"dark grey and pink running shoe\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 341 681 2 349 685 3 345 677 4 353 681 6 349 669 7 357 673 9 353 662 10 361 666 12 369 662 14 365 650 15 373 654 17 376 646 19 380 638 21 388 634 23 392 626 [OBJ] 16 \"yellow and green cardboard crayon box\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 373 701 4 376 693 7 380 685 10 384 677 13 392 673 14 388 666 17 392 658 19 396 650 21 404 646 23 408 638 [OBJ] 17 \"yellow cardboard DVD set\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 408 697 2 412 689 5 416 681 7 419 673 10 423 666 13 427 658 16 431 650 19 435 642 22 439 634 [OBJ] 18 \"yellow cardboard DVD set\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 416 712 2 419 705 4 423 697 6 427 689 8 431 681 11 435 673 13 439 666 16 443 658 18 447 650 21 451 642 23 455 634 [OBJ] 19 \"dark grey metallic C-clamp\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 431 681 3 435 673 6 439 666 9 443 658 12 447 650 15 451 642 18 455 634 20 458 626 23 462 619 [OBJ] 20 \"yellow plastic chocolate powder canister\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 458 185 1 466 189 4 474 185 7 482 189 10 490 185 13 498 189 15 490 185 16 501 193 17 494 189 18 505 193 22 513 197 [OBJ] 21 \"purple and beige wooden dollhouse lamp\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 990 642 1 986 634 2 982 626 3 978 619 4 974 611 6 966 603 8 955 599 9 947 595 11 935 591 12 927 587 14 919 583 15 912 580 17 900 576 19 892 572 21 880 568 23 869 564 [OBJS-END] [INTERACTIONS-START] [INTERACT] 13 \"contact\" 0 4 [IN] 12 14 [OUT] 12 14 [INTERACT] 2 \"contact\" 0 11 [IN] 1 6 [OUT] 1 6 [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 4 [OUT] 1 4 [INTERACT] 3 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 22 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 23 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 24 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 25 \"contact\" 0 23 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 26 \"contact\" 0 23 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 27 \"contact\" 0 23 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 28 \"contact\" 0 23 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 29 \"contact\" 0 23 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 10 11 [OUT] 10 11 [INTERACT] 5 \"contact\" 0 23 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 6 \"contact\" 0 23 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 7 \"contact\" 0 23 [IN] 12 16 [OUT] 12 16 [INTERACT] 8 \"contact\" 0 23 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 9 \"contact\" 0 23 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 10 \"contact\" 0 23 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 11 \"contact\" 0 23 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 12 \"contact\" 0 23 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 16 \"contact\" 0 23 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 17 \"contact\" 0 23 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 18 \"contact\" 0 23 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 19 \"contact\" 0 23 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 20 \"contact\" 0 23 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 14 \"collision\" 6 6 [IN] 12 14 [OUT] 12 14 [INTERACT] 15 \"contact\" 7 23 [IN] 12 14 [OUT] 12 14 [INTERACT] 19 \"collision\" 12 12 [IN] 2 7 [OUT] 2 7 [INTERACT] 21 \"collision\" 12 12 [IN] 2 8 [OUT] 2 8 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"blue chopper\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 25 679 1 33 684 2 42 688 3 55 685 4 70 691 5 82 690 6 94 675 7 100 684 8 116 688 9 128 674 10 137 686 11 149 673 12 161 671 13 173 669 14 183 668 15 194 667 16 186 681 17 213 670 18 225 667 19 235 679 20 248 661 21 239 678 22 269 660 23 279 660 [OBJ] 1 \"brown scooter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 301 873 1 304 869 2 305 867 3 310 859 4 313 855 5 314 853 6 319 846 7 323 842 8 323 840 9 330 831 10 333 827 11 334 825 12 339 819 13 341 816 14 344 811 15 347 807 16 350 802 17 353 798 18 355 795 19 360 789 20 362 785 21 365 781 22 369 776 23 372 772 [OBJ] 2 \"green dirtbike\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 319 721 3 317 723 4 319 722 8 325 716 10 297 735 11 331 709 12 298 734 13 301 732 14 316 726 15 299 732 16 302 731 17 326 716 18 326 718 20 329 716 21 330 715 22 332 714 23 333 713 [OBJ] 3 \"yellow airliner\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 472 182 4 476 181 8 481 180 12 480 181 13 485 180 14 479 181 15 488 180 16 484 180 17 490 179 23 494 178 [OBJ] 4 \"purple utility\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 995 614 2 989 613 3 980 614 4 972 615 5 961 614 6 953 615 7 980 627 8 933 616 9 976 589 10 964 590 11 940 630 12 943 591 13 920 629 14 872 619 15 912 592 16 850 619 17 893 592 18 829 620 19 820 623 20 861 593 21 848 594 22 838 594 23 812 634 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "light blue fabric backpack with owl and bird pattern",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 658
},
{
"t": 2,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "light blue plastic conditioner tube",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 16,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 16,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 705
},
{
"t": 1,
"x": 1,
"y": 716
},
{
"t": 2,
"x": 5,
"y": 732
},
{
"t": 3,
"x": 9,
"y": 748
},
{
"t": 4,
"x": 13,
"y": 763
},
{
"t": 5,
"x": 17,
"y": 775
},
{
"t": 6,
"x": 21,
"y": 787
},
{
"t": 7,
"x": 25,
"y": 798
},
{
"t": 8,
"x": 29,
"y": 806
},
{
"t": 9,
"x": 33,
"y": 818
},
{
"t": 10,
"x": 37,
"y": 826
},
{
"t": 11,
"x": 41,
"y": 833
},
{
"t": 12,
"x": 44,
"y": 841
},
{
"t": 13,
"x": 48,
"y": 849
},
{
"t": 14,
"x": 52,
"y": 857
},
{
"t": 15,
"x": 60,
"y": 869
},
{
"t": 16,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "light blue plastic conditioner tube",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 720
},
{
"t": 1,
"x": 9,
"y": 732
},
{
"t": 2,
"x": 17,
"y": 740
},
{
"t": 3,
"x": 25,
"y": 748
},
{
"t": 4,
"x": 33,
"y": 755
},
{
"t": 5,
"x": 41,
"y": 763
},
{
"t": 6,
"x": 48,
"y": 771
},
{
"t": 7,
"x": 56,
"y": 779
},
{
"t": 8,
"x": 64,
"y": 787
},
{
"t": 9,
"x": 72,
"y": 794
},
{
"t": 10,
"x": 80,
"y": 802
},
{
"t": 11,
"x": 87,
"y": 806
},
{
"t": 12,
"x": 95,
"y": 814
},
{
"t": 13,
"x": 103,
"y": 818
},
{
"t": 14,
"x": 111,
"y": 826
},
{
"t": 15,
"x": 119,
"y": 830
},
{
"t": 16,
"x": 126,
"y": 837
},
{
"t": 17,
"x": 134,
"y": 841
},
{
"t": 18,
"x": 142,
"y": 845
},
{
"t": 19,
"x": 150,
"y": 849
},
{
"t": 20,
"x": 158,
"y": 853
},
{
"t": 21,
"x": 166,
"y": 857
},
{
"t": 22,
"x": 173,
"y": 861
},
{
"t": 23,
"x": 181,
"y": 865
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "light blue plastic conditioner tube",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 9,
"y": 666
},
{
"t": 1,
"x": 17,
"y": 673
},
{
"t": 2,
"x": 21,
"y": 681
},
{
"t": 3,
"x": 29,
"y": 689
},
{
"t": 4,
"x": 33,
"y": 697
},
{
"t": 5,
"x": 41,
"y": 705
},
{
"t": 6,
"x": 44,
"y": 712
},
{
"t": 7,
"x": 52,
"y": 716
},
{
"t": 8,
"x": 60,
"y": 724
},
{
"t": 9,
"x": 68,
"y": 728
},
{
"t": 10,
"x": 76,
"y": 736
},
{
"t": 11,
"x": 83,
"y": 740
},
{
"t": 12,
"x": 91,
"y": 744
},
{
"t": 13,
"x": 99,
"y": 748
},
{
"t": 14,
"x": 107,
"y": 751
},
{
"t": 15,
"x": 115,
"y": 755
},
{
"t": 16,
"x": 123,
"y": 759
},
{
"t": 17,
"x": 130,
"y": 763
},
{
"t": 18,
"x": 138,
"y": 767
},
{
"t": 19,
"x": 146,
"y": 771
},
{
"t": 20,
"x": 154,
"y": 775
},
{
"t": 21,
"x": 162,
"y": 779
},
{
"t": 22,
"x": 169,
"y": 783
},
{
"t": 23,
"x": 177,
"y": 787
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "dark grey and light grey plastic monster action figure",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 19,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 19,
"x": 111,
"y": 720
},
{
"t": 20,
"x": 119,
"y": 724
},
{
"t": 21,
"x": 126,
"y": 732
},
{
"t": 22,
"x": 134,
"y": 736
},
{
"t": 23,
"x": 142,
"y": 744
}
],
"birth": 0,
"death": 24
},
{
"id": 5,
"type": "light blue plastic conditioner tube",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 17,
"y": 650
},
{
"t": 1,
"x": 37,
"y": 654
},
{
"t": 2,
"x": 60,
"y": 658
},
{
"t": 3,
"x": 80,
"y": 662
},
{
"t": 4,
"x": 103,
"y": 666
},
{
"t": 5,
"x": 115,
"y": 669
},
{
"t": 6,
"x": 134,
"y": 673
},
{
"t": 7,
"x": 150,
"y": 677
},
{
"t": 8,
"x": 166,
"y": 681
},
{
"t": 9,
"x": 177,
"y": 681
},
{
"t": 10,
"x": 193,
"y": 685
},
{
"t": 11,
"x": 205,
"y": 689
},
{
"t": 12,
"x": 216,
"y": 689
},
{
"t": 13,
"x": 228,
"y": 693
},
{
"t": 14,
"x": 240,
"y": 693
},
{
"t": 15,
"x": 251,
"y": 697
},
{
"t": 16,
"x": 263,
"y": 697
},
{
"t": 17,
"x": 275,
"y": 701
},
{
"t": 18,
"x": 287,
"y": 701
},
{
"t": 19,
"x": 298,
"y": 705
},
{
"t": 20,
"x": 310,
"y": 705
},
{
"t": 21,
"x": 322,
"y": 705
},
{
"t": 22,
"x": 333,
"y": 708
},
{
"t": 23,
"x": 345,
"y": 708
}
],
"birth": 0,
"death": 24
},
{
"id": 6,
"type": "dark green plastic coffee container",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 13,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 13,
"x": 162,
"y": 705
},
{
"t": 14,
"x": 169,
"y": 708
},
{
"t": 15,
"x": 177,
"y": 712
},
{
"t": 16,
"x": 189,
"y": 712
},
{
"t": 17,
"x": 201,
"y": 716
},
{
"t": 18,
"x": 212,
"y": 716
},
{
"t": 19,
"x": 224,
"y": 720
},
{
"t": 20,
"x": 236,
"y": 720
},
{
"t": 21,
"x": 248,
"y": 724
},
{
"t": 22,
"x": 259,
"y": 724
},
{
"t": 23,
"x": 271,
"y": 724
}
],
"birth": 0,
"death": 24
},
{
"id": 7,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 13,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 13,
"x": 177,
"y": 720
},
{
"t": 14,
"x": 189,
"y": 720
},
{
"t": 15,
"x": 201,
"y": 724
},
{
"t": 16,
"x": 212,
"y": 724
},
{
"t": 17,
"x": 224,
"y": 724
},
{
"t": 18,
"x": 236,
"y": 728
},
{
"t": 19,
"x": 248,
"y": 728
},
{
"t": 20,
"x": 259,
"y": 728
},
{
"t": 21,
"x": 271,
"y": 728
},
{
"t": 22,
"x": 283,
"y": 728
},
{
"t": 23,
"x": 294,
"y": 728
}
],
"birth": 0,
"death": 24
},
{
"id": 8,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 12,
"x": 212,
"y": 705
},
{
"t": 14,
"x": 228,
"y": 705
},
{
"t": 15,
"x": 240,
"y": 705
},
{
"t": 16,
"x": 251,
"y": 705
},
{
"t": 17,
"x": 263,
"y": 705
},
{
"t": 18,
"x": 275,
"y": 705
},
{
"t": 19,
"x": 287,
"y": 705
},
{
"t": 20,
"x": 298,
"y": 705
},
{
"t": 21,
"x": 310,
"y": 705
},
{
"t": 22,
"x": 322,
"y": 705
},
{
"t": 23,
"x": 333,
"y": 705
}
],
"birth": 0,
"death": 24
},
{
"id": 9,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 12,
"x": 251,
"y": 705
},
{
"t": 14,
"x": 267,
"y": 705
},
{
"t": 15,
"x": 279,
"y": 705
},
{
"t": 16,
"x": 291,
"y": 705
},
{
"t": 17,
"x": 302,
"y": 705
},
{
"t": 18,
"x": 314,
"y": 705
},
{
"t": 19,
"x": 326,
"y": 705
},
{
"t": 20,
"x": 337,
"y": 705
},
{
"t": 21,
"x": 349,
"y": 705
},
{
"t": 22,
"x": 361,
"y": 705
},
{
"t": 23,
"x": 373,
"y": 705
}
],
"birth": 0,
"death": 24
},
{
"id": 10,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 12,
"x": 271,
"y": 701
},
{
"t": 13,
"x": 283,
"y": 701
},
{
"t": 14,
"x": 294,
"y": 701
},
{
"t": 15,
"x": 306,
"y": 701
},
{
"t": 16,
"x": 318,
"y": 701
},
{
"t": 17,
"x": 330,
"y": 701
},
{
"t": 18,
"x": 341,
"y": 701
},
{
"t": 19,
"x": 353,
"y": 701
},
{
"t": 20,
"x": 365,
"y": 701
},
{
"t": 21,
"x": 376,
"y": 701
},
{
"t": 22,
"x": 388,
"y": 701
},
{
"t": 23,
"x": 396,
"y": 697
}
],
"birth": 0,
"death": 24
},
{
"id": 11,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 12,
"x": 314,
"y": 697
},
{
"t": 13,
"x": 322,
"y": 693
},
{
"t": 14,
"x": 333,
"y": 693
},
{
"t": 15,
"x": 345,
"y": 693
},
{
"t": 16,
"x": 353,
"y": 689
},
{
"t": 17,
"x": 365,
"y": 689
},
{
"t": 18,
"x": 376,
"y": 689
},
{
"t": 19,
"x": 384,
"y": 685
},
{
"t": 20,
"x": 396,
"y": 685
},
{
"t": 21,
"x": 408,
"y": 685
},
{
"t": 22,
"x": 416,
"y": 681
},
{
"t": 23,
"x": 427,
"y": 681
}
],
"birth": 0,
"death": 24
},
{
"id": 12,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 12,
"x": 345,
"y": 693
},
{
"t": 14,
"x": 357,
"y": 689
},
{
"t": 16,
"x": 369,
"y": 685
},
{
"t": 18,
"x": 380,
"y": 681
},
{
"t": 19,
"x": 392,
"y": 681
},
{
"t": 21,
"x": 404,
"y": 677
},
{
"t": 22,
"x": 412,
"y": 673
},
{
"t": 23,
"x": 423,
"y": 673
}
],
"birth": 0,
"death": 24
},
{
"id": 13,
"type": "yellow wooden toy roller",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 314,
"y": 896
},
{
"t": 1,
"x": 318,
"y": 888
},
{
"t": 2,
"x": 326,
"y": 880
},
{
"t": 3,
"x": 330,
"y": 873
},
{
"t": 4,
"x": 333,
"y": 865
},
{
"t": 5,
"x": 337,
"y": 857
},
{
"t": 6,
"x": 345,
"y": 849
},
{
"t": 7,
"x": 349,
"y": 841
},
{
"t": 8,
"x": 353,
"y": 833
},
{
"t": 9,
"x": 361,
"y": 826
},
{
"t": 10,
"x": 365,
"y": 818
},
{
"t": 11,
"x": 369,
"y": 810
},
{
"t": 12,
"x": 373,
"y": 802
},
{
"t": 13,
"x": 376,
"y": 794
},
{
"t": 14,
"x": 380,
"y": 787
},
{
"t": 15,
"x": 384,
"y": 779
},
{
"t": 16,
"x": 388,
"y": 771
},
{
"t": 17,
"x": 388,
"y": 759
},
{
"t": 18,
"x": 392,
"y": 751
},
{
"t": 19,
"x": 392,
"y": 740
},
{
"t": 20,
"x": 396,
"y": 732
},
{
"t": 21,
"x": 396,
"y": 720
},
{
"t": 22,
"x": 396,
"y": 708
},
{
"t": 23,
"x": 396,
"y": 697
}
],
"birth": 0,
"death": 24
},
{
"id": 14,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 318,
"y": 736
},
{
"t": 2,
"x": 322,
"y": 728
},
{
"t": 4,
"x": 310,
"y": 732
},
{
"t": 5,
"x": 318,
"y": 724
},
{
"t": 7,
"x": 310,
"y": 728
},
{
"t": 10,
"x": 306,
"y": 720
},
{
"t": 11,
"x": 314,
"y": 712
},
{
"t": 12,
"x": 306,
"y": 716
},
{
"t": 13,
"x": 314,
"y": 708
},
{
"t": 14,
"x": 322,
"y": 705
},
{
"t": 16,
"x": 330,
"y": 697
},
{
"t": 17,
"x": 314,
"y": 701
},
{
"t": 18,
"x": 322,
"y": 697
},
{
"t": 19,
"x": 330,
"y": 693
},
{
"t": 20,
"x": 341,
"y": 689
},
{
"t": 21,
"x": 353,
"y": 685
},
{
"t": 22,
"x": 345,
"y": 681
},
{
"t": 23,
"x": 357,
"y": 681
}
],
"birth": 0,
"death": 24
},
{
"id": 15,
"type": "dark grey and pink running shoe",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 341,
"y": 681
},
{
"t": 2,
"x": 349,
"y": 685
},
{
"t": 3,
"x": 345,
"y": 677
},
{
"t": 4,
"x": 353,
"y": 681
},
{
"t": 6,
"x": 349,
"y": 669
},
{
"t": 7,
"x": 357,
"y": 673
},
{
"t": 9,
"x": 353,
"y": 662
},
{
"t": 10,
"x": 361,
"y": 666
},
{
"t": 12,
"x": 369,
"y": 662
},
{
"t": 14,
"x": 365,
"y": 650
},
{
"t": 15,
"x": 373,
"y": 654
},
{
"t": 17,
"x": 376,
"y": 646
},
{
"t": 19,
"x": 380,
"y": 638
},
{
"t": 21,
"x": 388,
"y": 634
},
{
"t": 23,
"x": 392,
"y": 626
}
],
"birth": 0,
"death": 24
},
{
"id": 16,
"type": "yellow and green cardboard crayon box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 373,
"y": 701
},
{
"t": 4,
"x": 376,
"y": 693
},
{
"t": 7,
"x": 380,
"y": 685
},
{
"t": 10,
"x": 384,
"y": 677
},
{
"t": 13,
"x": 392,
"y": 673
},
{
"t": 14,
"x": 388,
"y": 666
},
{
"t": 17,
"x": 392,
"y": 658
},
{
"t": 19,
"x": 396,
"y": 650
},
{
"t": 21,
"x": 404,
"y": 646
},
{
"t": 23,
"x": 408,
"y": 638
}
],
"birth": 0,
"death": 24
},
{
"id": 17,
"type": "yellow cardboard DVD set",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 408,
"y": 697
},
{
"t": 2,
"x": 412,
"y": 689
},
{
"t": 5,
"x": 416,
"y": 681
},
{
"t": 7,
"x": 419,
"y": 673
},
{
"t": 10,
"x": 423,
"y": 666
},
{
"t": 13,
"x": 427,
"y": 658
},
{
"t": 16,
"x": 431,
"y": 650
},
{
"t": 19,
"x": 435,
"y": 642
},
{
"t": 22,
"x": 439,
"y": 634
}
],
"birth": 0,
"death": 24
},
{
"id": 18,
"type": "yellow cardboard DVD set",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 416,
"y": 712
},
{
"t": 2,
"x": 419,
"y": 705
},
{
"t": 4,
"x": 423,
"y": 697
},
{
"t": 6,
"x": 427,
"y": 689
},
{
"t": 8,
"x": 431,
"y": 681
},
{
"t": 11,
"x": 435,
"y": 673
},
{
"t": 13,
"x": 439,
"y": 666
},
{
"t": 16,
"x": 443,
"y": 658
},
{
"t": 18,
"x": 447,
"y": 650
},
{
"t": 21,
"x": 451,
"y": 642
},
{
"t": 23,
"x": 455,
"y": 634
}
],
"birth": 0,
"death": 24
},
{
"id": 19,
"type": "dark grey metallic C-clamp",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 431,
"y": 681
},
{
"t": 3,
"x": 435,
"y": 673
},
{
"t": 6,
"x": 439,
"y": 666
},
{
"t": 9,
"x": 443,
"y": 658
},
{
"t": 12,
"x": 447,
"y": 650
},
{
"t": 15,
"x": 451,
"y": 642
},
{
"t": 18,
"x": 455,
"y": 634
},
{
"t": 20,
"x": 458,
"y": 626
},
{
"t": 23,
"x": 462,
"y": 619
}
],
"birth": 0,
"death": 24
},
{
"id": 20,
"type": "yellow plastic chocolate powder canister",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 458,
"y": 185
},
{
"t": 1,
"x": 466,
"y": 189
},
{
"t": 4,
"x": 474,
"y": 185
},
{
"t": 7,
"x": 482,
"y": 189
},
{
"t": 10,
"x": 490,
"y": 185
},
{
"t": 13,
"x": 498,
"y": 189
},
{
"t": 15,
"x": 490,
"y": 185
},
{
"t": 16,
"x": 501,
"y": 193
},
{
"t": 17,
"x": 494,
"y": 189
},
{
"t": 18,
"x": 505,
"y": 193
},
{
"t": 22,
"x": 513,
"y": 197
}
],
"birth": 0,
"death": 24
},
{
"id": 21,
"type": "purple and beige wooden dollhouse lamp",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 990,
"y": 642
},
{
"t": 1,
"x": 986,
"y": 634
},
{
"t": 2,
"x": 982,
"y": 626
},
{
"t": 3,
"x": 978,
"y": 619
},
{
"t": 4,
"x": 974,
"y": 611
},
{
"t": 6,
"x": 966,
"y": 603
},
{
"t": 8,
"x": 955,
"y": 599
},
{
"t": 9,
"x": 947,
"y": 595
},
{
"t": 11,
"x": 935,
"y": 591
},
{
"t": 12,
"x": 927,
"y": 587
},
{
"t": 14,
"x": 919,
"y": 583
},
{
"t": 15,
"x": 912,
"y": 580
},
{
"t": 17,
"x": 900,
"y": 576
},
{
"t": 19,
"x": 892,
"y": 572
},
{
"t": 21,
"x": 880,
"y": 568
},
{
"t": 23,
"x": 869,
"y": 564
}
],
"birth": 0,
"death": 24
}
],
"interactions": [
{
"id": 13,
"inputs": [
12,
14
],
"outputs": [
12,
14
],
"begins_at": 0,
"ends_at": 4,
"type": "contact"
},
{
"id": 2,
"inputs": [
1,
6
],
"outputs": [
1,
6
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
4
],
"outputs": [
1,
4
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 22,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 23,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 24,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 25,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 26,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 27,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 28,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 29,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 4,
"inputs": [
10,
11
],
"outputs": [
10,
11
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 5,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 6,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 7,
"inputs": [
12,
16
],
"outputs": [
12,
16
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 8,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 9,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 10,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 11,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 12,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 16,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 17,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 18,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 19,
"inputs": [
20,
65535
],
"outputs": [
20,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 20,
"inputs": [
21,
65535
],
"outputs": [
21,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 14,
"inputs": [
12,
14
],
"outputs": [
12,
14
],
"begins_at": 6,
"ends_at": 6,
"type": "collision"
},
{
"id": 15,
"inputs": [
12,
14
],
"outputs": [
12,
14
],
"begins_at": 7,
"ends_at": 23,
"type": "contact"
},
{
"id": 19,
"inputs": [
2,
7
],
"outputs": [
2,
7
],
"begins_at": 12,
"ends_at": 12,
"type": "collision"
},
{
"id": 21,
"inputs": [
2,
8
],
"outputs": [
2,
8
],
"begins_at": 12,
"ends_at": 12,
"type": "collision"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "blue chopper",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 25,
"y": 679
},
{
"t": 1,
"x": 33,
"y": 684
},
{
"t": 2,
"x": 42,
"y": 688
},
{
"t": 3,
"x": 55,
"y": 685
},
{
"t": 4,
"x": 70,
"y": 691
},
{
"t": 5,
"x": 82,
"y": 690
},
{
"t": 6,
"x": 94,
"y": 675
},
{
"t": 7,
"x": 100,
"y": 684
},
{
"t": 8,
"x": 116,
"y": 688
},
{
"t": 9,
"x": 128,
"y": 674
},
{
"t": 10,
"x": 137,
"y": 686
},
{
"t": 11,
"x": 149,
"y": 673
},
{
"t": 12,
"x": 161,
"y": 671
},
{
"t": 13,
"x": 173,
"y": 669
},
{
"t": 14,
"x": 183,
"y": 668
},
{
"t": 15,
"x": 194,
"y": 667
},
{
"t": 16,
"x": 186,
"y": 681
},
{
"t": 17,
"x": 213,
"y": 670
},
{
"t": 18,
"x": 225,
"y": 667
},
{
"t": 19,
"x": 235,
"y": 679
},
{
"t": 20,
"x": 248,
"y": 661
},
{
"t": 21,
"x": 239,
"y": 678
},
{
"t": 22,
"x": 269,
"y": 660
},
{
"t": 23,
"x": 279,
"y": 660
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "brown scooter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 301,
"y": 873
},
{
"t": 1,
"x": 304,
"y": 869
},
{
"t": 2,
"x": 305,
"y": 867
},
{
"t": 3,
"x": 310,
"y": 859
},
{
"t": 4,
"x": 313,
"y": 855
},
{
"t": 5,
"x": 314,
"y": 853
},
{
"t": 6,
"x": 319,
"y": 846
},
{
"t": 7,
"x": 323,
"y": 842
},
{
"t": 8,
"x": 323,
"y": 840
},
{
"t": 9,
"x": 330,
"y": 831
},
{
"t": 10,
"x": 333,
"y": 827
},
{
"t": 11,
"x": 334,
"y": 825
},
{
"t": 12,
"x": 339,
"y": 819
},
{
"t": 13,
"x": 341,
"y": 816
},
{
"t": 14,
"x": 344,
"y": 811
},
{
"t": 15,
"x": 347,
"y": 807
},
{
"t": 16,
"x": 350,
"y": 802
},
{
"t": 17,
"x": 353,
"y": 798
},
{
"t": 18,
"x": 355,
"y": 795
},
{
"t": 19,
"x": 360,
"y": 789
},
{
"t": 20,
"x": 362,
"y": 785
},
{
"t": 21,
"x": 365,
"y": 781
},
{
"t": 22,
"x": 369,
"y": 776
},
{
"t": 23,
"x": 372,
"y": 772
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "green dirtbike",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 319,
"y": 721
},
{
"t": 3,
"x": 317,
"y": 723
},
{
"t": 4,
"x": 319,
"y": 722
},
{
"t": 8,
"x": 325,
"y": 716
},
{
"t": 10,
"x": 297,
"y": 735
},
{
"t": 11,
"x": 331,
"y": 709
},
{
"t": 12,
"x": 298,
"y": 734
},
{
"t": 13,
"x": 301,
"y": 732
},
{
"t": 14,
"x": 316,
"y": 726
},
{
"t": 15,
"x": 299,
"y": 732
},
{
"t": 16,
"x": 302,
"y": 731
},
{
"t": 17,
"x": 326,
"y": 716
},
{
"t": 18,
"x": 326,
"y": 718
},
{
"t": 20,
"x": 329,
"y": 716
},
{
"t": 21,
"x": 330,
"y": 715
},
{
"t": 22,
"x": 332,
"y": 714
},
{
"t": 23,
"x": 333,
"y": 713
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "yellow airliner",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 472,
"y": 182
},
{
"t": 4,
"x": 476,
"y": 181
},
{
"t": 8,
"x": 481,
"y": 180
},
{
"t": 12,
"x": 480,
"y": 181
},
{
"t": 13,
"x": 485,
"y": 180
},
{
"t": 14,
"x": 479,
"y": 181
},
{
"t": 15,
"x": 488,
"y": 180
},
{
"t": 16,
"x": 484,
"y": 180
},
{
"t": 17,
"x": 490,
"y": 179
},
{
"t": 23,
"x": 494,
"y": 178
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "purple utility",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 995,
"y": 614
},
{
"t": 2,
"x": 989,
"y": 613
},
{
"t": 3,
"x": 980,
"y": 614
},
{
"t": 4,
"x": 972,
"y": 615
},
{
"t": 5,
"x": 961,
"y": 614
},
{
"t": 6,
"x": 953,
"y": 615
},
{
"t": 7,
"x": 980,
"y": 627
},
{
"t": 8,
"x": 933,
"y": 616
},
{
"t": 9,
"x": 976,
"y": 589
},
{
"t": 10,
"x": 964,
"y": 590
},
{
"t": 11,
"x": 940,
"y": 630
},
{
"t": 12,
"x": 943,
"y": 591
},
{
"t": 13,
"x": 920,
"y": 629
},
{
"t": 14,
"x": 872,
"y": 619
},
{
"t": 15,
"x": 912,
"y": 592
},
{
"t": 16,
"x": 850,
"y": 619
},
{
"t": 17,
"x": 893,
"y": 592
},
{
"t": 18,
"x": 829,
"y": 620
},
{
"t": 19,
"x": 820,
"y": 623
},
{
"t": 20,
"x": 861,
"y": 593
},
{
"t": 21,
"x": 848,
"y": 594
},
{
"t": 22,
"x": 838,
"y": 594
},
{
"t": 23,
"x": 812,
"y": 634
}
],
"birth": 0,
"death": 24
}
],
"interactions": [
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.3776942753266542,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 61.59078354439656,
"avg_image_position_segmentation_acc": 0.35,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.35,
"avg_image_position_segmentation_acc_large": null,
"avg_image_position_segmentation_acc_first_frame": 0.4,
"avg_image_position_segmentation_visible_acc": 0.35,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null,
"num_matches": 5,
"num_unmatched_pred": 17,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 5,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 5,
"avg_image_position_segmentation_acc": 5,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 5,
"avg_image_position_segmentation_acc_large": 0,
"avg_image_position_segmentation_acc_first_frame": 5,
"avg_image_position_segmentation_visible_acc": 5,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 5,
"visibility_true_acc": 5,
"visibility_false_acc": 0,
"is_in_frame_acc": 5,
"is_in_frame_true_acc": 5,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 5,
"vis_acc_from_px_true_acc": 5,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 5,
"target_id": 0,
"pred_id": 5,
"cost": 1.2083333333333335,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.7083333333333334,
"weighted_cost": 0.7083333333333334,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 15,
"target_id": 1,
"pred_id": 15,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 14,
"target_id": 2,
"pred_id": 14,
"cost": 0.5416666666666666,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.5416666666666666,
"weighted_cost": 0.5416666666666666,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 20,
"target_id": 3,
"pred_id": 20,
"cost": 0.5833333333333334,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.08333333333333333,
"weighted_cost": 0.08333333333333333,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 21,
"target_id": 4,
"pred_id": 21,
"cost": 0.9166666666666666,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.9166666666666666,
"weighted_cost": 0.9166666666666666,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
2,
3,
4,
6,
7,
8,
9,
10,
11,
12,
13,
16,
17,
18,
19
],
"target_type_counts": {
"blue chopper": 1,
"brown scooter": 1,
"green dirtbike": 1,
"yellow airliner": 1,
"purple utility": 1
},
"pred_type_counts": {
"light blue fabric backpack with owl and bird pattern": 1,
"light blue plastic conditioner tube": 4,
"dark grey and light grey plastic monster action figure": 1,
"dark green plastic coffee container": 1,
"yellow plastic toy school bus": 6,
"yellow wooden toy roller": 1,
"green wooden toy train": 1,
"dark grey and pink running shoe": 1,
"yellow and green cardboard crayon box": 1,
"yellow cardboard DVD set": 2,
"dark grey metallic C-clamp": 1,
"yellow plastic chocolate powder canister": 1,
"purple and beige wooden dollhouse lamp": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 5,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 2,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 1.0,
"1.0": 0.6,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 5,
"false_positives": 17,
"false_negatives": 0
},
"0.5": {
"true_positives": 5,
"false_positives": 17,
"false_negatives": 0
},
"0.75": {
"true_positives": 5,
"false_positives": 17,
"false_negatives": 0
},
"1.0": {
"true_positives": 5,
"false_positives": 17,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 0.22727272727272727,
"0.5": 0.22727272727272727,
"0.75": 0.22727272727272727,
"1.0": 0.22727272727272727
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 0.37037037037037035,
"0.5": 0.37037037037037035,
"0.75": 0.37037037037037035,
"1.0": 0.37037037037037035
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 17,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 17,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 17,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 17,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 0.05555555555555555,
"0.5": 0.05555555555555555,
"0.75": 0.05555555555555555,
"1.0": 0.05555555555555555
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 0.10526315789473684,
"0.5": 0.10526315789473684,
"0.75": 0.10526315789473684,
"1.0": 0.10526315789473684
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 19,
"false_negatives": 2
},
"0.5": {
"true_positives": 1,
"false_positives": 21,
"false_negatives": 4
},
"0.75": {
"true_positives": 1,
"false_positives": 21,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 5
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.13636363636363635,
"0.5": 0.045454545454545456,
"0.75": 0.045454545454545456,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.6,
"0.5": 0.2,
"0.75": 0.2,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.22222222222222218,
"0.5": 0.07407407407407407,
"0.75": 0.07407407407407407,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.05681818181818182,
"segmentation_frame_average_recall": 0.25,
"segmentation_frame_average_f1": 0.09259259259259259,
"segmentation_first_frame_counts": {
"true_positives": 2,
"false_positives": 20,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.09090909090909091,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.16666666666666669,
"segmentation_by_motion_group": {
"static": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.5,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.125,
"segmentation_frame_average_recall": 0.125,
"segmentation_frame_average_f1": 0.125,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.25,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.4
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 17,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 17,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 17,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 18,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.05555555555555555,
"0.5": 0.05555555555555555,
"0.75": 0.05555555555555555,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.10526315789473684,
"0.5": 0.10526315789473684,
"0.75": 0.10526315789473684,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.041666666666666664,
"segmentation_frame_average_recall": 0.75,
"segmentation_frame_average_f1": 0.07894736842105263,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 17,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.05555555555555555,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.10526315789473684
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 5,
"cost": 1.2083333333333335,
"type_similarity": 0.2778313012796404,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 69.28029547997828,
"image_position_segmentation_acc": 0.2916666666666667,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.2916666666666667,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.2916666666666667,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 15,
"cost": 1.0,
"type_similarity": 0.39859342936778025,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 160.1413105627159,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 14,
"cost": 0.5416666666666666,
"type_similarity": 0.4455958173368902,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 22.355559998176076,
"image_position_segmentation_acc": 0.4583333333333333,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.4583333333333333,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.4583333333333333,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 20,
"cost": 0.5833333333333334,
"type_similarity": 0.3942463902143516,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 15.536666617319405,
"image_position_segmentation_acc": 0.9166666666666666,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.9166666666666666,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.9166666666666666,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 4,
"pred_id": 21,
"cost": 0.9166666666666666,
"type_similarity": 0.37220443843460893,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 40.64008506379317,
"image_position_segmentation_acc": 0.08333333333333333,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.08333333333333333,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.08333333333333333,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": 1.0,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 27,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.12903225806451613,
"0.5": 0.12903225806451613,
"0.75": 0.12903225806451613,
"1.0": 0.12903225806451613
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.2285714285714286,
"0.5": 0.2285714285714286,
"0.75": 0.2285714285714286,
"1.0": 0.2285714285714286
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.12903225806451613,
"0.5": 0.12903225806451613,
"0.75": 0.12903225806451613,
"1.0": 0.12903225806451613
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.2285714285714286,
"0.5": 0.2285714285714286,
"0.75": 0.2285714285714286,
"1.0": 0.2285714285714286
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 27,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.12903225806451613,
"0.5": 0.12903225806451613,
"0.75": 0.12903225806451613,
"1.0": 0.12903225806451613
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.2285714285714286,
"0.5": 0.2285714285714286,
"0.75": 0.2285714285714286,
"1.0": 0.2285714285714286
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 26,
"target_id": 0,
"pred_id": 25,
"cost": 0.0,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 11,
"target_id": 1,
"pred_id": 11,
"cost": 0.0,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 10,
"target_id": 2,
"pred_id": 10,
"cost": 0.0,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 21,
"target_id": 3,
"pred_id": 20,
"cost": 0.0,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
12,
13,
14,
15,
16,
17,
18,
19,
19,
21,
22,
23,
24,
26,
27,
28,
29
],
"target_type_counts": {
"contact": 4
},
"pred_type_counts": {
"contact": 28,
"collision": 3
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 26,
"target_id": 0,
"pred_id": 25,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.0,
"temporal_span_iou": 1.0,
"temporal_span_giou": 1.0,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 1.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 1,
"pred_index": 11,
"target_id": 1,
"pred_id": 11,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.0,
"temporal_span_iou": 1.0,
"temporal_span_giou": 1.0,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 1.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 2,
"pred_index": 10,
"target_id": 2,
"pred_id": 10,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.0,
"temporal_span_iou": 1.0,
"temporal_span_giou": 1.0,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 1.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 3,
"pred_index": 21,
"target_id": 3,
"pred_id": 20,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.0,
"temporal_span_iou": 1.0,
"temporal_span_giou": 1.0,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 1.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 1.0,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 18,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.18181818181818182,
"0.5": 0.18181818181818182,
"0.75": 0.18181818181818182,
"1.0": 0.18181818181818182
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.3076923076923077,
"0.5": 0.3076923076923077,
"0.75": 0.3076923076923077,
"1.0": 0.3076923076923077
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.18181818181818182,
"0.5": 0.18181818181818182,
"0.75": 0.18181818181818182,
"1.0": 0.18181818181818182
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.3076923076923077,
"0.5": 0.3076923076923077,
"0.75": 0.3076923076923077,
"1.0": 0.3076923076923077
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 18,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.18181818181818182,
"0.5": 0.18181818181818182,
"0.75": 0.18181818181818182,
"1.0": 0.18181818181818182
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.3076923076923077,
"0.5": 0.3076923076923077,
"0.75": 0.3076923076923077,
"1.0": 0.3076923076923077
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 9,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 3,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 3,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": 1.0,
"avg_interaction_input_object_jaccard": 1.0,
"avg_interaction_output_object_jaccard": 1.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 24,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.14285714285714285,
"0.5": 0.14285714285714285,
"0.75": 0.14285714285714285,
"1.0": 0.14285714285714285
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.25,
"0.5": 0.25,
"0.75": 0.25,
"1.0": 0.25
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.14285714285714285,
"0.5": 0.14285714285714285,
"0.75": 0.14285714285714285,
"1.0": 0.14285714285714285
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.25,
"0.5": 0.25,
"0.75": 0.25,
"1.0": 0.25
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 24,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.14285714285714285,
"0.5": 0.14285714285714285,
"0.75": 0.14285714285714285,
"1.0": 0.14285714285714285
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.25,
"0.5": 0.25,
"0.75": 0.25,
"1.0": 0.25
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"contact": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}