STRIVE · 08/28/2026
Dyn generalization · val_24 · super_clevr_1066
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"blue truck\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 340 674 [OBJ] 1 \"red minivan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 486 675 5 489 672 [OBJ] 2 \"cyan jet\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 659 77 1 647 85 2 637 91 3 625 100 4 614 108 5 604 118 6 573 151 7 563 161 8 555 172 9 552 177 10 558 171 11 548 184 12 540 198 13 531 211 14 523 226 15 518 234 16 512 246 17 505 258 18 500 270 19 494 280 20 489 293 21 486 301 22 480 315 23 475 329 [OBJ] 3 \"cyan minivan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 701 610 1 698 613 2 696 614 3 694 615 5 691 616 6 689 617 7 687 619 8 684 620 10 682 621 11 680 622 12 677 623 13 675 625 14 673 626 15 670 627 16 668 628 17 666 630 18 663 631 19 661 632 20 658 633 21 664 628 22 654 636 [OBJ] 4 \"brown fighter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 951 321 1 942 321 2 987 320 3 984 315 4 976 315 5 968 315 6 961 314 7 951 315 8 944 313 9 937 311 10 929 311 11 920 310 12 911 310 13 901 310 14 892 310 15 883 310 16 874 309 17 864 309 18 855 309 19 845 309 20 836 308 21 826 308 22 816 308 23 806 308 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/val_24/super_clevr_1066/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"blue and red running shoe\" 0 12 [VIS] 0 true 10 false [IN-FRAME] 0 true 10 false [PX] 0 1 470 10 null null [OBJ] 1 \"blue wooden toy kitchen set\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 17 521 9 9 525 [OBJ] 2 \"light blue plastic conditioner tube\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 318 662 1 330 666 2 318 666 4 330 669 5 318 669 7 330 673 8 318 673 10 330 677 11 318 677 13 330 681 14 318 681 16 330 685 17 318 685 19 330 689 20 318 689 22 330 693 23 318 693 [OBJ] 3 \"red fabric dish drying mat\" 0 12 [VIS] 0 true 2 false 3 true 4 false [IN-FRAME] 0 true [PX] 0 412 626 2 null null 3 412 626 4 null null [OBJ] 4 \"red fabric storage cube\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 451 658 2 466 654 3 455 658 4 466 654 5 455 658 7 466 654 8 455 658 10 466 654 11 455 658 12 466 654 [OBJ] 5 \"red and blue plastic Spider-Man action figure\" 0 12 [VIS] 0 true 5 false 6 true 7 false 8 true 9 false 10 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 458 615 5 null null 6 458 615 7 null null 8 458 615 9 null null 10 458 615 11 null null 12 458 615 [OBJ] 6 \"red plastic hair dryer\" 0 12 [VIS] 0 true 4 false 5 true 9 false 10 true [IN-FRAME] 0 true [PX] 0 462 603 4 null null 5 462 607 9 null null 10 458 603 [OBJ] 7 \"red and black plastic handheld game console\" 0 12 [VIS] 0 true 4 false 5 true 9 false 10 true [IN-FRAME] 0 true [PX] 0 478 615 4 null null 5 478 619 9 null null 10 474 619 [OBJ] 8 \"red and blue plastic Spider-Man action figure\" 0 12 [VIS] 0 true 4 false 5 true 9 false 10 true [IN-FRAME] 0 true [PX] 0 482 607 4 null null 5 482 611 9 null null 10 478 615 [OBJ] 9 \"red and blue plastic Spider-Man action figure\" 0 12 [VIS] 0 true 4 false 5 true 9 false 10 true [IN-FRAME] 0 true [PX] 0 486 619 4 null null 5 486 623 9 null null 10 482 626 [OBJ] 10 \"red and blue plastic Spider-Man action figure\" 0 12 [VIS] 0 true 4 false 5 true 9 false 10 true [IN-FRAME] 0 true [PX] 0 494 626 4 null null 5 494 630 9 null null 10 490 638 [OBJ] 11 \"dark green plastic coffee container\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 583 505 8 591 509 [OBJ] 12 \"teal and beige wooden toy boat\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 595 482 10 591 474 [OBJ] 13 \"red and black cardboard hair color box\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 634 486 1 646 482 2 634 486 9 642 482 10 634 486 [OBJ] 14 \"teal and pink high-top sneaker\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 634 494 3 646 490 4 634 494 10 646 490 11 634 494 [OBJ] 15 \"red and black plastic handheld game console\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 654 486 1 null null [OBJ] 16 \"teal and pink high-top sneaker\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 654 490 1 null null [OBJ] 17 \"red and black plastic handheld game console\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 658 486 1 null null [OBJ] 18 \"red and blue plastic Spider-Man action figure\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 662 482 1 null null [OBJ] 19 \"dark grey cardboard motherboard box\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 666 486 1 null null [OBJ] 20 \"red leather boot\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 669 482 1 null null [OBJ] 21 \"teal and lime green sneaker\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 681 482 1 null null [OBJ] 22 \"red plastic New Super Mario Bros. Wii game case\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 685 486 1 null null [OBJ] 23 \"teal and lime green sneaker\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 693 482 1 null null [OBJ] 24 \"light blue cardboard argan oil treatment box\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 712 615 1 705 623 3 701 630 4 693 634 6 685 638 8 681 646 9 673 650 11 669 658 12 662 662 14 654 666 16 646 669 18 638 673 20 634 681 22 626 685 23 619 689 [OBJ] 25 \"yellow and green cardboard crayon box\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 728 482 1 null null [OBJ] 26 \"black and grey athletic shoe\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 732 486 1 null null [OBJ] 27 \"black cardboard motherboard box\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 744 486 1 null null [OBJ] 28 \"white cardboard ink cartridge box\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 748 490 1 null null [OBJ] 29 \"yellow and blue plastic soccer cleat\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 755 490 1 null null [OBJ] 30 \"yellow and blue plastic soccer cleat\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 763 486 1 null null [OBJ] 31 \"yellow and blue plastic soccer cleat\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 775 486 1 null null [OBJ] 32 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 818 498 1 null null [OBJ] 33 \"yellow and blue plastic soccer cleat\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 826 490 1 null null [OBJ] 34 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 837 501 1 null null [OBJ] 35 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 849 501 1 null null [OBJ] 36 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 865 505 1 null null [OBJ] 37 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 880 509 1 null null [OBJ] 38 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 900 513 1 null null [OBJ] 39 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 912 517 1 null null [OBJ] 40 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 927 517 1 null null [OBJ] 41 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 935 521 1 null null [OBJ] 42 \"yellow and blue plastic Spongebob toy set\" 0 12 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 947 521 1 null null [OBJ] 43 \"yellow rubber domed drop foot\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 990 314 1 986 322 2 982 330 3 978 337 4 974 345 5 970 353 6 966 361 7 962 369 8 955 373 9 947 376 10 939 380 11 931 384 12 923 388 13 916 392 14 908 396 15 900 400 16 892 404 17 884 408 18 876 412 19 869 416 20 861 419 21 853 423 22 845 427 23 837 431 [OBJS-END] [INTERACTIONS-START] [INTERACT] 3 \"contact\" 0 3 [IN] 10 11 [OUT] 10 11 [INTERACT] 0 \"contact\" 0 11 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 11 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 4 \"contact\" 0 11 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 2 \"contact\" 0 11 [IN] 10 24 [OUT] 10 24 [INTERACT] 30 \"contact\" 0 11 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 31 \"contact\" 0 11 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 32 \"contact\" 0 11 [IN] 26 65535 [OUT] 26 65535 [INTERACT] 33 \"contact\" 0 11 [IN] 27 65535 [OUT] 27 65535 [INTERACT] 34 \"contact\" 0 11 [IN] 28 65535 [OUT] 28 65535 [INTERACT] 35 \"contact\" 0 11 [IN] 29 65535 [OUT] 29 65535 [INTERACT] 36 \"contact\" 0 11 [IN] 31 65535 [OUT] 31 65535 [INTERACT] 37 \"contact\" 0 11 [IN] 32 65535 [OUT] 32 65535 [INTERACT] 38 \"contact\" 0 11 [IN] 33 65535 [OUT] 33 65535 [INTERACT] 39 \"contact\" 0 11 [IN] 34 65535 [OUT] 34 65535 [INTERACT] 40 \"contact\" 0 11 [IN] 35 65535 [OUT] 35 65535 [INTERACT] 41 \"contact\" 0 11 [IN] 36 65535 [OUT] 36 65535 [INTERACT] 15 \"contact\" 0 11 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 16 \"contact\" 0 11 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 17 \"contact\" 0 11 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 18 \"contact\" 0 11 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 19 \"contact\" 0 11 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 20 \"contact\" 0 11 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 21 \"contact\" 0 11 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 22 \"contact\" 0 11 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 23 \"contact\" 0 11 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 24 \"contact\" 0 11 [IN] 22 65535 [OUT] 22 65535 [INTERACT] 25 \"contact\" 0 11 [IN] 23 65535 [OUT] 23 65535 [INTERACT] 28 \"contact\" 0 11 [IN] 24 25 [OUT] 24 25 [INTERACT] 29 \"contact\" 0 11 [IN] 24 41 [OUT] 24 41 [INTERACT] 11 \"contact\" 0 11 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 12 \"contact\" 0 11 [IN] 13 15 [OUT] 13 15 [INTERACT] 13 \"contact\" 0 11 [IN] 13 16 [OUT] 13 16 [INTERACT] 14 \"contact\" 0 11 [IN] 13 22 [OUT] 13 22 [INTERACT] 8 \"collision\" 5 5 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 9 \"collision\" 7 7 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 10 \"contact\" 8 11 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 42 \"collision\" 10 10 [IN] 37 65535 [OUT] 37 65535 [INTERACT] 43 \"contact\" 11 12 [IN] 37 65535 [OUT] 37 65535 [INTERACT] 26 \"collision\" 11 11 [IN] 24 25 [OUT] 24 25 [INTERACT] 27 \"contact\" 12 13 [IN] 24 25 [OUT] 24 25 [INTERACT] 5 \"collision\" 13 13 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 6 \"collision\" 15 15 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 7 \"contact\" 16 23 [IN] 11 65535 [OUT] 11 65535 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"blue truck\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 340 674 [OBJ] 1 \"red minivan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 486 675 5 489 672 [OBJ] 2 \"cyan jet\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 659 77 1 647 85 2 637 91 3 625 100 4 614 108 5 604 118 6 573 151 7 563 161 8 555 172 9 552 177 10 558 171 11 548 184 12 540 198 13 531 211 14 523 226 15 518 234 16 512 246 17 505 258 18 500 270 19 494 280 20 489 293 21 486 301 22 480 315 23 475 329 [OBJ] 3 \"cyan minivan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 701 610 1 698 613 2 696 614 3 694 615 5 691 616 6 689 617 7 687 619 8 684 620 10 682 621 11 680 622 12 677 623 13 675 625 14 673 626 15 670 627 16 668 628 17 666 630 18 663 631 19 661 632 20 658 633 21 664 628 22 654 636 [OBJ] 4 \"brown fighter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 951 321 1 942 321 2 987 320 3 984 315 4 976 315 5 968 315 6 961 314 7 951 315 8 944 313 9 937 311 10 929 311 11 920 310 12 911 310 13 901 310 14 892 310 15 883 310 16 874 309 17 864 309 18 855 309 19 845 309 20 836 308 21 826 308 22 816 308 23 806 308 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "blue and red running shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 10,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 10,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 470
},
{
"t": 10,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 1,
"type": "blue wooden toy kitchen set",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 17,
"y": 521
},
{
"t": 9,
"x": 9,
"y": 525
}
],
"birth": 0,
"death": 12
},
{
"id": 2,
"type": "light blue plastic conditioner tube",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 318,
"y": 662
},
{
"t": 1,
"x": 330,
"y": 666
},
{
"t": 2,
"x": 318,
"y": 666
},
{
"t": 4,
"x": 330,
"y": 669
},
{
"t": 5,
"x": 318,
"y": 669
},
{
"t": 7,
"x": 330,
"y": 673
},
{
"t": 8,
"x": 318,
"y": 673
},
{
"t": 10,
"x": 330,
"y": 677
},
{
"t": 11,
"x": 318,
"y": 677
},
{
"t": 13,
"x": 330,
"y": 681
},
{
"t": 14,
"x": 318,
"y": 681
},
{
"t": 16,
"x": 330,
"y": 685
},
{
"t": 17,
"x": 318,
"y": 685
},
{
"t": 19,
"x": 330,
"y": 689
},
{
"t": 20,
"x": 318,
"y": 689
},
{
"t": 22,
"x": 330,
"y": 693
},
{
"t": 23,
"x": 318,
"y": 693
}
],
"birth": 0,
"death": 12
},
{
"id": 3,
"type": "red fabric dish drying mat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
},
{
"t": 3,
"v": true
},
{
"t": 4,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 412,
"y": 626
},
{
"t": 2,
"x": null,
"y": null
},
{
"t": 3,
"x": 412,
"y": 626
},
{
"t": 4,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 4,
"type": "red fabric storage cube",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 451,
"y": 658
},
{
"t": 2,
"x": 466,
"y": 654
},
{
"t": 3,
"x": 455,
"y": 658
},
{
"t": 4,
"x": 466,
"y": 654
},
{
"t": 5,
"x": 455,
"y": 658
},
{
"t": 7,
"x": 466,
"y": 654
},
{
"t": 8,
"x": 455,
"y": 658
},
{
"t": 10,
"x": 466,
"y": 654
},
{
"t": 11,
"x": 455,
"y": 658
},
{
"t": 12,
"x": 466,
"y": 654
}
],
"birth": 0,
"death": 12
},
{
"id": 5,
"type": "red and blue plastic Spider-Man action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 5,
"v": false
},
{
"t": 6,
"v": true
},
{
"t": 7,
"v": false
},
{
"t": 8,
"v": true
},
{
"t": 9,
"v": false
},
{
"t": 10,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 458,
"y": 615
},
{
"t": 5,
"x": null,
"y": null
},
{
"t": 6,
"x": 458,
"y": 615
},
{
"t": 7,
"x": null,
"y": null
},
{
"t": 8,
"x": 458,
"y": 615
},
{
"t": 9,
"x": null,
"y": null
},
{
"t": 10,
"x": 458,
"y": 615
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 458,
"y": 615
}
],
"birth": 0,
"death": 12
},
{
"id": 6,
"type": "red plastic hair dryer",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 4,
"v": false
},
{
"t": 5,
"v": true
},
{
"t": 9,
"v": false
},
{
"t": 10,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 462,
"y": 603
},
{
"t": 4,
"x": null,
"y": null
},
{
"t": 5,
"x": 462,
"y": 607
},
{
"t": 9,
"x": null,
"y": null
},
{
"t": 10,
"x": 458,
"y": 603
}
],
"birth": 0,
"death": 12
},
{
"id": 7,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 4,
"v": false
},
{
"t": 5,
"v": true
},
{
"t": 9,
"v": false
},
{
"t": 10,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 478,
"y": 615
},
{
"t": 4,
"x": null,
"y": null
},
{
"t": 5,
"x": 478,
"y": 619
},
{
"t": 9,
"x": null,
"y": null
},
{
"t": 10,
"x": 474,
"y": 619
}
],
"birth": 0,
"death": 12
},
{
"id": 8,
"type": "red and blue plastic Spider-Man action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 4,
"v": false
},
{
"t": 5,
"v": true
},
{
"t": 9,
"v": false
},
{
"t": 10,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 482,
"y": 607
},
{
"t": 4,
"x": null,
"y": null
},
{
"t": 5,
"x": 482,
"y": 611
},
{
"t": 9,
"x": null,
"y": null
},
{
"t": 10,
"x": 478,
"y": 615
}
],
"birth": 0,
"death": 12
},
{
"id": 9,
"type": "red and blue plastic Spider-Man action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 4,
"v": false
},
{
"t": 5,
"v": true
},
{
"t": 9,
"v": false
},
{
"t": 10,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 486,
"y": 619
},
{
"t": 4,
"x": null,
"y": null
},
{
"t": 5,
"x": 486,
"y": 623
},
{
"t": 9,
"x": null,
"y": null
},
{
"t": 10,
"x": 482,
"y": 626
}
],
"birth": 0,
"death": 12
},
{
"id": 10,
"type": "red and blue plastic Spider-Man action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 4,
"v": false
},
{
"t": 5,
"v": true
},
{
"t": 9,
"v": false
},
{
"t": 10,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 494,
"y": 626
},
{
"t": 4,
"x": null,
"y": null
},
{
"t": 5,
"x": 494,
"y": 630
},
{
"t": 9,
"x": null,
"y": null
},
{
"t": 10,
"x": 490,
"y": 638
}
],
"birth": 0,
"death": 12
},
{
"id": 11,
"type": "dark green plastic coffee container",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 583,
"y": 505
},
{
"t": 8,
"x": 591,
"y": 509
}
],
"birth": 0,
"death": 12
},
{
"id": 12,
"type": "teal and beige wooden toy boat",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 595,
"y": 482
},
{
"t": 10,
"x": 591,
"y": 474
}
],
"birth": 0,
"death": 12
},
{
"id": 13,
"type": "red and black cardboard hair color box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 634,
"y": 486
},
{
"t": 1,
"x": 646,
"y": 482
},
{
"t": 2,
"x": 634,
"y": 486
},
{
"t": 9,
"x": 642,
"y": 482
},
{
"t": 10,
"x": 634,
"y": 486
}
],
"birth": 0,
"death": 12
},
{
"id": 14,
"type": "teal and pink high-top sneaker",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 634,
"y": 494
},
{
"t": 3,
"x": 646,
"y": 490
},
{
"t": 4,
"x": 634,
"y": 494
},
{
"t": 10,
"x": 646,
"y": 490
},
{
"t": 11,
"x": 634,
"y": 494
}
],
"birth": 0,
"death": 12
},
{
"id": 15,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 654,
"y": 486
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 16,
"type": "teal and pink high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 654,
"y": 490
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 17,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 658,
"y": 486
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 18,
"type": "red and blue plastic Spider-Man action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 662,
"y": 482
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 19,
"type": "dark grey cardboard motherboard box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 666,
"y": 486
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 20,
"type": "red leather boot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 669,
"y": 482
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 21,
"type": "teal and lime green sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 681,
"y": 482
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 22,
"type": "red plastic New Super Mario Bros. Wii game case",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 685,
"y": 486
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 23,
"type": "teal and lime green sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 693,
"y": 482
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 24,
"type": "light blue cardboard argan oil treatment box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 712,
"y": 615
},
{
"t": 1,
"x": 705,
"y": 623
},
{
"t": 3,
"x": 701,
"y": 630
},
{
"t": 4,
"x": 693,
"y": 634
},
{
"t": 6,
"x": 685,
"y": 638
},
{
"t": 8,
"x": 681,
"y": 646
},
{
"t": 9,
"x": 673,
"y": 650
},
{
"t": 11,
"x": 669,
"y": 658
},
{
"t": 12,
"x": 662,
"y": 662
},
{
"t": 14,
"x": 654,
"y": 666
},
{
"t": 16,
"x": 646,
"y": 669
},
{
"t": 18,
"x": 638,
"y": 673
},
{
"t": 20,
"x": 634,
"y": 681
},
{
"t": 22,
"x": 626,
"y": 685
},
{
"t": 23,
"x": 619,
"y": 689
}
],
"birth": 0,
"death": 12
},
{
"id": 25,
"type": "yellow and green cardboard crayon box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 728,
"y": 482
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 26,
"type": "black and grey athletic shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 732,
"y": 486
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 27,
"type": "black cardboard motherboard box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 744,
"y": 486
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 28,
"type": "white cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 748,
"y": 490
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 29,
"type": "yellow and blue plastic soccer cleat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 755,
"y": 490
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 30,
"type": "yellow and blue plastic soccer cleat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 763,
"y": 486
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 31,
"type": "yellow and blue plastic soccer cleat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 775,
"y": 486
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 32,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 818,
"y": 498
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 33,
"type": "yellow and blue plastic soccer cleat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 826,
"y": 490
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 34,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 837,
"y": 501
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 35,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 849,
"y": 501
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 36,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 865,
"y": 505
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 37,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 880,
"y": 509
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 38,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 900,
"y": 513
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 39,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 912,
"y": 517
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 40,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 927,
"y": 517
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 41,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 935,
"y": 521
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 42,
"type": "yellow and blue plastic Spongebob toy set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 947,
"y": 521
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 43,
"type": "yellow rubber domed drop foot",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 990,
"y": 314
},
{
"t": 1,
"x": 986,
"y": 322
},
{
"t": 2,
"x": 982,
"y": 330
},
{
"t": 3,
"x": 978,
"y": 337
},
{
"t": 4,
"x": 974,
"y": 345
},
{
"t": 5,
"x": 970,
"y": 353
},
{
"t": 6,
"x": 966,
"y": 361
},
{
"t": 7,
"x": 962,
"y": 369
},
{
"t": 8,
"x": 955,
"y": 373
},
{
"t": 9,
"x": 947,
"y": 376
},
{
"t": 10,
"x": 939,
"y": 380
},
{
"t": 11,
"x": 931,
"y": 384
},
{
"t": 12,
"x": 923,
"y": 388
},
{
"t": 13,
"x": 916,
"y": 392
},
{
"t": 14,
"x": 908,
"y": 396
},
{
"t": 15,
"x": 900,
"y": 400
},
{
"t": 16,
"x": 892,
"y": 404
},
{
"t": 17,
"x": 884,
"y": 408
},
{
"t": 18,
"x": 876,
"y": 412
},
{
"t": 19,
"x": 869,
"y": 416
},
{
"t": 20,
"x": 861,
"y": 419
},
{
"t": 21,
"x": 853,
"y": 423
},
{
"t": 22,
"x": 845,
"y": 427
},
{
"t": 23,
"x": 837,
"y": 431
}
],
"birth": 0,
"death": 12
}
],
"interactions": [
{
"id": 3,
"inputs": [
10,
11
],
"outputs": [
10,
11
],
"begins_at": 0,
"ends_at": 3,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 4,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 2,
"inputs": [
10,
24
],
"outputs": [
10,
24
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 30,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 31,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 32,
"inputs": [
26,
65535
],
"outputs": [
26,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 33,
"inputs": [
27,
65535
],
"outputs": [
27,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 34,
"inputs": [
28,
65535
],
"outputs": [
28,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 35,
"inputs": [
29,
65535
],
"outputs": [
29,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 36,
"inputs": [
31,
65535
],
"outputs": [
31,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 37,
"inputs": [
32,
65535
],
"outputs": [
32,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 38,
"inputs": [
33,
65535
],
"outputs": [
33,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 39,
"inputs": [
34,
65535
],
"outputs": [
34,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 40,
"inputs": [
35,
65535
],
"outputs": [
35,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 41,
"inputs": [
36,
65535
],
"outputs": [
36,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 15,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 16,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 17,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 18,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 19,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 20,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 21,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 22,
"inputs": [
20,
65535
],
"outputs": [
20,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 23,
"inputs": [
21,
65535
],
"outputs": [
21,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 24,
"inputs": [
22,
65535
],
"outputs": [
22,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 25,
"inputs": [
23,
65535
],
"outputs": [
23,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 28,
"inputs": [
24,
25
],
"outputs": [
24,
25
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 29,
"inputs": [
24,
41
],
"outputs": [
24,
41
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 11,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 12,
"inputs": [
13,
15
],
"outputs": [
13,
15
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 13,
"inputs": [
13,
16
],
"outputs": [
13,
16
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 14,
"inputs": [
13,
22
],
"outputs": [
13,
22
],
"begins_at": 0,
"ends_at": 11,
"type": "contact"
},
{
"id": 8,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 5,
"ends_at": 5,
"type": "collision"
},
{
"id": 9,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 7,
"ends_at": 7,
"type": "collision"
},
{
"id": 10,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 8,
"ends_at": 11,
"type": "contact"
},
{
"id": 42,
"inputs": [
37,
65535
],
"outputs": [
37,
65535
],
"begins_at": 10,
"ends_at": 10,
"type": "collision"
},
{
"id": 43,
"inputs": [
37,
65535
],
"outputs": [
37,
65535
],
"begins_at": 11,
"ends_at": 12,
"type": "contact"
},
{
"id": 26,
"inputs": [
24,
25
],
"outputs": [
24,
25
],
"begins_at": 11,
"ends_at": 11,
"type": "collision"
},
{
"id": 27,
"inputs": [
24,
25
],
"outputs": [
24,
25
],
"begins_at": 12,
"ends_at": 13,
"type": "contact"
},
{
"id": 5,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 13,
"ends_at": 13,
"type": "collision"
},
{
"id": 6,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 15,
"ends_at": 15,
"type": "collision"
},
{
"id": 7,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 16,
"ends_at": 23,
"type": "contact"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "blue truck",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 340,
"y": 674
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "red minivan",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 486,
"y": 675
},
{
"t": 5,
"x": 489,
"y": 672
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "cyan jet",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 659,
"y": 77
},
{
"t": 1,
"x": 647,
"y": 85
},
{
"t": 2,
"x": 637,
"y": 91
},
{
"t": 3,
"x": 625,
"y": 100
},
{
"t": 4,
"x": 614,
"y": 108
},
{
"t": 5,
"x": 604,
"y": 118
},
{
"t": 6,
"x": 573,
"y": 151
},
{
"t": 7,
"x": 563,
"y": 161
},
{
"t": 8,
"x": 555,
"y": 172
},
{
"t": 9,
"x": 552,
"y": 177
},
{
"t": 10,
"x": 558,
"y": 171
},
{
"t": 11,
"x": 548,
"y": 184
},
{
"t": 12,
"x": 540,
"y": 198
},
{
"t": 13,
"x": 531,
"y": 211
},
{
"t": 14,
"x": 523,
"y": 226
},
{
"t": 15,
"x": 518,
"y": 234
},
{
"t": 16,
"x": 512,
"y": 246
},
{
"t": 17,
"x": 505,
"y": 258
},
{
"t": 18,
"x": 500,
"y": 270
},
{
"t": 19,
"x": 494,
"y": 280
},
{
"t": 20,
"x": 489,
"y": 293
},
{
"t": 21,
"x": 486,
"y": 301
},
{
"t": 22,
"x": 480,
"y": 315
},
{
"t": 23,
"x": 475,
"y": 329
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "cyan minivan",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 701,
"y": 610
},
{
"t": 1,
"x": 698,
"y": 613
},
{
"t": 2,
"x": 696,
"y": 614
},
{
"t": 3,
"x": 694,
"y": 615
},
{
"t": 5,
"x": 691,
"y": 616
},
{
"t": 6,
"x": 689,
"y": 617
},
{
"t": 7,
"x": 687,
"y": 619
},
{
"t": 8,
"x": 684,
"y": 620
},
{
"t": 10,
"x": 682,
"y": 621
},
{
"t": 11,
"x": 680,
"y": 622
},
{
"t": 12,
"x": 677,
"y": 623
},
{
"t": 13,
"x": 675,
"y": 625
},
{
"t": 14,
"x": 673,
"y": 626
},
{
"t": 15,
"x": 670,
"y": 627
},
{
"t": 16,
"x": 668,
"y": 628
},
{
"t": 17,
"x": 666,
"y": 630
},
{
"t": 18,
"x": 663,
"y": 631
},
{
"t": 19,
"x": 661,
"y": 632
},
{
"t": 20,
"x": 658,
"y": 633
},
{
"t": 21,
"x": 664,
"y": 628
},
{
"t": 22,
"x": 654,
"y": 636
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "brown fighter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 951,
"y": 321
},
{
"t": 1,
"x": 942,
"y": 321
},
{
"t": 2,
"x": 987,
"y": 320
},
{
"t": 3,
"x": 984,
"y": 315
},
{
"t": 4,
"x": 976,
"y": 315
},
{
"t": 5,
"x": 968,
"y": 315
},
{
"t": 6,
"x": 961,
"y": 314
},
{
"t": 7,
"x": 951,
"y": 315
},
{
"t": 8,
"x": 944,
"y": 313
},
{
"t": 9,
"x": 937,
"y": 311
},
{
"t": 10,
"x": 929,
"y": 311
},
{
"t": 11,
"x": 920,
"y": 310
},
{
"t": 12,
"x": 911,
"y": 310
},
{
"t": 13,
"x": 901,
"y": 310
},
{
"t": 14,
"x": 892,
"y": 310
},
{
"t": 15,
"x": 883,
"y": 310
},
{
"t": 16,
"x": 874,
"y": 309
},
{
"t": 17,
"x": 864,
"y": 309
},
{
"t": 18,
"x": 855,
"y": 309
},
{
"t": 19,
"x": 845,
"y": 309
},
{
"t": 20,
"x": 836,
"y": 308
},
{
"t": 21,
"x": 826,
"y": 308
},
{
"t": 22,
"x": 816,
"y": 308
},
{
"t": 23,
"x": 806,
"y": 308
}
],
"birth": 0,
"death": 24
}
],
"interactions": [
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.26847685787888065,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 162.03111480766924,
"avg_image_position_segmentation_acc": 0.5916666666666667,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.0,
"avg_image_position_segmentation_acc_large": 0.7395833333333334,
"avg_image_position_segmentation_acc_first_frame": 0.6,
"avg_image_position_segmentation_visible_acc": 0.5916666666666667,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 0.6166666666666666,
"visibility_true_acc": 0.6166666666666666,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.6166666666666666,
"vis_acc_from_px_true_acc": 0.6166666666666666,
"vis_acc_from_px_false_acc": null,
"num_matches": 5,
"num_unmatched_pred": 39,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 5,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 5,
"avg_image_position_segmentation_acc": 5,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 1,
"avg_image_position_segmentation_acc_large": 4,
"avg_image_position_segmentation_acc_first_frame": 5,
"avg_image_position_segmentation_visible_acc": 5,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 5,
"visibility_true_acc": 5,
"visibility_false_acc": 0,
"is_in_frame_acc": 5,
"is_in_frame_true_acc": 5,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 5,
"vis_acc_from_px_true_acc": 5,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 2,
"target_id": 0,
"pred_id": 2,
"cost": 0.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 4,
"target_id": 1,
"pred_id": 4,
"cost": 0.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 21,
"target_id": 2,
"pred_id": 21,
"cost": 1.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 24,
"target_id": 3,
"pred_id": 24,
"cost": 1.0416666666666667,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.041666666666666664,
"weighted_cost": 0.041666666666666664,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 26,
"target_id": 4,
"pred_id": 26,
"cost": 1.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
3,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
22,
23,
25,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43
],
"target_type_counts": {
"blue truck": 1,
"red minivan": 1,
"cyan jet": 1,
"cyan minivan": 1,
"brown fighter": 1
},
"pred_type_counts": {
"blue and red running shoe": 1,
"blue wooden toy kitchen set": 1,
"light blue plastic conditioner tube": 1,
"red fabric dish drying mat": 1,
"red fabric storage cube": 1,
"red and blue plastic Spider-Man action figure": 5,
"red plastic hair dryer": 1,
"red and black plastic handheld game console": 3,
"dark green plastic coffee container": 1,
"teal and beige wooden toy boat": 1,
"red and black cardboard hair color box": 1,
"teal and pink high-top sneaker": 2,
"dark grey cardboard motherboard box": 1,
"red leather boot": 1,
"teal and lime green sneaker": 2,
"red plastic New Super Mario Bros. Wii game case": 1,
"light blue cardboard argan oil treatment box": 1,
"yellow and green cardboard crayon box": 1,
"black and grey athletic shoe": 1,
"black cardboard motherboard box": 1,
"white cardboard ink cartridge box": 1,
"yellow and blue plastic soccer cleat": 4,
"yellow and blue plastic Spongebob toy set": 10,
"yellow rubber domed drop foot": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 3,
"false_positives": 2,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 3,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 0.6,
"1.0": 0.4,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 2
},
"0.5": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 2
},
"0.75": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 2
},
"1.0": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 0.06818181818181818,
"0.5": 0.06818181818181818,
"0.75": 0.06818181818181818,
"1.0": 0.06818181818181818
},
"visibility_recall_by_threshold": {
"0.25": 0.6,
"0.5": 0.6,
"0.75": 0.6,
"1.0": 0.6
},
"visibility_f1_by_threshold": {
"0.25": 0.1224489795918367,
"0.5": 0.1224489795918367,
"0.75": 0.1224489795918367,
"1.0": 0.1224489795918367
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 0,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 0,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 0,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 2
},
"0.5": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 2
},
"0.75": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 2
},
"1.0": {
"true_positives": 2,
"false_positives": 42,
"false_negatives": 3
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.06818181818181818,
"0.5": 0.06818181818181818,
"0.75": 0.06818181818181818,
"1.0": 0.045454545454545456
},
"segmentation_recall_by_threshold": {
"0.25": 0.6,
"0.5": 0.6,
"0.75": 0.6,
"1.0": 0.4
},
"segmentation_f1_by_threshold": {
"0.25": 0.1224489795918367,
"0.5": 0.1224489795918367,
"0.75": 0.1224489795918367,
"1.0": 0.0816326530612245
},
"segmentation_frame_average_precision": 0.0625,
"segmentation_frame_average_recall": 0.5499999999999999,
"segmentation_frame_average_f1": 0.11224489795918366,
"segmentation_first_frame_counts": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.06818181818181818,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.1276595744680851,
"segmentation_by_motion_group": {
"static": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 0,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 0,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 1,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.6666666666666666
},
"segmentation_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.6666666666666666
},
"segmentation_f1_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.6666666666666666
},
"segmentation_frame_average_precision": 0.9166666666666666,
"segmentation_frame_average_recall": 0.9166666666666666,
"segmentation_frame_average_f1": 0.9166666666666666,
"segmentation_first_frame_counts": {
"true_positives": 3,
"false_positives": 0,
"false_negatives": 0
},
"segmentation_first_frame_precision": 1.0,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 1.0
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 2
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 2,
"cost": 0.5,
"type_similarity": 0.31636008956640593,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 25.01757910997748,
"image_position_segmentation_acc": 1.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 1.0,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 1.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 4,
"cost": 0.5,
"type_similarity": 0.3146100338982116,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 37.84710030632858,
"image_position_segmentation_acc": 1.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 1.0,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 1.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 21,
"cost": 1.5,
"type_similarity": 0.26881613215599703,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 389.4129210349857,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.0,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.041666666666666664,
"visibility_true_acc": 0.041666666666666664,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.041666666666666664,
"vis_acc_from_px_true_acc": 0.041666666666666664,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 24,
"cost": 1.0416666666666667,
"type_similarity": 0.15316870003188193,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 36.17176378376207,
"image_position_segmentation_acc": 0.9583333333333334,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.9583333333333334,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.9583333333333334,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 4,
"pred_id": 26,
"cost": 1.5,
"type_similarity": 0.2894293337419067,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 321.70620980329244,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.041666666666666664,
"visibility_true_acc": 0.041666666666666664,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.041666666666666664,
"vis_acc_from_px_true_acc": 0.041666666666666664,
"vis_acc_from_px_false_acc": null
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": 0.5,
"avg_interaction_input_object_jaccard": 0.5555555555555555,
"avg_interaction_output_object_jaccard": 0.5555555555555555,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 3,
"num_unmatched_pred_interactions": 41,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 44,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 44,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.06818181818181818,
"0.5": 0.06818181818181818,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.1276595744680851,
"0.5": 0.1276595744680851,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 43,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 43,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 43,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.06818181818181818,
"0.5": 0.022727272727272728,
"0.75": 0.022727272727272728,
"1.0": 0.022727272727272728
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.1276595744680851,
"0.5": 0.044444444444444446,
"0.75": 0.044444444444444446,
"1.0": 0.044444444444444446
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 41,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 43,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 43,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 43,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.06818181818181818,
"0.5": 0.022727272727272728,
"0.75": 0.022727272727272728,
"1.0": 0.022727272727272728
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.1276595744680851,
"0.5": 0.044444444444444446,
"0.75": 0.044444444444444446,
"1.0": 0.044444444444444446
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 3,
"avg_interaction_input_object_jaccard": 3,
"avg_interaction_output_object_jaccard": 3,
"avg_interaction_type_exact_match_acc": 3
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 0,
"target_id": 0,
"pred_id": 0,
"cost": 0.25,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.25,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 1,
"target_id": 1,
"pred_id": 1,
"cost": 0.25,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.25,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 30,
"target_id": 2,
"pred_id": 30,
"cost": 0.25,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.25,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43
],
"target_type_counts": {
"contact": 3
},
"pred_type_counts": {
"contact": 38,
"collision": 6
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 0,
"target_id": 0,
"pred_id": 0,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.25,
"temporal_span_iou": 0.5,
"temporal_span_giou": 0.5,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 1,
"pred_index": 1,
"target_id": 1,
"pred_id": 1,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.25,
"temporal_span_iou": 0.5,
"temporal_span_giou": 0.5,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 2,
"pred_index": 30,
"target_id": 2,
"pred_id": 30,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.25,
"temporal_span_iou": 0.5,
"temporal_span_giou": 0.5,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0.5,
"avg_interaction_input_object_jaccard": 0.5555555555555555,
"avg_interaction_output_object_jaccard": 0.5555555555555555,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 3,
"num_unmatched_pred_interactions": 32,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 32,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 32,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 35,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 35,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.08571428571428572,
"0.5": 0.08571428571428572,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.15789473684210528,
"0.5": 0.15789473684210528,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 32,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 34,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 34,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 34,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.08571428571428572,
"0.5": 0.02857142857142857,
"0.75": 0.02857142857142857,
"1.0": 0.02857142857142857
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.15789473684210528,
"0.5": 0.05555555555555556,
"0.75": 0.05555555555555556,
"1.0": 0.05555555555555556
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 32,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 34,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 34,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 34,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.08571428571428572,
"0.5": 0.02857142857142857,
"0.75": 0.02857142857142857,
"1.0": 0.02857142857142857
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.15789473684210528,
"0.5": 0.05555555555555556,
"0.75": 0.05555555555555556,
"1.0": 0.05555555555555556
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 9,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 9,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 3,
"avg_interaction_input_object_jaccard": 3,
"avg_interaction_output_object_jaccard": 3,
"avg_interaction_type_exact_match_acc": 3
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 6,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 6,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": 0.5,
"avg_interaction_input_object_jaccard": 0.5555555555555555,
"avg_interaction_output_object_jaccard": 0.5555555555555555,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 3,
"num_unmatched_pred_interactions": 35,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 35,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 38,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 38,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.07894736842105263,
"0.5": 0.07894736842105263,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.14634146341463414,
"0.5": 0.14634146341463414,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 37,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 37,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 37,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.07894736842105263,
"0.5": 0.02631578947368421,
"0.75": 0.02631578947368421,
"1.0": 0.02631578947368421
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.14634146341463414,
"0.5": 0.05128205128205127,
"0.75": 0.05128205128205127,
"1.0": 0.05128205128205127
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 35,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 37,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 37,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 37,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.07894736842105263,
"0.5": 0.02631578947368421,
"0.75": 0.02631578947368421,
"1.0": 0.02631578947368421
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.14634146341463414,
"0.5": 0.05128205128205127,
"0.75": 0.05128205128205127,
"1.0": 0.05128205128205127
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"contact": {
"avg_interaction_temporal_span_giou": 3,
"avg_interaction_input_object_jaccard": 3,
"avg_interaction_output_object_jaccard": 3,
"avg_interaction_type_exact_match_acc": 3
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}