STRIVE · 08/28/2026
Dyn generalization · val_24 · super_clevr_1088
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow truck\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 40 683 1 44 683 2 50 683 3 55 683 4 60 683 5 66 683 6 72 683 7 92 682 8 96 682 9 90 682 10 102 682 11 105 682 12 108 682 13 114 682 14 120 682 15 126 682 16 133 682 17 139 682 18 156 681 19 159 681 20 163 681 21 166 681 22 172 681 23 179 681 [OBJ] 1 \"green sedan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 346 544 2 348 546 3 349 548 5 351 551 7 358 554 8 354 556 10 355 559 11 358 561 12 362 563 13 358 565 14 364 567 15 362 569 16 362 571 18 364 574 19 367 576 21 368 580 22 369 583 23 371 584 [OBJ] 2 \"red truck\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 433 541 1 430 541 3 431 543 7 431 546 9 433 548 13 433 551 16 433 554 19 434 557 22 435 560 [OBJ] 3 \"gray wagon\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 773 706 [OBJ] 4 \"green jet\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 976 329 1 985 325 2 969 329 3 984 323 5 984 320 6 983 319 7 983 316 8 981 315 9 979 314 10 975 314 11 971 314 12 967 314 13 963 313 14 958 314 15 955 313 16 950 313 17 945 314 18 941 314 19 936 314 20 931 314 21 926 314 22 921 314 23 916 314 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/val_24/super_clevr_1088/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow plastic toy school bus\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 33 701 1 33 689 2 33 677 3 33 666 5 33 654 6 33 642 8 33 626 10 33 611 12 33 595 13 33 583 15 33 568 17 33 552 19 33 537 21 33 521 23 33 505 [OBJ] 1 \"yellow plastic toy school bus\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 37 728 1 37 716 3 37 705 4 37 693 6 37 681 7 37 669 9 37 654 11 37 638 12 37 626 14 37 611 15 37 599 17 37 583 19 37 568 20 37 556 22 37 541 23 37 529 [OBJ] 2 \"yellow plastic toy school bus\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 41 744 1 41 732 3 41 716 5 41 701 7 41 685 9 41 669 11 41 654 13 41 638 15 41 623 17 41 607 19 41 591 21 41 576 23 41 560 [OBJ] 3 \"yellow and blue plastic soccer cleat\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 99 697 1 103 689 2 111 685 3 115 677 4 119 669 5 126 666 6 130 658 7 134 650 8 138 642 9 146 638 10 150 630 11 158 626 12 162 619 13 166 611 14 169 603 15 173 595 16 177 587 17 181 580 19 189 572 20 193 564 21 197 556 23 205 544 [OBJ] 4 \"yellow tennis shoe\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 111 681 1 119 677 2 123 669 3 null null [OBJ] 5 \"yellow hoodie dollhouse boy\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 111 693 2 119 685 3 null null [OBJ] 6 \"yellow hoodie dollhouse boy\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 115 689 2 123 685 3 null null [OBJ] 7 \"yellow hoodie dollhouse boy\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 119 697 1 123 689 2 126 681 3 null null [OBJ] 8 \"yellow hoodie dollhouse boy\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 123 685 2 130 677 3 null null [OBJ] 9 \"yellow hoodie dollhouse boy\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 126 689 2 134 681 3 null null [OBJ] 10 \"yellow hoodie dollhouse boy\" 0 12 [VIS] 0 true 4 false [IN-FRAME] 0 true [PX] 0 126 701 2 134 693 4 null null [OBJ] 11 \"yellow hoodie dollhouse boy\" 0 12 [VIS] 0 true 4 false [IN-FRAME] 0 true [PX] 0 126 716 1 134 712 2 138 705 4 null null [OBJ] 12 \"blue and yellow basketball shoe\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 134 658 2 142 654 3 null null [OBJ] 13 \"yellow tennis shoe\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 138 677 2 146 673 3 null null [OBJ] 14 \"white and brown leather golf shoe\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 154 650 3 null null [OBJ] 15 \"yellow tennis shoe\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 162 689 2 169 685 3 null null [OBJ] 16 \"green plastic plant pot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 162 669 2 169 666 3 null null [OBJ] 17 \"green and yellow cardboard ink cartridge box\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 169 658 3 null null [OBJ] 18 \"green and white striped fabric dishtowel\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 181 662 3 null null [OBJ] 19 \"green and white striped fabric dishtowel\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 189 654 3 null null [OBJ] 20 \"green plastic plant pot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 212 642 3 null null [OBJ] 21 \"blue and yellow fabric sneaker\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 216 650 3 null null [OBJ] 22 \"green and white striped fabric dishtowel\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 232 646 3 null null [OBJ] 23 \"green wooden dollhouse bedroom furniture set\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 240 646 3 null null [OBJ] 24 \"green wooden toy train\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 244 654 3 null null [OBJ] 25 \"green wooden toy train\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 251 650 3 null null [OBJ] 26 \"green plastic plant pot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 267 638 3 null null [OBJ] 27 \"light yellow and dark blue cardboard whey protein box\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 275 634 3 null null [OBJ] 28 \"light blue and yellow high-top sneaker\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 291 638 3 null null [OBJ] 29 \"green and yellow cardboard ink cartridge box\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 302 630 3 null null [OBJ] 30 \"green Lego Android robot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 306 638 3 null null [OBJ] 31 \"green Lego Android robot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 310 630 3 null null [OBJ] 32 \"green plastic plant pot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 314 642 3 null null [OBJ] 33 \"green wooden dollhouse bedroom furniture set\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 326 630 3 null null [OBJ] 34 \"light blue cardboard argan oil treatment box\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 333 630 3 null null [OBJ] 35 \"green plastic plant pot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 341 626 3 null null [OBJ] 36 \"light blue cardboard argan oil treatment box\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 349 623 3 null null [OBJ] 37 \"green plastic plant pot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 353 630 3 null null [OBJ] 38 \"green and yellow cardboard ink cartridge box\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 361 619 3 null null [OBJ] 39 \"green plastic plant pot\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 373 615 3 null null [OBJ] 40 \"red plastic New Super Mario Bros. Wii game case\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 380 611 3 null null [OBJ] 41 \"light grey canvas shoe\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 384 619 3 null null [OBJ] 42 \"green and white cardboard electric toothbrush box\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 392 611 3 null null [OBJ] 43 \"green wooden dollhouse bedroom furniture set\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 396 619 3 null null [OBJ] 44 \"green and white plastic portable speaker\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 400 607 3 null null [OBJ] 45 \"green and white cardboard electric toothbrush box\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 404 623 3 null null [OBJ] 46 \"light blue fabric lunch bag with butterfly print\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 412 615 3 null null [OBJ] 47 \"red and black plastic handheld game console\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 419 619 3 null null [OBJ] 48 \"white plastic supplement bottle\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 423 607 3 null null [OBJ] 49 \"red and black plastic handheld game console\" 0 12 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 431 607 3 null null [OBJ] 50 \"red fabric chukka boot\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 435 552 3 439 560 6 439 576 7 439 587 8 435 599 9 443 580 10 439 591 11 439 580 12 435 568 [OBJ] 51 \"light grey and black sneaker\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 841 732 1 822 732 2 833 732 3 818 732 5 830 732 6 814 728 8 826 728 9 810 728 10 822 728 11 806 724 12 818 724 [OBJ] 52 \"green wooden toy train\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 330 1 986 333 2 982 341 3 974 345 4 970 353 5 962 357 6 955 361 7 947 365 9 939 369 10 931 373 11 923 376 12 916 380 13 908 384 14 900 388 16 892 392 17 884 396 19 873 400 21 865 404 22 857 408 23 849 412 [OBJ] 53 \"turquoise plastic plant pot\" 0 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 537 1 994 525 2 990 541 3 990 529 4 990 517 5 986 533 6 986 521 7 986 509 8 982 529 9 982 517 10 982 505 11 978 525 12 978 513 13 978 501 14 974 521 15 974 509 16 974 498 17 970 521 18 970 509 19 970 498 20 966 521 21 966 509 22 966 498 23 962 521 [OBJ] 54 \"red and green cardboard ink cartridge box\" 0 12 [VIS] 0 true 9 false [IN-FRAME] 0 true 9 false [PX] 0 998 306 1 994 314 3 990 322 4 986 330 6 982 337 7 978 345 9 null null [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 12 [IN] 0 14 [OUT] 0 14 [INTERACT] 1 \"contact\" 0 12 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 2 \"contact\" 0 12 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 38 \"contact\" 0 12 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 39 \"contact\" 0 12 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 40 \"contact\" 0 12 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 41 \"contact\" 0 12 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 42 \"contact\" 0 12 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 43 \"contact\" 0 12 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 44 \"contact\" 0 12 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 3 \"contact\" 0 12 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 4 \"contact\" 0 12 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 5 \"contact\" 0 12 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 6 \"contact\" 0 12 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 7 \"contact\" 0 12 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 8 \"contact\" 0 12 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 9 \"contact\" 0 12 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 10 \"contact\" 0 12 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 11 \"contact\" 0 12 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 12 \"contact\" 0 12 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 13 \"contact\" 0 12 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 14 \"contact\" 0 12 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 15 \"contact\" 0 12 [IN] 22 65535 [OUT] 22 65535 [INTERACT] 16 \"contact\" 0 12 [IN] 23 65535 [OUT] 23 65535 [INTERACT] 17 \"contact\" 0 12 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 18 \"contact\" 0 12 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 19 \"contact\" 0 12 [IN] 26 65535 [OUT] 26 65535 [INTERACT] 20 \"contact\" 0 12 [IN] 27 65535 [OUT] 27 65535 [INTERACT] 21 \"contact\" 0 12 [IN] 28 65535 [OUT] 28 65535 [INTERACT] 22 \"contact\" 0 12 [IN] 29 65535 [OUT] 29 65535 [INTERACT] 23 \"contact\" 0 12 [IN] 31 65535 [OUT] 31 65535 [INTERACT] 24 \"contact\" 0 12 [IN] 32 65535 [OUT] 32 65535 [INTERACT] 25 \"contact\" 0 12 [IN] 33 65535 [OUT] 33 65535 [INTERACT] 26 \"contact\" 0 12 [IN] 34 65535 [OUT] 34 65535 [INTERACT] 27 \"contact\" 0 12 [IN] 35 65535 [OUT] 35 65535 [INTERACT] 28 \"contact\" 0 12 [IN] 36 65535 [OUT] 36 65535 [INTERACT] 29 \"contact\" 0 12 [IN] 37 65535 [OUT] 37 65535 [INTERACT] 30 \"contact\" 0 12 [IN] 38 65535 [OUT] 38 65535 [INTERACT] 31 \"contact\" 0 12 [IN] 39 65535 [OUT] 39 65535 [INTERACT] 32 \"contact\" 0 12 [IN] 40 65535 [OUT] 40 65535 [INTERACT] 33 \"contact\" 0 12 [IN] 41 65535 [OUT] 41 65535 [INTERACT] 34 \"contact\" 0 12 [IN] 42 65535 [OUT] 42 65535 [INTERACT] 35 \"contact\" 0 12 [IN] 43 65535 [OUT] 43 65535 [INTERACT] 36 \"contact\" 0 12 [IN] 44 65535 [OUT] 44 65535 [INTERACT] 37 \"contact\" 0 12 [IN] 45 65535 [OUT] 45 65535 [INTERACT] 3 \"collision\" 2 2 [IN] 10 14 [OUT] 10 14 [INTERACT] 4 \"collision\" 5 5 [IN] 10 17 [OUT] 10 17 [INTERACT] 5 \"collision\" 7 7 [IN] 10 14 [OUT] 10 14 [INTERACT] 6 \"contact\" 8 11 [IN] 10 14 [OUT] 10 14 [INTERACT] 1 \"collision\" 10 10 [IN] 0 14 [OUT] 0 14 [INTERACT] 2 \"collision\" 11 12 [IN] 0 15 [OUT] 0 15 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow truck\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 40 683 1 44 683 2 50 683 3 55 683 4 60 683 5 66 683 6 72 683 7 92 682 8 96 682 9 90 682 10 102 682 11 105 682 12 108 682 13 114 682 14 120 682 15 126 682 16 133 682 17 139 682 18 156 681 19 159 681 20 163 681 21 166 681 22 172 681 23 179 681 [OBJ] 1 \"green sedan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 346 544 2 348 546 3 349 548 5 351 551 7 358 554 8 354 556 10 355 559 11 358 561 12 362 563 13 358 565 14 364 567 15 362 569 16 362 571 18 364 574 19 367 576 21 368 580 22 369 583 23 371 584 [OBJ] 2 \"red truck\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 433 541 1 430 541 3 431 543 7 431 546 9 433 548 13 433 551 16 433 554 19 434 557 22 435 560 [OBJ] 3 \"gray wagon\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 773 706 [OBJ] 4 \"green jet\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 976 329 1 985 325 2 969 329 3 984 323 5 984 320 6 983 319 7 983 316 8 981 315 9 979 314 10 975 314 11 971 314 12 967 314 13 963 313 14 958 314 15 955 313 16 950 313 17 945 314 18 941 314 19 936 314 20 931 314 21 926 314 22 921 314 23 916 314 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 33,
"y": 701
},
{
"t": 1,
"x": 33,
"y": 689
},
{
"t": 2,
"x": 33,
"y": 677
},
{
"t": 3,
"x": 33,
"y": 666
},
{
"t": 5,
"x": 33,
"y": 654
},
{
"t": 6,
"x": 33,
"y": 642
},
{
"t": 8,
"x": 33,
"y": 626
},
{
"t": 10,
"x": 33,
"y": 611
},
{
"t": 12,
"x": 33,
"y": 595
},
{
"t": 13,
"x": 33,
"y": 583
},
{
"t": 15,
"x": 33,
"y": 568
},
{
"t": 17,
"x": 33,
"y": 552
},
{
"t": 19,
"x": 33,
"y": 537
},
{
"t": 21,
"x": 33,
"y": 521
},
{
"t": 23,
"x": 33,
"y": 505
}
],
"birth": 0,
"death": 12
},
{
"id": 1,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 37,
"y": 728
},
{
"t": 1,
"x": 37,
"y": 716
},
{
"t": 3,
"x": 37,
"y": 705
},
{
"t": 4,
"x": 37,
"y": 693
},
{
"t": 6,
"x": 37,
"y": 681
},
{
"t": 7,
"x": 37,
"y": 669
},
{
"t": 9,
"x": 37,
"y": 654
},
{
"t": 11,
"x": 37,
"y": 638
},
{
"t": 12,
"x": 37,
"y": 626
},
{
"t": 14,
"x": 37,
"y": 611
},
{
"t": 15,
"x": 37,
"y": 599
},
{
"t": 17,
"x": 37,
"y": 583
},
{
"t": 19,
"x": 37,
"y": 568
},
{
"t": 20,
"x": 37,
"y": 556
},
{
"t": 22,
"x": 37,
"y": 541
},
{
"t": 23,
"x": 37,
"y": 529
}
],
"birth": 0,
"death": 12
},
{
"id": 2,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 41,
"y": 744
},
{
"t": 1,
"x": 41,
"y": 732
},
{
"t": 3,
"x": 41,
"y": 716
},
{
"t": 5,
"x": 41,
"y": 701
},
{
"t": 7,
"x": 41,
"y": 685
},
{
"t": 9,
"x": 41,
"y": 669
},
{
"t": 11,
"x": 41,
"y": 654
},
{
"t": 13,
"x": 41,
"y": 638
},
{
"t": 15,
"x": 41,
"y": 623
},
{
"t": 17,
"x": 41,
"y": 607
},
{
"t": 19,
"x": 41,
"y": 591
},
{
"t": 21,
"x": 41,
"y": 576
},
{
"t": 23,
"x": 41,
"y": 560
}
],
"birth": 0,
"death": 12
},
{
"id": 3,
"type": "yellow and blue plastic soccer cleat",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 99,
"y": 697
},
{
"t": 1,
"x": 103,
"y": 689
},
{
"t": 2,
"x": 111,
"y": 685
},
{
"t": 3,
"x": 115,
"y": 677
},
{
"t": 4,
"x": 119,
"y": 669
},
{
"t": 5,
"x": 126,
"y": 666
},
{
"t": 6,
"x": 130,
"y": 658
},
{
"t": 7,
"x": 134,
"y": 650
},
{
"t": 8,
"x": 138,
"y": 642
},
{
"t": 9,
"x": 146,
"y": 638
},
{
"t": 10,
"x": 150,
"y": 630
},
{
"t": 11,
"x": 158,
"y": 626
},
{
"t": 12,
"x": 162,
"y": 619
},
{
"t": 13,
"x": 166,
"y": 611
},
{
"t": 14,
"x": 169,
"y": 603
},
{
"t": 15,
"x": 173,
"y": 595
},
{
"t": 16,
"x": 177,
"y": 587
},
{
"t": 17,
"x": 181,
"y": 580
},
{
"t": 19,
"x": 189,
"y": 572
},
{
"t": 20,
"x": 193,
"y": 564
},
{
"t": 21,
"x": 197,
"y": 556
},
{
"t": 23,
"x": 205,
"y": 544
}
],
"birth": 0,
"death": 12
},
{
"id": 4,
"type": "yellow tennis shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 111,
"y": 681
},
{
"t": 1,
"x": 119,
"y": 677
},
{
"t": 2,
"x": 123,
"y": 669
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 5,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 111,
"y": 693
},
{
"t": 2,
"x": 119,
"y": 685
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 6,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 115,
"y": 689
},
{
"t": 2,
"x": 123,
"y": 685
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 7,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 119,
"y": 697
},
{
"t": 1,
"x": 123,
"y": 689
},
{
"t": 2,
"x": 126,
"y": 681
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 8,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 123,
"y": 685
},
{
"t": 2,
"x": 130,
"y": 677
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 9,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 126,
"y": 689
},
{
"t": 2,
"x": 134,
"y": 681
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 10,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 4,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 126,
"y": 701
},
{
"t": 2,
"x": 134,
"y": 693
},
{
"t": 4,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 11,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 4,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 126,
"y": 716
},
{
"t": 1,
"x": 134,
"y": 712
},
{
"t": 2,
"x": 138,
"y": 705
},
{
"t": 4,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 12,
"type": "blue and yellow basketball shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 134,
"y": 658
},
{
"t": 2,
"x": 142,
"y": 654
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 13,
"type": "yellow tennis shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 138,
"y": 677
},
{
"t": 2,
"x": 146,
"y": 673
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 14,
"type": "white and brown leather golf shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 154,
"y": 650
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 15,
"type": "yellow tennis shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 162,
"y": 689
},
{
"t": 2,
"x": 169,
"y": 685
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 16,
"type": "green plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 162,
"y": 669
},
{
"t": 2,
"x": 169,
"y": 666
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 17,
"type": "green and yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 169,
"y": 658
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 18,
"type": "green and white striped fabric dishtowel",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 181,
"y": 662
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 19,
"type": "green and white striped fabric dishtowel",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 189,
"y": 654
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 20,
"type": "green plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 212,
"y": 642
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 21,
"type": "blue and yellow fabric sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 216,
"y": 650
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 22,
"type": "green and white striped fabric dishtowel",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 232,
"y": 646
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 23,
"type": "green wooden dollhouse bedroom furniture set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 240,
"y": 646
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 24,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 244,
"y": 654
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 25,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 251,
"y": 650
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 26,
"type": "green plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 267,
"y": 638
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 27,
"type": "light yellow and dark blue cardboard whey protein box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 275,
"y": 634
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 28,
"type": "light blue and yellow high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 291,
"y": 638
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 29,
"type": "green and yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 302,
"y": 630
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 30,
"type": "green Lego Android robot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 306,
"y": 638
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 31,
"type": "green Lego Android robot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 310,
"y": 630
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 32,
"type": "green plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 314,
"y": 642
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 33,
"type": "green wooden dollhouse bedroom furniture set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 326,
"y": 630
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 34,
"type": "light blue cardboard argan oil treatment box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 333,
"y": 630
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 35,
"type": "green plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 341,
"y": 626
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 36,
"type": "light blue cardboard argan oil treatment box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 349,
"y": 623
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 37,
"type": "green plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 353,
"y": 630
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 38,
"type": "green and yellow cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 361,
"y": 619
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 39,
"type": "green plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 373,
"y": 615
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 40,
"type": "red plastic New Super Mario Bros. Wii game case",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 380,
"y": 611
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 41,
"type": "light grey canvas shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 384,
"y": 619
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 42,
"type": "green and white cardboard electric toothbrush box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 392,
"y": 611
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 43,
"type": "green wooden dollhouse bedroom furniture set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 396,
"y": 619
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 44,
"type": "green and white plastic portable speaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 400,
"y": 607
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 45,
"type": "green and white cardboard electric toothbrush box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 404,
"y": 623
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 46,
"type": "light blue fabric lunch bag with butterfly print",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 412,
"y": 615
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 47,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 419,
"y": 619
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 48,
"type": "white plastic supplement bottle",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 423,
"y": 607
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 49,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 3,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 431,
"y": 607
},
{
"t": 3,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
},
{
"id": 50,
"type": "red fabric chukka boot",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 435,
"y": 552
},
{
"t": 3,
"x": 439,
"y": 560
},
{
"t": 6,
"x": 439,
"y": 576
},
{
"t": 7,
"x": 439,
"y": 587
},
{
"t": 8,
"x": 435,
"y": 599
},
{
"t": 9,
"x": 443,
"y": 580
},
{
"t": 10,
"x": 439,
"y": 591
},
{
"t": 11,
"x": 439,
"y": 580
},
{
"t": 12,
"x": 435,
"y": 568
}
],
"birth": 0,
"death": 12
},
{
"id": 51,
"type": "light grey and black sneaker",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 841,
"y": 732
},
{
"t": 1,
"x": 822,
"y": 732
},
{
"t": 2,
"x": 833,
"y": 732
},
{
"t": 3,
"x": 818,
"y": 732
},
{
"t": 5,
"x": 830,
"y": 732
},
{
"t": 6,
"x": 814,
"y": 728
},
{
"t": 8,
"x": 826,
"y": 728
},
{
"t": 9,
"x": 810,
"y": 728
},
{
"t": 10,
"x": 822,
"y": 728
},
{
"t": 11,
"x": 806,
"y": 724
},
{
"t": 12,
"x": 818,
"y": 724
}
],
"birth": 0,
"death": 12
},
{
"id": 52,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 994,
"y": 330
},
{
"t": 1,
"x": 986,
"y": 333
},
{
"t": 2,
"x": 982,
"y": 341
},
{
"t": 3,
"x": 974,
"y": 345
},
{
"t": 4,
"x": 970,
"y": 353
},
{
"t": 5,
"x": 962,
"y": 357
},
{
"t": 6,
"x": 955,
"y": 361
},
{
"t": 7,
"x": 947,
"y": 365
},
{
"t": 9,
"x": 939,
"y": 369
},
{
"t": 10,
"x": 931,
"y": 373
},
{
"t": 11,
"x": 923,
"y": 376
},
{
"t": 12,
"x": 916,
"y": 380
},
{
"t": 13,
"x": 908,
"y": 384
},
{
"t": 14,
"x": 900,
"y": 388
},
{
"t": 16,
"x": 892,
"y": 392
},
{
"t": 17,
"x": 884,
"y": 396
},
{
"t": 19,
"x": 873,
"y": 400
},
{
"t": 21,
"x": 865,
"y": 404
},
{
"t": 22,
"x": 857,
"y": 408
},
{
"t": 23,
"x": 849,
"y": 412
}
],
"birth": 0,
"death": 12
},
{
"id": 53,
"type": "turquoise plastic plant pot",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 994,
"y": 537
},
{
"t": 1,
"x": 994,
"y": 525
},
{
"t": 2,
"x": 990,
"y": 541
},
{
"t": 3,
"x": 990,
"y": 529
},
{
"t": 4,
"x": 990,
"y": 517
},
{
"t": 5,
"x": 986,
"y": 533
},
{
"t": 6,
"x": 986,
"y": 521
},
{
"t": 7,
"x": 986,
"y": 509
},
{
"t": 8,
"x": 982,
"y": 529
},
{
"t": 9,
"x": 982,
"y": 517
},
{
"t": 10,
"x": 982,
"y": 505
},
{
"t": 11,
"x": 978,
"y": 525
},
{
"t": 12,
"x": 978,
"y": 513
},
{
"t": 13,
"x": 978,
"y": 501
},
{
"t": 14,
"x": 974,
"y": 521
},
{
"t": 15,
"x": 974,
"y": 509
},
{
"t": 16,
"x": 974,
"y": 498
},
{
"t": 17,
"x": 970,
"y": 521
},
{
"t": 18,
"x": 970,
"y": 509
},
{
"t": 19,
"x": 970,
"y": 498
},
{
"t": 20,
"x": 966,
"y": 521
},
{
"t": 21,
"x": 966,
"y": 509
},
{
"t": 22,
"x": 966,
"y": 498
},
{
"t": 23,
"x": 962,
"y": 521
}
],
"birth": 0,
"death": 12
},
{
"id": 54,
"type": "red and green cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 9,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 9,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 306
},
{
"t": 1,
"x": 994,
"y": 314
},
{
"t": 3,
"x": 990,
"y": 322
},
{
"t": 4,
"x": 986,
"y": 330
},
{
"t": 6,
"x": 982,
"y": 337
},
{
"t": 7,
"x": 978,
"y": 345
},
{
"t": 9,
"x": null,
"y": null
}
],
"birth": 0,
"death": 12
}
],
"interactions": [
{
"id": 0,
"inputs": [
0,
14
],
"outputs": [
0,
14
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 1,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 2,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 38,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 39,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 40,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 41,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 42,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 43,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 44,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 3,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 4,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 5,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 6,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 7,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 8,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 9,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 10,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 11,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 12,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 13,
"inputs": [
20,
65535
],
"outputs": [
20,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 14,
"inputs": [
21,
65535
],
"outputs": [
21,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 15,
"inputs": [
22,
65535
],
"outputs": [
22,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 16,
"inputs": [
23,
65535
],
"outputs": [
23,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 17,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 18,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 19,
"inputs": [
26,
65535
],
"outputs": [
26,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 20,
"inputs": [
27,
65535
],
"outputs": [
27,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 21,
"inputs": [
28,
65535
],
"outputs": [
28,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 22,
"inputs": [
29,
65535
],
"outputs": [
29,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 23,
"inputs": [
31,
65535
],
"outputs": [
31,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 24,
"inputs": [
32,
65535
],
"outputs": [
32,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 25,
"inputs": [
33,
65535
],
"outputs": [
33,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 26,
"inputs": [
34,
65535
],
"outputs": [
34,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 27,
"inputs": [
35,
65535
],
"outputs": [
35,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 28,
"inputs": [
36,
65535
],
"outputs": [
36,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 29,
"inputs": [
37,
65535
],
"outputs": [
37,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 30,
"inputs": [
38,
65535
],
"outputs": [
38,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 31,
"inputs": [
39,
65535
],
"outputs": [
39,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 32,
"inputs": [
40,
65535
],
"outputs": [
40,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 33,
"inputs": [
41,
65535
],
"outputs": [
41,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 34,
"inputs": [
42,
65535
],
"outputs": [
42,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 35,
"inputs": [
43,
65535
],
"outputs": [
43,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 36,
"inputs": [
44,
65535
],
"outputs": [
44,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 37,
"inputs": [
45,
65535
],
"outputs": [
45,
65535
],
"begins_at": 0,
"ends_at": 12,
"type": "contact"
},
{
"id": 3,
"inputs": [
10,
14
],
"outputs": [
10,
14
],
"begins_at": 2,
"ends_at": 2,
"type": "collision"
},
{
"id": 4,
"inputs": [
10,
17
],
"outputs": [
10,
17
],
"begins_at": 5,
"ends_at": 5,
"type": "collision"
},
{
"id": 5,
"inputs": [
10,
14
],
"outputs": [
10,
14
],
"begins_at": 7,
"ends_at": 7,
"type": "collision"
},
{
"id": 6,
"inputs": [
10,
14
],
"outputs": [
10,
14
],
"begins_at": 8,
"ends_at": 11,
"type": "contact"
},
{
"id": 1,
"inputs": [
0,
14
],
"outputs": [
0,
14
],
"begins_at": 10,
"ends_at": 10,
"type": "collision"
},
{
"id": 2,
"inputs": [
0,
15
],
"outputs": [
0,
15
],
"begins_at": 11,
"ends_at": 12,
"type": "collision"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "yellow truck",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 40,
"y": 683
},
{
"t": 1,
"x": 44,
"y": 683
},
{
"t": 2,
"x": 50,
"y": 683
},
{
"t": 3,
"x": 55,
"y": 683
},
{
"t": 4,
"x": 60,
"y": 683
},
{
"t": 5,
"x": 66,
"y": 683
},
{
"t": 6,
"x": 72,
"y": 683
},
{
"t": 7,
"x": 92,
"y": 682
},
{
"t": 8,
"x": 96,
"y": 682
},
{
"t": 9,
"x": 90,
"y": 682
},
{
"t": 10,
"x": 102,
"y": 682
},
{
"t": 11,
"x": 105,
"y": 682
},
{
"t": 12,
"x": 108,
"y": 682
},
{
"t": 13,
"x": 114,
"y": 682
},
{
"t": 14,
"x": 120,
"y": 682
},
{
"t": 15,
"x": 126,
"y": 682
},
{
"t": 16,
"x": 133,
"y": 682
},
{
"t": 17,
"x": 139,
"y": 682
},
{
"t": 18,
"x": 156,
"y": 681
},
{
"t": 19,
"x": 159,
"y": 681
},
{
"t": 20,
"x": 163,
"y": 681
},
{
"t": 21,
"x": 166,
"y": 681
},
{
"t": 22,
"x": 172,
"y": 681
},
{
"t": 23,
"x": 179,
"y": 681
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "green sedan",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 346,
"y": 544
},
{
"t": 2,
"x": 348,
"y": 546
},
{
"t": 3,
"x": 349,
"y": 548
},
{
"t": 5,
"x": 351,
"y": 551
},
{
"t": 7,
"x": 358,
"y": 554
},
{
"t": 8,
"x": 354,
"y": 556
},
{
"t": 10,
"x": 355,
"y": 559
},
{
"t": 11,
"x": 358,
"y": 561
},
{
"t": 12,
"x": 362,
"y": 563
},
{
"t": 13,
"x": 358,
"y": 565
},
{
"t": 14,
"x": 364,
"y": 567
},
{
"t": 15,
"x": 362,
"y": 569
},
{
"t": 16,
"x": 362,
"y": 571
},
{
"t": 18,
"x": 364,
"y": 574
},
{
"t": 19,
"x": 367,
"y": 576
},
{
"t": 21,
"x": 368,
"y": 580
},
{
"t": 22,
"x": 369,
"y": 583
},
{
"t": 23,
"x": 371,
"y": 584
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "red truck",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 433,
"y": 541
},
{
"t": 1,
"x": 430,
"y": 541
},
{
"t": 3,
"x": 431,
"y": 543
},
{
"t": 7,
"x": 431,
"y": 546
},
{
"t": 9,
"x": 433,
"y": 548
},
{
"t": 13,
"x": 433,
"y": 551
},
{
"t": 16,
"x": 433,
"y": 554
},
{
"t": 19,
"x": 434,
"y": 557
},
{
"t": 22,
"x": 435,
"y": 560
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "gray wagon",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 773,
"y": 706
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "green jet",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 976,
"y": 329
},
{
"t": 1,
"x": 985,
"y": 325
},
{
"t": 2,
"x": 969,
"y": 329
},
{
"t": 3,
"x": 984,
"y": 323
},
{
"t": 5,
"x": 984,
"y": 320
},
{
"t": 6,
"x": 983,
"y": 319
},
{
"t": 7,
"x": 983,
"y": 316
},
{
"t": 8,
"x": 981,
"y": 315
},
{
"t": 9,
"x": 979,
"y": 314
},
{
"t": 10,
"x": 975,
"y": 314
},
{
"t": 11,
"x": 971,
"y": 314
},
{
"t": 12,
"x": 967,
"y": 314
},
{
"t": 13,
"x": 963,
"y": 313
},
{
"t": 14,
"x": 958,
"y": 314
},
{
"t": 15,
"x": 955,
"y": 313
},
{
"t": 16,
"x": 950,
"y": 313
},
{
"t": 17,
"x": 945,
"y": 314
},
{
"t": 18,
"x": 941,
"y": 314
},
{
"t": 19,
"x": 936,
"y": 314
},
{
"t": 20,
"x": 931,
"y": 314
},
{
"t": 21,
"x": 926,
"y": 314
},
{
"t": 22,
"x": 921,
"y": 314
},
{
"t": 23,
"x": 916,
"y": 314
}
],
"birth": 0,
"death": 24
}
],
"interactions": [
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.4450987556826269,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 102.61066453926381,
"avg_image_position_segmentation_acc": 0.4916666666666667,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.36111111111111116,
"avg_image_position_segmentation_acc_large": 0.6875,
"avg_image_position_segmentation_acc_first_frame": 0.8,
"avg_image_position_segmentation_visible_acc": 0.4916666666666667,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 0.825,
"visibility_true_acc": 0.825,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.825,
"vis_acc_from_px_true_acc": 0.825,
"vis_acc_from_px_false_acc": null,
"num_matches": 5,
"num_unmatched_pred": 50,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 5,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 5,
"avg_image_position_segmentation_acc": 5,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 3,
"avg_image_position_segmentation_acc_large": 2,
"avg_image_position_segmentation_acc_first_frame": 5,
"avg_image_position_segmentation_visible_acc": 5,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 5,
"visibility_true_acc": 5,
"visibility_false_acc": 0,
"is_in_frame_acc": 5,
"is_in_frame_true_acc": 5,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 5,
"vis_acc_from_px_true_acc": 5,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 1,
"target_id": 0,
"pred_id": 1,
"cost": 0.625,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.625,
"weighted_cost": 0.625,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 18,
"target_id": 1,
"pred_id": 18,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 50,
"target_id": 2,
"pred_id": 50,
"cost": 0.16666666666666666,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.16666666666666666,
"weighted_cost": 0.16666666666666666,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 51,
"target_id": 3,
"pred_id": 51,
"cost": 0.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 52,
"target_id": 4,
"pred_id": 52,
"cost": 1.25,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.75,
"weighted_cost": 0.75,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44,
45,
46,
47,
48,
49,
53,
54
],
"target_type_counts": {
"yellow truck": 1,
"green sedan": 1,
"red truck": 1,
"gray wagon": 1,
"green jet": 1
},
"pred_type_counts": {
"yellow plastic toy school bus": 3,
"yellow and blue plastic soccer cleat": 1,
"yellow tennis shoe": 3,
"yellow hoodie dollhouse boy": 7,
"blue and yellow basketball shoe": 1,
"white and brown leather golf shoe": 1,
"green plastic plant pot": 7,
"green and yellow cardboard ink cartridge box": 3,
"green and white striped fabric dishtowel": 3,
"blue and yellow fabric sneaker": 1,
"green wooden dollhouse bedroom furniture set": 3,
"green wooden toy train": 3,
"light yellow and dark blue cardboard whey protein box": 1,
"light blue and yellow high-top sneaker": 1,
"green Lego Android robot": 2,
"light blue cardboard argan oil treatment box": 2,
"red plastic New Super Mario Bros. Wii game case": 1,
"light grey canvas shoe": 1,
"green and white cardboard electric toothbrush box": 2,
"green and white plastic portable speaker": 1,
"light blue fabric lunch bag with butterfly print": 1,
"red and black plastic handheld game console": 2,
"white plastic supplement bottle": 1,
"red fabric chukka boot": 1,
"light grey and black sneaker": 1,
"turquoise plastic plant pot": 1,
"red and green cardboard ink cartridge box": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 5,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 2,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 1.0,
"1.0": 0.6,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 51,
"false_negatives": 1
},
"0.5": {
"true_positives": 4,
"false_positives": 51,
"false_negatives": 1
},
"0.75": {
"true_positives": 4,
"false_positives": 51,
"false_negatives": 1
},
"1.0": {
"true_positives": 4,
"false_positives": 51,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.07272727272727272,
"0.5": 0.07272727272727272,
"0.75": 0.07272727272727272,
"1.0": 0.07272727272727272
},
"visibility_recall_by_threshold": {
"0.25": 0.8,
"0.5": 0.8,
"0.75": 0.8,
"1.0": 0.8
},
"visibility_f1_by_threshold": {
"0.25": 0.13333333333333333,
"0.5": 0.13333333333333333,
"0.75": 0.13333333333333333,
"1.0": 0.13333333333333333
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 3,
"false_positives": 1,
"false_negatives": 1
},
"0.75": {
"true_positives": 3,
"false_positives": 1,
"false_negatives": 1
},
"1.0": {
"true_positives": 3,
"false_positives": 1,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.75,
"0.5": 0.75,
"0.75": 0.75,
"1.0": 0.75
},
"visibility_recall_by_threshold": {
"0.25": 0.75,
"0.5": 0.75,
"0.75": 0.75,
"1.0": 0.75
},
"visibility_f1_by_threshold": {
"0.25": 0.75,
"0.5": 0.75,
"0.75": 0.75,
"1.0": 0.75
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 50,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 50,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 50,
"false_negatives": 0
},
"1.0": {
"true_positives": 1,
"false_positives": 50,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0196078431372549,
"0.5": 0.0196078431372549,
"0.75": 0.0196078431372549,
"1.0": 0.0196078431372549
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 0.038461538461538464,
"0.5": 0.038461538461538464,
"0.75": 0.038461538461538464,
"1.0": 0.038461538461538464
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 51,
"false_negatives": 1
},
"0.5": {
"true_positives": 2,
"false_positives": 53,
"false_negatives": 3
},
"0.75": {
"true_positives": 2,
"false_positives": 53,
"false_negatives": 3
},
"1.0": {
"true_positives": 1,
"false_positives": 54,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.07272727272727272,
"0.5": 0.03636363636363636,
"0.75": 0.03636363636363636,
"1.0": 0.01818181818181818
},
"segmentation_recall_by_threshold": {
"0.25": 0.8,
"0.5": 0.4,
"0.75": 0.4,
"1.0": 0.2
},
"segmentation_f1_by_threshold": {
"0.25": 0.13333333333333333,
"0.5": 0.06666666666666667,
"0.75": 0.06666666666666667,
"1.0": 0.03333333333333333
},
"segmentation_frame_average_precision": 0.04090909090909091,
"segmentation_frame_average_recall": 0.45,
"segmentation_frame_average_f1": 0.075,
"segmentation_first_frame_counts": {
"true_positives": 4,
"false_positives": 51,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.07272727272727272,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.13559322033898305,
"segmentation_by_motion_group": {
"static": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
},
"0.75": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
},
"1.0": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 3
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.75,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.25
},
"segmentation_recall_by_threshold": {
"0.25": 0.75,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.25
},
"segmentation_f1_by_threshold": {
"0.25": 0.75,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.25
},
"segmentation_frame_average_precision": 0.5,
"segmentation_frame_average_recall": 0.5,
"segmentation_frame_average_f1": 0.5,
"segmentation_first_frame_counts": {
"true_positives": 3,
"false_positives": 1,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.75,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.8571428571428571
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 50,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 51,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 51,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 51,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0196078431372549,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.038461538461538464,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.004901960784313725,
"segmentation_frame_average_recall": 0.25,
"segmentation_frame_average_f1": 0.009615384615384616,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 50,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0196078431372549,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.038461538461538464
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 1,
"cost": 0.625,
"type_similarity": 0.5009367133381388,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 111.57214187728435,
"image_position_segmentation_acc": 0.375,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.375,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.375,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 18,
"cost": 1.0,
"type_similarity": 0.4006974900964856,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 239.91579997223255,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.125,
"visibility_true_acc": 0.125,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.125,
"vis_acc_from_px_true_acc": 0.125,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 50,
"cost": 0.16666666666666666,
"type_similarity": 0.4524496841128007,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 20.08957598898697,
"image_position_segmentation_acc": 0.8333333333333334,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.8333333333333334,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.8333333333333334,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 51,
"cost": 0.5,
"type_similarity": 0.43530806517151177,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 62.90617616832532,
"image_position_segmentation_acc": 1.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 1.0,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 1.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 4,
"pred_id": 52,
"cost": 1.25,
"type_similarity": 0.43610182569419786,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 78.56962868948987,
"image_position_segmentation_acc": 0.25,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.25,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.25,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": 0.5416666666666666,
"avg_interaction_input_object_jaccard": 0.6666666666666667,
"avg_interaction_output_object_jaccard": 0.6666666666666667,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 47,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 47,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 47,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 51,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 51,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0784313725490196,
"0.5": 0.0784313725490196,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.14545454545454545,
"0.5": 0.14545454545454545,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 47,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 49,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 49,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 49,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0784313725490196,
"0.5": 0.0392156862745098,
"0.75": 0.0392156862745098,
"1.0": 0.0392156862745098
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.14545454545454545,
"0.5": 0.07547169811320754,
"0.75": 0.07547169811320754,
"1.0": 0.07547169811320754
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 47,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 49,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 49,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 49,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0784313725490196,
"0.5": 0.0392156862745098,
"0.75": 0.0392156862745098,
"1.0": 0.0392156862745098
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.14545454545454545,
"0.5": 0.07547169811320754,
"0.75": 0.07547169811320754,
"1.0": 0.07547169811320754
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 4,
"target_id": 0,
"pred_id": 2,
"cost": 0.22916666666666674,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.22916666666666674,
"weighted_cost": 0.22916666666666674,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 17,
"target_id": 1,
"pred_id": 11,
"cost": 0.22916666666666674,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.22916666666666674,
"weighted_cost": 0.22916666666666674,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 2,
"target_id": 2,
"pred_id": 1,
"cost": 0.22916666666666674,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.22916666666666674,
"weighted_cost": 0.22916666666666674,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 6,
"target_id": 3,
"pred_id": 3,
"cost": 0.22916666666666674,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.22916666666666674,
"weighted_cost": 0.22916666666666674,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
2,
3,
4,
4,
5,
5,
6,
6,
7,
8,
9,
10,
12,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41,
42,
43,
44
],
"target_type_counts": {
"contact": 4
},
"pred_type_counts": {
"contact": 46,
"collision": 5
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 4,
"target_id": 0,
"pred_id": 2,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.22916666666666674,
"temporal_span_iou": 0.5416666666666666,
"temporal_span_giou": 0.5416666666666666,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 1,
"pred_index": 17,
"target_id": 1,
"pred_id": 11,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.22916666666666674,
"temporal_span_iou": 0.5416666666666666,
"temporal_span_giou": 0.5416666666666666,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 2,
"pred_index": 2,
"target_id": 2,
"pred_id": 1,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.22916666666666674,
"temporal_span_iou": 0.5416666666666666,
"temporal_span_giou": 0.5416666666666666,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 3,
"pred_index": 6,
"target_id": 3,
"pred_id": 3,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.22916666666666674,
"temporal_span_iou": 0.5416666666666666,
"temporal_span_giou": 0.5416666666666666,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0.5416666666666666,
"avg_interaction_input_object_jaccard": 0.6666666666666667,
"avg_interaction_output_object_jaccard": 0.6666666666666667,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 40,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 40,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 40,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 44,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 44,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.09090909090909091,
"0.5": 0.09090909090909091,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.16666666666666669,
"0.5": 0.16666666666666669,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 40,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.09090909090909091,
"0.5": 0.045454545454545456,
"0.75": 0.045454545454545456,
"1.0": 0.045454545454545456
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.16666666666666669,
"0.5": 0.08695652173913045,
"0.75": 0.08695652173913045,
"1.0": 0.08695652173913045
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 40,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.09090909090909091,
"0.5": 0.045454545454545456,
"0.75": 0.045454545454545456,
"1.0": 0.045454545454545456
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.16666666666666669,
"0.5": 0.08695652173913045,
"0.75": 0.08695652173913045,
"1.0": 0.08695652173913045
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 7,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 7,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 5,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": 0.5416666666666666,
"avg_interaction_input_object_jaccard": 0.6666666666666667,
"avg_interaction_output_object_jaccard": 0.6666666666666667,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 42,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.08695652173913043,
"0.5": 0.08695652173913043,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.16,
"0.5": 0.16,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 44,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 44,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 44,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.08695652173913043,
"0.5": 0.043478260869565216,
"0.75": 0.043478260869565216,
"1.0": 0.043478260869565216
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.16,
"0.5": 0.08333333333333333,
"0.75": 0.08333333333333333,
"1.0": 0.08333333333333333
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 44,
"false_negatives": 0
},
"0.75": {
"true_positives": 2,
"false_positives": 44,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 44,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.08695652173913043,
"0.5": 0.043478260869565216,
"0.75": 0.043478260869565216,
"1.0": 0.043478260869565216
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.16,
"0.5": 0.08333333333333333,
"0.75": 0.08333333333333333,
"1.0": 0.08333333333333333
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"contact": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}