STRIVE · 08/28/2026
Dyn generalization · test_120 · super_clevr_1174
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow school\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 110 704 52 105 700 53 262 711 54 98 696 55 242 709 56 94 696 57 91 695 58 88 694 59 85 693 60 82 692 61 79 691 62 199 715 63 200 718 64 185 717 65 179 719 66 173 721 67 172 725 68 162 726 69 156 729 70 148 731 71 144 734 72 135 732 73 130 733 74 123 733 75 116 733 76 109 733 77 108 739 78 99 736 79 92 735 80 82 730 81 78 729 82 70 721 83 66 711 84 62 709 85 62 732 86 58 734 87 54 734 88 49 735 89 42 736 90 36 708 91 35 690 92 35 686 95 36 707 97 36 703 102 35 688 107 35 694 108 35 702 109 34 688 112 34 697 113 33 689 115 33 694 116 33 689 [OBJ] 1 \"gray truck\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 424 797 1 430 793 2 437 789 3 441 783 4 447 779 5 453 776 6 458 772 7 463 768 8 469 765 9 473 759 10 476 755 11 483 753 12 489 750 13 494 747 14 498 744 15 502 740 16 507 736 17 512 733 18 516 731 19 521 728 20 525 725 21 530 722 22 533 719 23 537 716 24 540 713 25 544 710 26 531 705 27 551 705 28 537 700 29 559 700 30 562 698 31 564 694 32 569 693 33 555 688 34 575 688 35 578 685 36 580 682 37 584 680 38 571 676 39 589 676 40 593 674 41 596 672 42 597 669 43 601 668 44 604 666 45 605 664 46 608 661 47 612 660 48 614 658 49 616 656 50 618 654 51 620 652 52 623 651 53 624 648 54 627 647 55 630 646 56 631 644 57 634 643 58 637 641 59 637 639 60 641 638 61 641 635 62 644 634 63 645 632 64 648 631 65 649 629 66 651 628 67 653 626 68 655 625 69 657 624 70 658 622 71 660 621 72 662 620 73 663 618 74 666 617 75 667 616 77 669 614 78 672 613 79 671 609 80 673 608 81 675 607 83 679 606 84 680 605 86 681 601 87 683 601 89 686 599 91 689 597 93 691 595 95 693 592 97 695 590 99 698 589 100 699 588 102 701 585 104 703 583 106 704 581 107 706 581 108 705 579 110 708 578 111 710 577 113 712 576 115 713 574 117 714 572 119 716 571 [OBJ] 2 \"green jet\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 543 896 6 542 898 13 541 899 15 540 897 17 538 898 19 537 897 21 536 894 22 537 904 23 508 869 25 507 868 27 505 866 30 503 863 32 501 860 34 500 858 36 498 856 38 497 854 39 495 852 41 493 849 42 492 847 44 490 845 46 488 842 47 487 841 48 508 863 49 484 836 50 483 835 51 504 857 52 480 832 53 501 855 54 500 852 55 498 851 56 497 850 57 495 848 58 494 845 59 493 844 60 490 842 62 487 839 63 486 836 64 485 834 65 483 832 66 481 830 67 462 805 68 460 804 69 459 802 70 458 800 71 474 820 72 455 797 73 454 795 74 452 792 75 451 790 76 449 789 77 448 786 79 445 783 80 443 780 81 441 778 82 441 776 83 439 775 84 437 772 85 436 770 86 434 768 87 433 766 88 431 765 89 430 763 90 428 760 91 427 758 92 426 756 93 423 754 94 422 752 95 420 750 96 419 748 97 418 746 98 416 744 99 415 742 100 413 740 101 412 738 102 410 735 104 408 732 105 406 730 106 404 727 107 402 725 108 401 723 109 400 721 110 398 719 112 396 716 113 394 714 114 392 711 115 391 709 116 390 707 117 388 705 118 387 703 119 385 701 [OBJ] 3 \"cyan jet\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 957 194 1 946 195 2 935 197 3 925 198 4 914 200 5 903 203 6 893 205 7 882 209 8 872 213 9 861 217 10 851 220 11 840 225 12 829 230 13 819 235 14 808 242 15 798 248 16 787 254 17 777 260 18 767 268 19 757 274 20 747 282 21 737 290 22 726 298 23 717 306 24 707 316 25 697 325 26 687 334 27 676 345 28 667 355 29 657 366 30 648 376 31 637 388 32 628 399 33 619 409 34 609 422 35 599 434 36 590 447 37 580 460 38 571 473 39 562 485 40 553 500 41 544 514 42 535 527 43 526 542 44 517 556 45 508 571 46 499 586 47 491 601 48 482 617 49 473 632 50 465 649 51 458 665 52 454 675 53 450 677 54 448 676 55 444 679 56 441 679 57 437 681 58 344 659 59 337 656 60 330 652 61 325 649 62 319 645 63 408 677 64 402 676 65 397 675 66 393 672 67 387 672 68 382 671 69 276 613 70 270 609 72 267 611 73 265 613 74 263 614 75 261 616 76 258 617 77 257 620 78 251 620 79 248 622 80 244 625 81 241 628 82 238 630 83 235 633 84 233 638 85 237 643 86 233 647 87 273 666 88 267 667 89 260 668 90 255 666 91 250 664 92 244 663 93 237 663 94 233 663 95 226 665 96 219 668 97 216 672 99 213 671 100 211 672 101 209 671 102 206 671 103 205 670 104 201 668 105 199 667 106 196 665 107 184 652 108 180 650 109 180 654 110 175 651 111 173 652 112 169 652 113 167 653 114 164 653 115 198 664 116 160 656 117 156 656 118 154 657 119 153 659 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 52 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 4 \"contact\" 0 119 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 5 \"contact\" 0 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 0 \"collision\" 51 51 [IN] 0 3 [OUT] 0 3 [INTERACT] 6 \"collision\" 53 53 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 2 \"collision\" 58 58 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 3 \"contact\" 59 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 7 \"collision\" 60 60 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 8 \"collision\" 72 72 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 9 \"collision\" 75 75 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 10 \"contact\" 76 90 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 11 \"collision\" 97 97 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 12 \"contact\" 98 119 [IN] 3 65535 [OUT] 3 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_full/super_clevr_1174/frames.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow plastic candy bar\" 0 24 [VIS] 0 true 17 false [IN-FRAME] 0 true 17 false [PX] 0 5 697 1 1 689 17 null null [OBJ] 1 \"yellow plastic candy bar\" 0 24 [VIS] 0 true 16 false [IN-FRAME] 0 true 16 false [PX] 0 5 716 1 5 740 2 5 759 3 1 744 4 1 759 5 1 771 6 1 783 7 1 794 8 1 806 9 1 818 10 1 830 11 1 841 12 1 853 13 1 865 14 1 876 15 1 888 16 null null [OBJ] 2 \"wooden toy tableware set\" 0 24 [VIS] 0 true 16 false [IN-FRAME] 0 true 16 false [PX] 0 1 830 3 1 841 5 1 853 7 1 865 9 1 876 11 1 888 13 1 900 15 null null [OBJ] 3 \"wooden honey dipper\" 0 24 [VIS] 0 true 13 false [IN-FRAME] 0 true 13 false [PX] 0 1 982 2 5 990 6 9 998 13 null null [OBJ] 4 \"yellow plastic candy bar\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 1 994 1 null null [OBJ] 5 \"wooden honey dipper\" 0 24 [VIS] 0 true 23 false [IN-FRAME] 0 true 23 false [PX] 0 9 658 2 9 669 5 5 658 7 5 669 10 5 681 14 1 673 17 1 685 20 1 697 23 null null [OBJ] 6 \"wooden honey dipper\" 0 24 [VIS] 0 true 23 false [IN-FRAME] 0 true 23 false [PX] 0 17 669 1 13 662 7 13 673 12 9 681 16 5 689 21 1 697 23 null null [OBJ] 7 \"wooden honey dipper\" 0 24 [VIS] 0 true 23 false [IN-FRAME] 0 true 23 false [PX] 0 21 693 4 17 701 9 13 708 13 9 716 17 5 724 20 1 732 22 1 744 23 null null [OBJ] 8 \"wooden honey dipper\" 0 24 [VIS] 0 true 23 false [IN-FRAME] 0 true 23 false [PX] 0 33 681 1 29 673 4 25 681 8 21 689 12 17 697 16 13 705 19 9 712 22 1 716 23 null null [OBJ] 9 \"yellow wooden toy roller\" 0 24 [VIS] 0 true 23 false [IN-FRAME] 0 true 23 false [PX] 0 33 716 2 29 708 6 25 716 10 21 724 14 17 732 18 13 740 22 1 740 23 null null [OBJ] 10 \"wooden honey dipper\" 0 24 [VIS] 0 true 23 false [IN-FRAME] 0 true 23 false [PX] 0 37 697 2 33 705 5 29 712 8 25 720 12 21 732 16 17 740 19 9 744 22 1 748 23 null null [OBJ] 11 \"yellow wooden toy roller\" 0 24 [VIS] 0 true 23 false [IN-FRAME] 0 true 23 false [PX] 0 44 736 1 41 728 4 37 720 8 33 712 11 29 705 14 25 697 17 21 689 20 17 681 23 null null [OBJ] 12 \"yellow plastic chocolate powder canister\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 91 716 1 115 720 2 103 720 4 115 724 5 103 724 7 115 728 8 103 728 10 115 732 11 103 732 13 115 736 14 103 736 16 115 740 17 103 740 19 115 744 20 103 744 22 115 748 23 103 748 [OBJ] 13 \"yellow plastic chocolate powder canister\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 263 716 3 271 720 7 263 716 10 271 720 11 263 716 15 271 720 16 263 716 20 275 720 21 263 716 [OBJ] 14 \"green wooden toy train\" 0 24 [VIS] 0 true 11 false [IN-FRAME] 0 true [PX] 0 310 650 11 null null [OBJ] 15 \"white cardboard box of fondant\" 0 24 [VIS] 0 true 11 false [IN-FRAME] 0 true [PX] 0 310 681 1 322 685 2 314 681 4 322 685 5 310 681 11 null null [OBJ] 16 \"white cardboard ink cartridge box\" 0 24 [VIS] 0 true 11 false [IN-FRAME] 0 true [PX] 0 314 666 2 322 669 7 314 666 11 null null [OBJ] 17 \"white cardboard ink cartridge box\" 0 24 [VIS] 0 true 12 false [IN-FRAME] 0 true [PX] 0 318 677 3 326 681 9 318 677 12 null null [OBJ] 18 \"white cardboard ink cartridge box\" 0 24 [VIS] 0 true 12 false [IN-FRAME] 0 true [PX] 0 326 689 2 333 693 3 322 689 7 330 693 12 null null [OBJ] 19 \"green ink cartridge cardboard box\" 0 24 [VIS] 0 true 12 false [IN-FRAME] 0 true [PX] 0 330 666 2 337 669 7 341 662 12 null null [OBJ] 20 \"green ink cartridge cardboard box\" 0 24 [VIS] 0 true 12 false [IN-FRAME] 0 true [PX] 0 337 681 1 345 685 4 333 677 11 345 681 12 null null [OBJ] 21 \"white cardboard ink cartridge box\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 408 775 2 416 779 3 423 775 4 431 771 5 439 767 6 451 763 7 458 759 8 470 755 9 478 751 10 486 748 11 494 744 12 505 740 13 513 736 14 521 732 15 533 728 16 541 724 17 548 720 18 560 716 19 568 712 20 576 708 21 583 705 22 591 701 23 599 697 [OBJ] 22 \"green wooden mushroom pounding toy\" 0 24 [VIS] 0 true 2 false [IN-FRAME] 0 true [PX] 0 470 833 2 null null [OBJ] 23 \"green wooden toy train\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 525 884 4 513 884 6 505 880 9 498 873 11 490 865 13 482 857 14 494 861 15 478 849 16 486 845 17 474 837 18 482 833 19 470 826 21 466 818 22 470 810 23 458 802 [OBJ] 24 \"white and red high-top sneaker\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 552 861 3 544 857 6 537 853 9 529 849 12 521 845 14 513 841 16 505 837 18 498 833 20 494 826 22 486 822 23 478 818 [OBJ] 25 \"dark grey and light grey plastic monster action figure\" 0 24 [VIS] 0 true 11 false [IN-FRAME] 0 true [PX] 0 568 857 3 560 853 6 552 849 9 544 845 11 null null [OBJ] 26 \"dark grey and purple plastic Shredder action figure\" 0 24 [VIS] 0 true 5 false [IN-FRAME] 0 true [PX] 0 607 849 1 603 857 5 null null [OBJ] 27 \"dark grey leather boat shoe\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 732 646 4 654 654 7 642 654 8 634 658 9 615 662 10 599 662 11 587 662 12 576 666 13 564 666 14 552 669 15 541 669 16 529 673 17 517 673 18 505 673 19 494 673 20 482 673 21 470 673 22 458 673 23 447 673 [OBJ] 28 \"teal and pink high-top sneaker\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 736 650 1 null null [OBJ] 29 \"light grey and black sneaker\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 794 666 1 null null [OBJ] 30 \"green and red plastic ninja turtle action figure\" 0 24 [VIS] 0 true 2 false [IN-FRAME] 0 true [PX] 0 810 662 2 null null [OBJ] 31 \"green and red plastic ninja turtle action figure\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 849 654 1 null null [OBJ] 32 \"teal and lime running shoe\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 853 662 1 null null [OBJ] 33 \"light grey fabric chukka boot\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 919 658 1 912 662 2 923 658 3 912 658 4 923 658 5 912 658 6 900 658 7 912 654 8 900 654 9 888 654 10 876 654 11 888 650 12 876 650 13 865 650 14 853 650 15 865 646 16 853 646 17 841 646 18 830 646 19 841 642 20 830 642 21 818 642 22 806 642 23 794 642 [OBJS-END] [INTERACTIONS-START] [INTERACT] 23 \"contact\" 0 1 [IN] 26 27 [OUT] 26 27 [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 31 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 39 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 40 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 41 \"contact\" 0 23 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 42 \"contact\" 0 23 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 43 \"contact\" 0 23 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 44 \"contact\" 0 23 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 45 \"contact\" 0 23 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 5 \"contact\" 0 23 [IN] 13 14 [OUT] 13 14 [INTERACT] 6 \"contact\" 0 23 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 7 \"contact\" 0 23 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 8 \"contact\" 0 23 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 9 \"contact\" 0 23 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 10 \"contact\" 0 23 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 11 \"contact\" 0 23 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 12 \"contact\" 0 23 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 13 \"contact\" 0 23 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 14 \"contact\" 0 23 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 15 \"contact\" 0 23 [IN] 22 65535 [OUT] 22 65535 [INTERACT] 16 \"contact\" 0 23 [IN] 23 65535 [OUT] 23 65535 [INTERACT] 24 \"contact\" 0 23 [IN] 26 65535 [OUT] 26 65535 [INTERACT] 25 \"contact\" 0 23 [IN] 27 65535 [OUT] 27 65535 [INTERACT] 26 \"contact\" 0 23 [IN] 28 65535 [OUT] 28 65535 [INTERACT] 27 \"contact\" 0 23 [IN] 29 65535 [OUT] 29 65535 [INTERACT] 28 \"contact\" 0 23 [IN] 30 65535 [OUT] 30 65535 [INTERACT] 29 \"contact\" 0 23 [IN] 31 65535 [OUT] 31 65535 [INTERACT] 30 \"contact\" 0 23 [IN] 32 65535 [OUT] 32 65535 [INTERACT] 32 \"collision\" 1 1 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 33 \"contact\" 2 23 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 19 \"collision\" 5 5 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 20 \"collision\" 7 7 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 21 \"collision\" 8 8 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 22 \"contact\" 9 23 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 36 \"collision\" 14 14 [IN] 26 27 [OUT] 26 27 [INTERACT] 37 \"contact\" 15 17 [IN] 26 27 [OUT] 26 27 [INTERACT] 17 \"collision\" 18 18 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 38 \"collision\" 19 19 [IN] 26 27 [OUT] 26 27 [INTERACT] 18 \"contact\" 19 23 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 34 \"collision\" 20 20 [IN] 26 27 [OUT] 26 27 [INTERACT] 35 \"contact\" 21 23 [IN] 26 27 [OUT] 26 27 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow school\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 110 704 52 105 700 53 262 711 54 98 696 55 242 709 56 94 696 57 91 695 58 88 694 59 85 693 60 82 692 61 79 691 62 199 715 63 200 718 64 185 717 65 179 719 66 173 721 67 172 725 68 162 726 69 156 729 70 148 731 71 144 734 72 135 732 73 130 733 74 123 733 75 116 733 76 109 733 77 108 739 78 99 736 79 92 735 80 82 730 81 78 729 82 70 721 83 66 711 84 62 709 85 62 732 86 58 734 87 54 734 88 49 735 89 42 736 90 36 708 91 35 690 92 35 686 95 36 707 97 36 703 102 35 688 107 35 694 108 35 702 109 34 688 112 34 697 113 33 689 115 33 694 116 33 689 [OBJ] 1 \"gray truck\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 424 797 1 430 793 2 437 789 3 441 783 4 447 779 5 453 776 6 458 772 7 463 768 8 469 765 9 473 759 10 476 755 11 483 753 12 489 750 13 494 747 14 498 744 15 502 740 16 507 736 17 512 733 18 516 731 19 521 728 20 525 725 21 530 722 22 533 719 23 537 716 24 540 713 25 544 710 26 531 705 27 551 705 28 537 700 29 559 700 30 562 698 31 564 694 32 569 693 33 555 688 34 575 688 35 578 685 36 580 682 37 584 680 38 571 676 39 589 676 40 593 674 41 596 672 42 597 669 43 601 668 44 604 666 45 605 664 46 608 661 47 612 660 48 614 658 49 616 656 50 618 654 51 620 652 52 623 651 53 624 648 54 627 647 55 630 646 56 631 644 57 634 643 58 637 641 59 637 639 60 641 638 61 641 635 62 644 634 63 645 632 64 648 631 65 649 629 66 651 628 67 653 626 68 655 625 69 657 624 70 658 622 71 660 621 72 662 620 73 663 618 74 666 617 75 667 616 77 669 614 78 672 613 79 671 609 80 673 608 81 675 607 83 679 606 84 680 605 86 681 601 87 683 601 89 686 599 91 689 597 93 691 595 95 693 592 97 695 590 99 698 589 100 699 588 102 701 585 104 703 583 106 704 581 107 706 581 108 705 579 110 708 578 111 710 577 113 712 576 115 713 574 117 714 572 119 716 571 [OBJ] 2 \"green jet\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 543 896 6 542 898 13 541 899 15 540 897 17 538 898 19 537 897 21 536 894 22 537 904 23 508 869 25 507 868 27 505 866 30 503 863 32 501 860 34 500 858 36 498 856 38 497 854 39 495 852 41 493 849 42 492 847 44 490 845 46 488 842 47 487 841 48 508 863 49 484 836 50 483 835 51 504 857 52 480 832 53 501 855 54 500 852 55 498 851 56 497 850 57 495 848 58 494 845 59 493 844 60 490 842 62 487 839 63 486 836 64 485 834 65 483 832 66 481 830 67 462 805 68 460 804 69 459 802 70 458 800 71 474 820 72 455 797 73 454 795 74 452 792 75 451 790 76 449 789 77 448 786 79 445 783 80 443 780 81 441 778 82 441 776 83 439 775 84 437 772 85 436 770 86 434 768 87 433 766 88 431 765 89 430 763 90 428 760 91 427 758 92 426 756 93 423 754 94 422 752 95 420 750 96 419 748 97 418 746 98 416 744 99 415 742 100 413 740 101 412 738 102 410 735 104 408 732 105 406 730 106 404 727 107 402 725 108 401 723 109 400 721 110 398 719 112 396 716 113 394 714 114 392 711 115 391 709 116 390 707 117 388 705 118 387 703 119 385 701 [OBJ] 3 \"cyan jet\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 957 194 1 946 195 2 935 197 3 925 198 4 914 200 5 903 203 6 893 205 7 882 209 8 872 213 9 861 217 10 851 220 11 840 225 12 829 230 13 819 235 14 808 242 15 798 248 16 787 254 17 777 260 18 767 268 19 757 274 20 747 282 21 737 290 22 726 298 23 717 306 24 707 316 25 697 325 26 687 334 27 676 345 28 667 355 29 657 366 30 648 376 31 637 388 32 628 399 33 619 409 34 609 422 35 599 434 36 590 447 37 580 460 38 571 473 39 562 485 40 553 500 41 544 514 42 535 527 43 526 542 44 517 556 45 508 571 46 499 586 47 491 601 48 482 617 49 473 632 50 465 649 51 458 665 52 454 675 53 450 677 54 448 676 55 444 679 56 441 679 57 437 681 58 344 659 59 337 656 60 330 652 61 325 649 62 319 645 63 408 677 64 402 676 65 397 675 66 393 672 67 387 672 68 382 671 69 276 613 70 270 609 72 267 611 73 265 613 74 263 614 75 261 616 76 258 617 77 257 620 78 251 620 79 248 622 80 244 625 81 241 628 82 238 630 83 235 633 84 233 638 85 237 643 86 233 647 87 273 666 88 267 667 89 260 668 90 255 666 91 250 664 92 244 663 93 237 663 94 233 663 95 226 665 96 219 668 97 216 672 99 213 671 100 211 672 101 209 671 102 206 671 103 205 670 104 201 668 105 199 667 106 196 665 107 184 652 108 180 650 109 180 654 110 175 651 111 173 652 112 169 652 113 167 653 114 164 653 115 198 664 116 160 656 117 156 656 118 154 657 119 153 659 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 52 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 4 \"contact\" 0 119 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 5 \"contact\" 0 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 0 \"collision\" 51 51 [IN] 0 3 [OUT] 0 3 [INTERACT] 6 \"collision\" 53 53 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 2 \"collision\" 58 58 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 3 \"contact\" 59 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 7 \"collision\" 60 60 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 8 \"collision\" 72 72 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 9 \"collision\" 75 75 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 10 \"contact\" 76 90 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 11 \"collision\" 97 97 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 12 \"contact\" 98 119 [IN] 3 65535 [OUT] 3 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "yellow plastic candy bar",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 17,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 17,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 5,
"y": 697
},
{
"t": 1,
"x": 1,
"y": 689
},
{
"t": 17,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "yellow plastic candy bar",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 16,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 16,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 5,
"y": 716
},
{
"t": 1,
"x": 5,
"y": 740
},
{
"t": 2,
"x": 5,
"y": 759
},
{
"t": 3,
"x": 1,
"y": 744
},
{
"t": 4,
"x": 1,
"y": 759
},
{
"t": 5,
"x": 1,
"y": 771
},
{
"t": 6,
"x": 1,
"y": 783
},
{
"t": 7,
"x": 1,
"y": 794
},
{
"t": 8,
"x": 1,
"y": 806
},
{
"t": 9,
"x": 1,
"y": 818
},
{
"t": 10,
"x": 1,
"y": 830
},
{
"t": 11,
"x": 1,
"y": 841
},
{
"t": 12,
"x": 1,
"y": 853
},
{
"t": 13,
"x": 1,
"y": 865
},
{
"t": 14,
"x": 1,
"y": 876
},
{
"t": 15,
"x": 1,
"y": 888
},
{
"t": 16,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "wooden toy tableware set",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 16,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 16,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 830
},
{
"t": 3,
"x": 1,
"y": 841
},
{
"t": 5,
"x": 1,
"y": 853
},
{
"t": 7,
"x": 1,
"y": 865
},
{
"t": 9,
"x": 1,
"y": 876
},
{
"t": 11,
"x": 1,
"y": 888
},
{
"t": 13,
"x": 1,
"y": 900
},
{
"t": 15,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "wooden honey dipper",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 13,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 13,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 982
},
{
"t": 2,
"x": 5,
"y": 990
},
{
"t": 6,
"x": 9,
"y": 998
},
{
"t": 13,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "yellow plastic candy bar",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 1,
"y": 994
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 5,
"type": "wooden honey dipper",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 9,
"y": 658
},
{
"t": 2,
"x": 9,
"y": 669
},
{
"t": 5,
"x": 5,
"y": 658
},
{
"t": 7,
"x": 5,
"y": 669
},
{
"t": 10,
"x": 5,
"y": 681
},
{
"t": 14,
"x": 1,
"y": 673
},
{
"t": 17,
"x": 1,
"y": 685
},
{
"t": 20,
"x": 1,
"y": 697
},
{
"t": 23,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 6,
"type": "wooden honey dipper",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 17,
"y": 669
},
{
"t": 1,
"x": 13,
"y": 662
},
{
"t": 7,
"x": 13,
"y": 673
},
{
"t": 12,
"x": 9,
"y": 681
},
{
"t": 16,
"x": 5,
"y": 689
},
{
"t": 21,
"x": 1,
"y": 697
},
{
"t": 23,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 7,
"type": "wooden honey dipper",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 21,
"y": 693
},
{
"t": 4,
"x": 17,
"y": 701
},
{
"t": 9,
"x": 13,
"y": 708
},
{
"t": 13,
"x": 9,
"y": 716
},
{
"t": 17,
"x": 5,
"y": 724
},
{
"t": 20,
"x": 1,
"y": 732
},
{
"t": 22,
"x": 1,
"y": 744
},
{
"t": 23,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 8,
"type": "wooden honey dipper",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 33,
"y": 681
},
{
"t": 1,
"x": 29,
"y": 673
},
{
"t": 4,
"x": 25,
"y": 681
},
{
"t": 8,
"x": 21,
"y": 689
},
{
"t": 12,
"x": 17,
"y": 697
},
{
"t": 16,
"x": 13,
"y": 705
},
{
"t": 19,
"x": 9,
"y": 712
},
{
"t": 22,
"x": 1,
"y": 716
},
{
"t": 23,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 9,
"type": "yellow wooden toy roller",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 33,
"y": 716
},
{
"t": 2,
"x": 29,
"y": 708
},
{
"t": 6,
"x": 25,
"y": 716
},
{
"t": 10,
"x": 21,
"y": 724
},
{
"t": 14,
"x": 17,
"y": 732
},
{
"t": 18,
"x": 13,
"y": 740
},
{
"t": 22,
"x": 1,
"y": 740
},
{
"t": 23,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 10,
"type": "wooden honey dipper",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 37,
"y": 697
},
{
"t": 2,
"x": 33,
"y": 705
},
{
"t": 5,
"x": 29,
"y": 712
},
{
"t": 8,
"x": 25,
"y": 720
},
{
"t": 12,
"x": 21,
"y": 732
},
{
"t": 16,
"x": 17,
"y": 740
},
{
"t": 19,
"x": 9,
"y": 744
},
{
"t": 22,
"x": 1,
"y": 748
},
{
"t": 23,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 11,
"type": "yellow wooden toy roller",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 44,
"y": 736
},
{
"t": 1,
"x": 41,
"y": 728
},
{
"t": 4,
"x": 37,
"y": 720
},
{
"t": 8,
"x": 33,
"y": 712
},
{
"t": 11,
"x": 29,
"y": 705
},
{
"t": 14,
"x": 25,
"y": 697
},
{
"t": 17,
"x": 21,
"y": 689
},
{
"t": 20,
"x": 17,
"y": 681
},
{
"t": 23,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 12,
"type": "yellow plastic chocolate powder canister",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 91,
"y": 716
},
{
"t": 1,
"x": 115,
"y": 720
},
{
"t": 2,
"x": 103,
"y": 720
},
{
"t": 4,
"x": 115,
"y": 724
},
{
"t": 5,
"x": 103,
"y": 724
},
{
"t": 7,
"x": 115,
"y": 728
},
{
"t": 8,
"x": 103,
"y": 728
},
{
"t": 10,
"x": 115,
"y": 732
},
{
"t": 11,
"x": 103,
"y": 732
},
{
"t": 13,
"x": 115,
"y": 736
},
{
"t": 14,
"x": 103,
"y": 736
},
{
"t": 16,
"x": 115,
"y": 740
},
{
"t": 17,
"x": 103,
"y": 740
},
{
"t": 19,
"x": 115,
"y": 744
},
{
"t": 20,
"x": 103,
"y": 744
},
{
"t": 22,
"x": 115,
"y": 748
},
{
"t": 23,
"x": 103,
"y": 748
}
],
"birth": 0,
"death": 24
},
{
"id": 13,
"type": "yellow plastic chocolate powder canister",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 263,
"y": 716
},
{
"t": 3,
"x": 271,
"y": 720
},
{
"t": 7,
"x": 263,
"y": 716
},
{
"t": 10,
"x": 271,
"y": 720
},
{
"t": 11,
"x": 263,
"y": 716
},
{
"t": 15,
"x": 271,
"y": 720
},
{
"t": 16,
"x": 263,
"y": 716
},
{
"t": 20,
"x": 275,
"y": 720
},
{
"t": 21,
"x": 263,
"y": 716
}
],
"birth": 0,
"death": 24
},
{
"id": 14,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 310,
"y": 650
},
{
"t": 11,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 15,
"type": "white cardboard box of fondant",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 310,
"y": 681
},
{
"t": 1,
"x": 322,
"y": 685
},
{
"t": 2,
"x": 314,
"y": 681
},
{
"t": 4,
"x": 322,
"y": 685
},
{
"t": 5,
"x": 310,
"y": 681
},
{
"t": 11,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 16,
"type": "white cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 314,
"y": 666
},
{
"t": 2,
"x": 322,
"y": 669
},
{
"t": 7,
"x": 314,
"y": 666
},
{
"t": 11,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 17,
"type": "white cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 12,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 318,
"y": 677
},
{
"t": 3,
"x": 326,
"y": 681
},
{
"t": 9,
"x": 318,
"y": 677
},
{
"t": 12,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 18,
"type": "white cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 12,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 326,
"y": 689
},
{
"t": 2,
"x": 333,
"y": 693
},
{
"t": 3,
"x": 322,
"y": 689
},
{
"t": 7,
"x": 330,
"y": 693
},
{
"t": 12,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 19,
"type": "green ink cartridge cardboard box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 12,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 330,
"y": 666
},
{
"t": 2,
"x": 337,
"y": 669
},
{
"t": 7,
"x": 341,
"y": 662
},
{
"t": 12,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 20,
"type": "green ink cartridge cardboard box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 12,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 337,
"y": 681
},
{
"t": 1,
"x": 345,
"y": 685
},
{
"t": 4,
"x": 333,
"y": 677
},
{
"t": 11,
"x": 345,
"y": 681
},
{
"t": 12,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 21,
"type": "white cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 408,
"y": 775
},
{
"t": 2,
"x": 416,
"y": 779
},
{
"t": 3,
"x": 423,
"y": 775
},
{
"t": 4,
"x": 431,
"y": 771
},
{
"t": 5,
"x": 439,
"y": 767
},
{
"t": 6,
"x": 451,
"y": 763
},
{
"t": 7,
"x": 458,
"y": 759
},
{
"t": 8,
"x": 470,
"y": 755
},
{
"t": 9,
"x": 478,
"y": 751
},
{
"t": 10,
"x": 486,
"y": 748
},
{
"t": 11,
"x": 494,
"y": 744
},
{
"t": 12,
"x": 505,
"y": 740
},
{
"t": 13,
"x": 513,
"y": 736
},
{
"t": 14,
"x": 521,
"y": 732
},
{
"t": 15,
"x": 533,
"y": 728
},
{
"t": 16,
"x": 541,
"y": 724
},
{
"t": 17,
"x": 548,
"y": 720
},
{
"t": 18,
"x": 560,
"y": 716
},
{
"t": 19,
"x": 568,
"y": 712
},
{
"t": 20,
"x": 576,
"y": 708
},
{
"t": 21,
"x": 583,
"y": 705
},
{
"t": 22,
"x": 591,
"y": 701
},
{
"t": 23,
"x": 599,
"y": 697
}
],
"birth": 0,
"death": 24
},
{
"id": 22,
"type": "green wooden mushroom pounding toy",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 470,
"y": 833
},
{
"t": 2,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 23,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 525,
"y": 884
},
{
"t": 4,
"x": 513,
"y": 884
},
{
"t": 6,
"x": 505,
"y": 880
},
{
"t": 9,
"x": 498,
"y": 873
},
{
"t": 11,
"x": 490,
"y": 865
},
{
"t": 13,
"x": 482,
"y": 857
},
{
"t": 14,
"x": 494,
"y": 861
},
{
"t": 15,
"x": 478,
"y": 849
},
{
"t": 16,
"x": 486,
"y": 845
},
{
"t": 17,
"x": 474,
"y": 837
},
{
"t": 18,
"x": 482,
"y": 833
},
{
"t": 19,
"x": 470,
"y": 826
},
{
"t": 21,
"x": 466,
"y": 818
},
{
"t": 22,
"x": 470,
"y": 810
},
{
"t": 23,
"x": 458,
"y": 802
}
],
"birth": 0,
"death": 24
},
{
"id": 24,
"type": "white and red high-top sneaker",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 552,
"y": 861
},
{
"t": 3,
"x": 544,
"y": 857
},
{
"t": 6,
"x": 537,
"y": 853
},
{
"t": 9,
"x": 529,
"y": 849
},
{
"t": 12,
"x": 521,
"y": 845
},
{
"t": 14,
"x": 513,
"y": 841
},
{
"t": 16,
"x": 505,
"y": 837
},
{
"t": 18,
"x": 498,
"y": 833
},
{
"t": 20,
"x": 494,
"y": 826
},
{
"t": 22,
"x": 486,
"y": 822
},
{
"t": 23,
"x": 478,
"y": 818
}
],
"birth": 0,
"death": 24
},
{
"id": 25,
"type": "dark grey and light grey plastic monster action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 568,
"y": 857
},
{
"t": 3,
"x": 560,
"y": 853
},
{
"t": 6,
"x": 552,
"y": 849
},
{
"t": 9,
"x": 544,
"y": 845
},
{
"t": 11,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 26,
"type": "dark grey and purple plastic Shredder action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 5,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 607,
"y": 849
},
{
"t": 1,
"x": 603,
"y": 857
},
{
"t": 5,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 27,
"type": "dark grey leather boat shoe",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 732,
"y": 646
},
{
"t": 4,
"x": 654,
"y": 654
},
{
"t": 7,
"x": 642,
"y": 654
},
{
"t": 8,
"x": 634,
"y": 658
},
{
"t": 9,
"x": 615,
"y": 662
},
{
"t": 10,
"x": 599,
"y": 662
},
{
"t": 11,
"x": 587,
"y": 662
},
{
"t": 12,
"x": 576,
"y": 666
},
{
"t": 13,
"x": 564,
"y": 666
},
{
"t": 14,
"x": 552,
"y": 669
},
{
"t": 15,
"x": 541,
"y": 669
},
{
"t": 16,
"x": 529,
"y": 673
},
{
"t": 17,
"x": 517,
"y": 673
},
{
"t": 18,
"x": 505,
"y": 673
},
{
"t": 19,
"x": 494,
"y": 673
},
{
"t": 20,
"x": 482,
"y": 673
},
{
"t": 21,
"x": 470,
"y": 673
},
{
"t": 22,
"x": 458,
"y": 673
},
{
"t": 23,
"x": 447,
"y": 673
}
],
"birth": 0,
"death": 24
},
{
"id": 28,
"type": "teal and pink high-top sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 736,
"y": 650
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 29,
"type": "light grey and black sneaker",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 794,
"y": 666
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 30,
"type": "green and red plastic ninja turtle action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 810,
"y": 662
},
{
"t": 2,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 31,
"type": "green and red plastic ninja turtle action figure",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 849,
"y": 654
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 32,
"type": "teal and lime running shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 853,
"y": 662
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 33,
"type": "light grey fabric chukka boot",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 919,
"y": 658
},
{
"t": 1,
"x": 912,
"y": 662
},
{
"t": 2,
"x": 923,
"y": 658
},
{
"t": 3,
"x": 912,
"y": 658
},
{
"t": 4,
"x": 923,
"y": 658
},
{
"t": 5,
"x": 912,
"y": 658
},
{
"t": 6,
"x": 900,
"y": 658
},
{
"t": 7,
"x": 912,
"y": 654
},
{
"t": 8,
"x": 900,
"y": 654
},
{
"t": 9,
"x": 888,
"y": 654
},
{
"t": 10,
"x": 876,
"y": 654
},
{
"t": 11,
"x": 888,
"y": 650
},
{
"t": 12,
"x": 876,
"y": 650
},
{
"t": 13,
"x": 865,
"y": 650
},
{
"t": 14,
"x": 853,
"y": 650
},
{
"t": 15,
"x": 865,
"y": 646
},
{
"t": 16,
"x": 853,
"y": 646
},
{
"t": 17,
"x": 841,
"y": 646
},
{
"t": 18,
"x": 830,
"y": 646
},
{
"t": 19,
"x": 841,
"y": 642
},
{
"t": 20,
"x": 830,
"y": 642
},
{
"t": 21,
"x": 818,
"y": 642
},
{
"t": 22,
"x": 806,
"y": 642
},
{
"t": 23,
"x": 794,
"y": 642
}
],
"birth": 0,
"death": 24
}
],
"interactions": [
{
"id": 23,
"inputs": [
26,
27
],
"outputs": [
26,
27
],
"begins_at": 0,
"ends_at": 1,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 31,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 39,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 40,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 41,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 42,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 43,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 44,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 45,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 4,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 5,
"inputs": [
13,
14
],
"outputs": [
13,
14
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 6,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 7,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 8,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 9,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 10,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 11,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 12,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 13,
"inputs": [
20,
65535
],
"outputs": [
20,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 14,
"inputs": [
21,
65535
],
"outputs": [
21,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 15,
"inputs": [
22,
65535
],
"outputs": [
22,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 16,
"inputs": [
23,
65535
],
"outputs": [
23,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 24,
"inputs": [
26,
65535
],
"outputs": [
26,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 25,
"inputs": [
27,
65535
],
"outputs": [
27,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 26,
"inputs": [
28,
65535
],
"outputs": [
28,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 27,
"inputs": [
29,
65535
],
"outputs": [
29,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 28,
"inputs": [
30,
65535
],
"outputs": [
30,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 29,
"inputs": [
31,
65535
],
"outputs": [
31,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 30,
"inputs": [
32,
65535
],
"outputs": [
32,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 32,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 1,
"ends_at": 1,
"type": "collision"
},
{
"id": 33,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 2,
"ends_at": 23,
"type": "contact"
},
{
"id": 19,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 5,
"ends_at": 5,
"type": "collision"
},
{
"id": 20,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 7,
"ends_at": 7,
"type": "collision"
},
{
"id": 21,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 8,
"ends_at": 8,
"type": "collision"
},
{
"id": 22,
"inputs": [
25,
65535
],
"outputs": [
25,
65535
],
"begins_at": 9,
"ends_at": 23,
"type": "contact"
},
{
"id": 36,
"inputs": [
26,
27
],
"outputs": [
26,
27
],
"begins_at": 14,
"ends_at": 14,
"type": "collision"
},
{
"id": 37,
"inputs": [
26,
27
],
"outputs": [
26,
27
],
"begins_at": 15,
"ends_at": 17,
"type": "contact"
},
{
"id": 17,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 18,
"ends_at": 18,
"type": "collision"
},
{
"id": 38,
"inputs": [
26,
27
],
"outputs": [
26,
27
],
"begins_at": 19,
"ends_at": 19,
"type": "collision"
},
{
"id": 18,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 19,
"ends_at": 23,
"type": "contact"
},
{
"id": 34,
"inputs": [
26,
27
],
"outputs": [
26,
27
],
"begins_at": 20,
"ends_at": 20,
"type": "collision"
},
{
"id": 35,
"inputs": [
26,
27
],
"outputs": [
26,
27
],
"begins_at": 21,
"ends_at": 23,
"type": "contact"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "yellow school",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 110,
"y": 704
},
{
"t": 52,
"x": 105,
"y": 700
},
{
"t": 53,
"x": 262,
"y": 711
},
{
"t": 54,
"x": 98,
"y": 696
},
{
"t": 55,
"x": 242,
"y": 709
},
{
"t": 56,
"x": 94,
"y": 696
},
{
"t": 57,
"x": 91,
"y": 695
},
{
"t": 58,
"x": 88,
"y": 694
},
{
"t": 59,
"x": 85,
"y": 693
},
{
"t": 60,
"x": 82,
"y": 692
},
{
"t": 61,
"x": 79,
"y": 691
},
{
"t": 62,
"x": 199,
"y": 715
},
{
"t": 63,
"x": 200,
"y": 718
},
{
"t": 64,
"x": 185,
"y": 717
},
{
"t": 65,
"x": 179,
"y": 719
},
{
"t": 66,
"x": 173,
"y": 721
},
{
"t": 67,
"x": 172,
"y": 725
},
{
"t": 68,
"x": 162,
"y": 726
},
{
"t": 69,
"x": 156,
"y": 729
},
{
"t": 70,
"x": 148,
"y": 731
},
{
"t": 71,
"x": 144,
"y": 734
},
{
"t": 72,
"x": 135,
"y": 732
},
{
"t": 73,
"x": 130,
"y": 733
},
{
"t": 74,
"x": 123,
"y": 733
},
{
"t": 75,
"x": 116,
"y": 733
},
{
"t": 76,
"x": 109,
"y": 733
},
{
"t": 77,
"x": 108,
"y": 739
},
{
"t": 78,
"x": 99,
"y": 736
},
{
"t": 79,
"x": 92,
"y": 735
},
{
"t": 80,
"x": 82,
"y": 730
},
{
"t": 81,
"x": 78,
"y": 729
},
{
"t": 82,
"x": 70,
"y": 721
},
{
"t": 83,
"x": 66,
"y": 711
},
{
"t": 84,
"x": 62,
"y": 709
},
{
"t": 85,
"x": 62,
"y": 732
},
{
"t": 86,
"x": 58,
"y": 734
},
{
"t": 87,
"x": 54,
"y": 734
},
{
"t": 88,
"x": 49,
"y": 735
},
{
"t": 89,
"x": 42,
"y": 736
},
{
"t": 90,
"x": 36,
"y": 708
},
{
"t": 91,
"x": 35,
"y": 690
},
{
"t": 92,
"x": 35,
"y": 686
},
{
"t": 95,
"x": 36,
"y": 707
},
{
"t": 97,
"x": 36,
"y": 703
},
{
"t": 102,
"x": 35,
"y": 688
},
{
"t": 107,
"x": 35,
"y": 694
},
{
"t": 108,
"x": 35,
"y": 702
},
{
"t": 109,
"x": 34,
"y": 688
},
{
"t": 112,
"x": 34,
"y": 697
},
{
"t": 113,
"x": 33,
"y": 689
},
{
"t": 115,
"x": 33,
"y": 694
},
{
"t": 116,
"x": 33,
"y": 689
}
],
"birth": 0,
"death": 120
},
{
"id": 1,
"type": "gray truck",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 424,
"y": 797
},
{
"t": 1,
"x": 430,
"y": 793
},
{
"t": 2,
"x": 437,
"y": 789
},
{
"t": 3,
"x": 441,
"y": 783
},
{
"t": 4,
"x": 447,
"y": 779
},
{
"t": 5,
"x": 453,
"y": 776
},
{
"t": 6,
"x": 458,
"y": 772
},
{
"t": 7,
"x": 463,
"y": 768
},
{
"t": 8,
"x": 469,
"y": 765
},
{
"t": 9,
"x": 473,
"y": 759
},
{
"t": 10,
"x": 476,
"y": 755
},
{
"t": 11,
"x": 483,
"y": 753
},
{
"t": 12,
"x": 489,
"y": 750
},
{
"t": 13,
"x": 494,
"y": 747
},
{
"t": 14,
"x": 498,
"y": 744
},
{
"t": 15,
"x": 502,
"y": 740
},
{
"t": 16,
"x": 507,
"y": 736
},
{
"t": 17,
"x": 512,
"y": 733
},
{
"t": 18,
"x": 516,
"y": 731
},
{
"t": 19,
"x": 521,
"y": 728
},
{
"t": 20,
"x": 525,
"y": 725
},
{
"t": 21,
"x": 530,
"y": 722
},
{
"t": 22,
"x": 533,
"y": 719
},
{
"t": 23,
"x": 537,
"y": 716
},
{
"t": 24,
"x": 540,
"y": 713
},
{
"t": 25,
"x": 544,
"y": 710
},
{
"t": 26,
"x": 531,
"y": 705
},
{
"t": 27,
"x": 551,
"y": 705
},
{
"t": 28,
"x": 537,
"y": 700
},
{
"t": 29,
"x": 559,
"y": 700
},
{
"t": 30,
"x": 562,
"y": 698
},
{
"t": 31,
"x": 564,
"y": 694
},
{
"t": 32,
"x": 569,
"y": 693
},
{
"t": 33,
"x": 555,
"y": 688
},
{
"t": 34,
"x": 575,
"y": 688
},
{
"t": 35,
"x": 578,
"y": 685
},
{
"t": 36,
"x": 580,
"y": 682
},
{
"t": 37,
"x": 584,
"y": 680
},
{
"t": 38,
"x": 571,
"y": 676
},
{
"t": 39,
"x": 589,
"y": 676
},
{
"t": 40,
"x": 593,
"y": 674
},
{
"t": 41,
"x": 596,
"y": 672
},
{
"t": 42,
"x": 597,
"y": 669
},
{
"t": 43,
"x": 601,
"y": 668
},
{
"t": 44,
"x": 604,
"y": 666
},
{
"t": 45,
"x": 605,
"y": 664
},
{
"t": 46,
"x": 608,
"y": 661
},
{
"t": 47,
"x": 612,
"y": 660
},
{
"t": 48,
"x": 614,
"y": 658
},
{
"t": 49,
"x": 616,
"y": 656
},
{
"t": 50,
"x": 618,
"y": 654
},
{
"t": 51,
"x": 620,
"y": 652
},
{
"t": 52,
"x": 623,
"y": 651
},
{
"t": 53,
"x": 624,
"y": 648
},
{
"t": 54,
"x": 627,
"y": 647
},
{
"t": 55,
"x": 630,
"y": 646
},
{
"t": 56,
"x": 631,
"y": 644
},
{
"t": 57,
"x": 634,
"y": 643
},
{
"t": 58,
"x": 637,
"y": 641
},
{
"t": 59,
"x": 637,
"y": 639
},
{
"t": 60,
"x": 641,
"y": 638
},
{
"t": 61,
"x": 641,
"y": 635
},
{
"t": 62,
"x": 644,
"y": 634
},
{
"t": 63,
"x": 645,
"y": 632
},
{
"t": 64,
"x": 648,
"y": 631
},
{
"t": 65,
"x": 649,
"y": 629
},
{
"t": 66,
"x": 651,
"y": 628
},
{
"t": 67,
"x": 653,
"y": 626
},
{
"t": 68,
"x": 655,
"y": 625
},
{
"t": 69,
"x": 657,
"y": 624
},
{
"t": 70,
"x": 658,
"y": 622
},
{
"t": 71,
"x": 660,
"y": 621
},
{
"t": 72,
"x": 662,
"y": 620
},
{
"t": 73,
"x": 663,
"y": 618
},
{
"t": 74,
"x": 666,
"y": 617
},
{
"t": 75,
"x": 667,
"y": 616
},
{
"t": 77,
"x": 669,
"y": 614
},
{
"t": 78,
"x": 672,
"y": 613
},
{
"t": 79,
"x": 671,
"y": 609
},
{
"t": 80,
"x": 673,
"y": 608
},
{
"t": 81,
"x": 675,
"y": 607
},
{
"t": 83,
"x": 679,
"y": 606
},
{
"t": 84,
"x": 680,
"y": 605
},
{
"t": 86,
"x": 681,
"y": 601
},
{
"t": 87,
"x": 683,
"y": 601
},
{
"t": 89,
"x": 686,
"y": 599
},
{
"t": 91,
"x": 689,
"y": 597
},
{
"t": 93,
"x": 691,
"y": 595
},
{
"t": 95,
"x": 693,
"y": 592
},
{
"t": 97,
"x": 695,
"y": 590
},
{
"t": 99,
"x": 698,
"y": 589
},
{
"t": 100,
"x": 699,
"y": 588
},
{
"t": 102,
"x": 701,
"y": 585
},
{
"t": 104,
"x": 703,
"y": 583
},
{
"t": 106,
"x": 704,
"y": 581
},
{
"t": 107,
"x": 706,
"y": 581
},
{
"t": 108,
"x": 705,
"y": 579
},
{
"t": 110,
"x": 708,
"y": 578
},
{
"t": 111,
"x": 710,
"y": 577
},
{
"t": 113,
"x": 712,
"y": 576
},
{
"t": 115,
"x": 713,
"y": 574
},
{
"t": 117,
"x": 714,
"y": 572
},
{
"t": 119,
"x": 716,
"y": 571
}
],
"birth": 0,
"death": 120
},
{
"id": 2,
"type": "green jet",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 543,
"y": 896
},
{
"t": 6,
"x": 542,
"y": 898
},
{
"t": 13,
"x": 541,
"y": 899
},
{
"t": 15,
"x": 540,
"y": 897
},
{
"t": 17,
"x": 538,
"y": 898
},
{
"t": 19,
"x": 537,
"y": 897
},
{
"t": 21,
"x": 536,
"y": 894
},
{
"t": 22,
"x": 537,
"y": 904
},
{
"t": 23,
"x": 508,
"y": 869
},
{
"t": 25,
"x": 507,
"y": 868
},
{
"t": 27,
"x": 505,
"y": 866
},
{
"t": 30,
"x": 503,
"y": 863
},
{
"t": 32,
"x": 501,
"y": 860
},
{
"t": 34,
"x": 500,
"y": 858
},
{
"t": 36,
"x": 498,
"y": 856
},
{
"t": 38,
"x": 497,
"y": 854
},
{
"t": 39,
"x": 495,
"y": 852
},
{
"t": 41,
"x": 493,
"y": 849
},
{
"t": 42,
"x": 492,
"y": 847
},
{
"t": 44,
"x": 490,
"y": 845
},
{
"t": 46,
"x": 488,
"y": 842
},
{
"t": 47,
"x": 487,
"y": 841
},
{
"t": 48,
"x": 508,
"y": 863
},
{
"t": 49,
"x": 484,
"y": 836
},
{
"t": 50,
"x": 483,
"y": 835
},
{
"t": 51,
"x": 504,
"y": 857
},
{
"t": 52,
"x": 480,
"y": 832
},
{
"t": 53,
"x": 501,
"y": 855
},
{
"t": 54,
"x": 500,
"y": 852
},
{
"t": 55,
"x": 498,
"y": 851
},
{
"t": 56,
"x": 497,
"y": 850
},
{
"t": 57,
"x": 495,
"y": 848
},
{
"t": 58,
"x": 494,
"y": 845
},
{
"t": 59,
"x": 493,
"y": 844
},
{
"t": 60,
"x": 490,
"y": 842
},
{
"t": 62,
"x": 487,
"y": 839
},
{
"t": 63,
"x": 486,
"y": 836
},
{
"t": 64,
"x": 485,
"y": 834
},
{
"t": 65,
"x": 483,
"y": 832
},
{
"t": 66,
"x": 481,
"y": 830
},
{
"t": 67,
"x": 462,
"y": 805
},
{
"t": 68,
"x": 460,
"y": 804
},
{
"t": 69,
"x": 459,
"y": 802
},
{
"t": 70,
"x": 458,
"y": 800
},
{
"t": 71,
"x": 474,
"y": 820
},
{
"t": 72,
"x": 455,
"y": 797
},
{
"t": 73,
"x": 454,
"y": 795
},
{
"t": 74,
"x": 452,
"y": 792
},
{
"t": 75,
"x": 451,
"y": 790
},
{
"t": 76,
"x": 449,
"y": 789
},
{
"t": 77,
"x": 448,
"y": 786
},
{
"t": 79,
"x": 445,
"y": 783
},
{
"t": 80,
"x": 443,
"y": 780
},
{
"t": 81,
"x": 441,
"y": 778
},
{
"t": 82,
"x": 441,
"y": 776
},
{
"t": 83,
"x": 439,
"y": 775
},
{
"t": 84,
"x": 437,
"y": 772
},
{
"t": 85,
"x": 436,
"y": 770
},
{
"t": 86,
"x": 434,
"y": 768
},
{
"t": 87,
"x": 433,
"y": 766
},
{
"t": 88,
"x": 431,
"y": 765
},
{
"t": 89,
"x": 430,
"y": 763
},
{
"t": 90,
"x": 428,
"y": 760
},
{
"t": 91,
"x": 427,
"y": 758
},
{
"t": 92,
"x": 426,
"y": 756
},
{
"t": 93,
"x": 423,
"y": 754
},
{
"t": 94,
"x": 422,
"y": 752
},
{
"t": 95,
"x": 420,
"y": 750
},
{
"t": 96,
"x": 419,
"y": 748
},
{
"t": 97,
"x": 418,
"y": 746
},
{
"t": 98,
"x": 416,
"y": 744
},
{
"t": 99,
"x": 415,
"y": 742
},
{
"t": 100,
"x": 413,
"y": 740
},
{
"t": 101,
"x": 412,
"y": 738
},
{
"t": 102,
"x": 410,
"y": 735
},
{
"t": 104,
"x": 408,
"y": 732
},
{
"t": 105,
"x": 406,
"y": 730
},
{
"t": 106,
"x": 404,
"y": 727
},
{
"t": 107,
"x": 402,
"y": 725
},
{
"t": 108,
"x": 401,
"y": 723
},
{
"t": 109,
"x": 400,
"y": 721
},
{
"t": 110,
"x": 398,
"y": 719
},
{
"t": 112,
"x": 396,
"y": 716
},
{
"t": 113,
"x": 394,
"y": 714
},
{
"t": 114,
"x": 392,
"y": 711
},
{
"t": 115,
"x": 391,
"y": 709
},
{
"t": 116,
"x": 390,
"y": 707
},
{
"t": 117,
"x": 388,
"y": 705
},
{
"t": 118,
"x": 387,
"y": 703
},
{
"t": 119,
"x": 385,
"y": 701
}
],
"birth": 0,
"death": 120
},
{
"id": 3,
"type": "cyan jet",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 957,
"y": 194
},
{
"t": 1,
"x": 946,
"y": 195
},
{
"t": 2,
"x": 935,
"y": 197
},
{
"t": 3,
"x": 925,
"y": 198
},
{
"t": 4,
"x": 914,
"y": 200
},
{
"t": 5,
"x": 903,
"y": 203
},
{
"t": 6,
"x": 893,
"y": 205
},
{
"t": 7,
"x": 882,
"y": 209
},
{
"t": 8,
"x": 872,
"y": 213
},
{
"t": 9,
"x": 861,
"y": 217
},
{
"t": 10,
"x": 851,
"y": 220
},
{
"t": 11,
"x": 840,
"y": 225
},
{
"t": 12,
"x": 829,
"y": 230
},
{
"t": 13,
"x": 819,
"y": 235
},
{
"t": 14,
"x": 808,
"y": 242
},
{
"t": 15,
"x": 798,
"y": 248
},
{
"t": 16,
"x": 787,
"y": 254
},
{
"t": 17,
"x": 777,
"y": 260
},
{
"t": 18,
"x": 767,
"y": 268
},
{
"t": 19,
"x": 757,
"y": 274
},
{
"t": 20,
"x": 747,
"y": 282
},
{
"t": 21,
"x": 737,
"y": 290
},
{
"t": 22,
"x": 726,
"y": 298
},
{
"t": 23,
"x": 717,
"y": 306
},
{
"t": 24,
"x": 707,
"y": 316
},
{
"t": 25,
"x": 697,
"y": 325
},
{
"t": 26,
"x": 687,
"y": 334
},
{
"t": 27,
"x": 676,
"y": 345
},
{
"t": 28,
"x": 667,
"y": 355
},
{
"t": 29,
"x": 657,
"y": 366
},
{
"t": 30,
"x": 648,
"y": 376
},
{
"t": 31,
"x": 637,
"y": 388
},
{
"t": 32,
"x": 628,
"y": 399
},
{
"t": 33,
"x": 619,
"y": 409
},
{
"t": 34,
"x": 609,
"y": 422
},
{
"t": 35,
"x": 599,
"y": 434
},
{
"t": 36,
"x": 590,
"y": 447
},
{
"t": 37,
"x": 580,
"y": 460
},
{
"t": 38,
"x": 571,
"y": 473
},
{
"t": 39,
"x": 562,
"y": 485
},
{
"t": 40,
"x": 553,
"y": 500
},
{
"t": 41,
"x": 544,
"y": 514
},
{
"t": 42,
"x": 535,
"y": 527
},
{
"t": 43,
"x": 526,
"y": 542
},
{
"t": 44,
"x": 517,
"y": 556
},
{
"t": 45,
"x": 508,
"y": 571
},
{
"t": 46,
"x": 499,
"y": 586
},
{
"t": 47,
"x": 491,
"y": 601
},
{
"t": 48,
"x": 482,
"y": 617
},
{
"t": 49,
"x": 473,
"y": 632
},
{
"t": 50,
"x": 465,
"y": 649
},
{
"t": 51,
"x": 458,
"y": 665
},
{
"t": 52,
"x": 454,
"y": 675
},
{
"t": 53,
"x": 450,
"y": 677
},
{
"t": 54,
"x": 448,
"y": 676
},
{
"t": 55,
"x": 444,
"y": 679
},
{
"t": 56,
"x": 441,
"y": 679
},
{
"t": 57,
"x": 437,
"y": 681
},
{
"t": 58,
"x": 344,
"y": 659
},
{
"t": 59,
"x": 337,
"y": 656
},
{
"t": 60,
"x": 330,
"y": 652
},
{
"t": 61,
"x": 325,
"y": 649
},
{
"t": 62,
"x": 319,
"y": 645
},
{
"t": 63,
"x": 408,
"y": 677
},
{
"t": 64,
"x": 402,
"y": 676
},
{
"t": 65,
"x": 397,
"y": 675
},
{
"t": 66,
"x": 393,
"y": 672
},
{
"t": 67,
"x": 387,
"y": 672
},
{
"t": 68,
"x": 382,
"y": 671
},
{
"t": 69,
"x": 276,
"y": 613
},
{
"t": 70,
"x": 270,
"y": 609
},
{
"t": 72,
"x": 267,
"y": 611
},
{
"t": 73,
"x": 265,
"y": 613
},
{
"t": 74,
"x": 263,
"y": 614
},
{
"t": 75,
"x": 261,
"y": 616
},
{
"t": 76,
"x": 258,
"y": 617
},
{
"t": 77,
"x": 257,
"y": 620
},
{
"t": 78,
"x": 251,
"y": 620
},
{
"t": 79,
"x": 248,
"y": 622
},
{
"t": 80,
"x": 244,
"y": 625
},
{
"t": 81,
"x": 241,
"y": 628
},
{
"t": 82,
"x": 238,
"y": 630
},
{
"t": 83,
"x": 235,
"y": 633
},
{
"t": 84,
"x": 233,
"y": 638
},
{
"t": 85,
"x": 237,
"y": 643
},
{
"t": 86,
"x": 233,
"y": 647
},
{
"t": 87,
"x": 273,
"y": 666
},
{
"t": 88,
"x": 267,
"y": 667
},
{
"t": 89,
"x": 260,
"y": 668
},
{
"t": 90,
"x": 255,
"y": 666
},
{
"t": 91,
"x": 250,
"y": 664
},
{
"t": 92,
"x": 244,
"y": 663
},
{
"t": 93,
"x": 237,
"y": 663
},
{
"t": 94,
"x": 233,
"y": 663
},
{
"t": 95,
"x": 226,
"y": 665
},
{
"t": 96,
"x": 219,
"y": 668
},
{
"t": 97,
"x": 216,
"y": 672
},
{
"t": 99,
"x": 213,
"y": 671
},
{
"t": 100,
"x": 211,
"y": 672
},
{
"t": 101,
"x": 209,
"y": 671
},
{
"t": 102,
"x": 206,
"y": 671
},
{
"t": 103,
"x": 205,
"y": 670
},
{
"t": 104,
"x": 201,
"y": 668
},
{
"t": 105,
"x": 199,
"y": 667
},
{
"t": 106,
"x": 196,
"y": 665
},
{
"t": 107,
"x": 184,
"y": 652
},
{
"t": 108,
"x": 180,
"y": 650
},
{
"t": 109,
"x": 180,
"y": 654
},
{
"t": 110,
"x": 175,
"y": 651
},
{
"t": 111,
"x": 173,
"y": 652
},
{
"t": 112,
"x": 169,
"y": 652
},
{
"t": 113,
"x": 167,
"y": 653
},
{
"t": 114,
"x": 164,
"y": 653
},
{
"t": 115,
"x": 198,
"y": 664
},
{
"t": 116,
"x": 160,
"y": 656
},
{
"t": 117,
"x": 156,
"y": 656
},
{
"t": 118,
"x": 154,
"y": 657
},
{
"t": 119,
"x": 153,
"y": 659
}
],
"birth": 0,
"death": 120
}
],
"interactions": [
{
"id": 1,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 52,
"type": "contact"
},
{
"id": 4,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 5,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
3
],
"outputs": [
0,
3
],
"begins_at": 51,
"ends_at": 51,
"type": "collision"
},
{
"id": 6,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 53,
"ends_at": 53,
"type": "collision"
},
{
"id": 2,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 58,
"ends_at": 58,
"type": "collision"
},
{
"id": 3,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 59,
"ends_at": 119,
"type": "contact"
},
{
"id": 7,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 60,
"ends_at": 60,
"type": "collision"
},
{
"id": 8,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 72,
"ends_at": 72,
"type": "collision"
},
{
"id": 9,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 75,
"ends_at": 75,
"type": "collision"
},
{
"id": 10,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 76,
"ends_at": 90,
"type": "contact"
},
{
"id": 11,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 97,
"ends_at": 97,
"type": "collision"
},
{
"id": 12,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 98,
"ends_at": 119,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.3838260792941223,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 275.4212693898943,
"avg_image_position_segmentation_acc": 0.31666666666666665,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.0,
"avg_image_position_segmentation_acc_large": 0.6333333333333333,
"avg_image_position_segmentation_acc_first_frame": 0.5,
"avg_image_position_segmentation_visible_acc": 0.31666666666666665,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 0.50625,
"visibility_true_acc": 0.50625,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.50625,
"vis_acc_from_px_true_acc": 0.50625,
"vis_acc_from_px_false_acc": null,
"num_matches": 4,
"num_unmatched_pred": 30,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 4,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 4,
"avg_image_position_segmentation_acc": 4,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 2,
"avg_image_position_segmentation_acc_large": 2,
"avg_image_position_segmentation_acc_first_frame": 4,
"avg_image_position_segmentation_visible_acc": 4,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 4,
"visibility_true_acc": 4,
"visibility_false_acc": 0,
"is_in_frame_acc": 4,
"is_in_frame_true_acc": 4,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 4,
"vis_acc_from_px_true_acc": 4,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 12,
"target_id": 0,
"pred_id": 12,
"cost": 0.8833333333333333,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.38333333333333336,
"weighted_cost": 0.38333333333333336,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 29,
"target_id": 1,
"pred_id": 29,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 23,
"target_id": 2,
"pred_id": 23,
"cost": 0.35,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.35,
"weighted_cost": 0.35,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 30,
"target_id": 3,
"pred_id": 30,
"cost": 1.5,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
13,
14,
15,
16,
17,
18,
19,
20,
21,
22,
24,
25,
26,
27,
28,
31,
32,
33
],
"target_type_counts": {
"yellow school": 1,
"gray truck": 1,
"green jet": 1,
"cyan jet": 1
},
"pred_type_counts": {
"yellow plastic candy bar": 3,
"wooden toy tableware set": 1,
"wooden honey dipper": 6,
"yellow wooden toy roller": 2,
"yellow plastic chocolate powder canister": 2,
"green wooden toy train": 2,
"white cardboard box of fondant": 1,
"white cardboard ink cartridge box": 4,
"green ink cartridge cardboard box": 2,
"green wooden mushroom pounding toy": 1,
"white and red high-top sneaker": 1,
"dark grey and light grey plastic monster action figure": 1,
"dark grey and purple plastic Shredder action figure": 1,
"dark grey leather boat shoe": 1,
"teal and pink high-top sneaker": 1,
"light grey and black sneaker": 1,
"green and red plastic ninja turtle action figure": 2,
"teal and lime running shoe": 1,
"light grey fabric chukka boot": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 120
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 3,
"false_positives": 1,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 0.75,
"1.0": 0.5,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 32,
"false_negatives": 2
},
"0.5": {
"true_positives": 2,
"false_positives": 32,
"false_negatives": 2
},
"0.75": {
"true_positives": 2,
"false_positives": 32,
"false_negatives": 2
},
"1.0": {
"true_positives": 2,
"false_positives": 32,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 0.058823529411764705,
"0.5": 0.058823529411764705,
"0.75": 0.058823529411764705,
"1.0": 0.058823529411764705
},
"visibility_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
},
"visibility_f1_by_threshold": {
"0.25": 0.10526315789473684,
"0.5": 0.10526315789473684,
"0.75": 0.10526315789473684,
"1.0": 0.10526315789473684
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.75": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"1.0": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
},
"visibility_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
},
"visibility_f1_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
}
},
"dynamic": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.75": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"1.0": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
},
"visibility_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
},
"visibility_f1_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.5,
"1.0": 0.5
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 32,
"false_negatives": 2
},
"0.5": {
"true_positives": 2,
"false_positives": 32,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 34,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 34,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.058823529411764705,
"0.5": 0.058823529411764705,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.10526315789473684,
"0.5": 0.10526315789473684,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.029411764705882353,
"segmentation_frame_average_recall": 0.25,
"segmentation_frame_average_f1": 0.05263157894736842,
"segmentation_first_frame_counts": {
"true_positives": 2,
"false_positives": 32,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.058823529411764705,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.1111111111111111,
"segmentation_by_motion_group": {
"static": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 2
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.25,
"segmentation_frame_average_recall": 0.25,
"segmentation_frame_average_f1": 0.25,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.5,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.6666666666666666
},
"dynamic": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.5": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 2,
"false_negatives": 2
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.5,
"0.5": 0.5,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.25,
"segmentation_frame_average_recall": 0.25,
"segmentation_frame_average_f1": 0.25,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 1,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.5,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.6666666666666666
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 30,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 12,
"cost": 0.8833333333333333,
"type_similarity": 0.2846333025828356,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 62.202861656702005,
"image_position_segmentation_acc": 0.6166666666666667,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.6166666666666667,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.6166666666666667,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 29,
"cost": 1.0,
"type_similarity": 0.5159292751607654,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 489.62156666245835,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.008333333333333333,
"visibility_true_acc": 0.008333333333333333,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.008333333333333333,
"vis_acc_from_px_true_acc": 0.008333333333333333,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 23,
"cost": 0.35,
"type_similarity": 0.43610183224766974,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 66.33661589370939,
"image_position_segmentation_acc": 0.65,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.65,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.65,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 30,
"cost": 1.5,
"type_similarity": 0.29863990718521854,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 483.5240333467075,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.016666666666666666,
"visibility_true_acc": 0.016666666666666666,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.016666666666666666,
"vis_acc_from_px_true_acc": 0.016666666666666666,
"vis_acc_from_px_false_acc": null
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": -0.5642073127142976,
"avg_interaction_input_object_jaccard": 0.46153846153846145,
"avg_interaction_output_object_jaccard": 0.46153846153846145,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 13,
"num_unmatched_pred_interactions": 33,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 45,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 46,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.021739130434782608,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.042553191489361694,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 10,
"false_positives": 36,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.21739130434782608,
"0.5": 0.08695652173913043,
"0.75": 0.08695652173913043,
"1.0": 0.08695652173913043
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.3571428571428571,
"0.5": 0.16,
"0.75": 0.16,
"1.0": 0.16
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 10,
"false_positives": 36,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 42,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.21739130434782608,
"0.5": 0.08695652173913043,
"0.75": 0.08695652173913043,
"1.0": 0.08695652173913043
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.3571428571428571,
"0.5": 0.16,
"0.75": 0.16,
"1.0": 0.16
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 13,
"avg_interaction_input_object_jaccard": 13,
"avg_interaction_output_object_jaccard": 13,
"avg_interaction_type_exact_match_acc": 13
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 34,
"target_id": 0,
"pred_id": 34,
"cost": 1.46875,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.96875,
"weighted_cost": 0.96875,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 4,
"target_id": 1,
"pred_id": 4,
"cost": 0.2735849056603774,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.2735849056603774,
"weighted_cost": 0.2735849056603774,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 17,
"target_id": 2,
"pred_id": 17,
"cost": 1.475609756097561,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.975609756097561,
"weighted_cost": 0.975609756097561,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 1,
"target_id": 3,
"pred_id": 1,
"cost": 1.1458333333333335,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.6458333333333334,
"weighted_cost": 0.6458333333333334,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 27,
"target_id": 4,
"pred_id": 27,
"cost": 0.4,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.4,
"weighted_cost": 0.4,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 5,
"pred_index": 16,
"target_id": 5,
"pred_id": 16,
"cost": 0.4,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.4,
"weighted_cost": 0.4,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 6,
"pred_index": 38,
"target_id": 6,
"pred_id": 38,
"cost": 1.4714285714285715,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9714285714285714,
"weighted_cost": 0.9714285714285714,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 7,
"pred_index": 36,
"target_id": 7,
"pred_id": 36,
"cost": 1.4787234042553192,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9787234042553192,
"weighted_cost": 0.9787234042553192,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 8,
"pred_index": 21,
"target_id": 8,
"pred_id": 21,
"cost": 1.4846153846153847,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9846153846153847,
"weighted_cost": 0.9846153846153847,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 9,
"pred_index": 20,
"target_id": 9,
"pred_id": 20,
"cost": 1.4855072463768115,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9855072463768115,
"weighted_cost": 0.9855072463768115,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 10,
"pred_index": 28,
"target_id": 10,
"pred_id": 28,
"cost": 0.7857142857142857,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7857142857142857,
"weighted_cost": 0.7857142857142857,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 11,
"pred_index": 19,
"target_id": 11,
"pred_id": 19,
"cost": 1.489247311827957,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.989247311827957,
"weighted_cost": 0.989247311827957,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 12,
"pred_index": 2,
"target_id": 12,
"pred_id": 2,
"cost": 1.3083333333333333,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8083333333333333,
"weighted_cost": 0.8083333333333333,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
3,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
18,
22,
23,
24,
25,
26,
29,
30,
31,
32,
33,
35,
37,
39,
40,
41,
42,
43,
44,
45
],
"target_type_counts": {
"collision": 7,
"contact": 6
},
"pred_type_counts": {
"contact": 38,
"collision": 8
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 34,
"target_id": 0,
"pred_id": 34,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.46875,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9375,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 1,
"pred_index": 4,
"target_id": 1,
"pred_id": 4,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.2735849056603774,
"temporal_span_iou": 0.4528301886792453,
"temporal_span_giou": 0.4528301886792453,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 2,
"pred_index": 17,
"target_id": 2,
"pred_id": 17,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.475609756097561,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9512195121951219,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 3,
"pred_index": 1,
"target_id": 3,
"pred_id": 1,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.1458333333333335,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.2916666666666667,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 4,
"pred_index": 27,
"target_id": 4,
"pred_id": 27,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.4,
"temporal_span_iou": 0.2,
"temporal_span_giou": 0.2,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 5,
"pred_index": 16,
"target_id": 5,
"pred_id": 16,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.4,
"temporal_span_iou": 0.2,
"temporal_span_giou": 0.2,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 6,
"pred_index": 38,
"target_id": 6,
"pred_id": 38,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4714285714285715,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9428571428571428,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 7,
"pred_index": 36,
"target_id": 7,
"pred_id": 36,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4787234042553192,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9574468085106383,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 8,
"pred_index": 21,
"target_id": 8,
"pred_id": 21,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4846153846153847,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9692307692307692,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 9,
"pred_index": 20,
"target_id": 9,
"pred_id": 20,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4855072463768115,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9710144927536232,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 10,
"pred_index": 28,
"target_id": 10,
"pred_id": 28,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.7857142857142857,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.5714285714285714,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 11,
"pred_index": 19,
"target_id": 11,
"pred_id": 19,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.489247311827957,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.978494623655914,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 12,
"pred_index": 2,
"target_id": 12,
"pred_id": 2,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.3083333333333333,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.6166666666666667,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": -0.5330995887738225,
"avg_interaction_input_object_jaccard": 0.49999999999999994,
"avg_interaction_output_object_jaccard": 0.49999999999999994,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 12,
"num_unmatched_pred_interactions": 29,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 40,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 41,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.024390243902439025,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.047619047619047616,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 10,
"false_positives": 31,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 37,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 37,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 37,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.24390243902439024,
"0.5": 0.0975609756097561,
"0.75": 0.0975609756097561,
"1.0": 0.0975609756097561
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.39215686274509803,
"0.5": 0.17777777777777776,
"0.75": 0.17777777777777776,
"1.0": 0.17777777777777776
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 10,
"false_positives": 31,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 37,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 37,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 37,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.24390243902439024,
"0.5": 0.0975609756097561,
"0.75": 0.0975609756097561,
"1.0": 0.0975609756097561
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.39215686274509803,
"0.5": 0.17777777777777776,
"0.75": 0.17777777777777776,
"1.0": 0.17777777777777776
}
},
"object_object": {
"avg_interaction_temporal_span_giou": -0.9375,
"avg_interaction_input_object_jaccard": 0.0,
"avg_interaction_output_object_jaccard": 0.0,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 1,
"num_unmatched_pred_interactions": 4,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 12,
"avg_interaction_input_object_jaccard": 12,
"avg_interaction_output_object_jaccard": 12,
"avg_interaction_type_exact_match_acc": 12
},
"object_object": {
"avg_interaction_temporal_span_giou": 1,
"avg_interaction_input_object_jaccard": 1,
"avg_interaction_output_object_jaccard": 1,
"avg_interaction_type_exact_match_acc": 1
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": -0.95825190702903,
"avg_interaction_input_object_jaccard": 0.19047619047619047,
"avg_interaction_output_object_jaccard": 0.19047619047619047,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 7,
"num_unmatched_pred_interactions": 1,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 4,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.6666666666666666,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 4,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.5,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.6666666666666666,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": -0.10448861934710992,
"avg_interaction_input_object_jaccard": 0.7777777777777777,
"avg_interaction_output_object_jaccard": 0.7777777777777777,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 6,
"num_unmatched_pred_interactions": 32,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 37,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 38,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 38,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 38,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.02631578947368421,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.05128205128205127,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 6,
"false_positives": 32,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 34,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 34,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 34,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.15789473684210525,
"0.5": 0.10526315789473684,
"0.75": 0.10526315789473684,
"1.0": 0.10526315789473684
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.2727272727272727,
"0.5": 0.1904761904761905,
"0.75": 0.1904761904761905,
"1.0": 0.1904761904761905
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 6,
"false_positives": 32,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 34,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 34,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 34,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.15789473684210525,
"0.5": 0.10526315789473684,
"0.75": 0.10526315789473684,
"1.0": 0.10526315789473684
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.2727272727272727,
"0.5": 0.1904761904761905,
"0.75": 0.1904761904761905,
"1.0": 0.1904761904761905
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 7,
"avg_interaction_input_object_jaccard": 7,
"avg_interaction_output_object_jaccard": 7,
"avg_interaction_type_exact_match_acc": 7
},
"contact": {
"avg_interaction_temporal_span_giou": 6,
"avg_interaction_input_object_jaccard": 6,
"avg_interaction_output_object_jaccard": 6,
"avg_interaction_type_exact_match_acc": 6
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}