STRIVE · 08/28/2026
Dyn generalization · test_120 · super_clevr_1184
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"blue chopper\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 25 679 1 33 684 2 42 688 3 55 685 4 70 691 5 82 690 6 94 675 7 100 684 8 116 688 9 128 674 10 137 686 11 149 673 12 161 671 13 173 669 14 183 668 15 194 667 16 186 681 17 213 670 18 225 667 19 235 679 20 248 661 21 239 678 22 269 660 23 279 660 24 287 676 25 299 658 26 309 658 27 300 675 28 330 656 29 340 647 30 349 646 31 360 654 32 372 641 33 379 653 34 369 670 35 377 670 36 390 670 37 399 669 38 406 668 39 415 666 40 448 635 41 456 638 42 464 639 43 473 646 44 479 636 45 488 633 46 494 630 47 453 651 48 512 624 49 505 634 50 526 614 51 526 621 52 530 619 53 532 622 54 508 632 55 569 605 56 518 629 57 579 601 59 578 599 60 582 599 61 557 619 62 566 617 63 572 617 64 607 595 65 583 617 66 609 595 68 597 615 69 601 614 70 607 613 71 624 592 72 628 592 73 632 592 74 635 592 75 638 592 76 641 593 77 645 593 78 646 599 79 650 596 80 654 590 81 658 590 82 662 591 83 676 588 84 678 585 85 682 583 86 686 580 87 687 579 88 691 578 89 696 576 90 698 575 91 704 574 92 705 572 94 711 570 95 713 568 96 718 567 97 714 565 98 722 563 100 726 560 101 730 558 102 732 557 103 737 556 105 740 553 106 743 552 107 746 551 109 750 547 110 755 547 112 759 545 113 762 543 114 766 542 115 768 541 116 771 541 117 773 540 118 776 539 119 778 538 [OBJ] 1 \"brown scooter\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 301 873 1 304 869 2 305 867 3 310 859 4 313 855 5 314 853 6 319 846 7 323 842 8 323 840 9 330 831 10 333 827 11 334 825 12 339 819 13 341 816 14 344 811 15 347 807 16 350 802 17 353 798 18 355 795 19 360 789 20 362 785 21 365 781 22 369 776 23 372 772 24 375 768 25 376 766 26 381 760 27 383 758 28 387 754 29 391 750 30 394 746 31 396 744 32 400 740 33 402 738 34 405 734 35 408 731 36 411 729 37 414 726 38 416 723 39 419 721 40 422 719 41 426 716 42 427 717 43 431 713 44 430 718 45 438 706 46 439 708 47 441 708 48 444 707 50 448 708 51 450 709 52 451 708 54 453 707 56 455 708 58 455 710 59 457 711 60 455 714 62 458 715 64 459 716 65 458 718 66 460 720 68 462 724 69 457 731 70 455 727 71 472 732 73 473 734 75 462 744 76 474 738 78 476 738 80 478 739 84 481 740 86 482 742 88 483 744 90 483 747 92 484 749 95 486 750 97 485 752 99 487 753 104 487 755 107 488 757 110 489 759 112 491 760 118 491 763 [OBJ] 2 \"green dirtbike\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 319 721 3 317 723 4 319 722 8 325 716 10 297 735 11 331 709 12 298 734 13 301 732 14 316 726 15 299 732 16 302 731 17 326 716 18 326 718 20 329 716 21 330 715 22 332 714 23 333 713 24 326 721 25 327 718 26 340 701 28 327 715 29 333 704 31 331 705 32 318 724 36 335 699 37 334 701 38 333 704 39 332 707 40 330 713 41 323 729 42 320 735 43 319 741 44 318 742 45 312 749 46 318 739 47 311 746 48 318 736 49 319 734 55 319 732 63 319 729 65 313 738 68 318 729 73 315 732 79 318 722 86 317 720 93 317 717 102 316 718 118 316 715 [OBJ] 3 \"yellow airliner\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 472 182 4 476 181 8 481 180 12 480 181 13 485 180 14 479 181 15 488 180 16 484 180 17 490 179 23 494 178 26 498 177 31 502 176 35 501 177 37 506 176 40 512 175 44 516 174 47 519 173 50 525 172 54 523 173 56 528 172 59 533 171 62 537 170 66 541 169 67 548 168 68 543 170 69 545 168 71 551 167 73 549 168 77 553 167 79 558 166 80 568 165 82 562 165 83 573 164 85 575 163 86 567 165 87 571 163 88 580 161 89 583 163 90 570 164 91 586 161 93 588 160 94 591 160 96 593 159 97 596 160 98 589 159 99 598 158 100 601 159 102 605 158 104 608 158 106 612 157 108 615 156 109 618 156 111 620 155 112 623 156 113 626 155 115 628 154 116 630 154 117 633 154 118 636 154 [OBJ] 4 \"purple utility\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 995 614 2 989 613 3 980 614 4 972 615 5 961 614 6 953 615 7 980 627 8 933 616 9 976 589 10 964 590 11 940 630 12 943 591 13 920 629 14 872 619 15 912 592 16 850 619 17 893 592 18 829 620 19 820 623 20 861 593 21 848 594 22 838 594 23 812 634 24 816 595 25 791 633 26 793 596 27 731 628 28 719 628 29 760 597 30 696 629 31 737 599 32 725 599 33 697 640 34 690 630 35 691 602 36 677 604 37 648 644 38 651 606 39 623 644 40 628 610 41 616 613 42 603 615 43 590 615 44 579 617 45 566 619 46 555 621 47 544 624 48 534 627 49 526 630 50 519 632 51 451 644 52 501 638 53 496 639 54 484 641 55 475 642 56 466 640 57 457 638 58 448 634 59 438 630 60 430 627 61 420 625 62 411 622 63 401 620 64 391 618 65 383 619 66 373 617 67 366 632 68 361 636 69 344 617 70 350 648 71 327 619 72 316 618 73 307 619 74 308 639 75 301 643 76 292 646 77 283 645 78 275 633 79 250 606 80 256 633 81 244 629 82 237 632 83 243 663 84 233 669 85 229 667 86 202 629 87 194 633 88 188 640 89 183 644 90 192 683 91 185 678 92 178 688 93 170 679 94 112 686 95 105 686 96 148 683 97 140 683 98 82 688 99 120 682 100 67 686 101 59 686 102 52 685 103 45 685 104 38 685 105 30 684 106 24 683 107 76 665 108 12 682 109 62 664 110 57 664 111 51 657 112 44 664 113 38 664 114 32 664 115 27 661 116 20 660 117 14 660 118 11 654 119 6 653 [OBJS-END] [INTERACTIONS-START] [INTERACT] 16 \"contact\" 0 45 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 2 \"contact\" 0 47 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 22 \"contact\" 0 48 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 15 \"contact\" 0 119 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 8 \"collision\" 24 24 [IN] 1 2 [OUT] 1 2 [INTERACT] 9 \"contact\" 25 26 [IN] 1 2 [OUT] 1 2 [INTERACT] 0 \"collision\" 43 43 [IN] 0 4 [OUT] 0 4 [INTERACT] 1 \"collision\" 48 49 [IN] 0 4 [OUT] 0 4 [INTERACT] 10 \"collision\" 49 49 [IN] 1 4 [OUT] 1 4 [INTERACT] 23 \"collision\" 50 50 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 11 \"contact\" 50 51 [IN] 1 4 [OUT] 1 4 [INTERACT] 24 \"contact\" 51 54 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 17 \"collision\" 53 53 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 18 \"collision\" 55 55 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 12 \"collision\" 56 56 [IN] 1 4 [OUT] 1 4 [INTERACT] 19 \"contact\" 56 69 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 13 \"collision\" 58 58 [IN] 1 4 [OUT] 1 4 [INTERACT] 14 \"contact\" 59 60 [IN] 1 4 [OUT] 1 4 [INTERACT] 3 \"collision\" 65 65 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 20 \"collision\" 71 71 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 4 \"collision\" 72 73 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 21 \"contact\" 72 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 5 \"collision\" 78 79 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 6 \"collision\" 81 81 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 25 \"collision\" 82 82 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 7 \"contact\" 82 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 26 \"collision\" 89 89 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 27 \"contact\" 90 93 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 28 \"collision\" 98 98 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 29 \"contact\" 99 119 [IN] 4 65535 [OUT] 4 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_full/super_clevr_1184/frames.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"light blue fabric sports backpack\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 33 681 1 52 685 2 37 685 3 56 685 4 68 685 5 80 685 6 91 685 7 103 685 8 115 685 9 126 685 10 138 685 11 150 685 12 162 685 13 173 685 14 185 685 15 197 685 16 337 666 17 351 669 18 361 669 19 373 673 20 384 673 21 396 677 22 404 673 23 416 677 [OBJ] 1 \"dark grey and purple plastic Shredder action figure\" 0 24 [VIS] 0 false 10 true [IN-FRAME] 0 true [PX] 0 null null 10 197 650 11 205 654 12 216 654 13 228 658 14 240 662 15 251 666 16 259 669 17 267 673 18 279 677 19 291 677 20 298 681 21 306 685 22 314 689 23 322 685 [OBJ] 2 \"black and blue athletic shoe\" 0 24 [VIS] 0 false 10 true [IN-FRAME] 0 true [PX] 0 null null 10 275 654 12 283 662 13 291 666 14 302 666 16 314 669 17 322 673 19 333 677 20 341 681 22 349 685 [OBJ] 3 \"yellow hoodie dollhouse boy\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 310 861 1 314 869 2 322 845 3 326 853 4 333 837 5 341 826 6 349 814 7 353 806 8 357 794 9 361 787 10 369 775 11 373 767 12 380 763 13 384 755 14 388 748 15 396 740 16 400 732 17 408 728 18 416 724 19 419 716 20 427 712 21 435 708 22 443 705 23 451 701 [OBJ] 4 \"green wooden toy train\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 322 775 2 326 767 4 330 759 6 337 755 8 341 748 10 345 740 12 349 732 14 353 724 16 357 716 19 361 708 22 365 701 [OBJ] 5 \"green wooden toy train\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 341 724 3 345 716 5 349 708 8 353 701 11 357 693 15 361 685 19 365 677 23 369 669 [OBJ] 6 \"yellow plastic candy bar\" 0 24 [VIS] 0 true 2 false [IN-FRAME] 0 true [PX] 0 361 685 1 361 697 2 null null [OBJ] 7 \"yellow plastic candy bar\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 404 689 1 null null [OBJ] 8 \"yellow and blue plastic soccer cleat\" 0 24 [VIS] 0 true 2 false [IN-FRAME] 0 true [PX] 0 431 681 2 null null [OBJ] 9 \"yellow plastic chocolate powder canister\" 0 24 [VIS] 0 true 2 false [IN-FRAME] 0 true [PX] 0 435 685 1 435 697 2 null null [OBJ] 10 \"dark grey and pink running shoe\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 458 677 1 null null [OBJ] 11 \"yellow plastic chocolate powder jar\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 474 677 1 474 689 4 478 681 9 474 697 10 478 681 12 478 693 13 482 673 16 482 689 17 486 669 20 486 685 21 490 666 24 490 681 [OBJ] 12 \"dark grey and yellow running shoe\" 0 24 [VIS] 0 true 2 false [IN-FRAME] 0 true [PX] 0 486 681 1 490 689 2 null null [OBJ] 13 \"yellow and blue plastic soccer cleat\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 490 669 4 494 662 9 498 654 13 501 646 18 505 638 22 509 630 [OBJ] 14 \"yellow cardboard curl cream box\" 0 24 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 505 689 1 null null [OBJ] 15 \"yellow plastic chocolate powder jar\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 505 693 3 509 685 7 513 677 11 517 669 16 521 662 20 525 654 23 529 646 [OBJ] 16 \"white and blue track spike\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 509 638 3 513 630 7 517 623 10 521 615 13 525 607 16 533 603 19 541 599 22 544 591 [OBJ] 17 \"yellow plastic chocolate powder jar\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 517 673 3 521 666 7 525 658 11 529 650 15 533 642 19 537 634 23 541 626 [OBJ] 18 \"white plastic supplement bottle\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 525 638 1 537 634 2 529 638 3 541 634 6 548 630 10 552 623 14 560 619 17 564 611 20 572 607 23 580 603 [OBJ] 19 \"purple and beige wooden dollhouse lamp\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 615 1 982 623 2 978 615 3 974 623 4 966 626 5 958 630 6 951 634 7 943 638 8 935 634 9 927 638 10 916 638 11 908 634 12 896 638 13 884 638 14 876 634 15 865 634 16 853 634 17 845 630 18 833 630 19 826 626 20 818 623 21 806 623 22 798 619 23 791 615 [OBJ] 20 \"white and light blue athletic shoe\" 17 24 [VIS] 0 false 17 true [IN-FRAME] 0 false 17 true [PX] 0 null null 17 1 662 19 5 669 20 9 677 22 13 685 23 17 693 [OBJS-END] [INTERACTIONS-START] [INTERACT] 2 \"contact\" 0 0 [IN] 1 2 [OUT] 1 2 [INTERACT] 22 \"contact\" 0 2 [IN] 2 5 [OUT] 2 5 [INTERACT] 25 \"contact\" 0 2 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 32 \"contact\" 0 13 [IN] 4 5 [OUT] 4 5 [INTERACT] 0 \"contact\" 0 23 [IN] 0 2 [OUT] 0 2 [INTERACT] 1 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 10 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 28 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 33 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 34 \"contact\" 0 23 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 35 \"contact\" 0 23 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 36 \"contact\" 0 23 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 37 \"contact\" 0 23 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 38 \"contact\" 0 23 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 7 \"contact\" 0 23 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 8 \"contact\" 0 23 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 9 \"contact\" 0 23 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 11 \"contact\" 0 23 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 12 \"contact\" 0 23 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 13 \"contact\" 0 23 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 14 \"contact\" 0 23 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 15 \"contact\" 0 23 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 16 \"contact\" 0 23 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 17 \"contact\" 0 23 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 18 \"contact\" 0 23 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 26 \"collision\" 4 4 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 27 \"contact\" 5 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 19 \"collision\" 6 6 [IN] 2 4 [OUT] 2 4 [INTERACT] 23 \"collision\" 6 6 [IN] 2 5 [OUT] 2 5 [INTERACT] 20 \"contact\" 7 23 [IN] 2 4 [OUT] 2 4 [INTERACT] 24 \"contact\" 7 23 [IN] 2 5 [OUT] 2 5 [INTERACT] 3 \"collision\" 11 11 [IN] 1 2 [OUT] 1 2 [INTERACT] 29 \"collision\" 11 11 [IN] 3 7 [OUT] 3 7 [INTERACT] 4 \"contact\" 12 15 [IN] 1 2 [OUT] 1 2 [INTERACT] 30 \"collision\" 14 14 [IN] 3 7 [OUT] 3 7 [INTERACT] 31 \"contact\" 15 17 [IN] 3 7 [OUT] 3 7 [INTERACT] 5 \"collision\" 17 17 [IN] 1 2 [OUT] 1 2 [INTERACT] 6 \"contact\" 18 23 [IN] 1 2 [OUT] 1 2 [INTERACT] 21 \"collision\" 20 20 [IN] 2 5 [OUT] 2 5 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"blue chopper\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 25 679 1 33 684 2 42 688 3 55 685 4 70 691 5 82 690 6 94 675 7 100 684 8 116 688 9 128 674 10 137 686 11 149 673 12 161 671 13 173 669 14 183 668 15 194 667 16 186 681 17 213 670 18 225 667 19 235 679 20 248 661 21 239 678 22 269 660 23 279 660 24 287 676 25 299 658 26 309 658 27 300 675 28 330 656 29 340 647 30 349 646 31 360 654 32 372 641 33 379 653 34 369 670 35 377 670 36 390 670 37 399 669 38 406 668 39 415 666 40 448 635 41 456 638 42 464 639 43 473 646 44 479 636 45 488 633 46 494 630 47 453 651 48 512 624 49 505 634 50 526 614 51 526 621 52 530 619 53 532 622 54 508 632 55 569 605 56 518 629 57 579 601 59 578 599 60 582 599 61 557 619 62 566 617 63 572 617 64 607 595 65 583 617 66 609 595 68 597 615 69 601 614 70 607 613 71 624 592 72 628 592 73 632 592 74 635 592 75 638 592 76 641 593 77 645 593 78 646 599 79 650 596 80 654 590 81 658 590 82 662 591 83 676 588 84 678 585 85 682 583 86 686 580 87 687 579 88 691 578 89 696 576 90 698 575 91 704 574 92 705 572 94 711 570 95 713 568 96 718 567 97 714 565 98 722 563 100 726 560 101 730 558 102 732 557 103 737 556 105 740 553 106 743 552 107 746 551 109 750 547 110 755 547 112 759 545 113 762 543 114 766 542 115 768 541 116 771 541 117 773 540 118 776 539 119 778 538 [OBJ] 1 \"brown scooter\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 301 873 1 304 869 2 305 867 3 310 859 4 313 855 5 314 853 6 319 846 7 323 842 8 323 840 9 330 831 10 333 827 11 334 825 12 339 819 13 341 816 14 344 811 15 347 807 16 350 802 17 353 798 18 355 795 19 360 789 20 362 785 21 365 781 22 369 776 23 372 772 24 375 768 25 376 766 26 381 760 27 383 758 28 387 754 29 391 750 30 394 746 31 396 744 32 400 740 33 402 738 34 405 734 35 408 731 36 411 729 37 414 726 38 416 723 39 419 721 40 422 719 41 426 716 42 427 717 43 431 713 44 430 718 45 438 706 46 439 708 47 441 708 48 444 707 50 448 708 51 450 709 52 451 708 54 453 707 56 455 708 58 455 710 59 457 711 60 455 714 62 458 715 64 459 716 65 458 718 66 460 720 68 462 724 69 457 731 70 455 727 71 472 732 73 473 734 75 462 744 76 474 738 78 476 738 80 478 739 84 481 740 86 482 742 88 483 744 90 483 747 92 484 749 95 486 750 97 485 752 99 487 753 104 487 755 107 488 757 110 489 759 112 491 760 118 491 763 [OBJ] 2 \"green dirtbike\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 319 721 3 317 723 4 319 722 8 325 716 10 297 735 11 331 709 12 298 734 13 301 732 14 316 726 15 299 732 16 302 731 17 326 716 18 326 718 20 329 716 21 330 715 22 332 714 23 333 713 24 326 721 25 327 718 26 340 701 28 327 715 29 333 704 31 331 705 32 318 724 36 335 699 37 334 701 38 333 704 39 332 707 40 330 713 41 323 729 42 320 735 43 319 741 44 318 742 45 312 749 46 318 739 47 311 746 48 318 736 49 319 734 55 319 732 63 319 729 65 313 738 68 318 729 73 315 732 79 318 722 86 317 720 93 317 717 102 316 718 118 316 715 [OBJ] 3 \"yellow airliner\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 472 182 4 476 181 8 481 180 12 480 181 13 485 180 14 479 181 15 488 180 16 484 180 17 490 179 23 494 178 26 498 177 31 502 176 35 501 177 37 506 176 40 512 175 44 516 174 47 519 173 50 525 172 54 523 173 56 528 172 59 533 171 62 537 170 66 541 169 67 548 168 68 543 170 69 545 168 71 551 167 73 549 168 77 553 167 79 558 166 80 568 165 82 562 165 83 573 164 85 575 163 86 567 165 87 571 163 88 580 161 89 583 163 90 570 164 91 586 161 93 588 160 94 591 160 96 593 159 97 596 160 98 589 159 99 598 158 100 601 159 102 605 158 104 608 158 106 612 157 108 615 156 109 618 156 111 620 155 112 623 156 113 626 155 115 628 154 116 630 154 117 633 154 118 636 154 [OBJ] 4 \"purple utility\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 995 614 2 989 613 3 980 614 4 972 615 5 961 614 6 953 615 7 980 627 8 933 616 9 976 589 10 964 590 11 940 630 12 943 591 13 920 629 14 872 619 15 912 592 16 850 619 17 893 592 18 829 620 19 820 623 20 861 593 21 848 594 22 838 594 23 812 634 24 816 595 25 791 633 26 793 596 27 731 628 28 719 628 29 760 597 30 696 629 31 737 599 32 725 599 33 697 640 34 690 630 35 691 602 36 677 604 37 648 644 38 651 606 39 623 644 40 628 610 41 616 613 42 603 615 43 590 615 44 579 617 45 566 619 46 555 621 47 544 624 48 534 627 49 526 630 50 519 632 51 451 644 52 501 638 53 496 639 54 484 641 55 475 642 56 466 640 57 457 638 58 448 634 59 438 630 60 430 627 61 420 625 62 411 622 63 401 620 64 391 618 65 383 619 66 373 617 67 366 632 68 361 636 69 344 617 70 350 648 71 327 619 72 316 618 73 307 619 74 308 639 75 301 643 76 292 646 77 283 645 78 275 633 79 250 606 80 256 633 81 244 629 82 237 632 83 243 663 84 233 669 85 229 667 86 202 629 87 194 633 88 188 640 89 183 644 90 192 683 91 185 678 92 178 688 93 170 679 94 112 686 95 105 686 96 148 683 97 140 683 98 82 688 99 120 682 100 67 686 101 59 686 102 52 685 103 45 685 104 38 685 105 30 684 106 24 683 107 76 665 108 12 682 109 62 664 110 57 664 111 51 657 112 44 664 113 38 664 114 32 664 115 27 661 116 20 660 117 14 660 118 11 654 119 6 653 [OBJS-END] [INTERACTIONS-START] [INTERACT] 16 \"contact\" 0 45 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 2 \"contact\" 0 47 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 22 \"contact\" 0 48 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 15 \"contact\" 0 119 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 8 \"collision\" 24 24 [IN] 1 2 [OUT] 1 2 [INTERACT] 9 \"contact\" 25 26 [IN] 1 2 [OUT] 1 2 [INTERACT] 0 \"collision\" 43 43 [IN] 0 4 [OUT] 0 4 [INTERACT] 1 \"collision\" 48 49 [IN] 0 4 [OUT] 0 4 [INTERACT] 10 \"collision\" 49 49 [IN] 1 4 [OUT] 1 4 [INTERACT] 23 \"collision\" 50 50 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 11 \"contact\" 50 51 [IN] 1 4 [OUT] 1 4 [INTERACT] 24 \"contact\" 51 54 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 17 \"collision\" 53 53 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 18 \"collision\" 55 55 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 12 \"collision\" 56 56 [IN] 1 4 [OUT] 1 4 [INTERACT] 19 \"contact\" 56 69 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 13 \"collision\" 58 58 [IN] 1 4 [OUT] 1 4 [INTERACT] 14 \"contact\" 59 60 [IN] 1 4 [OUT] 1 4 [INTERACT] 3 \"collision\" 65 65 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 20 \"collision\" 71 71 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 4 \"collision\" 72 73 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 21 \"contact\" 72 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 5 \"collision\" 78 79 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 6 \"collision\" 81 81 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 25 \"collision\" 82 82 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 7 \"contact\" 82 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 26 \"collision\" 89 89 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 27 \"contact\" 90 93 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 28 \"collision\" 98 98 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 29 \"contact\" 99 119 [IN] 4 65535 [OUT] 4 65535 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "light blue fabric sports backpack",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 33,
"y": 681
},
{
"t": 1,
"x": 52,
"y": 685
},
{
"t": 2,
"x": 37,
"y": 685
},
{
"t": 3,
"x": 56,
"y": 685
},
{
"t": 4,
"x": 68,
"y": 685
},
{
"t": 5,
"x": 80,
"y": 685
},
{
"t": 6,
"x": 91,
"y": 685
},
{
"t": 7,
"x": 103,
"y": 685
},
{
"t": 8,
"x": 115,
"y": 685
},
{
"t": 9,
"x": 126,
"y": 685
},
{
"t": 10,
"x": 138,
"y": 685
},
{
"t": 11,
"x": 150,
"y": 685
},
{
"t": 12,
"x": 162,
"y": 685
},
{
"t": 13,
"x": 173,
"y": 685
},
{
"t": 14,
"x": 185,
"y": 685
},
{
"t": 15,
"x": 197,
"y": 685
},
{
"t": 16,
"x": 337,
"y": 666
},
{
"t": 17,
"x": 351,
"y": 669
},
{
"t": 18,
"x": 361,
"y": 669
},
{
"t": 19,
"x": 373,
"y": 673
},
{
"t": 20,
"x": 384,
"y": 673
},
{
"t": 21,
"x": 396,
"y": 677
},
{
"t": 22,
"x": 404,
"y": 673
},
{
"t": 23,
"x": 416,
"y": 677
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "dark grey and purple plastic Shredder action figure",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 10,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 10,
"x": 197,
"y": 650
},
{
"t": 11,
"x": 205,
"y": 654
},
{
"t": 12,
"x": 216,
"y": 654
},
{
"t": 13,
"x": 228,
"y": 658
},
{
"t": 14,
"x": 240,
"y": 662
},
{
"t": 15,
"x": 251,
"y": 666
},
{
"t": 16,
"x": 259,
"y": 669
},
{
"t": 17,
"x": 267,
"y": 673
},
{
"t": 18,
"x": 279,
"y": 677
},
{
"t": 19,
"x": 291,
"y": 677
},
{
"t": 20,
"x": 298,
"y": 681
},
{
"t": 21,
"x": 306,
"y": 685
},
{
"t": 22,
"x": 314,
"y": 689
},
{
"t": 23,
"x": 322,
"y": 685
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "black and blue athletic shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 10,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 10,
"x": 275,
"y": 654
},
{
"t": 12,
"x": 283,
"y": 662
},
{
"t": 13,
"x": 291,
"y": 666
},
{
"t": 14,
"x": 302,
"y": 666
},
{
"t": 16,
"x": 314,
"y": 669
},
{
"t": 17,
"x": 322,
"y": 673
},
{
"t": 19,
"x": 333,
"y": 677
},
{
"t": 20,
"x": 341,
"y": 681
},
{
"t": 22,
"x": 349,
"y": 685
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "yellow hoodie dollhouse boy",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 310,
"y": 861
},
{
"t": 1,
"x": 314,
"y": 869
},
{
"t": 2,
"x": 322,
"y": 845
},
{
"t": 3,
"x": 326,
"y": 853
},
{
"t": 4,
"x": 333,
"y": 837
},
{
"t": 5,
"x": 341,
"y": 826
},
{
"t": 6,
"x": 349,
"y": 814
},
{
"t": 7,
"x": 353,
"y": 806
},
{
"t": 8,
"x": 357,
"y": 794
},
{
"t": 9,
"x": 361,
"y": 787
},
{
"t": 10,
"x": 369,
"y": 775
},
{
"t": 11,
"x": 373,
"y": 767
},
{
"t": 12,
"x": 380,
"y": 763
},
{
"t": 13,
"x": 384,
"y": 755
},
{
"t": 14,
"x": 388,
"y": 748
},
{
"t": 15,
"x": 396,
"y": 740
},
{
"t": 16,
"x": 400,
"y": 732
},
{
"t": 17,
"x": 408,
"y": 728
},
{
"t": 18,
"x": 416,
"y": 724
},
{
"t": 19,
"x": 419,
"y": 716
},
{
"t": 20,
"x": 427,
"y": 712
},
{
"t": 21,
"x": 435,
"y": 708
},
{
"t": 22,
"x": 443,
"y": 705
},
{
"t": 23,
"x": 451,
"y": 701
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 322,
"y": 775
},
{
"t": 2,
"x": 326,
"y": 767
},
{
"t": 4,
"x": 330,
"y": 759
},
{
"t": 6,
"x": 337,
"y": 755
},
{
"t": 8,
"x": 341,
"y": 748
},
{
"t": 10,
"x": 345,
"y": 740
},
{
"t": 12,
"x": 349,
"y": 732
},
{
"t": 14,
"x": 353,
"y": 724
},
{
"t": 16,
"x": 357,
"y": 716
},
{
"t": 19,
"x": 361,
"y": 708
},
{
"t": 22,
"x": 365,
"y": 701
}
],
"birth": 0,
"death": 24
},
{
"id": 5,
"type": "green wooden toy train",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 341,
"y": 724
},
{
"t": 3,
"x": 345,
"y": 716
},
{
"t": 5,
"x": 349,
"y": 708
},
{
"t": 8,
"x": 353,
"y": 701
},
{
"t": 11,
"x": 357,
"y": 693
},
{
"t": 15,
"x": 361,
"y": 685
},
{
"t": 19,
"x": 365,
"y": 677
},
{
"t": 23,
"x": 369,
"y": 669
}
],
"birth": 0,
"death": 24
},
{
"id": 6,
"type": "yellow plastic candy bar",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 361,
"y": 685
},
{
"t": 1,
"x": 361,
"y": 697
},
{
"t": 2,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 7,
"type": "yellow plastic candy bar",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 404,
"y": 689
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 8,
"type": "yellow and blue plastic soccer cleat",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 431,
"y": 681
},
{
"t": 2,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 9,
"type": "yellow plastic chocolate powder canister",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 435,
"y": 685
},
{
"t": 1,
"x": 435,
"y": 697
},
{
"t": 2,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 10,
"type": "dark grey and pink running shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 458,
"y": 677
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 11,
"type": "yellow plastic chocolate powder jar",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 474,
"y": 677
},
{
"t": 1,
"x": 474,
"y": 689
},
{
"t": 4,
"x": 478,
"y": 681
},
{
"t": 9,
"x": 474,
"y": 697
},
{
"t": 10,
"x": 478,
"y": 681
},
{
"t": 12,
"x": 478,
"y": 693
},
{
"t": 13,
"x": 482,
"y": 673
},
{
"t": 16,
"x": 482,
"y": 689
},
{
"t": 17,
"x": 486,
"y": 669
},
{
"t": 20,
"x": 486,
"y": 685
},
{
"t": 21,
"x": 490,
"y": 666
},
{
"t": 24,
"x": 490,
"y": 681
}
],
"birth": 0,
"death": 24
},
{
"id": 12,
"type": "dark grey and yellow running shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 2,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 486,
"y": 681
},
{
"t": 1,
"x": 490,
"y": 689
},
{
"t": 2,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 13,
"type": "yellow and blue plastic soccer cleat",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 490,
"y": 669
},
{
"t": 4,
"x": 494,
"y": 662
},
{
"t": 9,
"x": 498,
"y": 654
},
{
"t": 13,
"x": 501,
"y": 646
},
{
"t": 18,
"x": 505,
"y": 638
},
{
"t": 22,
"x": 509,
"y": 630
}
],
"birth": 0,
"death": 24
},
{
"id": 14,
"type": "yellow cardboard curl cream box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 1,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 505,
"y": 689
},
{
"t": 1,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 15,
"type": "yellow plastic chocolate powder jar",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 505,
"y": 693
},
{
"t": 3,
"x": 509,
"y": 685
},
{
"t": 7,
"x": 513,
"y": 677
},
{
"t": 11,
"x": 517,
"y": 669
},
{
"t": 16,
"x": 521,
"y": 662
},
{
"t": 20,
"x": 525,
"y": 654
},
{
"t": 23,
"x": 529,
"y": 646
}
],
"birth": 0,
"death": 24
},
{
"id": 16,
"type": "white and blue track spike",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 509,
"y": 638
},
{
"t": 3,
"x": 513,
"y": 630
},
{
"t": 7,
"x": 517,
"y": 623
},
{
"t": 10,
"x": 521,
"y": 615
},
{
"t": 13,
"x": 525,
"y": 607
},
{
"t": 16,
"x": 533,
"y": 603
},
{
"t": 19,
"x": 541,
"y": 599
},
{
"t": 22,
"x": 544,
"y": 591
}
],
"birth": 0,
"death": 24
},
{
"id": 17,
"type": "yellow plastic chocolate powder jar",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 517,
"y": 673
},
{
"t": 3,
"x": 521,
"y": 666
},
{
"t": 7,
"x": 525,
"y": 658
},
{
"t": 11,
"x": 529,
"y": 650
},
{
"t": 15,
"x": 533,
"y": 642
},
{
"t": 19,
"x": 537,
"y": 634
},
{
"t": 23,
"x": 541,
"y": 626
}
],
"birth": 0,
"death": 24
},
{
"id": 18,
"type": "white plastic supplement bottle",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 525,
"y": 638
},
{
"t": 1,
"x": 537,
"y": 634
},
{
"t": 2,
"x": 529,
"y": 638
},
{
"t": 3,
"x": 541,
"y": 634
},
{
"t": 6,
"x": 548,
"y": 630
},
{
"t": 10,
"x": 552,
"y": 623
},
{
"t": 14,
"x": 560,
"y": 619
},
{
"t": 17,
"x": 564,
"y": 611
},
{
"t": 20,
"x": 572,
"y": 607
},
{
"t": 23,
"x": 580,
"y": 603
}
],
"birth": 0,
"death": 24
},
{
"id": 19,
"type": "purple and beige wooden dollhouse lamp",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 986,
"y": 615
},
{
"t": 1,
"x": 982,
"y": 623
},
{
"t": 2,
"x": 978,
"y": 615
},
{
"t": 3,
"x": 974,
"y": 623
},
{
"t": 4,
"x": 966,
"y": 626
},
{
"t": 5,
"x": 958,
"y": 630
},
{
"t": 6,
"x": 951,
"y": 634
},
{
"t": 7,
"x": 943,
"y": 638
},
{
"t": 8,
"x": 935,
"y": 634
},
{
"t": 9,
"x": 927,
"y": 638
},
{
"t": 10,
"x": 916,
"y": 638
},
{
"t": 11,
"x": 908,
"y": 634
},
{
"t": 12,
"x": 896,
"y": 638
},
{
"t": 13,
"x": 884,
"y": 638
},
{
"t": 14,
"x": 876,
"y": 634
},
{
"t": 15,
"x": 865,
"y": 634
},
{
"t": 16,
"x": 853,
"y": 634
},
{
"t": 17,
"x": 845,
"y": 630
},
{
"t": 18,
"x": 833,
"y": 630
},
{
"t": 19,
"x": 826,
"y": 626
},
{
"t": 20,
"x": 818,
"y": 623
},
{
"t": 21,
"x": 806,
"y": 623
},
{
"t": 22,
"x": 798,
"y": 619
},
{
"t": 23,
"x": 791,
"y": 615
}
],
"birth": 0,
"death": 24
},
{
"id": 20,
"type": "white and light blue athletic shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 17,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 17,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 17,
"x": 1,
"y": 662
},
{
"t": 19,
"x": 5,
"y": 669
},
{
"t": 20,
"x": 9,
"y": 677
},
{
"t": 22,
"x": 13,
"y": 685
},
{
"t": 23,
"x": 17,
"y": 693
}
],
"birth": 17,
"death": 24
}
],
"interactions": [
{
"id": 2,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 0,
"ends_at": 0,
"type": "contact"
},
{
"id": 22,
"inputs": [
2,
5
],
"outputs": [
2,
5
],
"begins_at": 0,
"ends_at": 2,
"type": "contact"
},
{
"id": 25,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 2,
"type": "contact"
},
{
"id": 32,
"inputs": [
4,
5
],
"outputs": [
4,
5
],
"begins_at": 0,
"ends_at": 13,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
2
],
"outputs": [
0,
2
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 10,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 28,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 33,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 34,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 35,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 36,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 37,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 38,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 7,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 8,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 9,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 11,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 12,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 13,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 14,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 15,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 16,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 17,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 18,
"inputs": [
20,
65535
],
"outputs": [
20,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 26,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 4,
"ends_at": 4,
"type": "collision"
},
{
"id": 27,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 5,
"ends_at": 23,
"type": "contact"
},
{
"id": 19,
"inputs": [
2,
4
],
"outputs": [
2,
4
],
"begins_at": 6,
"ends_at": 6,
"type": "collision"
},
{
"id": 23,
"inputs": [
2,
5
],
"outputs": [
2,
5
],
"begins_at": 6,
"ends_at": 6,
"type": "collision"
},
{
"id": 20,
"inputs": [
2,
4
],
"outputs": [
2,
4
],
"begins_at": 7,
"ends_at": 23,
"type": "contact"
},
{
"id": 24,
"inputs": [
2,
5
],
"outputs": [
2,
5
],
"begins_at": 7,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 11,
"ends_at": 11,
"type": "collision"
},
{
"id": 29,
"inputs": [
3,
7
],
"outputs": [
3,
7
],
"begins_at": 11,
"ends_at": 11,
"type": "collision"
},
{
"id": 4,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 12,
"ends_at": 15,
"type": "contact"
},
{
"id": 30,
"inputs": [
3,
7
],
"outputs": [
3,
7
],
"begins_at": 14,
"ends_at": 14,
"type": "collision"
},
{
"id": 31,
"inputs": [
3,
7
],
"outputs": [
3,
7
],
"begins_at": 15,
"ends_at": 17,
"type": "contact"
},
{
"id": 5,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 17,
"ends_at": 17,
"type": "collision"
},
{
"id": 6,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 18,
"ends_at": 23,
"type": "contact"
},
{
"id": 21,
"inputs": [
2,
5
],
"outputs": [
2,
5
],
"begins_at": 20,
"ends_at": 20,
"type": "collision"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "blue chopper",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 25,
"y": 679
},
{
"t": 1,
"x": 33,
"y": 684
},
{
"t": 2,
"x": 42,
"y": 688
},
{
"t": 3,
"x": 55,
"y": 685
},
{
"t": 4,
"x": 70,
"y": 691
},
{
"t": 5,
"x": 82,
"y": 690
},
{
"t": 6,
"x": 94,
"y": 675
},
{
"t": 7,
"x": 100,
"y": 684
},
{
"t": 8,
"x": 116,
"y": 688
},
{
"t": 9,
"x": 128,
"y": 674
},
{
"t": 10,
"x": 137,
"y": 686
},
{
"t": 11,
"x": 149,
"y": 673
},
{
"t": 12,
"x": 161,
"y": 671
},
{
"t": 13,
"x": 173,
"y": 669
},
{
"t": 14,
"x": 183,
"y": 668
},
{
"t": 15,
"x": 194,
"y": 667
},
{
"t": 16,
"x": 186,
"y": 681
},
{
"t": 17,
"x": 213,
"y": 670
},
{
"t": 18,
"x": 225,
"y": 667
},
{
"t": 19,
"x": 235,
"y": 679
},
{
"t": 20,
"x": 248,
"y": 661
},
{
"t": 21,
"x": 239,
"y": 678
},
{
"t": 22,
"x": 269,
"y": 660
},
{
"t": 23,
"x": 279,
"y": 660
},
{
"t": 24,
"x": 287,
"y": 676
},
{
"t": 25,
"x": 299,
"y": 658
},
{
"t": 26,
"x": 309,
"y": 658
},
{
"t": 27,
"x": 300,
"y": 675
},
{
"t": 28,
"x": 330,
"y": 656
},
{
"t": 29,
"x": 340,
"y": 647
},
{
"t": 30,
"x": 349,
"y": 646
},
{
"t": 31,
"x": 360,
"y": 654
},
{
"t": 32,
"x": 372,
"y": 641
},
{
"t": 33,
"x": 379,
"y": 653
},
{
"t": 34,
"x": 369,
"y": 670
},
{
"t": 35,
"x": 377,
"y": 670
},
{
"t": 36,
"x": 390,
"y": 670
},
{
"t": 37,
"x": 399,
"y": 669
},
{
"t": 38,
"x": 406,
"y": 668
},
{
"t": 39,
"x": 415,
"y": 666
},
{
"t": 40,
"x": 448,
"y": 635
},
{
"t": 41,
"x": 456,
"y": 638
},
{
"t": 42,
"x": 464,
"y": 639
},
{
"t": 43,
"x": 473,
"y": 646
},
{
"t": 44,
"x": 479,
"y": 636
},
{
"t": 45,
"x": 488,
"y": 633
},
{
"t": 46,
"x": 494,
"y": 630
},
{
"t": 47,
"x": 453,
"y": 651
},
{
"t": 48,
"x": 512,
"y": 624
},
{
"t": 49,
"x": 505,
"y": 634
},
{
"t": 50,
"x": 526,
"y": 614
},
{
"t": 51,
"x": 526,
"y": 621
},
{
"t": 52,
"x": 530,
"y": 619
},
{
"t": 53,
"x": 532,
"y": 622
},
{
"t": 54,
"x": 508,
"y": 632
},
{
"t": 55,
"x": 569,
"y": 605
},
{
"t": 56,
"x": 518,
"y": 629
},
{
"t": 57,
"x": 579,
"y": 601
},
{
"t": 59,
"x": 578,
"y": 599
},
{
"t": 60,
"x": 582,
"y": 599
},
{
"t": 61,
"x": 557,
"y": 619
},
{
"t": 62,
"x": 566,
"y": 617
},
{
"t": 63,
"x": 572,
"y": 617
},
{
"t": 64,
"x": 607,
"y": 595
},
{
"t": 65,
"x": 583,
"y": 617
},
{
"t": 66,
"x": 609,
"y": 595
},
{
"t": 68,
"x": 597,
"y": 615
},
{
"t": 69,
"x": 601,
"y": 614
},
{
"t": 70,
"x": 607,
"y": 613
},
{
"t": 71,
"x": 624,
"y": 592
},
{
"t": 72,
"x": 628,
"y": 592
},
{
"t": 73,
"x": 632,
"y": 592
},
{
"t": 74,
"x": 635,
"y": 592
},
{
"t": 75,
"x": 638,
"y": 592
},
{
"t": 76,
"x": 641,
"y": 593
},
{
"t": 77,
"x": 645,
"y": 593
},
{
"t": 78,
"x": 646,
"y": 599
},
{
"t": 79,
"x": 650,
"y": 596
},
{
"t": 80,
"x": 654,
"y": 590
},
{
"t": 81,
"x": 658,
"y": 590
},
{
"t": 82,
"x": 662,
"y": 591
},
{
"t": 83,
"x": 676,
"y": 588
},
{
"t": 84,
"x": 678,
"y": 585
},
{
"t": 85,
"x": 682,
"y": 583
},
{
"t": 86,
"x": 686,
"y": 580
},
{
"t": 87,
"x": 687,
"y": 579
},
{
"t": 88,
"x": 691,
"y": 578
},
{
"t": 89,
"x": 696,
"y": 576
},
{
"t": 90,
"x": 698,
"y": 575
},
{
"t": 91,
"x": 704,
"y": 574
},
{
"t": 92,
"x": 705,
"y": 572
},
{
"t": 94,
"x": 711,
"y": 570
},
{
"t": 95,
"x": 713,
"y": 568
},
{
"t": 96,
"x": 718,
"y": 567
},
{
"t": 97,
"x": 714,
"y": 565
},
{
"t": 98,
"x": 722,
"y": 563
},
{
"t": 100,
"x": 726,
"y": 560
},
{
"t": 101,
"x": 730,
"y": 558
},
{
"t": 102,
"x": 732,
"y": 557
},
{
"t": 103,
"x": 737,
"y": 556
},
{
"t": 105,
"x": 740,
"y": 553
},
{
"t": 106,
"x": 743,
"y": 552
},
{
"t": 107,
"x": 746,
"y": 551
},
{
"t": 109,
"x": 750,
"y": 547
},
{
"t": 110,
"x": 755,
"y": 547
},
{
"t": 112,
"x": 759,
"y": 545
},
{
"t": 113,
"x": 762,
"y": 543
},
{
"t": 114,
"x": 766,
"y": 542
},
{
"t": 115,
"x": 768,
"y": 541
},
{
"t": 116,
"x": 771,
"y": 541
},
{
"t": 117,
"x": 773,
"y": 540
},
{
"t": 118,
"x": 776,
"y": 539
},
{
"t": 119,
"x": 778,
"y": 538
}
],
"birth": 0,
"death": 120
},
{
"id": 1,
"type": "brown scooter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 301,
"y": 873
},
{
"t": 1,
"x": 304,
"y": 869
},
{
"t": 2,
"x": 305,
"y": 867
},
{
"t": 3,
"x": 310,
"y": 859
},
{
"t": 4,
"x": 313,
"y": 855
},
{
"t": 5,
"x": 314,
"y": 853
},
{
"t": 6,
"x": 319,
"y": 846
},
{
"t": 7,
"x": 323,
"y": 842
},
{
"t": 8,
"x": 323,
"y": 840
},
{
"t": 9,
"x": 330,
"y": 831
},
{
"t": 10,
"x": 333,
"y": 827
},
{
"t": 11,
"x": 334,
"y": 825
},
{
"t": 12,
"x": 339,
"y": 819
},
{
"t": 13,
"x": 341,
"y": 816
},
{
"t": 14,
"x": 344,
"y": 811
},
{
"t": 15,
"x": 347,
"y": 807
},
{
"t": 16,
"x": 350,
"y": 802
},
{
"t": 17,
"x": 353,
"y": 798
},
{
"t": 18,
"x": 355,
"y": 795
},
{
"t": 19,
"x": 360,
"y": 789
},
{
"t": 20,
"x": 362,
"y": 785
},
{
"t": 21,
"x": 365,
"y": 781
},
{
"t": 22,
"x": 369,
"y": 776
},
{
"t": 23,
"x": 372,
"y": 772
},
{
"t": 24,
"x": 375,
"y": 768
},
{
"t": 25,
"x": 376,
"y": 766
},
{
"t": 26,
"x": 381,
"y": 760
},
{
"t": 27,
"x": 383,
"y": 758
},
{
"t": 28,
"x": 387,
"y": 754
},
{
"t": 29,
"x": 391,
"y": 750
},
{
"t": 30,
"x": 394,
"y": 746
},
{
"t": 31,
"x": 396,
"y": 744
},
{
"t": 32,
"x": 400,
"y": 740
},
{
"t": 33,
"x": 402,
"y": 738
},
{
"t": 34,
"x": 405,
"y": 734
},
{
"t": 35,
"x": 408,
"y": 731
},
{
"t": 36,
"x": 411,
"y": 729
},
{
"t": 37,
"x": 414,
"y": 726
},
{
"t": 38,
"x": 416,
"y": 723
},
{
"t": 39,
"x": 419,
"y": 721
},
{
"t": 40,
"x": 422,
"y": 719
},
{
"t": 41,
"x": 426,
"y": 716
},
{
"t": 42,
"x": 427,
"y": 717
},
{
"t": 43,
"x": 431,
"y": 713
},
{
"t": 44,
"x": 430,
"y": 718
},
{
"t": 45,
"x": 438,
"y": 706
},
{
"t": 46,
"x": 439,
"y": 708
},
{
"t": 47,
"x": 441,
"y": 708
},
{
"t": 48,
"x": 444,
"y": 707
},
{
"t": 50,
"x": 448,
"y": 708
},
{
"t": 51,
"x": 450,
"y": 709
},
{
"t": 52,
"x": 451,
"y": 708
},
{
"t": 54,
"x": 453,
"y": 707
},
{
"t": 56,
"x": 455,
"y": 708
},
{
"t": 58,
"x": 455,
"y": 710
},
{
"t": 59,
"x": 457,
"y": 711
},
{
"t": 60,
"x": 455,
"y": 714
},
{
"t": 62,
"x": 458,
"y": 715
},
{
"t": 64,
"x": 459,
"y": 716
},
{
"t": 65,
"x": 458,
"y": 718
},
{
"t": 66,
"x": 460,
"y": 720
},
{
"t": 68,
"x": 462,
"y": 724
},
{
"t": 69,
"x": 457,
"y": 731
},
{
"t": 70,
"x": 455,
"y": 727
},
{
"t": 71,
"x": 472,
"y": 732
},
{
"t": 73,
"x": 473,
"y": 734
},
{
"t": 75,
"x": 462,
"y": 744
},
{
"t": 76,
"x": 474,
"y": 738
},
{
"t": 78,
"x": 476,
"y": 738
},
{
"t": 80,
"x": 478,
"y": 739
},
{
"t": 84,
"x": 481,
"y": 740
},
{
"t": 86,
"x": 482,
"y": 742
},
{
"t": 88,
"x": 483,
"y": 744
},
{
"t": 90,
"x": 483,
"y": 747
},
{
"t": 92,
"x": 484,
"y": 749
},
{
"t": 95,
"x": 486,
"y": 750
},
{
"t": 97,
"x": 485,
"y": 752
},
{
"t": 99,
"x": 487,
"y": 753
},
{
"t": 104,
"x": 487,
"y": 755
},
{
"t": 107,
"x": 488,
"y": 757
},
{
"t": 110,
"x": 489,
"y": 759
},
{
"t": 112,
"x": 491,
"y": 760
},
{
"t": 118,
"x": 491,
"y": 763
}
],
"birth": 0,
"death": 120
},
{
"id": 2,
"type": "green dirtbike",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 319,
"y": 721
},
{
"t": 3,
"x": 317,
"y": 723
},
{
"t": 4,
"x": 319,
"y": 722
},
{
"t": 8,
"x": 325,
"y": 716
},
{
"t": 10,
"x": 297,
"y": 735
},
{
"t": 11,
"x": 331,
"y": 709
},
{
"t": 12,
"x": 298,
"y": 734
},
{
"t": 13,
"x": 301,
"y": 732
},
{
"t": 14,
"x": 316,
"y": 726
},
{
"t": 15,
"x": 299,
"y": 732
},
{
"t": 16,
"x": 302,
"y": 731
},
{
"t": 17,
"x": 326,
"y": 716
},
{
"t": 18,
"x": 326,
"y": 718
},
{
"t": 20,
"x": 329,
"y": 716
},
{
"t": 21,
"x": 330,
"y": 715
},
{
"t": 22,
"x": 332,
"y": 714
},
{
"t": 23,
"x": 333,
"y": 713
},
{
"t": 24,
"x": 326,
"y": 721
},
{
"t": 25,
"x": 327,
"y": 718
},
{
"t": 26,
"x": 340,
"y": 701
},
{
"t": 28,
"x": 327,
"y": 715
},
{
"t": 29,
"x": 333,
"y": 704
},
{
"t": 31,
"x": 331,
"y": 705
},
{
"t": 32,
"x": 318,
"y": 724
},
{
"t": 36,
"x": 335,
"y": 699
},
{
"t": 37,
"x": 334,
"y": 701
},
{
"t": 38,
"x": 333,
"y": 704
},
{
"t": 39,
"x": 332,
"y": 707
},
{
"t": 40,
"x": 330,
"y": 713
},
{
"t": 41,
"x": 323,
"y": 729
},
{
"t": 42,
"x": 320,
"y": 735
},
{
"t": 43,
"x": 319,
"y": 741
},
{
"t": 44,
"x": 318,
"y": 742
},
{
"t": 45,
"x": 312,
"y": 749
},
{
"t": 46,
"x": 318,
"y": 739
},
{
"t": 47,
"x": 311,
"y": 746
},
{
"t": 48,
"x": 318,
"y": 736
},
{
"t": 49,
"x": 319,
"y": 734
},
{
"t": 55,
"x": 319,
"y": 732
},
{
"t": 63,
"x": 319,
"y": 729
},
{
"t": 65,
"x": 313,
"y": 738
},
{
"t": 68,
"x": 318,
"y": 729
},
{
"t": 73,
"x": 315,
"y": 732
},
{
"t": 79,
"x": 318,
"y": 722
},
{
"t": 86,
"x": 317,
"y": 720
},
{
"t": 93,
"x": 317,
"y": 717
},
{
"t": 102,
"x": 316,
"y": 718
},
{
"t": 118,
"x": 316,
"y": 715
}
],
"birth": 0,
"death": 120
},
{
"id": 3,
"type": "yellow airliner",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 472,
"y": 182
},
{
"t": 4,
"x": 476,
"y": 181
},
{
"t": 8,
"x": 481,
"y": 180
},
{
"t": 12,
"x": 480,
"y": 181
},
{
"t": 13,
"x": 485,
"y": 180
},
{
"t": 14,
"x": 479,
"y": 181
},
{
"t": 15,
"x": 488,
"y": 180
},
{
"t": 16,
"x": 484,
"y": 180
},
{
"t": 17,
"x": 490,
"y": 179
},
{
"t": 23,
"x": 494,
"y": 178
},
{
"t": 26,
"x": 498,
"y": 177
},
{
"t": 31,
"x": 502,
"y": 176
},
{
"t": 35,
"x": 501,
"y": 177
},
{
"t": 37,
"x": 506,
"y": 176
},
{
"t": 40,
"x": 512,
"y": 175
},
{
"t": 44,
"x": 516,
"y": 174
},
{
"t": 47,
"x": 519,
"y": 173
},
{
"t": 50,
"x": 525,
"y": 172
},
{
"t": 54,
"x": 523,
"y": 173
},
{
"t": 56,
"x": 528,
"y": 172
},
{
"t": 59,
"x": 533,
"y": 171
},
{
"t": 62,
"x": 537,
"y": 170
},
{
"t": 66,
"x": 541,
"y": 169
},
{
"t": 67,
"x": 548,
"y": 168
},
{
"t": 68,
"x": 543,
"y": 170
},
{
"t": 69,
"x": 545,
"y": 168
},
{
"t": 71,
"x": 551,
"y": 167
},
{
"t": 73,
"x": 549,
"y": 168
},
{
"t": 77,
"x": 553,
"y": 167
},
{
"t": 79,
"x": 558,
"y": 166
},
{
"t": 80,
"x": 568,
"y": 165
},
{
"t": 82,
"x": 562,
"y": 165
},
{
"t": 83,
"x": 573,
"y": 164
},
{
"t": 85,
"x": 575,
"y": 163
},
{
"t": 86,
"x": 567,
"y": 165
},
{
"t": 87,
"x": 571,
"y": 163
},
{
"t": 88,
"x": 580,
"y": 161
},
{
"t": 89,
"x": 583,
"y": 163
},
{
"t": 90,
"x": 570,
"y": 164
},
{
"t": 91,
"x": 586,
"y": 161
},
{
"t": 93,
"x": 588,
"y": 160
},
{
"t": 94,
"x": 591,
"y": 160
},
{
"t": 96,
"x": 593,
"y": 159
},
{
"t": 97,
"x": 596,
"y": 160
},
{
"t": 98,
"x": 589,
"y": 159
},
{
"t": 99,
"x": 598,
"y": 158
},
{
"t": 100,
"x": 601,
"y": 159
},
{
"t": 102,
"x": 605,
"y": 158
},
{
"t": 104,
"x": 608,
"y": 158
},
{
"t": 106,
"x": 612,
"y": 157
},
{
"t": 108,
"x": 615,
"y": 156
},
{
"t": 109,
"x": 618,
"y": 156
},
{
"t": 111,
"x": 620,
"y": 155
},
{
"t": 112,
"x": 623,
"y": 156
},
{
"t": 113,
"x": 626,
"y": 155
},
{
"t": 115,
"x": 628,
"y": 154
},
{
"t": 116,
"x": 630,
"y": 154
},
{
"t": 117,
"x": 633,
"y": 154
},
{
"t": 118,
"x": 636,
"y": 154
}
],
"birth": 0,
"death": 120
},
{
"id": 4,
"type": "purple utility",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 995,
"y": 614
},
{
"t": 2,
"x": 989,
"y": 613
},
{
"t": 3,
"x": 980,
"y": 614
},
{
"t": 4,
"x": 972,
"y": 615
},
{
"t": 5,
"x": 961,
"y": 614
},
{
"t": 6,
"x": 953,
"y": 615
},
{
"t": 7,
"x": 980,
"y": 627
},
{
"t": 8,
"x": 933,
"y": 616
},
{
"t": 9,
"x": 976,
"y": 589
},
{
"t": 10,
"x": 964,
"y": 590
},
{
"t": 11,
"x": 940,
"y": 630
},
{
"t": 12,
"x": 943,
"y": 591
},
{
"t": 13,
"x": 920,
"y": 629
},
{
"t": 14,
"x": 872,
"y": 619
},
{
"t": 15,
"x": 912,
"y": 592
},
{
"t": 16,
"x": 850,
"y": 619
},
{
"t": 17,
"x": 893,
"y": 592
},
{
"t": 18,
"x": 829,
"y": 620
},
{
"t": 19,
"x": 820,
"y": 623
},
{
"t": 20,
"x": 861,
"y": 593
},
{
"t": 21,
"x": 848,
"y": 594
},
{
"t": 22,
"x": 838,
"y": 594
},
{
"t": 23,
"x": 812,
"y": 634
},
{
"t": 24,
"x": 816,
"y": 595
},
{
"t": 25,
"x": 791,
"y": 633
},
{
"t": 26,
"x": 793,
"y": 596
},
{
"t": 27,
"x": 731,
"y": 628
},
{
"t": 28,
"x": 719,
"y": 628
},
{
"t": 29,
"x": 760,
"y": 597
},
{
"t": 30,
"x": 696,
"y": 629
},
{
"t": 31,
"x": 737,
"y": 599
},
{
"t": 32,
"x": 725,
"y": 599
},
{
"t": 33,
"x": 697,
"y": 640
},
{
"t": 34,
"x": 690,
"y": 630
},
{
"t": 35,
"x": 691,
"y": 602
},
{
"t": 36,
"x": 677,
"y": 604
},
{
"t": 37,
"x": 648,
"y": 644
},
{
"t": 38,
"x": 651,
"y": 606
},
{
"t": 39,
"x": 623,
"y": 644
},
{
"t": 40,
"x": 628,
"y": 610
},
{
"t": 41,
"x": 616,
"y": 613
},
{
"t": 42,
"x": 603,
"y": 615
},
{
"t": 43,
"x": 590,
"y": 615
},
{
"t": 44,
"x": 579,
"y": 617
},
{
"t": 45,
"x": 566,
"y": 619
},
{
"t": 46,
"x": 555,
"y": 621
},
{
"t": 47,
"x": 544,
"y": 624
},
{
"t": 48,
"x": 534,
"y": 627
},
{
"t": 49,
"x": 526,
"y": 630
},
{
"t": 50,
"x": 519,
"y": 632
},
{
"t": 51,
"x": 451,
"y": 644
},
{
"t": 52,
"x": 501,
"y": 638
},
{
"t": 53,
"x": 496,
"y": 639
},
{
"t": 54,
"x": 484,
"y": 641
},
{
"t": 55,
"x": 475,
"y": 642
},
{
"t": 56,
"x": 466,
"y": 640
},
{
"t": 57,
"x": 457,
"y": 638
},
{
"t": 58,
"x": 448,
"y": 634
},
{
"t": 59,
"x": 438,
"y": 630
},
{
"t": 60,
"x": 430,
"y": 627
},
{
"t": 61,
"x": 420,
"y": 625
},
{
"t": 62,
"x": 411,
"y": 622
},
{
"t": 63,
"x": 401,
"y": 620
},
{
"t": 64,
"x": 391,
"y": 618
},
{
"t": 65,
"x": 383,
"y": 619
},
{
"t": 66,
"x": 373,
"y": 617
},
{
"t": 67,
"x": 366,
"y": 632
},
{
"t": 68,
"x": 361,
"y": 636
},
{
"t": 69,
"x": 344,
"y": 617
},
{
"t": 70,
"x": 350,
"y": 648
},
{
"t": 71,
"x": 327,
"y": 619
},
{
"t": 72,
"x": 316,
"y": 618
},
{
"t": 73,
"x": 307,
"y": 619
},
{
"t": 74,
"x": 308,
"y": 639
},
{
"t": 75,
"x": 301,
"y": 643
},
{
"t": 76,
"x": 292,
"y": 646
},
{
"t": 77,
"x": 283,
"y": 645
},
{
"t": 78,
"x": 275,
"y": 633
},
{
"t": 79,
"x": 250,
"y": 606
},
{
"t": 80,
"x": 256,
"y": 633
},
{
"t": 81,
"x": 244,
"y": 629
},
{
"t": 82,
"x": 237,
"y": 632
},
{
"t": 83,
"x": 243,
"y": 663
},
{
"t": 84,
"x": 233,
"y": 669
},
{
"t": 85,
"x": 229,
"y": 667
},
{
"t": 86,
"x": 202,
"y": 629
},
{
"t": 87,
"x": 194,
"y": 633
},
{
"t": 88,
"x": 188,
"y": 640
},
{
"t": 89,
"x": 183,
"y": 644
},
{
"t": 90,
"x": 192,
"y": 683
},
{
"t": 91,
"x": 185,
"y": 678
},
{
"t": 92,
"x": 178,
"y": 688
},
{
"t": 93,
"x": 170,
"y": 679
},
{
"t": 94,
"x": 112,
"y": 686
},
{
"t": 95,
"x": 105,
"y": 686
},
{
"t": 96,
"x": 148,
"y": 683
},
{
"t": 97,
"x": 140,
"y": 683
},
{
"t": 98,
"x": 82,
"y": 688
},
{
"t": 99,
"x": 120,
"y": 682
},
{
"t": 100,
"x": 67,
"y": 686
},
{
"t": 101,
"x": 59,
"y": 686
},
{
"t": 102,
"x": 52,
"y": 685
},
{
"t": 103,
"x": 45,
"y": 685
},
{
"t": 104,
"x": 38,
"y": 685
},
{
"t": 105,
"x": 30,
"y": 684
},
{
"t": 106,
"x": 24,
"y": 683
},
{
"t": 107,
"x": 76,
"y": 665
},
{
"t": 108,
"x": 12,
"y": 682
},
{
"t": 109,
"x": 62,
"y": 664
},
{
"t": 110,
"x": 57,
"y": 664
},
{
"t": 111,
"x": 51,
"y": 657
},
{
"t": 112,
"x": 44,
"y": 664
},
{
"t": 113,
"x": 38,
"y": 664
},
{
"t": 114,
"x": 32,
"y": 664
},
{
"t": 115,
"x": 27,
"y": 661
},
{
"t": 116,
"x": 20,
"y": 660
},
{
"t": 117,
"x": 14,
"y": 660
},
{
"t": 118,
"x": 11,
"y": 654
},
{
"t": 119,
"x": 6,
"y": 653
}
],
"birth": 0,
"death": 120
}
],
"interactions": [
{
"id": 16,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 45,
"type": "contact"
},
{
"id": 2,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 47,
"type": "contact"
},
{
"id": 22,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 48,
"type": "contact"
},
{
"id": 15,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 8,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 24,
"ends_at": 24,
"type": "collision"
},
{
"id": 9,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 25,
"ends_at": 26,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
4
],
"outputs": [
0,
4
],
"begins_at": 43,
"ends_at": 43,
"type": "collision"
},
{
"id": 1,
"inputs": [
0,
4
],
"outputs": [
0,
4
],
"begins_at": 48,
"ends_at": 49,
"type": "collision"
},
{
"id": 10,
"inputs": [
1,
4
],
"outputs": [
1,
4
],
"begins_at": 49,
"ends_at": 49,
"type": "collision"
},
{
"id": 23,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 50,
"ends_at": 50,
"type": "collision"
},
{
"id": 11,
"inputs": [
1,
4
],
"outputs": [
1,
4
],
"begins_at": 50,
"ends_at": 51,
"type": "contact"
},
{
"id": 24,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 51,
"ends_at": 54,
"type": "contact"
},
{
"id": 17,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 53,
"ends_at": 53,
"type": "collision"
},
{
"id": 18,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 55,
"ends_at": 55,
"type": "collision"
},
{
"id": 12,
"inputs": [
1,
4
],
"outputs": [
1,
4
],
"begins_at": 56,
"ends_at": 56,
"type": "collision"
},
{
"id": 19,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 56,
"ends_at": 69,
"type": "contact"
},
{
"id": 13,
"inputs": [
1,
4
],
"outputs": [
1,
4
],
"begins_at": 58,
"ends_at": 58,
"type": "collision"
},
{
"id": 14,
"inputs": [
1,
4
],
"outputs": [
1,
4
],
"begins_at": 59,
"ends_at": 60,
"type": "contact"
},
{
"id": 3,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 65,
"ends_at": 65,
"type": "collision"
},
{
"id": 20,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 71,
"ends_at": 71,
"type": "collision"
},
{
"id": 4,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 72,
"ends_at": 73,
"type": "collision"
},
{
"id": 21,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 72,
"ends_at": 119,
"type": "contact"
},
{
"id": 5,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 78,
"ends_at": 79,
"type": "collision"
},
{
"id": 6,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 81,
"ends_at": 81,
"type": "collision"
},
{
"id": 25,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 82,
"ends_at": 82,
"type": "collision"
},
{
"id": 7,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 82,
"ends_at": 119,
"type": "contact"
},
{
"id": 26,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 89,
"ends_at": 89,
"type": "collision"
},
{
"id": 27,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 90,
"ends_at": 93,
"type": "contact"
},
{
"id": 28,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 98,
"ends_at": 98,
"type": "collision"
},
{
"id": 29,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 99,
"ends_at": 119,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.4148888695547403,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 250.26449920040062,
"avg_image_position_segmentation_acc": 0.11833333333333332,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.11833333333333332,
"avg_image_position_segmentation_acc_large": null,
"avg_image_position_segmentation_acc_first_frame": 0.2,
"avg_image_position_segmentation_visible_acc": 0.11833333333333332,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 0.7699999999999999,
"visibility_true_acc": 0.7699999999999999,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.7699999999999999,
"vis_acc_from_px_true_acc": 0.7699999999999999,
"vis_acc_from_px_false_acc": null,
"num_matches": 5,
"num_unmatched_pred": 16,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 5,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 5,
"avg_image_position_segmentation_acc": 5,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 5,
"avg_image_position_segmentation_acc_large": 0,
"avg_image_position_segmentation_acc_first_frame": 5,
"avg_image_position_segmentation_visible_acc": 5,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 5,
"visibility_true_acc": 5,
"visibility_false_acc": 0,
"is_in_frame_acc": 5,
"is_in_frame_true_acc": 5,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 5,
"vis_acc_from_px_true_acc": 5,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 0,
"target_id": 0,
"pred_id": 0,
"cost": 1.825,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.825,
"weighted_cost": 0.825,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 2,
"target_id": 1,
"pred_id": 2,
"cost": 0.9833333333333333,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.9833333333333333,
"weighted_cost": 0.9833333333333333,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 4,
"target_id": 2,
"pred_id": 4,
"cost": 0.6083333333333333,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.6083333333333333,
"weighted_cost": 0.6083333333333333,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 8,
"target_id": 3,
"pred_id": 8,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 1,
"target_id": 4,
"pred_id": 1,
"cost": 0.9916666666666667,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.9916666666666667,
"weighted_cost": 0.9916666666666667,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
3,
5,
6,
7,
9,
10,
11,
12,
13,
14,
15,
16,
17,
18,
19,
20
],
"target_type_counts": {
"blue chopper": 1,
"brown scooter": 1,
"green dirtbike": 1,
"yellow airliner": 1,
"purple utility": 1
},
"pred_type_counts": {
"light blue fabric sports backpack": 1,
"dark grey and purple plastic Shredder action figure": 1,
"black and blue athletic shoe": 1,
"yellow hoodie dollhouse boy": 1,
"green wooden toy train": 2,
"yellow plastic candy bar": 2,
"yellow and blue plastic soccer cleat": 2,
"yellow plastic chocolate powder canister": 1,
"dark grey and pink running shoe": 1,
"yellow plastic chocolate powder jar": 3,
"dark grey and yellow running shoe": 1,
"yellow cardboard curl cream box": 1,
"white and blue track spike": 1,
"white plastic supplement bottle": 1,
"purple and beige wooden dollhouse lamp": 1,
"white and light blue athletic shoe": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 120
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 4,
"false_positives": 1,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 2,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 0.8,
"1.0": 0.6,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 17,
"false_negatives": 1
},
"0.5": {
"true_positives": 4,
"false_positives": 17,
"false_negatives": 1
},
"0.75": {
"true_positives": 4,
"false_positives": 17,
"false_negatives": 1
},
"1.0": {
"true_positives": 2,
"false_positives": 19,
"false_negatives": 3
}
},
"visibility_precision_by_threshold": {
"0.25": 0.19047619047619047,
"0.5": 0.19047619047619047,
"0.75": 0.19047619047619047,
"1.0": 0.09523809523809523
},
"visibility_recall_by_threshold": {
"0.25": 0.8,
"0.5": 0.8,
"0.75": 0.8,
"1.0": 0.4
},
"visibility_f1_by_threshold": {
"0.25": 0.3076923076923077,
"0.5": 0.3076923076923077,
"0.75": 0.3076923076923077,
"1.0": 0.15384615384615385
},
"visibility_by_motion_group": {
"dynamic": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.5
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.5
},
"visibility_f1_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.5
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 20,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 21,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 21,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 21,
"false_negatives": 5
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.047619047619047616,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.2,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.07692307692307693,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.011904761904761904,
"segmentation_frame_average_recall": 0.05,
"segmentation_frame_average_f1": 0.019230769230769232,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 20,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.047619047619047616,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.0909090909090909,
"segmentation_by_motion_group": {
"dynamic": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.25,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.25,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.25,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0625,
"segmentation_frame_average_recall": 0.0625,
"segmentation_frame_average_f1": 0.0625,
"segmentation_first_frame_counts": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.25,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.4
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 0,
"cost": 1.825,
"type_similarity": 0.3143252456242868,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 225.47908009085847,
"image_position_segmentation_acc": 0.175,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.175,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.175,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 2,
"cost": 0.9833333333333333,
"type_similarity": 0.41364326695986986,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 147.4999685425684,
"image_position_segmentation_acc": 0.016666666666666666,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.016666666666666666,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.016666666666666666,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.9166666666666666,
"visibility_true_acc": 0.9166666666666666,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.9166666666666666,
"vis_acc_from_px_true_acc": 0.9166666666666666,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 4,
"cost": 0.6083333333333333,
"type_similarity": 0.44559581767487577,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 59.72971765594537,
"image_position_segmentation_acc": 0.39166666666666666,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.39166666666666666,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.39166666666666666,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 8,
"cost": 1.0,
"type_similarity": 0.4521251320989993,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 481.4055851379375,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.0,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.016666666666666666,
"visibility_true_acc": 0.016666666666666666,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.016666666666666666,
"vis_acc_from_px_true_acc": 0.016666666666666666,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 4,
"pred_id": 1,
"cost": 0.9916666666666667,
"type_similarity": 0.4487548854156699,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 337.20814457469334,
"image_position_segmentation_acc": 0.008333333333333333,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.008333333333333333,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.008333333333333333,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.9166666666666666,
"visibility_true_acc": 0.9166666666666666,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.9166666666666666,
"vis_acc_from_px_true_acc": 0.9166666666666666,
"vis_acc_from_px_false_acc": null
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": -0.5361151535496744,
"avg_interaction_input_object_jaccard": 0.5222222222222225,
"avg_interaction_output_object_jaccard": 0.5222222222222225,
"avg_interaction_type_exact_match_acc": 0.7,
"num_interaction_matches": 30,
"num_unmatched_pred_interactions": 9,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 36,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 37,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 39,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.07692307692307693,
"0.5": 0.05128205128205128,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.14285714285714288,
"0.5": 0.09756097560975609,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 27,
"false_positives": 12,
"false_negatives": 0
},
"0.5": {
"true_positives": 10,
"false_positives": 29,
"false_negatives": 0
},
"0.75": {
"true_positives": 10,
"false_positives": 29,
"false_negatives": 0
},
"1.0": {
"true_positives": 10,
"false_positives": 29,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.6923076923076923,
"0.5": 0.2564102564102564,
"0.75": 0.2564102564102564,
"1.0": 0.2564102564102564
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.8181818181818181,
"0.5": 0.4081632653061224,
"0.75": 0.4081632653061224,
"1.0": 0.4081632653061224
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 27,
"false_positives": 12,
"false_negatives": 0
},
"0.5": {
"true_positives": 10,
"false_positives": 29,
"false_negatives": 0
},
"0.75": {
"true_positives": 10,
"false_positives": 29,
"false_negatives": 0
},
"1.0": {
"true_positives": 10,
"false_positives": 29,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.6923076923076923,
"0.5": 0.2564102564102564,
"0.75": 0.2564102564102564,
"1.0": 0.2564102564102564
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.8181818181818181,
"0.5": 0.4081632653061224,
"0.75": 0.4081632653061224,
"1.0": 0.4081632653061224
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 30,
"avg_interaction_input_object_jaccard": 30,
"avg_interaction_output_object_jaccard": 30,
"avg_interaction_type_exact_match_acc": 30
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 0,
"target_id": 0,
"pred_id": 0,
"cost": 1.7159090909090908,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7159090909090909,
"weighted_cost": 0.7159090909090909,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 21,
"target_id": 1,
"pred_id": 21,
"cost": 1.95,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.95,
"weighted_cost": 0.95,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 1.0,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 1.0,
"weighted_cost": 0.5,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 1,
"target_id": 2,
"pred_id": 1,
"cost": 0.25,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.25,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 12,
"target_id": 3,
"pred_id": 12,
"cost": 2.3106060606060606,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8106060606060606,
"weighted_cost": 0.8106060606060606,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 17,
"target_id": 4,
"pred_id": 17,
"cost": 2.3243243243243246,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8243243243243243,
"weighted_cost": 0.8243243243243243,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 5,
"pred_index": 18,
"target_id": 5,
"pred_id": 18,
"cost": 2.3375,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8375,
"weighted_cost": 0.8375,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 6,
"pred_index": 34,
"target_id": 6,
"pred_id": 34,
"cost": 2.347560975609756,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8475609756097561,
"weighted_cost": 0.8475609756097561,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 7,
"pred_index": 7,
"target_id": 7,
"pred_id": 7,
"cost": 1.2416666666666667,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7416666666666667,
"weighted_cost": 0.7416666666666667,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 8,
"pred_index": 19,
"target_id": 8,
"pred_id": 19,
"cost": 0.9473684210526316,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9473684210526316,
"weighted_cost": 0.9473684210526316,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 9,
"pred_index": 20,
"target_id": 9,
"pred_id": 20,
"cost": 0.525,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.525,
"weighted_cost": 0.525,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 10,
"pred_index": 5,
"target_id": 10,
"pred_id": 5,
"cost": 0.9696969696969697,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9696969696969697,
"weighted_cost": 0.9696969696969697,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 11,
"pred_index": 6,
"target_id": 11,
"pred_id": 6,
"cost": 0.8823529411764706,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8823529411764706,
"weighted_cost": 0.8823529411764706,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 12,
"pred_index": 3,
"target_id": 12,
"pred_id": 3,
"cost": 0.9782608695652174,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9782608695652174,
"weighted_cost": 0.9782608695652174,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 13,
"pred_index": 23,
"target_id": 13,
"pred_id": 23,
"cost": 1.4811320754716981,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9811320754716981,
"weighted_cost": 0.9811320754716981,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 14,
"pred_index": 4,
"target_id": 14,
"pred_id": 4,
"cost": 0.9387755102040816,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9387755102040816,
"weighted_cost": 0.9387755102040816,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 15,
"pred_index": 27,
"target_id": 15,
"pred_id": 27,
"cost": 0.4208333333333334,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.4208333333333334,
"weighted_cost": 0.4208333333333334,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 16,
"pred_index": 33,
"target_id": 16,
"pred_id": 33,
"cost": 0.23913043478260865,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.23913043478260865,
"weighted_cost": 0.23913043478260865,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 17,
"pred_index": 9,
"target_id": 17,
"pred_id": 9,
"cost": 2.2685185185185186,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7685185185185186,
"weighted_cost": 0.7685185185185186,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 18,
"pred_index": 8,
"target_id": 18,
"pred_id": 8,
"cost": 2.2767857142857144,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7767857142857143,
"weighted_cost": 0.7767857142857143,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 19,
"pred_index": 11,
"target_id": 19,
"pred_id": 11,
"cost": 1.2285714285714286,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7285714285714285,
"weighted_cost": 0.7285714285714285,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 20,
"pred_index": 14,
"target_id": 20,
"pred_id": 14,
"cost": 2.326388888888889,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8263888888888888,
"weighted_cost": 0.8263888888888888,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 21,
"pred_index": 13,
"target_id": 21,
"pred_id": 13,
"cost": 1.2,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7,
"weighted_cost": 0.7,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 22,
"pred_index": 10,
"target_id": 22,
"pred_id": 10,
"cost": 0.2551020408163265,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.2551020408163265,
"weighted_cost": 0.2551020408163265,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 23,
"pred_index": 15,
"target_id": 23,
"pred_id": 15,
"cost": 2.2549019607843137,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7549019607843137,
"weighted_cost": 0.7549019607843137,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 24,
"pred_index": 16,
"target_id": 24,
"pred_id": 16,
"cost": 1.2454545454545456,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7454545454545455,
"weighted_cost": 0.7454545454545455,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 25,
"pred_index": 30,
"target_id": 25,
"pred_id": 30,
"cost": 1.4855072463768115,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9855072463768115,
"weighted_cost": 0.9855072463768115,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 26,
"pred_index": 29,
"target_id": 26,
"pred_id": 29,
"cost": 1.4873417721518987,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9873417721518987,
"weighted_cost": 0.9873417721518987,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 27,
"pred_index": 28,
"target_id": 27,
"pred_id": 28,
"cost": 1.351063829787234,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.851063829787234,
"weighted_cost": 0.851063829787234,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 28,
"pred_index": 26,
"target_id": 28,
"pred_id": 26,
"cost": 1.9894736842105263,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9894736842105263,
"weighted_cost": 0.9894736842105263,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 1.0,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 1.0,
"weighted_cost": 0.5,
"used_missing_value": false
}
]
},
{
"target_index": 29,
"pred_index": 35,
"target_id": 29,
"pred_id": 35,
"cost": 1.3125,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8125,
"weighted_cost": 0.8125,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
2,
22,
24,
25,
31,
32,
36,
37,
38
],
"target_type_counts": {
"collision": 17,
"contact": 13
},
"pred_type_counts": {
"contact": 31,
"collision": 8
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 0,
"target_id": 0,
"pred_id": 0,
"target_type": "collision",
"pred_type": "contact",
"cost": 1.7159090909090908,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.4318181818181818,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 1,
"pred_index": 21,
"target_id": 1,
"pred_id": 21,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.95,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 2,
"pred_index": 1,
"target_id": 2,
"pred_id": 1,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.25,
"temporal_span_iou": 0.5,
"temporal_span_giou": 0.5,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 3,
"pred_index": 12,
"target_id": 3,
"pred_id": 12,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.3106060606060606,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.6212121212121212,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 4,
"pred_index": 17,
"target_id": 4,
"pred_id": 17,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.3243243243243246,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.6486486486486487,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 5,
"pred_index": 18,
"target_id": 5,
"pred_id": 18,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.3375,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.675,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 6,
"pred_index": 34,
"target_id": 6,
"pred_id": 34,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.347560975609756,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.6951219512195121,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 7,
"pred_index": 7,
"target_id": 7,
"pred_id": 7,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.2416666666666667,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.48333333333333334,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 8,
"pred_index": 19,
"target_id": 8,
"pred_id": 19,
"target_type": "collision",
"pred_type": "collision",
"cost": 0.9473684210526316,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.8947368421052632,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 9,
"pred_index": 20,
"target_id": 9,
"pred_id": 20,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.525,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.05,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 10,
"pred_index": 5,
"target_id": 10,
"pred_id": 5,
"target_type": "collision",
"pred_type": "collision",
"cost": 0.9696969696969697,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9393939393939394,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 11,
"pred_index": 6,
"target_id": 11,
"pred_id": 6,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.8823529411764706,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.7647058823529411,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 12,
"pred_index": 3,
"target_id": 12,
"pred_id": 3,
"target_type": "collision",
"pred_type": "collision",
"cost": 0.9782608695652174,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9565217391304348,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 13,
"pred_index": 23,
"target_id": 13,
"pred_id": 23,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4811320754716981,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9622641509433962,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 14,
"pred_index": 4,
"target_id": 14,
"pred_id": 4,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.9387755102040816,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.8775510204081632,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 15,
"pred_index": 27,
"target_id": 15,
"pred_id": 27,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.4208333333333334,
"temporal_span_iou": 0.15833333333333333,
"temporal_span_giou": 0.15833333333333333,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 16,
"pred_index": 33,
"target_id": 16,
"pred_id": 33,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.23913043478260865,
"temporal_span_iou": 0.5217391304347826,
"temporal_span_giou": 0.5217391304347826,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 17,
"pred_index": 9,
"target_id": 17,
"pred_id": 9,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.2685185185185186,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.5370370370370371,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 18,
"pred_index": 8,
"target_id": 18,
"pred_id": 8,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.2767857142857144,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.5535714285714286,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 19,
"pred_index": 11,
"target_id": 19,
"pred_id": 11,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.2285714285714286,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.45714285714285713,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 20,
"pred_index": 14,
"target_id": 20,
"pred_id": 14,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.326388888888889,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.6527777777777778,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 21,
"pred_index": 13,
"target_id": 21,
"pred_id": 13,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.2,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.4,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 22,
"pred_index": 10,
"target_id": 22,
"pred_id": 10,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.2551020408163265,
"temporal_span_iou": 0.4897959183673469,
"temporal_span_giou": 0.4897959183673469,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 23,
"pred_index": 15,
"target_id": 23,
"pred_id": 15,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.2549019607843137,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.5098039215686274,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 24,
"pred_index": 16,
"target_id": 24,
"pred_id": 16,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.2454545454545456,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.4909090909090909,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 25,
"pred_index": 30,
"target_id": 25,
"pred_id": 30,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4855072463768115,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9710144927536232,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 26,
"pred_index": 29,
"target_id": 26,
"pred_id": 29,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4873417721518987,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9746835443037974,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 27,
"pred_index": 28,
"target_id": 27,
"pred_id": 28,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.351063829787234,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.7021276595744681,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 28,
"pred_index": 26,
"target_id": 28,
"pred_id": 26,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.9894736842105263,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9789473684210527,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 29,
"pred_index": 35,
"target_id": 29,
"pred_id": 35,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.3125,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.625,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": -0.4431648976351387,
"avg_interaction_input_object_jaccard": 0.4285714285714285,
"avg_interaction_output_object_jaccard": 0.4285714285714285,
"avg_interaction_type_exact_match_acc": 0.6190476190476191,
"num_interaction_matches": 21,
"num_unmatched_pred_interactions": 4,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 22,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 23,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 25,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 25,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.12,
"0.5": 0.08,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.21428571428571425,
"0.5": 0.14814814814814814,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 19,
"false_positives": 6,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 21,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 21,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 21,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.76,
"0.5": 0.16,
"0.75": 0.16,
"1.0": 0.16
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.8636363636363636,
"0.5": 0.2758620689655173,
"0.75": 0.2758620689655173,
"1.0": 0.2758620689655173
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 19,
"false_positives": 6,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 21,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 21,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 21,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.76,
"0.5": 0.16,
"0.75": 0.16,
"1.0": 0.16
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.8636363636363636,
"0.5": 0.2758620689655173,
"0.75": 0.2758620689655173,
"1.0": 0.2758620689655173
}
},
"object_object": {
"avg_interaction_temporal_span_giou": -0.7529990840169244,
"avg_interaction_input_object_jaccard": 0.7407407407407407,
"avg_interaction_output_object_jaccard": 0.7407407407407407,
"avg_interaction_type_exact_match_acc": 0.8888888888888888,
"num_interaction_matches": 9,
"num_unmatched_pred_interactions": 5,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 14,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 8,
"false_positives": 6,
"false_negatives": 0
},
"0.5": {
"true_positives": 6,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 6,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 6,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.5714285714285714,
"0.5": 0.42857142857142855,
"0.75": 0.42857142857142855,
"1.0": 0.42857142857142855
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.7272727272727273,
"0.5": 0.6,
"0.75": 0.6,
"1.0": 0.6
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 8,
"false_positives": 6,
"false_negatives": 0
},
"0.5": {
"true_positives": 6,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 6,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 6,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.5714285714285714,
"0.5": 0.42857142857142855,
"0.75": 0.42857142857142855,
"1.0": 0.42857142857142855
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.7272727272727273,
"0.5": 0.6,
"0.75": 0.6,
"1.0": 0.6
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 21,
"avg_interaction_input_object_jaccard": 21,
"avg_interaction_output_object_jaccard": 21,
"avg_interaction_type_exact_match_acc": 21
},
"object_object": {
"avg_interaction_temporal_span_giou": 9,
"avg_interaction_input_object_jaccard": 9,
"avg_interaction_output_object_jaccard": 9,
"avg_interaction_type_exact_match_acc": 9
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": -0.7589737144061672,
"avg_interaction_input_object_jaccard": 0.3921568627450979,
"avg_interaction_output_object_jaccard": 0.3921568627450979,
"avg_interaction_type_exact_match_acc": 0.47058823529411764,
"num_interaction_matches": 17,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 17,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 14,
"false_positives": 3,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 14,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 14,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 14,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.8235294117647058,
"0.5": 0.17647058823529413,
"0.75": 0.17647058823529413,
"1.0": 0.17647058823529413
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.9032258064516129,
"0.5": 0.3,
"0.75": 0.3,
"1.0": 0.3
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 14,
"false_positives": 3,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 14,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 14,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 14,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.8235294117647058,
"0.5": 0.17647058823529413,
"0.75": 0.17647058823529413,
"1.0": 0.17647058823529413
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.9032258064516129,
"0.5": 0.3,
"0.75": 0.3,
"1.0": 0.3
}
},
"contact": {
"avg_interaction_temporal_span_giou": -0.24468472781426082,
"avg_interaction_input_object_jaccard": 0.6923076923076923,
"avg_interaction_output_object_jaccard": 0.6923076923076923,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 13,
"num_unmatched_pred_interactions": 9,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 3,
"false_positives": 19,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 20,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.13636363636363635,
"0.5": 0.09090909090909091,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.24000000000000002,
"0.5": 0.16666666666666669,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 13,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 7,
"false_positives": 15,
"false_negatives": 0
},
"0.75": {
"true_positives": 7,
"false_positives": 15,
"false_negatives": 0
},
"1.0": {
"true_positives": 7,
"false_positives": 15,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.5909090909090909,
"0.5": 0.3181818181818182,
"0.75": 0.3181818181818182,
"1.0": 0.3181818181818182
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.7428571428571429,
"0.5": 0.4827586206896552,
"0.75": 0.4827586206896552,
"1.0": 0.4827586206896552
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 13,
"false_positives": 9,
"false_negatives": 0
},
"0.5": {
"true_positives": 7,
"false_positives": 15,
"false_negatives": 0
},
"0.75": {
"true_positives": 7,
"false_positives": 15,
"false_negatives": 0
},
"1.0": {
"true_positives": 7,
"false_positives": 15,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.5909090909090909,
"0.5": 0.3181818181818182,
"0.75": 0.3181818181818182,
"1.0": 0.3181818181818182
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.7428571428571429,
"0.5": 0.4827586206896552,
"0.75": 0.4827586206896552,
"1.0": 0.4827586206896552
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 17,
"avg_interaction_input_object_jaccard": 17,
"avg_interaction_output_object_jaccard": 17,
"avg_interaction_type_exact_match_acc": 17
},
"contact": {
"avg_interaction_temporal_span_giou": 13,
"avg_interaction_input_object_jaccard": 13,
"avg_interaction_output_object_jaccard": 13,
"avg_interaction_type_exact_match_acc": 13
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}