STRIVE · 08/28/2026
Dyn generalization · val_120 · super_clevr_1088
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow truck\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 40 683 1 44 683 2 50 683 3 55 683 4 60 683 5 66 683 6 72 683 7 92 682 8 96 682 9 90 682 10 102 682 11 105 682 12 108 682 13 114 682 14 120 682 15 126 682 16 133 682 17 139 682 18 156 681 19 159 681 20 163 681 21 166 681 22 172 681 23 179 681 24 186 681 25 192 681 26 199 681 27 206 680 28 220 680 29 224 680 30 229 680 31 235 680 32 242 680 33 250 679 34 257 679 35 265 679 36 278 679 37 282 679 38 287 679 39 294 679 40 301 678 41 309 678 42 316 678 43 324 678 44 337 678 45 341 678 46 347 678 47 358 680 49 374 683 51 383 686 52 392 689 53 427 699 54 433 703 55 437 707 56 441 710 58 442 714 59 446 718 60 448 721 61 450 724 62 455 727 63 457 730 64 459 734 65 462 738 66 466 741 67 478 742 68 469 745 69 473 748 70 483 751 71 487 755 72 495 758 73 500 763 74 506 766 75 509 768 76 516 772 77 522 778 78 527 780 79 533 786 80 540 793 81 547 802 82 551 807 83 555 810 84 560 816 85 566 819 86 574 814 87 580 820 88 587 822 89 594 825 90 601 829 91 609 831 92 616 836 93 622 843 94 628 848 95 631 858 96 636 869 97 647 872 98 645 881 99 648 886 100 655 892 101 661 897 102 677 903 103 674 904 104 681 906 105 691 908 106 698 911 107 707 914 108 718 916 109 726 919 110 805 915 111 817 921 112 830 924 113 843 928 114 858 931 115 787 934 116 796 938 117 806 941 118 817 944 119 829 947 [OBJ] 1 \"green sedan\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 346 544 2 348 546 3 349 548 5 351 551 7 358 554 8 354 556 10 355 559 11 358 561 12 362 563 13 358 565 14 364 567 15 362 569 16 362 571 18 364 574 19 367 576 21 368 580 22 369 583 23 371 584 24 373 588 25 373 590 26 375 592 27 376 595 28 377 597 29 380 600 30 380 602 31 382 604 32 384 607 33 387 610 35 387 616 36 388 619 37 389 621 38 390 625 39 391 627 40 399 630 41 405 630 42 406 627 44 413 628 45 397 628 46 407 630 47 400 631 48 408 632 49 416 635 50 423 638 51 430 641 52 437 643 53 441 643 54 446 645 55 393 636 56 405 635 57 405 639 58 483 676 59 489 678 60 495 681 61 507 688 62 512 690 64 516 690 65 525 691 66 523 690 67 529 692 68 533 693 69 539 695 70 547 696 71 554 697 72 562 698 73 544 690 74 548 690 75 551 690 76 555 690 77 560 692 78 564 692 79 571 694 80 579 696 81 489 703 82 497 705 83 505 707 84 512 710 85 521 713 86 532 713 87 541 715 88 551 716 89 559 717 90 576 711 91 585 714 92 594 715 93 606 714 94 617 716 95 627 718 96 638 719 97 646 718 98 651 719 99 655 722 102 662 722 103 658 723 104 664 723 105 669 724 106 675 725 107 679 725 108 683 725 109 687 725 110 692 726 111 696 726 112 700 727 113 702 726 114 708 728 115 711 728 116 715 729 117 719 729 118 722 729 119 724 729 [OBJ] 2 \"red truck\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 433 541 1 430 541 3 431 543 7 431 546 9 433 548 13 433 551 16 433 554 19 434 557 22 435 560 25 435 564 27 436 566 28 437 568 31 437 571 33 437 574 35 438 576 37 439 579 40 441 583 43 441 586 44 442 589 47 444 593 48 446 594 51 448 596 54 450 597 57 452 597 61 449 598 62 452 599 63 455 600 64 437 601 65 440 603 66 443 604 67 441 607 69 441 610 70 445 613 71 442 616 73 445 620 74 449 622 75 447 626 76 448 628 77 449 630 78 456 632 79 451 635 80 454 639 81 458 642 82 458 646 83 460 648 85 461 651 86 462 653 87 462 655 89 464 658 90 465 660 92 466 664 93 466 666 95 468 669 96 469 671 97 469 673 99 470 676 100 472 678 101 473 680 103 473 683 104 474 685 106 476 691 108 477 694 110 478 697 111 479 699 113 480 702 114 480 704 115 481 706 116 482 709 117 483 711 118 483 714 [OBJ] 3 \"gray wagon\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 773 706 89 782 706 90 768 707 91 773 707 92 783 707 93 799 708 94 813 708 95 822 708 96 802 709 97 816 709 98 826 709 99 818 710 100 829 710 101 839 710 102 811 714 103 841 713 104 850 713 105 858 713 106 855 714 107 862 714 108 870 714 109 880 714 110 874 715 111 870 717 112 889 716 113 898 716 114 908 716 115 900 717 116 907 717 117 914 717 118 883 722 119 898 721 [OBJ] 4 \"green jet\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 976 329 1 985 325 2 969 329 3 984 323 5 984 320 6 983 319 7 983 316 8 981 315 9 979 314 10 975 314 11 971 314 12 967 314 13 963 313 14 958 314 15 955 313 16 950 313 17 945 314 18 941 314 19 936 314 20 931 314 21 926 314 22 921 314 23 916 314 24 911 314 25 906 314 26 901 314 27 895 314 28 891 313 29 885 313 30 879 314 31 873 313 32 867 314 33 861 314 34 855 314 35 849 314 36 843 314 37 837 313 38 830 313 39 824 313 40 816 315 41 810 314 42 803 315 43 797 314 44 790 314 45 783 313 46 776 315 47 769 315 48 762 315 49 755 315 50 748 314 51 741 314 52 733 314 53 726 315 54 719 314 55 712 315 56 705 315 57 697 315 58 690 314 59 682 315 60 675 314 61 667 314 62 659 315 63 651 315 64 644 315 65 636 315 66 628 314 67 620 314 68 612 314 69 604 315 70 596 315 71 587 315 72 580 314 73 572 314 74 563 314 75 555 315 76 546 315 77 537 315 78 529 315 79 520 314 80 511 316 81 502 316 82 494 315 83 485 315 84 476 314 85 466 316 86 458 316 87 448 316 88 440 315 89 430 314 90 420 316 91 411 316 92 401 316 93 392 315 94 383 315 95 373 316 96 362 316 97 353 315 98 343 315 99 333 316 100 323 316 101 312 316 102 302 315 103 292 315 104 281 316 105 271 315 106 260 316 107 249 316 108 239 315 109 228 315 110 216 316 111 205 316 112 194 315 113 183 315 114 172 316 115 160 316 116 149 315 117 137 315 118 126 316 119 113 316 [OBJS-END] [INTERACTIONS-START] [INTERACT] 5 \"contact\" 0 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 13 \"contact\" 0 119 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 14 \"contact\" 0 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 15 \"contact\" 0 119 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 6 \"collision\" 33 33 [IN] 1 2 [OUT] 1 2 [INTERACT] 7 \"contact\" 34 36 [IN] 1 2 [OUT] 1 2 [INTERACT] 0 \"collision\" 46 46 [IN] 0 1 [OUT] 0 1 [INTERACT] 1 \"collision\" 53 53 [IN] 0 1 [OUT] 0 1 [INTERACT] 2 \"collision\" 55 55 [IN] 0 1 [OUT] 0 1 [INTERACT] 3 \"collision\" 57 57 [IN] 0 1 [OUT] 0 1 [INTERACT] 4 \"contact\" 58 59 [IN] 0 1 [OUT] 0 1 [INTERACT] 10 \"collision\" 89 89 [IN] 1 3 [OUT] 1 3 [INTERACT] 11 \"collision\" 93 93 [IN] 1 3 [OUT] 1 3 [INTERACT] 12 \"contact\" 94 108 [IN] 1 3 [OUT] 1 3 [INTERACT] 8 \"collision\" 113 113 [IN] 1 3 [OUT] 1 3 [INTERACT] 9 \"contact\" 114 117 [IN] 1 3 [OUT] 1 3 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/val_full/super_clevr_1088/frames.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow plastic toy school bus\" 0 24 [VIS] 0 true 23 false [IN-FRAME] 0 true 23 false [PX] 0 13 990 2 5 982 3 13 990 4 5 986 7 1 974 10 1 986 14 1 974 17 1 962 20 1 951 22 1 998 23 null null [OBJ] 1 \"yellow plastic toy school bus\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 56 693 1 76 693 2 48 697 3 64 697 4 103 697 5 68 701 6 80 701 7 115 701 8 80 705 9 91 705 10 119 705 11 130 705 12 142 705 13 154 705 14 166 705 15 177 705 16 189 705 17 201 705 18 212 705 19 224 705 20 236 705 21 251 701 22 263 701 23 275 701 [OBJ] 2 \"brown plastic plant pot\" 0 24 [VIS] 0 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 119 310 3 119 322 6 119 333 8 123 341 10 123 353 11 null null 12 123 357 15 123 369 17 126 380 21 126 392 [OBJ] 3 \"dark brown fabric loafer\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 134 474 1 138 482 4 134 474 7 138 482 14 134 474 15 138 482 19 134 474 21 138 482 [OBJ] 4 \"green plate\" 0 24 [VIS] 0 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 142 318 2 142 333 3 142 349 5 142 365 7 142 380 9 142 392 11 null null 12 142 404 14 142 416 16 142 427 18 142 439 21 142 451 [OBJ] 5 \"dark brown fabric loafer\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 146 482 1 150 490 4 154 501 8 154 513 11 154 525 14 154 537 17 154 548 20 154 560 22 150 568 [OBJ] 6 \"green plate\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 345 544 1 349 552 3 357 556 5 353 564 7 361 568 8 373 572 10 369 580 12 373 587 14 376 595 16 380 603 18 388 607 20 392 615 22 396 623 23 404 626 [OBJ] 7 \"dark red and black plastic curling wand\" 0 24 [VIS] 0 true 11 false 13 true [IN-FRAME] 0 true [PX] 0 365 548 2 369 556 5 373 564 8 376 572 10 380 580 11 null null 13 384 587 15 388 595 18 392 603 20 396 611 23 400 619 [OBJ] 8 \"red ceramic square butter dish\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 431 548 2 435 556 4 431 548 6 435 556 13 435 568 15 439 576 18 439 587 20 443 595 23 447 607 [OBJ] 9 \"dark grey and orange athletic shoe\" 0 24 [VIS] 0 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 435 521 1 443 525 2 439 533 4 443 541 6 447 548 9 451 556 11 null null 12 451 568 14 455 576 16 458 583 19 458 595 21 462 603 23 466 611 [OBJ] 10 \"white plastic Nintendo 3DS game case\" 0 24 [VIS] 0 true 11 false 12 true [IN-FRAME] 0 true [PX] 0 615 466 4 611 458 9 607 451 11 null null 12 607 447 15 603 439 18 599 431 21 595 423 23 591 416 [OBJ] 11 \"white metallic wall mount\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 658 1 2 666 5 4 673 9 6 681 13 9 689 17 12 697 21 14 705 25 17 712 29 20 720 33 [OBJ] 12 \"white and brown leather golf shoe\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 666 41 2 669 48 4 673 56 6 677 64 9 681 76 12 685 87 14 685 99 17 689 111 20 689 123 23 689 134 [OBJ] 13 \"white and grey cardboard desktop hard drive box\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 759 685 1 791 689 3 779 689 4 771 693 5 763 697 7 751 697 8 744 701 10 732 701 12 724 705 14 712 705 16 701 705 18 791 693 19 783 697 21 775 701 23 767 705 [OBJ] 14 \"dark green fabric loafer\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 982 314 2 974 318 4 966 322 5 962 330 6 958 337 7 951 345 8 943 353 9 935 357 10 927 361 11 919 365 12 912 369 13 904 373 14 896 376 15 888 380 16 880 384 17 873 388 18 865 392 19 857 396 20 849 400 22 837 408 23 830 412 [OBJ] 15 \"dark grey cardboard motherboard box\" 0 24 [VIS] 0 true 5 false [IN-FRAME] 0 true 5 false [PX] 0 998 541 1 998 552 2 998 564 3 998 576 4 998 591 5 null null [OBJ] 16 \"red and blue plastic Mario action figure\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 521 1 998 537 2 998 552 3 998 568 4 998 583 5 998 599 6 998 611 7 998 623 8 998 634 9 998 646 10 994 671 12 994 654 14 994 642 15 990 669 17 990 658 18 986 685 20 986 673 21 982 697 23 982 685 [OBJ] 17 \"red and green cardboard ink cartridge box\" 0 24 [VIS] 0 true 6 false [IN-FRAME] 0 true 6 false [PX] 0 998 544 1 998 556 2 998 568 3 998 580 4 998 591 5 998 603 6 null null [OBJ] 18 \"yellow plastic toy school bus\" 5 24 [VIS] 0 false 5 true [IN-FRAME] 0 false 5 true [PX] 0 null null 5 388 685 7 396 681 9 404 677 10 412 681 11 419 685 12 427 693 13 435 701 14 455 720 15 474 744 16 490 755 17 498 767 18 498 779 19 494 787 20 486 791 21 482 798 22 474 806 23 462 794 [OBJ] 19 \"red and black plastic handheld game console\" 6 24 [VIS] 0 false 6 true [IN-FRAME] 0 false 6 true [PX] 0 null null 6 91 341 8 91 353 11 91 365 13 95 373 16 95 384 18 99 392 20 99 404 22 103 412 23 107 419 [OBJ] 20 \"green and yellow running shoe\" 10 24 [VIS] 0 false 10 true [IN-FRAME] 0 false 10 true [PX] 0 null null 10 99 376 11 107 384 13 111 392 15 115 400 17 119 408 20 123 416 23 126 423 [OBJ] 21 \"green and yellow running shoe\" 12 24 [VIS] 0 false 12 true [IN-FRAME] 0 false 12 true [PX] 0 null null 12 1 591 13 1 580 15 5 587 16 5 576 18 9 583 19 9 572 21 13 580 22 13 568 23 13 556 [OBJ] 22 \"green and yellow running shoe\" 14 24 [VIS] 0 false 14 true [IN-FRAME] 0 false 14 true [PX] 0 null null 14 107 380 16 111 392 18 115 400 20 119 408 23 123 419 [OBJ] 23 \"green and grey plastic Doctor Doom action figure\" 18 24 [VIS] 0 false 18 true [IN-FRAME] 0 false 18 true [PX] 0 null null 18 1 455 19 1 443 21 5 458 22 5 447 23 5 435 [OBJ] 24 \"green and yellow running shoe\" 21 24 [VIS] 0 false 21 true [IN-FRAME] 0 false 21 true [PX] 0 null null 21 1 431 22 1 419 23 1 404 [OBJ] 25 \"green and yellow running shoe\" 23 24 [VIS] 0 false 23 true [IN-FRAME] 0 false 23 true [PX] 0 null null 23 1 423 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 2 [IN] 1 2 [OUT] 1 2 [INTERACT] 21 \"contact\" 0 4 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 29 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 30 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 31 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 32 \"contact\" 0 23 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 33 \"contact\" 0 23 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 34 \"contact\" 0 23 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 35 \"contact\" 0 23 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 36 \"contact\" 0 23 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 5 \"contact\" 0 23 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 6 \"contact\" 0 23 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 7 \"contact\" 0 23 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 8 \"contact\" 0 23 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 9 \"contact\" 0 23 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 10 \"contact\" 0 23 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 15 \"contact\" 0 23 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 16 \"contact\" 0 23 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 17 \"contact\" 0 23 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 18 \"contact\" 0 23 [IN] 22 65535 [OUT] 22 65535 [INTERACT] 19 \"contact\" 0 23 [IN] 23 65535 [OUT] 23 65535 [INTERACT] 20 \"contact\" 0 23 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 22 \"contact\" 0 23 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 23 \"collision\" 4 4 [IN] 19 20 [OUT] 19 20 [INTERACT] 24 \"contact\" 5 7 [IN] 19 20 [OUT] 19 20 [INTERACT] 14 \"collision\" 9 9 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 25 \"collision\" 10 10 [IN] 19 20 [OUT] 19 20 [INTERACT] 13 \"contact\" 10 23 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 26 \"collision\" 13 13 [IN] 19 20 [OUT] 19 20 [INTERACT] 27 \"collision\" 16 16 [IN] 19 20 [OUT] 19 20 [INTERACT] 28 \"contact\" 17 23 [IN] 19 20 [OUT] 19 20 [INTERACT] 11 \"collision\" 19 19 [IN] 18 19 [OUT] 18 19 [INTERACT] 12 \"contact\" 20 23 [IN] 18 19 [OUT] 18 19 [INTERACTIONS-END] [SCENE-END]",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow truck\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 40 683 1 44 683 2 50 683 3 55 683 4 60 683 5 66 683 6 72 683 7 92 682 8 96 682 9 90 682 10 102 682 11 105 682 12 108 682 13 114 682 14 120 682 15 126 682 16 133 682 17 139 682 18 156 681 19 159 681 20 163 681 21 166 681 22 172 681 23 179 681 24 186 681 25 192 681 26 199 681 27 206 680 28 220 680 29 224 680 30 229 680 31 235 680 32 242 680 33 250 679 34 257 679 35 265 679 36 278 679 37 282 679 38 287 679 39 294 679 40 301 678 41 309 678 42 316 678 43 324 678 44 337 678 45 341 678 46 347 678 47 358 680 49 374 683 51 383 686 52 392 689 53 427 699 54 433 703 55 437 707 56 441 710 58 442 714 59 446 718 60 448 721 61 450 724 62 455 727 63 457 730 64 459 734 65 462 738 66 466 741 67 478 742 68 469 745 69 473 748 70 483 751 71 487 755 72 495 758 73 500 763 74 506 766 75 509 768 76 516 772 77 522 778 78 527 780 79 533 786 80 540 793 81 547 802 82 551 807 83 555 810 84 560 816 85 566 819 86 574 814 87 580 820 88 587 822 89 594 825 90 601 829 91 609 831 92 616 836 93 622 843 94 628 848 95 631 858 96 636 869 97 647 872 98 645 881 99 648 886 100 655 892 101 661 897 102 677 903 103 674 904 104 681 906 105 691 908 106 698 911 107 707 914 108 718 916 109 726 919 110 805 915 111 817 921 112 830 924 113 843 928 114 858 931 115 787 934 116 796 938 117 806 941 118 817 944 119 829 947 [OBJ] 1 \"green sedan\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 346 544 2 348 546 3 349 548 5 351 551 7 358 554 8 354 556 10 355 559 11 358 561 12 362 563 13 358 565 14 364 567 15 362 569 16 362 571 18 364 574 19 367 576 21 368 580 22 369 583 23 371 584 24 373 588 25 373 590 26 375 592 27 376 595 28 377 597 29 380 600 30 380 602 31 382 604 32 384 607 33 387 610 35 387 616 36 388 619 37 389 621 38 390 625 39 391 627 40 399 630 41 405 630 42 406 627 44 413 628 45 397 628 46 407 630 47 400 631 48 408 632 49 416 635 50 423 638 51 430 641 52 437 643 53 441 643 54 446 645 55 393 636 56 405 635 57 405 639 58 483 676 59 489 678 60 495 681 61 507 688 62 512 690 64 516 690 65 525 691 66 523 690 67 529 692 68 533 693 69 539 695 70 547 696 71 554 697 72 562 698 73 544 690 74 548 690 75 551 690 76 555 690 77 560 692 78 564 692 79 571 694 80 579 696 81 489 703 82 497 705 83 505 707 84 512 710 85 521 713 86 532 713 87 541 715 88 551 716 89 559 717 90 576 711 91 585 714 92 594 715 93 606 714 94 617 716 95 627 718 96 638 719 97 646 718 98 651 719 99 655 722 102 662 722 103 658 723 104 664 723 105 669 724 106 675 725 107 679 725 108 683 725 109 687 725 110 692 726 111 696 726 112 700 727 113 702 726 114 708 728 115 711 728 116 715 729 117 719 729 118 722 729 119 724 729 [OBJ] 2 \"red truck\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 433 541 1 430 541 3 431 543 7 431 546 9 433 548 13 433 551 16 433 554 19 434 557 22 435 560 25 435 564 27 436 566 28 437 568 31 437 571 33 437 574 35 438 576 37 439 579 40 441 583 43 441 586 44 442 589 47 444 593 48 446 594 51 448 596 54 450 597 57 452 597 61 449 598 62 452 599 63 455 600 64 437 601 65 440 603 66 443 604 67 441 607 69 441 610 70 445 613 71 442 616 73 445 620 74 449 622 75 447 626 76 448 628 77 449 630 78 456 632 79 451 635 80 454 639 81 458 642 82 458 646 83 460 648 85 461 651 86 462 653 87 462 655 89 464 658 90 465 660 92 466 664 93 466 666 95 468 669 96 469 671 97 469 673 99 470 676 100 472 678 101 473 680 103 473 683 104 474 685 106 476 691 108 477 694 110 478 697 111 479 699 113 480 702 114 480 704 115 481 706 116 482 709 117 483 711 118 483 714 [OBJ] 3 \"gray wagon\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 773 706 89 782 706 90 768 707 91 773 707 92 783 707 93 799 708 94 813 708 95 822 708 96 802 709 97 816 709 98 826 709 99 818 710 100 829 710 101 839 710 102 811 714 103 841 713 104 850 713 105 858 713 106 855 714 107 862 714 108 870 714 109 880 714 110 874 715 111 870 717 112 889 716 113 898 716 114 908 716 115 900 717 116 907 717 117 914 717 118 883 722 119 898 721 [OBJ] 4 \"green jet\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 976 329 1 985 325 2 969 329 3 984 323 5 984 320 6 983 319 7 983 316 8 981 315 9 979 314 10 975 314 11 971 314 12 967 314 13 963 313 14 958 314 15 955 313 16 950 313 17 945 314 18 941 314 19 936 314 20 931 314 21 926 314 22 921 314 23 916 314 24 911 314 25 906 314 26 901 314 27 895 314 28 891 313 29 885 313 30 879 314 31 873 313 32 867 314 33 861 314 34 855 314 35 849 314 36 843 314 37 837 313 38 830 313 39 824 313 40 816 315 41 810 314 42 803 315 43 797 314 44 790 314 45 783 313 46 776 315 47 769 315 48 762 315 49 755 315 50 748 314 51 741 314 52 733 314 53 726 315 54 719 314 55 712 315 56 705 315 57 697 315 58 690 314 59 682 315 60 675 314 61 667 314 62 659 315 63 651 315 64 644 315 65 636 315 66 628 314 67 620 314 68 612 314 69 604 315 70 596 315 71 587 315 72 580 314 73 572 314 74 563 314 75 555 315 76 546 315 77 537 315 78 529 315 79 520 314 80 511 316 81 502 316 82 494 315 83 485 315 84 476 314 85 466 316 86 458 316 87 448 316 88 440 315 89 430 314 90 420 316 91 411 316 92 401 316 93 392 315 94 383 315 95 373 316 96 362 316 97 353 315 98 343 315 99 333 316 100 323 316 101 312 316 102 302 315 103 292 315 104 281 316 105 271 315 106 260 316 107 249 316 108 239 315 109 228 315 110 216 316 111 205 316 112 194 315 113 183 315 114 172 316 115 160 316 116 149 315 117 137 315 118 126 316 119 113 316 [OBJS-END] [INTERACTIONS-START] [INTERACT] 5 \"contact\" 0 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 13 \"contact\" 0 119 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 14 \"contact\" 0 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 15 \"contact\" 0 119 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 6 \"collision\" 33 33 [IN] 1 2 [OUT] 1 2 [INTERACT] 7 \"contact\" 34 36 [IN] 1 2 [OUT] 1 2 [INTERACT] 0 \"collision\" 46 46 [IN] 0 1 [OUT] 0 1 [INTERACT] 1 \"collision\" 53 53 [IN] 0 1 [OUT] 0 1 [INTERACT] 2 \"collision\" 55 55 [IN] 0 1 [OUT] 0 1 [INTERACT] 3 \"collision\" 57 57 [IN] 0 1 [OUT] 0 1 [INTERACT] 4 \"contact\" 58 59 [IN] 0 1 [OUT] 0 1 [INTERACT] 10 \"collision\" 89 89 [IN] 1 3 [OUT] 1 3 [INTERACT] 11 \"collision\" 93 93 [IN] 1 3 [OUT] 1 3 [INTERACT] 12 \"contact\" 94 108 [IN] 1 3 [OUT] 1 3 [INTERACT] 8 \"collision\" 113 113 [IN] 1 3 [OUT] 1 3 [INTERACT] 9 \"contact\" 114 117 [IN] 1 3 [OUT] 1 3 [INTERACTIONS-END] [SCENE-END]\n"
}Parsed prediction
{
"objects": [
{
"id": 0,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 23,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 13,
"y": 990
},
{
"t": 2,
"x": 5,
"y": 982
},
{
"t": 3,
"x": 13,
"y": 990
},
{
"t": 4,
"x": 5,
"y": 986
},
{
"t": 7,
"x": 1,
"y": 974
},
{
"t": 10,
"x": 1,
"y": 986
},
{
"t": 14,
"x": 1,
"y": 974
},
{
"t": 17,
"x": 1,
"y": 962
},
{
"t": 20,
"x": 1,
"y": 951
},
{
"t": 22,
"x": 1,
"y": 998
},
{
"t": 23,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 56,
"y": 693
},
{
"t": 1,
"x": 76,
"y": 693
},
{
"t": 2,
"x": 48,
"y": 697
},
{
"t": 3,
"x": 64,
"y": 697
},
{
"t": 4,
"x": 103,
"y": 697
},
{
"t": 5,
"x": 68,
"y": 701
},
{
"t": 6,
"x": 80,
"y": 701
},
{
"t": 7,
"x": 115,
"y": 701
},
{
"t": 8,
"x": 80,
"y": 705
},
{
"t": 9,
"x": 91,
"y": 705
},
{
"t": 10,
"x": 119,
"y": 705
},
{
"t": 11,
"x": 130,
"y": 705
},
{
"t": 12,
"x": 142,
"y": 705
},
{
"t": 13,
"x": 154,
"y": 705
},
{
"t": 14,
"x": 166,
"y": 705
},
{
"t": 15,
"x": 177,
"y": 705
},
{
"t": 16,
"x": 189,
"y": 705
},
{
"t": 17,
"x": 201,
"y": 705
},
{
"t": 18,
"x": 212,
"y": 705
},
{
"t": 19,
"x": 224,
"y": 705
},
{
"t": 20,
"x": 236,
"y": 705
},
{
"t": 21,
"x": 251,
"y": 701
},
{
"t": 22,
"x": 263,
"y": 701
},
{
"t": 23,
"x": 275,
"y": 701
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "brown plastic plant pot",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 119,
"y": 310
},
{
"t": 3,
"x": 119,
"y": 322
},
{
"t": 6,
"x": 119,
"y": 333
},
{
"t": 8,
"x": 123,
"y": 341
},
{
"t": 10,
"x": 123,
"y": 353
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 123,
"y": 357
},
{
"t": 15,
"x": 123,
"y": 369
},
{
"t": 17,
"x": 126,
"y": 380
},
{
"t": 21,
"x": 126,
"y": 392
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "dark brown fabric loafer",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 134,
"y": 474
},
{
"t": 1,
"x": 138,
"y": 482
},
{
"t": 4,
"x": 134,
"y": 474
},
{
"t": 7,
"x": 138,
"y": 482
},
{
"t": 14,
"x": 134,
"y": 474
},
{
"t": 15,
"x": 138,
"y": 482
},
{
"t": 19,
"x": 134,
"y": 474
},
{
"t": 21,
"x": 138,
"y": 482
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "green plate",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 142,
"y": 318
},
{
"t": 2,
"x": 142,
"y": 333
},
{
"t": 3,
"x": 142,
"y": 349
},
{
"t": 5,
"x": 142,
"y": 365
},
{
"t": 7,
"x": 142,
"y": 380
},
{
"t": 9,
"x": 142,
"y": 392
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 142,
"y": 404
},
{
"t": 14,
"x": 142,
"y": 416
},
{
"t": 16,
"x": 142,
"y": 427
},
{
"t": 18,
"x": 142,
"y": 439
},
{
"t": 21,
"x": 142,
"y": 451
}
],
"birth": 0,
"death": 24
},
{
"id": 5,
"type": "dark brown fabric loafer",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 146,
"y": 482
},
{
"t": 1,
"x": 150,
"y": 490
},
{
"t": 4,
"x": 154,
"y": 501
},
{
"t": 8,
"x": 154,
"y": 513
},
{
"t": 11,
"x": 154,
"y": 525
},
{
"t": 14,
"x": 154,
"y": 537
},
{
"t": 17,
"x": 154,
"y": 548
},
{
"t": 20,
"x": 154,
"y": 560
},
{
"t": 22,
"x": 150,
"y": 568
}
],
"birth": 0,
"death": 24
},
{
"id": 6,
"type": "green plate",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 345,
"y": 544
},
{
"t": 1,
"x": 349,
"y": 552
},
{
"t": 3,
"x": 357,
"y": 556
},
{
"t": 5,
"x": 353,
"y": 564
},
{
"t": 7,
"x": 361,
"y": 568
},
{
"t": 8,
"x": 373,
"y": 572
},
{
"t": 10,
"x": 369,
"y": 580
},
{
"t": 12,
"x": 373,
"y": 587
},
{
"t": 14,
"x": 376,
"y": 595
},
{
"t": 16,
"x": 380,
"y": 603
},
{
"t": 18,
"x": 388,
"y": 607
},
{
"t": 20,
"x": 392,
"y": 615
},
{
"t": 22,
"x": 396,
"y": 623
},
{
"t": 23,
"x": 404,
"y": 626
}
],
"birth": 0,
"death": 24
},
{
"id": 7,
"type": "dark red and black plastic curling wand",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 13,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 365,
"y": 548
},
{
"t": 2,
"x": 369,
"y": 556
},
{
"t": 5,
"x": 373,
"y": 564
},
{
"t": 8,
"x": 376,
"y": 572
},
{
"t": 10,
"x": 380,
"y": 580
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 13,
"x": 384,
"y": 587
},
{
"t": 15,
"x": 388,
"y": 595
},
{
"t": 18,
"x": 392,
"y": 603
},
{
"t": 20,
"x": 396,
"y": 611
},
{
"t": 23,
"x": 400,
"y": 619
}
],
"birth": 0,
"death": 24
},
{
"id": 8,
"type": "red ceramic square butter dish",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 431,
"y": 548
},
{
"t": 2,
"x": 435,
"y": 556
},
{
"t": 4,
"x": 431,
"y": 548
},
{
"t": 6,
"x": 435,
"y": 556
},
{
"t": 13,
"x": 435,
"y": 568
},
{
"t": 15,
"x": 439,
"y": 576
},
{
"t": 18,
"x": 439,
"y": 587
},
{
"t": 20,
"x": 443,
"y": 595
},
{
"t": 23,
"x": 447,
"y": 607
}
],
"birth": 0,
"death": 24
},
{
"id": 9,
"type": "dark grey and orange athletic shoe",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 435,
"y": 521
},
{
"t": 1,
"x": 443,
"y": 525
},
{
"t": 2,
"x": 439,
"y": 533
},
{
"t": 4,
"x": 443,
"y": 541
},
{
"t": 6,
"x": 447,
"y": 548
},
{
"t": 9,
"x": 451,
"y": 556
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 451,
"y": 568
},
{
"t": 14,
"x": 455,
"y": 576
},
{
"t": 16,
"x": 458,
"y": 583
},
{
"t": 19,
"x": 458,
"y": 595
},
{
"t": 21,
"x": 462,
"y": 603
},
{
"t": 23,
"x": 466,
"y": 611
}
],
"birth": 0,
"death": 24
},
{
"id": 10,
"type": "white plastic Nintendo 3DS game case",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 11,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 615,
"y": 466
},
{
"t": 4,
"x": 611,
"y": 458
},
{
"t": 9,
"x": 607,
"y": 451
},
{
"t": 11,
"x": null,
"y": null
},
{
"t": 12,
"x": 607,
"y": 447
},
{
"t": 15,
"x": 603,
"y": 439
},
{
"t": 18,
"x": 599,
"y": 431
},
{
"t": 21,
"x": 595,
"y": 423
},
{
"t": 23,
"x": 591,
"y": 416
}
],
"birth": 0,
"death": 24
},
{
"id": 11,
"type": "white metallic wall mount",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 658,
"y": 1
},
{
"t": 2,
"x": 666,
"y": 5
},
{
"t": 4,
"x": 673,
"y": 9
},
{
"t": 6,
"x": 681,
"y": 13
},
{
"t": 9,
"x": 689,
"y": 17
},
{
"t": 12,
"x": 697,
"y": 21
},
{
"t": 14,
"x": 705,
"y": 25
},
{
"t": 17,
"x": 712,
"y": 29
},
{
"t": 20,
"x": 720,
"y": 33
}
],
"birth": 0,
"death": 24
},
{
"id": 12,
"type": "white and brown leather golf shoe",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 666,
"y": 41
},
{
"t": 2,
"x": 669,
"y": 48
},
{
"t": 4,
"x": 673,
"y": 56
},
{
"t": 6,
"x": 677,
"y": 64
},
{
"t": 9,
"x": 681,
"y": 76
},
{
"t": 12,
"x": 685,
"y": 87
},
{
"t": 14,
"x": 685,
"y": 99
},
{
"t": 17,
"x": 689,
"y": 111
},
{
"t": 20,
"x": 689,
"y": 123
},
{
"t": 23,
"x": 689,
"y": 134
}
],
"birth": 0,
"death": 24
},
{
"id": 13,
"type": "white and grey cardboard desktop hard drive box",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 759,
"y": 685
},
{
"t": 1,
"x": 791,
"y": 689
},
{
"t": 3,
"x": 779,
"y": 689
},
{
"t": 4,
"x": 771,
"y": 693
},
{
"t": 5,
"x": 763,
"y": 697
},
{
"t": 7,
"x": 751,
"y": 697
},
{
"t": 8,
"x": 744,
"y": 701
},
{
"t": 10,
"x": 732,
"y": 701
},
{
"t": 12,
"x": 724,
"y": 705
},
{
"t": 14,
"x": 712,
"y": 705
},
{
"t": 16,
"x": 701,
"y": 705
},
{
"t": 18,
"x": 791,
"y": 693
},
{
"t": 19,
"x": 783,
"y": 697
},
{
"t": 21,
"x": 775,
"y": 701
},
{
"t": 23,
"x": 767,
"y": 705
}
],
"birth": 0,
"death": 24
},
{
"id": 14,
"type": "dark green fabric loafer",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 982,
"y": 314
},
{
"t": 2,
"x": 974,
"y": 318
},
{
"t": 4,
"x": 966,
"y": 322
},
{
"t": 5,
"x": 962,
"y": 330
},
{
"t": 6,
"x": 958,
"y": 337
},
{
"t": 7,
"x": 951,
"y": 345
},
{
"t": 8,
"x": 943,
"y": 353
},
{
"t": 9,
"x": 935,
"y": 357
},
{
"t": 10,
"x": 927,
"y": 361
},
{
"t": 11,
"x": 919,
"y": 365
},
{
"t": 12,
"x": 912,
"y": 369
},
{
"t": 13,
"x": 904,
"y": 373
},
{
"t": 14,
"x": 896,
"y": 376
},
{
"t": 15,
"x": 888,
"y": 380
},
{
"t": 16,
"x": 880,
"y": 384
},
{
"t": 17,
"x": 873,
"y": 388
},
{
"t": 18,
"x": 865,
"y": 392
},
{
"t": 19,
"x": 857,
"y": 396
},
{
"t": 20,
"x": 849,
"y": 400
},
{
"t": 22,
"x": 837,
"y": 408
},
{
"t": 23,
"x": 830,
"y": 412
}
],
"birth": 0,
"death": 24
},
{
"id": 15,
"type": "dark grey cardboard motherboard box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 5,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 5,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 541
},
{
"t": 1,
"x": 998,
"y": 552
},
{
"t": 2,
"x": 998,
"y": 564
},
{
"t": 3,
"x": 998,
"y": 576
},
{
"t": 4,
"x": 998,
"y": 591
},
{
"t": 5,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 16,
"type": "red and blue plastic Mario action figure",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 521
},
{
"t": 1,
"x": 998,
"y": 537
},
{
"t": 2,
"x": 998,
"y": 552
},
{
"t": 3,
"x": 998,
"y": 568
},
{
"t": 4,
"x": 998,
"y": 583
},
{
"t": 5,
"x": 998,
"y": 599
},
{
"t": 6,
"x": 998,
"y": 611
},
{
"t": 7,
"x": 998,
"y": 623
},
{
"t": 8,
"x": 998,
"y": 634
},
{
"t": 9,
"x": 998,
"y": 646
},
{
"t": 10,
"x": 994,
"y": 671
},
{
"t": 12,
"x": 994,
"y": 654
},
{
"t": 14,
"x": 994,
"y": 642
},
{
"t": 15,
"x": 990,
"y": 669
},
{
"t": 17,
"x": 990,
"y": 658
},
{
"t": 18,
"x": 986,
"y": 685
},
{
"t": 20,
"x": 986,
"y": 673
},
{
"t": 21,
"x": 982,
"y": 697
},
{
"t": 23,
"x": 982,
"y": 685
}
],
"birth": 0,
"death": 24
},
{
"id": 17,
"type": "red and green cardboard ink cartridge box",
"vis": [
{
"t": 0,
"v": true
},
{
"t": 6,
"v": false
}
],
"is_in_frame": [
{
"t": 0,
"v": true
},
{
"t": 6,
"v": false
}
],
"image_positions": [
{
"t": 0,
"x": 998,
"y": 544
},
{
"t": 1,
"x": 998,
"y": 556
},
{
"t": 2,
"x": 998,
"y": 568
},
{
"t": 3,
"x": 998,
"y": 580
},
{
"t": 4,
"x": 998,
"y": 591
},
{
"t": 5,
"x": 998,
"y": 603
},
{
"t": 6,
"x": null,
"y": null
}
],
"birth": 0,
"death": 24
},
{
"id": 18,
"type": "yellow plastic toy school bus",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 5,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 5,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 5,
"x": 388,
"y": 685
},
{
"t": 7,
"x": 396,
"y": 681
},
{
"t": 9,
"x": 404,
"y": 677
},
{
"t": 10,
"x": 412,
"y": 681
},
{
"t": 11,
"x": 419,
"y": 685
},
{
"t": 12,
"x": 427,
"y": 693
},
{
"t": 13,
"x": 435,
"y": 701
},
{
"t": 14,
"x": 455,
"y": 720
},
{
"t": 15,
"x": 474,
"y": 744
},
{
"t": 16,
"x": 490,
"y": 755
},
{
"t": 17,
"x": 498,
"y": 767
},
{
"t": 18,
"x": 498,
"y": 779
},
{
"t": 19,
"x": 494,
"y": 787
},
{
"t": 20,
"x": 486,
"y": 791
},
{
"t": 21,
"x": 482,
"y": 798
},
{
"t": 22,
"x": 474,
"y": 806
},
{
"t": 23,
"x": 462,
"y": 794
}
],
"birth": 5,
"death": 24
},
{
"id": 19,
"type": "red and black plastic handheld game console",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 6,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 6,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 6,
"x": 91,
"y": 341
},
{
"t": 8,
"x": 91,
"y": 353
},
{
"t": 11,
"x": 91,
"y": 365
},
{
"t": 13,
"x": 95,
"y": 373
},
{
"t": 16,
"x": 95,
"y": 384
},
{
"t": 18,
"x": 99,
"y": 392
},
{
"t": 20,
"x": 99,
"y": 404
},
{
"t": 22,
"x": 103,
"y": 412
},
{
"t": 23,
"x": 107,
"y": 419
}
],
"birth": 6,
"death": 24
},
{
"id": 20,
"type": "green and yellow running shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 10,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 10,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 10,
"x": 99,
"y": 376
},
{
"t": 11,
"x": 107,
"y": 384
},
{
"t": 13,
"x": 111,
"y": 392
},
{
"t": 15,
"x": 115,
"y": 400
},
{
"t": 17,
"x": 119,
"y": 408
},
{
"t": 20,
"x": 123,
"y": 416
},
{
"t": 23,
"x": 126,
"y": 423
}
],
"birth": 10,
"death": 24
},
{
"id": 21,
"type": "green and yellow running shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 12,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 12,
"x": 1,
"y": 591
},
{
"t": 13,
"x": 1,
"y": 580
},
{
"t": 15,
"x": 5,
"y": 587
},
{
"t": 16,
"x": 5,
"y": 576
},
{
"t": 18,
"x": 9,
"y": 583
},
{
"t": 19,
"x": 9,
"y": 572
},
{
"t": 21,
"x": 13,
"y": 580
},
{
"t": 22,
"x": 13,
"y": 568
},
{
"t": 23,
"x": 13,
"y": 556
}
],
"birth": 12,
"death": 24
},
{
"id": 22,
"type": "green and yellow running shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 14,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 14,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 14,
"x": 107,
"y": 380
},
{
"t": 16,
"x": 111,
"y": 392
},
{
"t": 18,
"x": 115,
"y": 400
},
{
"t": 20,
"x": 119,
"y": 408
},
{
"t": 23,
"x": 123,
"y": 419
}
],
"birth": 14,
"death": 24
},
{
"id": 23,
"type": "green and grey plastic Doctor Doom action figure",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 18,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 18,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 18,
"x": 1,
"y": 455
},
{
"t": 19,
"x": 1,
"y": 443
},
{
"t": 21,
"x": 5,
"y": 458
},
{
"t": 22,
"x": 5,
"y": 447
},
{
"t": 23,
"x": 5,
"y": 435
}
],
"birth": 18,
"death": 24
},
{
"id": 24,
"type": "green and yellow running shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 21,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 21,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 21,
"x": 1,
"y": 431
},
{
"t": 22,
"x": 1,
"y": 419
},
{
"t": 23,
"x": 1,
"y": 404
}
],
"birth": 21,
"death": 24
},
{
"id": 25,
"type": "green and yellow running shoe",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 23,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 23,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 23,
"x": 1,
"y": 423
}
],
"birth": 23,
"death": 24
}
],
"interactions": [
{
"id": 1,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 0,
"ends_at": 2,
"type": "contact"
},
{
"id": 21,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 4,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 29,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 30,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 31,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 32,
"inputs": [
5,
65535
],
"outputs": [
5,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 33,
"inputs": [
6,
65535
],
"outputs": [
6,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 34,
"inputs": [
7,
65535
],
"outputs": [
7,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 35,
"inputs": [
8,
65535
],
"outputs": [
8,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 36,
"inputs": [
9,
65535
],
"outputs": [
9,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
10,
65535
],
"outputs": [
10,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 4,
"inputs": [
11,
65535
],
"outputs": [
11,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 5,
"inputs": [
12,
65535
],
"outputs": [
12,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 6,
"inputs": [
13,
65535
],
"outputs": [
13,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 7,
"inputs": [
14,
65535
],
"outputs": [
14,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 8,
"inputs": [
15,
65535
],
"outputs": [
15,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 9,
"inputs": [
16,
65535
],
"outputs": [
16,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 10,
"inputs": [
17,
65535
],
"outputs": [
17,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 15,
"inputs": [
19,
65535
],
"outputs": [
19,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 16,
"inputs": [
20,
65535
],
"outputs": [
20,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 17,
"inputs": [
21,
65535
],
"outputs": [
21,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 18,
"inputs": [
22,
65535
],
"outputs": [
22,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 19,
"inputs": [
23,
65535
],
"outputs": [
23,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 20,
"inputs": [
24,
65535
],
"outputs": [
24,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 22,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 23,
"inputs": [
19,
20
],
"outputs": [
19,
20
],
"begins_at": 4,
"ends_at": 4,
"type": "collision"
},
{
"id": 24,
"inputs": [
19,
20
],
"outputs": [
19,
20
],
"begins_at": 5,
"ends_at": 7,
"type": "contact"
},
{
"id": 14,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 9,
"ends_at": 9,
"type": "collision"
},
{
"id": 25,
"inputs": [
19,
20
],
"outputs": [
19,
20
],
"begins_at": 10,
"ends_at": 10,
"type": "collision"
},
{
"id": 13,
"inputs": [
18,
65535
],
"outputs": [
18,
65535
],
"begins_at": 10,
"ends_at": 23,
"type": "contact"
},
{
"id": 26,
"inputs": [
19,
20
],
"outputs": [
19,
20
],
"begins_at": 13,
"ends_at": 13,
"type": "collision"
},
{
"id": 27,
"inputs": [
19,
20
],
"outputs": [
19,
20
],
"begins_at": 16,
"ends_at": 16,
"type": "collision"
},
{
"id": 28,
"inputs": [
19,
20
],
"outputs": [
19,
20
],
"begins_at": 17,
"ends_at": 23,
"type": "contact"
},
{
"id": 11,
"inputs": [
18,
19
],
"outputs": [
18,
19
],
"begins_at": 19,
"ends_at": 19,
"type": "collision"
},
{
"id": 12,
"inputs": [
18,
19
],
"outputs": [
18,
19
],
"begins_at": 20,
"ends_at": 23,
"type": "contact"
}
]
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "yellow truck",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 40,
"y": 683
},
{
"t": 1,
"x": 44,
"y": 683
},
{
"t": 2,
"x": 50,
"y": 683
},
{
"t": 3,
"x": 55,
"y": 683
},
{
"t": 4,
"x": 60,
"y": 683
},
{
"t": 5,
"x": 66,
"y": 683
},
{
"t": 6,
"x": 72,
"y": 683
},
{
"t": 7,
"x": 92,
"y": 682
},
{
"t": 8,
"x": 96,
"y": 682
},
{
"t": 9,
"x": 90,
"y": 682
},
{
"t": 10,
"x": 102,
"y": 682
},
{
"t": 11,
"x": 105,
"y": 682
},
{
"t": 12,
"x": 108,
"y": 682
},
{
"t": 13,
"x": 114,
"y": 682
},
{
"t": 14,
"x": 120,
"y": 682
},
{
"t": 15,
"x": 126,
"y": 682
},
{
"t": 16,
"x": 133,
"y": 682
},
{
"t": 17,
"x": 139,
"y": 682
},
{
"t": 18,
"x": 156,
"y": 681
},
{
"t": 19,
"x": 159,
"y": 681
},
{
"t": 20,
"x": 163,
"y": 681
},
{
"t": 21,
"x": 166,
"y": 681
},
{
"t": 22,
"x": 172,
"y": 681
},
{
"t": 23,
"x": 179,
"y": 681
},
{
"t": 24,
"x": 186,
"y": 681
},
{
"t": 25,
"x": 192,
"y": 681
},
{
"t": 26,
"x": 199,
"y": 681
},
{
"t": 27,
"x": 206,
"y": 680
},
{
"t": 28,
"x": 220,
"y": 680
},
{
"t": 29,
"x": 224,
"y": 680
},
{
"t": 30,
"x": 229,
"y": 680
},
{
"t": 31,
"x": 235,
"y": 680
},
{
"t": 32,
"x": 242,
"y": 680
},
{
"t": 33,
"x": 250,
"y": 679
},
{
"t": 34,
"x": 257,
"y": 679
},
{
"t": 35,
"x": 265,
"y": 679
},
{
"t": 36,
"x": 278,
"y": 679
},
{
"t": 37,
"x": 282,
"y": 679
},
{
"t": 38,
"x": 287,
"y": 679
},
{
"t": 39,
"x": 294,
"y": 679
},
{
"t": 40,
"x": 301,
"y": 678
},
{
"t": 41,
"x": 309,
"y": 678
},
{
"t": 42,
"x": 316,
"y": 678
},
{
"t": 43,
"x": 324,
"y": 678
},
{
"t": 44,
"x": 337,
"y": 678
},
{
"t": 45,
"x": 341,
"y": 678
},
{
"t": 46,
"x": 347,
"y": 678
},
{
"t": 47,
"x": 358,
"y": 680
},
{
"t": 49,
"x": 374,
"y": 683
},
{
"t": 51,
"x": 383,
"y": 686
},
{
"t": 52,
"x": 392,
"y": 689
},
{
"t": 53,
"x": 427,
"y": 699
},
{
"t": 54,
"x": 433,
"y": 703
},
{
"t": 55,
"x": 437,
"y": 707
},
{
"t": 56,
"x": 441,
"y": 710
},
{
"t": 58,
"x": 442,
"y": 714
},
{
"t": 59,
"x": 446,
"y": 718
},
{
"t": 60,
"x": 448,
"y": 721
},
{
"t": 61,
"x": 450,
"y": 724
},
{
"t": 62,
"x": 455,
"y": 727
},
{
"t": 63,
"x": 457,
"y": 730
},
{
"t": 64,
"x": 459,
"y": 734
},
{
"t": 65,
"x": 462,
"y": 738
},
{
"t": 66,
"x": 466,
"y": 741
},
{
"t": 67,
"x": 478,
"y": 742
},
{
"t": 68,
"x": 469,
"y": 745
},
{
"t": 69,
"x": 473,
"y": 748
},
{
"t": 70,
"x": 483,
"y": 751
},
{
"t": 71,
"x": 487,
"y": 755
},
{
"t": 72,
"x": 495,
"y": 758
},
{
"t": 73,
"x": 500,
"y": 763
},
{
"t": 74,
"x": 506,
"y": 766
},
{
"t": 75,
"x": 509,
"y": 768
},
{
"t": 76,
"x": 516,
"y": 772
},
{
"t": 77,
"x": 522,
"y": 778
},
{
"t": 78,
"x": 527,
"y": 780
},
{
"t": 79,
"x": 533,
"y": 786
},
{
"t": 80,
"x": 540,
"y": 793
},
{
"t": 81,
"x": 547,
"y": 802
},
{
"t": 82,
"x": 551,
"y": 807
},
{
"t": 83,
"x": 555,
"y": 810
},
{
"t": 84,
"x": 560,
"y": 816
},
{
"t": 85,
"x": 566,
"y": 819
},
{
"t": 86,
"x": 574,
"y": 814
},
{
"t": 87,
"x": 580,
"y": 820
},
{
"t": 88,
"x": 587,
"y": 822
},
{
"t": 89,
"x": 594,
"y": 825
},
{
"t": 90,
"x": 601,
"y": 829
},
{
"t": 91,
"x": 609,
"y": 831
},
{
"t": 92,
"x": 616,
"y": 836
},
{
"t": 93,
"x": 622,
"y": 843
},
{
"t": 94,
"x": 628,
"y": 848
},
{
"t": 95,
"x": 631,
"y": 858
},
{
"t": 96,
"x": 636,
"y": 869
},
{
"t": 97,
"x": 647,
"y": 872
},
{
"t": 98,
"x": 645,
"y": 881
},
{
"t": 99,
"x": 648,
"y": 886
},
{
"t": 100,
"x": 655,
"y": 892
},
{
"t": 101,
"x": 661,
"y": 897
},
{
"t": 102,
"x": 677,
"y": 903
},
{
"t": 103,
"x": 674,
"y": 904
},
{
"t": 104,
"x": 681,
"y": 906
},
{
"t": 105,
"x": 691,
"y": 908
},
{
"t": 106,
"x": 698,
"y": 911
},
{
"t": 107,
"x": 707,
"y": 914
},
{
"t": 108,
"x": 718,
"y": 916
},
{
"t": 109,
"x": 726,
"y": 919
},
{
"t": 110,
"x": 805,
"y": 915
},
{
"t": 111,
"x": 817,
"y": 921
},
{
"t": 112,
"x": 830,
"y": 924
},
{
"t": 113,
"x": 843,
"y": 928
},
{
"t": 114,
"x": 858,
"y": 931
},
{
"t": 115,
"x": 787,
"y": 934
},
{
"t": 116,
"x": 796,
"y": 938
},
{
"t": 117,
"x": 806,
"y": 941
},
{
"t": 118,
"x": 817,
"y": 944
},
{
"t": 119,
"x": 829,
"y": 947
}
],
"birth": 0,
"death": 120
},
{
"id": 1,
"type": "green sedan",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 346,
"y": 544
},
{
"t": 2,
"x": 348,
"y": 546
},
{
"t": 3,
"x": 349,
"y": 548
},
{
"t": 5,
"x": 351,
"y": 551
},
{
"t": 7,
"x": 358,
"y": 554
},
{
"t": 8,
"x": 354,
"y": 556
},
{
"t": 10,
"x": 355,
"y": 559
},
{
"t": 11,
"x": 358,
"y": 561
},
{
"t": 12,
"x": 362,
"y": 563
},
{
"t": 13,
"x": 358,
"y": 565
},
{
"t": 14,
"x": 364,
"y": 567
},
{
"t": 15,
"x": 362,
"y": 569
},
{
"t": 16,
"x": 362,
"y": 571
},
{
"t": 18,
"x": 364,
"y": 574
},
{
"t": 19,
"x": 367,
"y": 576
},
{
"t": 21,
"x": 368,
"y": 580
},
{
"t": 22,
"x": 369,
"y": 583
},
{
"t": 23,
"x": 371,
"y": 584
},
{
"t": 24,
"x": 373,
"y": 588
},
{
"t": 25,
"x": 373,
"y": 590
},
{
"t": 26,
"x": 375,
"y": 592
},
{
"t": 27,
"x": 376,
"y": 595
},
{
"t": 28,
"x": 377,
"y": 597
},
{
"t": 29,
"x": 380,
"y": 600
},
{
"t": 30,
"x": 380,
"y": 602
},
{
"t": 31,
"x": 382,
"y": 604
},
{
"t": 32,
"x": 384,
"y": 607
},
{
"t": 33,
"x": 387,
"y": 610
},
{
"t": 35,
"x": 387,
"y": 616
},
{
"t": 36,
"x": 388,
"y": 619
},
{
"t": 37,
"x": 389,
"y": 621
},
{
"t": 38,
"x": 390,
"y": 625
},
{
"t": 39,
"x": 391,
"y": 627
},
{
"t": 40,
"x": 399,
"y": 630
},
{
"t": 41,
"x": 405,
"y": 630
},
{
"t": 42,
"x": 406,
"y": 627
},
{
"t": 44,
"x": 413,
"y": 628
},
{
"t": 45,
"x": 397,
"y": 628
},
{
"t": 46,
"x": 407,
"y": 630
},
{
"t": 47,
"x": 400,
"y": 631
},
{
"t": 48,
"x": 408,
"y": 632
},
{
"t": 49,
"x": 416,
"y": 635
},
{
"t": 50,
"x": 423,
"y": 638
},
{
"t": 51,
"x": 430,
"y": 641
},
{
"t": 52,
"x": 437,
"y": 643
},
{
"t": 53,
"x": 441,
"y": 643
},
{
"t": 54,
"x": 446,
"y": 645
},
{
"t": 55,
"x": 393,
"y": 636
},
{
"t": 56,
"x": 405,
"y": 635
},
{
"t": 57,
"x": 405,
"y": 639
},
{
"t": 58,
"x": 483,
"y": 676
},
{
"t": 59,
"x": 489,
"y": 678
},
{
"t": 60,
"x": 495,
"y": 681
},
{
"t": 61,
"x": 507,
"y": 688
},
{
"t": 62,
"x": 512,
"y": 690
},
{
"t": 64,
"x": 516,
"y": 690
},
{
"t": 65,
"x": 525,
"y": 691
},
{
"t": 66,
"x": 523,
"y": 690
},
{
"t": 67,
"x": 529,
"y": 692
},
{
"t": 68,
"x": 533,
"y": 693
},
{
"t": 69,
"x": 539,
"y": 695
},
{
"t": 70,
"x": 547,
"y": 696
},
{
"t": 71,
"x": 554,
"y": 697
},
{
"t": 72,
"x": 562,
"y": 698
},
{
"t": 73,
"x": 544,
"y": 690
},
{
"t": 74,
"x": 548,
"y": 690
},
{
"t": 75,
"x": 551,
"y": 690
},
{
"t": 76,
"x": 555,
"y": 690
},
{
"t": 77,
"x": 560,
"y": 692
},
{
"t": 78,
"x": 564,
"y": 692
},
{
"t": 79,
"x": 571,
"y": 694
},
{
"t": 80,
"x": 579,
"y": 696
},
{
"t": 81,
"x": 489,
"y": 703
},
{
"t": 82,
"x": 497,
"y": 705
},
{
"t": 83,
"x": 505,
"y": 707
},
{
"t": 84,
"x": 512,
"y": 710
},
{
"t": 85,
"x": 521,
"y": 713
},
{
"t": 86,
"x": 532,
"y": 713
},
{
"t": 87,
"x": 541,
"y": 715
},
{
"t": 88,
"x": 551,
"y": 716
},
{
"t": 89,
"x": 559,
"y": 717
},
{
"t": 90,
"x": 576,
"y": 711
},
{
"t": 91,
"x": 585,
"y": 714
},
{
"t": 92,
"x": 594,
"y": 715
},
{
"t": 93,
"x": 606,
"y": 714
},
{
"t": 94,
"x": 617,
"y": 716
},
{
"t": 95,
"x": 627,
"y": 718
},
{
"t": 96,
"x": 638,
"y": 719
},
{
"t": 97,
"x": 646,
"y": 718
},
{
"t": 98,
"x": 651,
"y": 719
},
{
"t": 99,
"x": 655,
"y": 722
},
{
"t": 102,
"x": 662,
"y": 722
},
{
"t": 103,
"x": 658,
"y": 723
},
{
"t": 104,
"x": 664,
"y": 723
},
{
"t": 105,
"x": 669,
"y": 724
},
{
"t": 106,
"x": 675,
"y": 725
},
{
"t": 107,
"x": 679,
"y": 725
},
{
"t": 108,
"x": 683,
"y": 725
},
{
"t": 109,
"x": 687,
"y": 725
},
{
"t": 110,
"x": 692,
"y": 726
},
{
"t": 111,
"x": 696,
"y": 726
},
{
"t": 112,
"x": 700,
"y": 727
},
{
"t": 113,
"x": 702,
"y": 726
},
{
"t": 114,
"x": 708,
"y": 728
},
{
"t": 115,
"x": 711,
"y": 728
},
{
"t": 116,
"x": 715,
"y": 729
},
{
"t": 117,
"x": 719,
"y": 729
},
{
"t": 118,
"x": 722,
"y": 729
},
{
"t": 119,
"x": 724,
"y": 729
}
],
"birth": 0,
"death": 120
},
{
"id": 2,
"type": "red truck",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 433,
"y": 541
},
{
"t": 1,
"x": 430,
"y": 541
},
{
"t": 3,
"x": 431,
"y": 543
},
{
"t": 7,
"x": 431,
"y": 546
},
{
"t": 9,
"x": 433,
"y": 548
},
{
"t": 13,
"x": 433,
"y": 551
},
{
"t": 16,
"x": 433,
"y": 554
},
{
"t": 19,
"x": 434,
"y": 557
},
{
"t": 22,
"x": 435,
"y": 560
},
{
"t": 25,
"x": 435,
"y": 564
},
{
"t": 27,
"x": 436,
"y": 566
},
{
"t": 28,
"x": 437,
"y": 568
},
{
"t": 31,
"x": 437,
"y": 571
},
{
"t": 33,
"x": 437,
"y": 574
},
{
"t": 35,
"x": 438,
"y": 576
},
{
"t": 37,
"x": 439,
"y": 579
},
{
"t": 40,
"x": 441,
"y": 583
},
{
"t": 43,
"x": 441,
"y": 586
},
{
"t": 44,
"x": 442,
"y": 589
},
{
"t": 47,
"x": 444,
"y": 593
},
{
"t": 48,
"x": 446,
"y": 594
},
{
"t": 51,
"x": 448,
"y": 596
},
{
"t": 54,
"x": 450,
"y": 597
},
{
"t": 57,
"x": 452,
"y": 597
},
{
"t": 61,
"x": 449,
"y": 598
},
{
"t": 62,
"x": 452,
"y": 599
},
{
"t": 63,
"x": 455,
"y": 600
},
{
"t": 64,
"x": 437,
"y": 601
},
{
"t": 65,
"x": 440,
"y": 603
},
{
"t": 66,
"x": 443,
"y": 604
},
{
"t": 67,
"x": 441,
"y": 607
},
{
"t": 69,
"x": 441,
"y": 610
},
{
"t": 70,
"x": 445,
"y": 613
},
{
"t": 71,
"x": 442,
"y": 616
},
{
"t": 73,
"x": 445,
"y": 620
},
{
"t": 74,
"x": 449,
"y": 622
},
{
"t": 75,
"x": 447,
"y": 626
},
{
"t": 76,
"x": 448,
"y": 628
},
{
"t": 77,
"x": 449,
"y": 630
},
{
"t": 78,
"x": 456,
"y": 632
},
{
"t": 79,
"x": 451,
"y": 635
},
{
"t": 80,
"x": 454,
"y": 639
},
{
"t": 81,
"x": 458,
"y": 642
},
{
"t": 82,
"x": 458,
"y": 646
},
{
"t": 83,
"x": 460,
"y": 648
},
{
"t": 85,
"x": 461,
"y": 651
},
{
"t": 86,
"x": 462,
"y": 653
},
{
"t": 87,
"x": 462,
"y": 655
},
{
"t": 89,
"x": 464,
"y": 658
},
{
"t": 90,
"x": 465,
"y": 660
},
{
"t": 92,
"x": 466,
"y": 664
},
{
"t": 93,
"x": 466,
"y": 666
},
{
"t": 95,
"x": 468,
"y": 669
},
{
"t": 96,
"x": 469,
"y": 671
},
{
"t": 97,
"x": 469,
"y": 673
},
{
"t": 99,
"x": 470,
"y": 676
},
{
"t": 100,
"x": 472,
"y": 678
},
{
"t": 101,
"x": 473,
"y": 680
},
{
"t": 103,
"x": 473,
"y": 683
},
{
"t": 104,
"x": 474,
"y": 685
},
{
"t": 106,
"x": 476,
"y": 691
},
{
"t": 108,
"x": 477,
"y": 694
},
{
"t": 110,
"x": 478,
"y": 697
},
{
"t": 111,
"x": 479,
"y": 699
},
{
"t": 113,
"x": 480,
"y": 702
},
{
"t": 114,
"x": 480,
"y": 704
},
{
"t": 115,
"x": 481,
"y": 706
},
{
"t": 116,
"x": 482,
"y": 709
},
{
"t": 117,
"x": 483,
"y": 711
},
{
"t": 118,
"x": 483,
"y": 714
}
],
"birth": 0,
"death": 120
},
{
"id": 3,
"type": "gray wagon",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 773,
"y": 706
},
{
"t": 89,
"x": 782,
"y": 706
},
{
"t": 90,
"x": 768,
"y": 707
},
{
"t": 91,
"x": 773,
"y": 707
},
{
"t": 92,
"x": 783,
"y": 707
},
{
"t": 93,
"x": 799,
"y": 708
},
{
"t": 94,
"x": 813,
"y": 708
},
{
"t": 95,
"x": 822,
"y": 708
},
{
"t": 96,
"x": 802,
"y": 709
},
{
"t": 97,
"x": 816,
"y": 709
},
{
"t": 98,
"x": 826,
"y": 709
},
{
"t": 99,
"x": 818,
"y": 710
},
{
"t": 100,
"x": 829,
"y": 710
},
{
"t": 101,
"x": 839,
"y": 710
},
{
"t": 102,
"x": 811,
"y": 714
},
{
"t": 103,
"x": 841,
"y": 713
},
{
"t": 104,
"x": 850,
"y": 713
},
{
"t": 105,
"x": 858,
"y": 713
},
{
"t": 106,
"x": 855,
"y": 714
},
{
"t": 107,
"x": 862,
"y": 714
},
{
"t": 108,
"x": 870,
"y": 714
},
{
"t": 109,
"x": 880,
"y": 714
},
{
"t": 110,
"x": 874,
"y": 715
},
{
"t": 111,
"x": 870,
"y": 717
},
{
"t": 112,
"x": 889,
"y": 716
},
{
"t": 113,
"x": 898,
"y": 716
},
{
"t": 114,
"x": 908,
"y": 716
},
{
"t": 115,
"x": 900,
"y": 717
},
{
"t": 116,
"x": 907,
"y": 717
},
{
"t": 117,
"x": 914,
"y": 717
},
{
"t": 118,
"x": 883,
"y": 722
},
{
"t": 119,
"x": 898,
"y": 721
}
],
"birth": 0,
"death": 120
},
{
"id": 4,
"type": "green jet",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 976,
"y": 329
},
{
"t": 1,
"x": 985,
"y": 325
},
{
"t": 2,
"x": 969,
"y": 329
},
{
"t": 3,
"x": 984,
"y": 323
},
{
"t": 5,
"x": 984,
"y": 320
},
{
"t": 6,
"x": 983,
"y": 319
},
{
"t": 7,
"x": 983,
"y": 316
},
{
"t": 8,
"x": 981,
"y": 315
},
{
"t": 9,
"x": 979,
"y": 314
},
{
"t": 10,
"x": 975,
"y": 314
},
{
"t": 11,
"x": 971,
"y": 314
},
{
"t": 12,
"x": 967,
"y": 314
},
{
"t": 13,
"x": 963,
"y": 313
},
{
"t": 14,
"x": 958,
"y": 314
},
{
"t": 15,
"x": 955,
"y": 313
},
{
"t": 16,
"x": 950,
"y": 313
},
{
"t": 17,
"x": 945,
"y": 314
},
{
"t": 18,
"x": 941,
"y": 314
},
{
"t": 19,
"x": 936,
"y": 314
},
{
"t": 20,
"x": 931,
"y": 314
},
{
"t": 21,
"x": 926,
"y": 314
},
{
"t": 22,
"x": 921,
"y": 314
},
{
"t": 23,
"x": 916,
"y": 314
},
{
"t": 24,
"x": 911,
"y": 314
},
{
"t": 25,
"x": 906,
"y": 314
},
{
"t": 26,
"x": 901,
"y": 314
},
{
"t": 27,
"x": 895,
"y": 314
},
{
"t": 28,
"x": 891,
"y": 313
},
{
"t": 29,
"x": 885,
"y": 313
},
{
"t": 30,
"x": 879,
"y": 314
},
{
"t": 31,
"x": 873,
"y": 313
},
{
"t": 32,
"x": 867,
"y": 314
},
{
"t": 33,
"x": 861,
"y": 314
},
{
"t": 34,
"x": 855,
"y": 314
},
{
"t": 35,
"x": 849,
"y": 314
},
{
"t": 36,
"x": 843,
"y": 314
},
{
"t": 37,
"x": 837,
"y": 313
},
{
"t": 38,
"x": 830,
"y": 313
},
{
"t": 39,
"x": 824,
"y": 313
},
{
"t": 40,
"x": 816,
"y": 315
},
{
"t": 41,
"x": 810,
"y": 314
},
{
"t": 42,
"x": 803,
"y": 315
},
{
"t": 43,
"x": 797,
"y": 314
},
{
"t": 44,
"x": 790,
"y": 314
},
{
"t": 45,
"x": 783,
"y": 313
},
{
"t": 46,
"x": 776,
"y": 315
},
{
"t": 47,
"x": 769,
"y": 315
},
{
"t": 48,
"x": 762,
"y": 315
},
{
"t": 49,
"x": 755,
"y": 315
},
{
"t": 50,
"x": 748,
"y": 314
},
{
"t": 51,
"x": 741,
"y": 314
},
{
"t": 52,
"x": 733,
"y": 314
},
{
"t": 53,
"x": 726,
"y": 315
},
{
"t": 54,
"x": 719,
"y": 314
},
{
"t": 55,
"x": 712,
"y": 315
},
{
"t": 56,
"x": 705,
"y": 315
},
{
"t": 57,
"x": 697,
"y": 315
},
{
"t": 58,
"x": 690,
"y": 314
},
{
"t": 59,
"x": 682,
"y": 315
},
{
"t": 60,
"x": 675,
"y": 314
},
{
"t": 61,
"x": 667,
"y": 314
},
{
"t": 62,
"x": 659,
"y": 315
},
{
"t": 63,
"x": 651,
"y": 315
},
{
"t": 64,
"x": 644,
"y": 315
},
{
"t": 65,
"x": 636,
"y": 315
},
{
"t": 66,
"x": 628,
"y": 314
},
{
"t": 67,
"x": 620,
"y": 314
},
{
"t": 68,
"x": 612,
"y": 314
},
{
"t": 69,
"x": 604,
"y": 315
},
{
"t": 70,
"x": 596,
"y": 315
},
{
"t": 71,
"x": 587,
"y": 315
},
{
"t": 72,
"x": 580,
"y": 314
},
{
"t": 73,
"x": 572,
"y": 314
},
{
"t": 74,
"x": 563,
"y": 314
},
{
"t": 75,
"x": 555,
"y": 315
},
{
"t": 76,
"x": 546,
"y": 315
},
{
"t": 77,
"x": 537,
"y": 315
},
{
"t": 78,
"x": 529,
"y": 315
},
{
"t": 79,
"x": 520,
"y": 314
},
{
"t": 80,
"x": 511,
"y": 316
},
{
"t": 81,
"x": 502,
"y": 316
},
{
"t": 82,
"x": 494,
"y": 315
},
{
"t": 83,
"x": 485,
"y": 315
},
{
"t": 84,
"x": 476,
"y": 314
},
{
"t": 85,
"x": 466,
"y": 316
},
{
"t": 86,
"x": 458,
"y": 316
},
{
"t": 87,
"x": 448,
"y": 316
},
{
"t": 88,
"x": 440,
"y": 315
},
{
"t": 89,
"x": 430,
"y": 314
},
{
"t": 90,
"x": 420,
"y": 316
},
{
"t": 91,
"x": 411,
"y": 316
},
{
"t": 92,
"x": 401,
"y": 316
},
{
"t": 93,
"x": 392,
"y": 315
},
{
"t": 94,
"x": 383,
"y": 315
},
{
"t": 95,
"x": 373,
"y": 316
},
{
"t": 96,
"x": 362,
"y": 316
},
{
"t": 97,
"x": 353,
"y": 315
},
{
"t": 98,
"x": 343,
"y": 315
},
{
"t": 99,
"x": 333,
"y": 316
},
{
"t": 100,
"x": 323,
"y": 316
},
{
"t": 101,
"x": 312,
"y": 316
},
{
"t": 102,
"x": 302,
"y": 315
},
{
"t": 103,
"x": 292,
"y": 315
},
{
"t": 104,
"x": 281,
"y": 316
},
{
"t": 105,
"x": 271,
"y": 315
},
{
"t": 106,
"x": 260,
"y": 316
},
{
"t": 107,
"x": 249,
"y": 316
},
{
"t": 108,
"x": 239,
"y": 315
},
{
"t": 109,
"x": 228,
"y": 315
},
{
"t": 110,
"x": 216,
"y": 316
},
{
"t": 111,
"x": 205,
"y": 316
},
{
"t": 112,
"x": 194,
"y": 315
},
{
"t": 113,
"x": 183,
"y": 315
},
{
"t": 114,
"x": 172,
"y": 316
},
{
"t": 115,
"x": 160,
"y": 316
},
{
"t": 116,
"x": 149,
"y": 315
},
{
"t": 117,
"x": 137,
"y": 315
},
{
"t": 118,
"x": 126,
"y": 316
},
{
"t": 119,
"x": 113,
"y": 316
}
],
"birth": 0,
"death": 120
}
],
"interactions": [
{
"id": 5,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 13,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 14,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 15,
"inputs": [
3,
65535
],
"outputs": [
3,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 6,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 33,
"ends_at": 33,
"type": "collision"
},
{
"id": 7,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 34,
"ends_at": 36,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 46,
"ends_at": 46,
"type": "collision"
},
{
"id": 1,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 53,
"ends_at": 53,
"type": "collision"
},
{
"id": 2,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 55,
"ends_at": 55,
"type": "collision"
},
{
"id": 3,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 57,
"ends_at": 57,
"type": "collision"
},
{
"id": 4,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 58,
"ends_at": 59,
"type": "contact"
},
{
"id": 10,
"inputs": [
1,
3
],
"outputs": [
1,
3
],
"begins_at": 89,
"ends_at": 89,
"type": "collision"
},
{
"id": 11,
"inputs": [
1,
3
],
"outputs": [
1,
3
],
"begins_at": 93,
"ends_at": 93,
"type": "collision"
},
{
"id": 12,
"inputs": [
1,
3
],
"outputs": [
1,
3
],
"begins_at": 94,
"ends_at": 108,
"type": "contact"
},
{
"id": 8,
"inputs": [
1,
3
],
"outputs": [
1,
3
],
"begins_at": 113,
"ends_at": 113,
"type": "collision"
},
{
"id": 9,
"inputs": [
1,
3
],
"outputs": [
1,
3
],
"begins_at": 114,
"ends_at": 117,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": 0.4386401304645827,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": 229.32040277025573,
"avg_image_position_segmentation_acc": 0.4866666666666667,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": 0.6166666666666667,
"avg_image_position_segmentation_acc_large": 0.45416666666666666,
"avg_image_position_segmentation_acc_first_frame": 0.8,
"avg_image_position_segmentation_visible_acc": 0.4866666666666667,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": 0.9983333333333334,
"visibility_true_acc": 0.9983333333333334,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.9983333333333334,
"vis_acc_from_px_true_acc": 0.9983333333333334,
"vis_acc_from_px_false_acc": null,
"num_matches": 5,
"num_unmatched_pred": 21,
"num_unmatched_target": 0
},
"supports": {
"avg_type_similarity": 5,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 5,
"avg_image_position_segmentation_acc": 5,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 1,
"avg_image_position_segmentation_acc_large": 4,
"avg_image_position_segmentation_acc_first_frame": 5,
"avg_image_position_segmentation_visible_acc": 5,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 5,
"visibility_true_acc": 5,
"visibility_false_acc": 0,
"is_in_frame_acc": 5,
"is_in_frame_true_acc": 5,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 5,
"vis_acc_from_px_true_acc": 5,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [
{
"target_index": 0,
"pred_index": 1,
"target_id": 0,
"pred_id": 1,
"cost": 0.575,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.575,
"weighted_cost": 0.575,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 6,
"target_id": 1,
"pred_id": 6,
"cost": 1.0333333333333332,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.5333333333333333,
"weighted_cost": 0.5333333333333333,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 8,
"target_id": 2,
"pred_id": 8,
"cost": 0.8833333333333333,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.38333333333333336,
"weighted_cost": 0.38333333333333336,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 13,
"target_id": 3,
"pred_id": 13,
"cost": 0.575,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.5,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 0.075,
"weighted_cost": 0.075,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 4,
"target_id": 4,
"pred_id": 4,
"cost": 1.0,
"cost_breakdown": [
{
"name": "mutual_type",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "same_seg",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
2,
3,
5,
7,
9,
10,
11,
12,
14,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25
],
"target_type_counts": {
"yellow truck": 1,
"green sedan": 1,
"red truck": 1,
"gray wagon": 1,
"green jet": 1
},
"pred_type_counts": {
"yellow plastic toy school bus": 3,
"brown plastic plant pot": 1,
"dark brown fabric loafer": 2,
"green plate": 2,
"dark red and black plastic curling wand": 1,
"red ceramic square butter dish": 1,
"dark grey and orange athletic shoe": 1,
"white plastic Nintendo 3DS game case": 1,
"white metallic wall mount": 1,
"white and brown leather golf shoe": 1,
"white and grey cardboard desktop hard drive box": 1,
"dark green fabric loafer": 1,
"dark grey cardboard motherboard box": 1,
"red and blue plastic Mario action figure": 1,
"red and green cardboard ink cartridge box": 1,
"red and black plastic handheld game console": 1,
"green and yellow running shoe": 5,
"green and grey plastic Doctor Doom action figure": 1
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 120
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 5,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 2,
"false_negatives": 0
},
"2.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
},
"3.0": {
"true_positives": 0,
"false_positives": 5,
"false_negatives": 0
}
},
"jaccard_by_threshold": {
"0.5": 1.0,
"1.0": 0.6,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 5,
"false_positives": 21,
"false_negatives": 0
},
"0.5": {
"true_positives": 5,
"false_positives": 21,
"false_negatives": 0
},
"0.75": {
"true_positives": 5,
"false_positives": 21,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 22,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.19230769230769232,
"0.5": 0.19230769230769232,
"0.75": 0.19230769230769232,
"1.0": 0.15384615384615385
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.8
},
"visibility_f1_by_threshold": {
"0.25": 0.32258064516129037,
"0.5": 0.32258064516129037,
"0.75": 0.32258064516129037,
"1.0": 0.25806451612903225
},
"visibility_by_motion_group": {
"dynamic": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"0.5": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"0.75": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"1.0": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
}
},
"visibility_precision_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"visibility_f1_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 21,
"false_negatives": 0
},
"0.5": {
"true_positives": 1,
"false_positives": 21,
"false_negatives": 0
},
"0.75": {
"true_positives": 1,
"false_positives": 21,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.045454545454545456,
"0.5": 0.045454545454545456,
"0.75": 0.045454545454545456,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.08695652173913045,
"0.5": 0.08695652173913045,
"0.75": 0.08695652173913045,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 22,
"false_negatives": 1
},
"0.5": {
"true_positives": 2,
"false_positives": 24,
"false_negatives": 3
},
"0.75": {
"true_positives": 1,
"false_positives": 25,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 26,
"false_negatives": 5
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.15384615384615385,
"0.5": 0.07692307692307693,
"0.75": 0.038461538461538464,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.8,
"0.5": 0.4,
"0.75": 0.2,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.25806451612903225,
"0.5": 0.12903225806451613,
"0.75": 0.06451612903225806,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.06730769230769232,
"segmentation_frame_average_recall": 0.35000000000000003,
"segmentation_frame_average_f1": 0.11290322580645161,
"segmentation_first_frame_counts": {
"true_positives": 4,
"false_positives": 22,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.15384615384615385,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 0.2666666666666667,
"segmentation_by_motion_group": {
"dynamic": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"0.5": {
"true_positives": 2,
"false_positives": 2,
"false_negatives": 2
},
"0.75": {
"true_positives": 1,
"false_positives": 3,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 4,
"false_negatives": 4
}
},
"segmentation_precision_by_threshold": {
"0.25": 1.0,
"0.5": 0.5,
"0.75": 0.25,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.5,
"0.75": 0.25,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 1.0,
"0.5": 0.5,
"0.75": 0.25,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.4375,
"segmentation_frame_average_recall": 0.4375,
"segmentation_frame_average_f1": 0.4375,
"segmentation_first_frame_counts": {
"true_positives": 4,
"false_positives": 0,
"false_negatives": 0
},
"segmentation_first_frame_precision": 1.0,
"segmentation_first_frame_recall": 1.0,
"segmentation_first_frame_f1": 1.0
},
"unknown": {
"segmentation_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 1
}
},
"segmentation_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"segmentation_frame_average_precision": 0.0,
"segmentation_frame_average_recall": 0.0,
"segmentation_frame_average_f1": 0.0,
"segmentation_first_frame_counts": {
"true_positives": 0,
"false_positives": 22,
"false_negatives": 0
},
"segmentation_first_frame_precision": 0.0,
"segmentation_first_frame_recall": 0.0,
"segmentation_first_frame_f1": 0.0
}
}
},
"per_match_metrics": [
{
"target_id": 0,
"pred_id": 1,
"cost": 0.575,
"type_similarity": 0.5009367035182198,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 266.6224469020304,
"image_position_segmentation_acc": 0.425,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.425,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.425,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 1,
"pred_id": 6,
"cost": 1.0333333333333332,
"type_similarity": 0.5721037078119161,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 146.8921073973907,
"image_position_segmentation_acc": 0.4666666666666667,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.4666666666666667,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.4666666666666667,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 2,
"pred_id": 8,
"cost": 0.8833333333333333,
"type_similarity": 0.22305004779758078,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 37.923495432324714,
"image_position_segmentation_acc": 0.6166666666666667,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": 0.6166666666666667,
"image_position_segmentation_acc_large": null,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.6166666666666667,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 3,
"pred_id": 13,
"cost": 0.575,
"type_similarity": 0.31273277133042354,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 37.96422856006942,
"image_position_segmentation_acc": 0.925,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.925,
"image_position_segmentation_acc_first_frame": 1.0,
"image_position_segmentation_visible_acc": 0.925,
"quaternion_geodesic_deg": null,
"visibility_acc": 1.0,
"visibility_true_acc": 1.0,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 1.0,
"vis_acc_from_px_true_acc": 1.0,
"vis_acc_from_px_false_acc": null
},
{
"target_id": 4,
"pred_id": 4,
"cost": 1.0,
"type_similarity": 0.5843774218647733,
"size_diff": null,
"3d_iou": null,
"image_position_px_diff": 657.1997355594633,
"image_position_segmentation_acc": 0.0,
"image_position_segmentation_acc_small": null,
"image_position_segmentation_acc_medium": null,
"image_position_segmentation_acc_large": 0.0,
"image_position_segmentation_acc_first_frame": 0.0,
"image_position_segmentation_visible_acc": 0.0,
"quaternion_geodesic_deg": null,
"visibility_acc": 0.9916666666666667,
"visibility_true_acc": 0.9916666666666667,
"visibility_false_acc": null,
"is_in_frame_acc": 1.0,
"is_in_frame_true_acc": 1.0,
"is_in_frame_false_acc": null,
"vis_acc_from_px": 0.9916666666666667,
"vis_acc_from_px_true_acc": 0.9916666666666667,
"vis_acc_from_px_false_acc": null
}
],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": -0.553483067196655,
"avg_interaction_input_object_jaccard": 0.25,
"avg_interaction_output_object_jaccard": 0.25,
"avg_interaction_type_exact_match_acc": 0.875,
"num_interaction_matches": 16,
"num_unmatched_pred_interactions": 21,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 37,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 37,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 37,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 37,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 6,
"false_positives": 31,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 34,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 34,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 34,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.16216216216216217,
"0.5": 0.08108108108108109,
"0.75": 0.08108108108108109,
"1.0": 0.08108108108108109
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.27906976744186046,
"0.5": 0.15,
"0.75": 0.15,
"1.0": 0.15
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 6,
"false_positives": 31,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 34,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 34,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 34,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.16216216216216217,
"0.5": 0.08108108108108109,
"0.75": 0.08108108108108109,
"1.0": 0.08108108108108109
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.27906976744186046,
"0.5": 0.15,
"0.75": 0.15,
"1.0": 0.15
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 16,
"avg_interaction_input_object_jaccard": 16,
"avg_interaction_output_object_jaccard": 16,
"avg_interaction_type_exact_match_acc": 16
},
"interaction_matching": {
"matches": [
{
"target_index": 0,
"pred_index": 8,
"target_id": 0,
"pred_id": 8,
"cost": 2.7340425531914896,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.7340425531914894,
"weighted_cost": 0.7340425531914894,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 1.0,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 1.0,
"weighted_cost": 0.5,
"used_missing_value": false
}
]
},
{
"target_index": 1,
"pred_index": 11,
"target_id": 1,
"pred_id": 11,
"cost": 1.4714285714285715,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9714285714285714,
"weighted_cost": 0.9714285714285714,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 2,
"pred_index": 27,
"target_id": 2,
"pred_id": 27,
"cost": 1.475,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.975,
"weighted_cost": 0.975,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 3,
"pred_index": 26,
"target_id": 3,
"pred_id": 26,
"cost": 1.4777777777777779,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9777777777777779,
"weighted_cost": 0.9777777777777779,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 4,
"pred_index": 1,
"target_id": 4,
"pred_id": 1,
"cost": 0.9583333333333333,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9583333333333333,
"weighted_cost": 0.9583333333333333,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 5,
"pred_index": 2,
"target_id": 5,
"pred_id": 2,
"cost": 0.4,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.4,
"weighted_cost": 0.4,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 6,
"pred_index": 35,
"target_id": 6,
"pred_id": 35,
"cost": 2.1323529411764706,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 1.0,
"weighted_cost": 1.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.6323529411764706,
"weighted_cost": 0.6323529411764706,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 7,
"pred_index": 28,
"target_id": 7,
"pred_id": 28,
"cost": 1.25,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.75,
"weighted_cost": 0.75,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 8,
"pred_index": 23,
"target_id": 8,
"pred_id": 23,
"cost": 1.490909090909091,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.990909090909091,
"weighted_cost": 0.990909090909091,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 9,
"pred_index": 24,
"target_id": 9,
"pred_id": 24,
"cost": 1.4690265486725664,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9690265486725664,
"weighted_cost": 0.9690265486725664,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 10,
"pred_index": 25,
"target_id": 10,
"pred_id": 25,
"cost": 1.4875,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9875,
"weighted_cost": 0.9875,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 11,
"pred_index": 14,
"target_id": 11,
"pred_id": 14,
"cost": 1.988235294117647,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.9882352941176471,
"weighted_cost": 0.9882352941176471,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 1.0,
"weighted_cost": 0.5,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 1.0,
"weighted_cost": 0.5,
"used_missing_value": false
}
]
},
{
"target_index": 12,
"pred_index": 12,
"target_id": 12,
"pred_id": 12,
"cost": 1.3932584269662922,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.8932584269662922,
"weighted_cost": 0.8932584269662922,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 13,
"pred_index": 33,
"target_id": 13,
"pred_id": 33,
"cost": 0.4,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.4,
"weighted_cost": 0.4,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
},
{
"target_index": 14,
"pred_index": 10,
"target_id": 14,
"pred_id": 10,
"cost": 0.9,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.4,
"weighted_cost": 0.4,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.5,
"weighted_cost": 0.25,
"used_missing_value": false
}
]
},
{
"target_index": 15,
"pred_index": 6,
"target_id": 15,
"pred_id": 6,
"cost": 0.4,
"cost_breakdown": [
{
"name": "type_exact",
"weight": 1.0,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "temporal_span",
"weight": 1.0,
"raw_cost": 0.4,
"weighted_cost": 0.4,
"used_missing_value": false
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5,
"raw_cost": 0.0,
"weighted_cost": 0.0,
"used_missing_value": false
}
]
}
],
"unmatched_target_ids": [],
"unmatched_pred_ids": [
0,
3,
4,
5,
7,
9,
13,
15,
16,
17,
18,
19,
20,
21,
22,
29,
30,
31,
32,
34,
36
],
"target_type_counts": {
"collision": 8,
"contact": 8
},
"pred_type_counts": {
"contact": 31,
"collision": 6
},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [
{
"target_index": 0,
"pred_index": 8,
"target_id": 0,
"pred_id": 8,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.7340425531914896,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.46808510638297873,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 1,
"pred_index": 11,
"target_id": 1,
"pred_id": 11,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4714285714285715,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9428571428571428,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 2,
"pred_index": 27,
"target_id": 2,
"pred_id": 27,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.475,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.95,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 3,
"pred_index": 26,
"target_id": 3,
"pred_id": 26,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4777777777777779,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9555555555555556,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 4,
"pred_index": 1,
"target_id": 4,
"pred_id": 1,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.9583333333333333,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9166666666666666,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 5,
"pred_index": 2,
"target_id": 5,
"pred_id": 2,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.4,
"temporal_span_iou": 0.2,
"temporal_span_giou": 0.2,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 6,
"pred_index": 35,
"target_id": 6,
"pred_id": 35,
"target_type": "collision",
"pred_type": "contact",
"cost": 2.1323529411764706,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.2647058823529412,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 0.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 7,
"pred_index": 28,
"target_id": 7,
"pred_id": 28,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.25,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.5,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 8,
"pred_index": 23,
"target_id": 8,
"pred_id": 23,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.490909090909091,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9818181818181818,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 9,
"pred_index": 24,
"target_id": 9,
"pred_id": 24,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.4690265486725664,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9380530973451328,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 10,
"pred_index": 25,
"target_id": 10,
"pred_id": 25,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.4875,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.975,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 11,
"pred_index": 14,
"target_id": 11,
"pred_id": 14,
"target_type": "collision",
"pred_type": "collision",
"cost": 1.988235294117647,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.9764705882352941,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 12,
"pred_index": 12,
"target_id": 12,
"pred_id": 12,
"target_type": "contact",
"pred_type": "contact",
"cost": 1.3932584269662922,
"temporal_span_iou": 0.0,
"temporal_span_giou": -0.7865168539325843,
"input_object_jaccard": 0.0,
"output_object_jaccard": 0.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 13,
"pred_index": 33,
"target_id": 13,
"pred_id": 33,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.4,
"temporal_span_iou": 0.2,
"temporal_span_giou": 0.2,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
},
{
"target_index": 14,
"pred_index": 10,
"target_id": 14,
"pred_id": 10,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.9,
"temporal_span_iou": 0.2,
"temporal_span_giou": 0.2,
"input_object_jaccard": 0.3333333333333333,
"output_object_jaccard": 0.3333333333333333,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 0.0,
"output_object_set_exact_match": 0.0
},
{
"target_index": 15,
"pred_index": 6,
"target_id": 15,
"pred_id": 6,
"target_type": "contact",
"pred_type": "contact",
"cost": 0.4,
"temporal_span_iou": 0.2,
"temporal_span_giou": 0.2,
"input_object_jaccard": 1.0,
"output_object_jaccard": 1.0,
"type_exact_match": 1.0,
"temporal_span_exact_match": 0.0,
"input_object_set_exact_match": 1.0,
"output_object_set_exact_match": 1.0
}
],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0.2,
"avg_interaction_input_object_jaccard": 0.8333333333333334,
"avg_interaction_output_object_jaccard": 0.8333333333333334,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 4,
"num_unmatched_pred_interactions": 21,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 25,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 25,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 25,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 25,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 21,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 22,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 22,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 22,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.16,
"0.5": 0.12,
"0.75": 0.12,
"1.0": 0.12
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.2758620689655173,
"0.5": 0.21428571428571425,
"0.75": 0.21428571428571425,
"1.0": 0.21428571428571425
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 4,
"false_positives": 21,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 22,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 22,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 22,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.16,
"0.5": 0.12,
"0.75": 0.12,
"1.0": 0.12
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.2758620689655173,
"0.5": 0.21428571428571425,
"0.75": 0.21428571428571425,
"1.0": 0.21428571428571425
}
},
"object_object": {
"avg_interaction_temporal_span_giou": -0.8046440895955397,
"avg_interaction_input_object_jaccard": 0.05555555555555555,
"avg_interaction_output_object_jaccard": 0.05555555555555555,
"avg_interaction_type_exact_match_acc": 0.8333333333333334,
"num_interaction_matches": 12,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 10,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.16666666666666666,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.2857142857142857,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 2,
"false_positives": 10,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 12,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.16666666666666666,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.2857142857142857,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 4,
"avg_interaction_input_object_jaccard": 4,
"avg_interaction_output_object_jaccard": 4,
"avg_interaction_type_exact_match_acc": 4
},
"object_object": {
"avg_interaction_temporal_span_giou": 12,
"avg_interaction_input_object_jaccard": 12,
"avg_interaction_output_object_jaccard": 12,
"avg_interaction_type_exact_match_acc": 12
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": -0.8143115571502617,
"avg_interaction_input_object_jaccard": 0.041666666666666664,
"avg_interaction_output_object_jaccard": 0.041666666666666664,
"avg_interaction_type_exact_match_acc": 0.75,
"num_interaction_matches": 8,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 7,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.125,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.2222222222222222,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 1,
"false_positives": 7,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 8,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.125,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.2222222222222222,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": -0.2926545772430479,
"avg_interaction_input_object_jaccard": 0.4583333333333333,
"avg_interaction_output_object_jaccard": 0.4583333333333333,
"avg_interaction_type_exact_match_acc": 1.0,
"num_interaction_matches": 8,
"num_unmatched_pred_interactions": 21,
"num_unmatched_target_interactions": 0,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 29,
"false_negatives": 0
},
"0.5": {
"true_positives": 0,
"false_positives": 29,
"false_negatives": 0
},
"0.75": {
"true_positives": 0,
"false_positives": 29,
"false_negatives": 0
},
"1.0": {
"true_positives": 0,
"false_positives": 29,
"false_negatives": 0
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 5,
"false_positives": 24,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 26,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 26,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 26,
"false_negatives": 0
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.1724137931034483,
"0.5": 0.10344827586206896,
"0.75": 0.10344827586206896,
"1.0": 0.10344827586206896
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.29411764705882354,
"0.5": 0.1875,
"0.75": 0.1875,
"1.0": 0.1875
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 5,
"false_positives": 24,
"false_negatives": 0
},
"0.5": {
"true_positives": 3,
"false_positives": 26,
"false_negatives": 0
},
"0.75": {
"true_positives": 3,
"false_positives": 26,
"false_negatives": 0
},
"1.0": {
"true_positives": 3,
"false_positives": 26,
"false_negatives": 0
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.1724137931034483,
"0.5": 0.10344827586206896,
"0.75": 0.10344827586206896,
"1.0": 0.10344827586206896
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 1.0,
"0.5": 1.0,
"0.75": 1.0,
"1.0": 1.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.29411764705882354,
"0.5": 0.1875,
"0.75": 0.1875,
"1.0": 0.1875
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 8,
"avg_interaction_input_object_jaccard": 8,
"avg_interaction_output_object_jaccard": 8,
"avg_interaction_type_exact_match_acc": 8
},
"contact": {
"avg_interaction_temporal_span_giou": 8,
"avg_interaction_input_object_jaccard": 8,
"avg_interaction_output_object_jaccard": 8,
"avg_interaction_type_exact_match_acc": 8
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}