STRIVE · 08/28/2026
Dyn generalization · test_120 · super_clevr_1153
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow scooter\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 80 738 106 75 732 [OBJ] 1 \"blue mountain\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 383 819 7 386 865 10 383 818 12 382 814 19 387 866 20 385 858 21 382 810 23 383 797 24 382 808 26 383 828 27 381 805 28 381 800 30 380 802 31 383 846 32 380 799 33 380 795 34 380 818 35 380 797 36 379 793 37 382 836 38 380 790 39 380 811 40 380 790 41 380 786 43 380 828 44 379 784 45 379 781 46 377 780 48 378 777 49 380 820 50 376 775 53 378 811 54 376 767 55 376 770 56 375 765 59 375 761 60 376 799 61 374 758 63 376 793 64 373 754 66 375 786 67 373 748 69 373 745 70 372 743 72 373 774 73 371 736 75 369 774 76 369 718 77 364 771 78 361 769 79 359 760 80 358 756 81 363 719 82 352 748 83 351 746 84 352 740 85 359 714 87 355 715 88 348 733 89 354 713 91 352 714 92 356 709 93 352 714 94 355 710 95 354 713 96 340 724 97 348 723 98 354 721 99 329 730 100 326 733 103 326 728 104 326 725 105 325 722 106 324 720 108 307 732 109 304 731 110 305 729 111 299 731 112 298 730 113 297 728 114 294 728 115 293 727 116 330 707 117 284 723 119 282 721 [OBJ] 2 \"cyan suv\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 733 628 1 730 628 3 725 631 4 722 631 5 724 631 6 719 634 7 717 635 9 712 638 11 709 639 12 704 640 14 701 643 15 699 644 16 696 645 17 694 647 18 691 648 19 688 648 20 686 650 22 680 652 23 678 653 24 676 655 25 673 656 26 670 657 27 665 658 29 659 661 30 657 664 31 655 665 32 651 666 33 649 667 34 644 669 36 640 672 37 636 673 38 630 676 39 628 677 40 625 678 41 620 681 42 617 682 43 615 683 44 609 686 45 605 689 46 601 690 47 598 691 48 594 694 49 591 695 50 585 698 51 580 700 52 577 701 53 572 704 54 567 706 55 564 707 56 558 710 57 554 713 58 548 715 59 543 718 60 537 721 61 533 723 62 530 724 63 522 728 64 519 729 65 512 733 66 513 730 67 503 736 68 498 740 69 491 744 70 485 746 71 486 743 72 474 751 73 473 749 74 462 757 75 459 759 77 457 763 78 455 765 79 451 767 80 450 768 81 445 772 82 444 773 83 442 774 84 438 777 85 436 778 86 432 782 87 428 784 88 424 786 89 420 790 90 416 793 91 410 799 92 406 801 93 401 805 94 396 813 95 391 817 96 387 821 97 382 824 98 378 825 99 373 828 100 369 829 101 364 831 102 359 814 103 352 856 104 349 859 105 346 864 106 343 868 107 339 869 108 335 871 109 330 872 110 325 873 111 321 879 112 316 880 113 310 882 114 305 884 115 300 889 116 294 890 117 289 894 118 283 896 119 278 899 [OBJ] 3 \"gray fighter\" 9 120 [VIS] 0 false 9 true [IN-FRAME] 0 false 9 true [PX] 0 null null 9 0 308 10 2 307 11 5 307 13 7 305 14 8 302 15 10 299 16 12 296 17 23 296 18 35 296 19 39 295 20 55 295 21 61 296 22 12 304 23 26 305 24 41 305 25 56 304 26 71 304 27 12 301 28 16 295 29 29 294 30 43 294 31 57 294 32 71 294 33 87 294 34 101 294 35 115 294 36 129 294 37 141 291 38 155 290 39 168 289 40 175 280 41 197 291 42 202 280 43 222 288 44 230 280 45 243 280 46 255 279 47 270 282 48 282 280 49 295 281 50 309 283 51 321 282 52 334 283 53 348 284 54 360 285 55 373 285 56 385 286 57 398 288 58 410 290 59 422 290 60 435 292 61 446 292 62 458 292 63 470 293 64 482 293 65 494 294 66 505 294 67 516 295 68 527 295 69 538 295 70 549 296 71 560 296 72 571 297 73 582 298 74 593 297 75 604 298 76 613 298 77 624 298 78 634 299 79 644 299 80 656 301 81 666 301 82 678 303 83 689 304 84 698 304 85 708 304 86 717 304 87 726 304 88 736 304 89 745 305 90 755 304 91 763 305 92 772 305 93 781 305 94 790 305 95 798 305 96 807 305 97 816 305 98 825 305 99 833 305 100 841 306 101 850 306 102 858 306 103 866 306 104 874 306 105 883 306 106 891 306 107 899 306 108 908 307 109 915 307 110 923 307 111 930 307 112 938 307 113 946 307 114 953 307 115 961 307 116 968 307 117 976 307 118 983 307 119 988 309 [OBJS-END] [INTERACTIONS-START] [INTERACT] 2 \"contact\" 0 97 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 0 \"contact\" 0 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 8 \"contact\" 0 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 1 \"collision\" 75 75 [IN] 1 2 [OUT] 1 2 [INTERACT] 3 \"collision\" 103 103 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 4 \"collision\" 105 105 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 5 \"contact\" 106 109 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 6 \"collision\" 111 111 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 7 \"contact\" 112 119 [IN] 1 65535 [OUT] 1 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_full/super_clevr_1153/frames.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow cardboard Raisinets box\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 1 837 1 null null [OBJ] 1 \"yellow cardboard Raisinets box\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 1 873 1 null null [OBJ] 2 \"yellow plastic candy bar\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 1 986 1 null null [OBJ] 3 \"yellow plastic candy bar\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 1 994 1 null null [OBJ] 4 \"yellow wooden toy construction vehicle set\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 5 810 1 null null [OBJ] 5 \"yellow and blue plastic soccer cleat\" 0 26 [VIS] 0 true 23 false [IN-FRAME] 0 true [PX] 0 17 837 1 13 826 5 9 818 10 5 810 22 1 802 23 null null [OBJ] 6 \"yellow wooden toy construction vehicle set\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 5 998 1 null null [OBJ] 7 \"yellow plastic scooter\" 0 26 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 158 720 16 150 724 [OBJ] 8 \"blue and black hiking shoe\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 314 994 1 null null [OBJ] 9 \"blue and orange plastic Therizinosaurus figurine\" 0 26 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 330 763 3 333 771 4 326 763 5 333 771 11 326 763 12 330 771 13 326 763 14 333 771 16 330 763 17 337 771 18 330 763 19 337 771 20 330 763 21 337 771 22 330 763 23 337 771 24 330 763 25 337 771 26 330 763 27 337 771 28 330 763 29 337 771 30 330 763 31 337 771 32 330 763 33 337 771 34 330 763 35 337 771 36 330 763 37 337 771 38 330 763 39 337 771 40 330 763 41 337 771 42 330 763 43 337 771 44 330 763 45 337 771 46 330 763 47 337 771 48 330 763 49 337 771 50 330 763 51 337 771 52 330 763 53 337 771 54 330 763 55 337 771 56 330 763 57 337 771 58 330 763 59 337 771 60 330 763 61 337 771 62 330 763 63 337 771 64 330 763 65 337 771 66 330 763 67 337 771 68 330 763 69 337 771 70 330 763 71 337 771 72 330 763 73 337 771 74 330 763 75 337 771 76 330 763 77 337 771 78 330 763 79 337 771 80 330 763 81 337 771 82 330 763 83 337 771 84 330 763 85 337 771 86 330 763 87 337 771 88 330 763 89 337 771 90 330 763 91 337 771 92 330 763 93 337 771 94 330 763 95 337 771 96 330 763 97 337 771 98 330 763 99 337 771 100 330 763 101 337 771 102 330 763 103 337 771 104 330 763 105 337 771 106 330 763 107 337 771 108 330 763 109 337 771 110 330 763 111 337 771 112 330 763 113 337 771 114 330 763 115 337 771 116 330 763 117 337 771 118 330 763 119 337 771 [OBJ] 10 \"teal fabric mermaid backpack\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 349 998 1 null null [OBJ] 11 \"blue and black plastic DVD set\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 458 998 1 null null [OBJ] 12 \"black plastic gaming mouse\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 529 998 1 null null [OBJ] 13 \"black plastic USB flash drive\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 533 998 1 null null [OBJ] 14 \"white and grey cardboard desktop hard drive box\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 541 998 1 null null [OBJ] 15 \"grey and white plastic shark figurine\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 583 998 1 null null [OBJ] 16 \"grey and white plush shark\" 0 26 [VIS] 0 true 1 false [IN-FRAME] 0 true 1 false [PX] 0 587 994 1 null null [OBJ] 17 \"teal fabric mermaid backpack\" 0 26 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 740 630 4 728 634 6 720 638 9 708 642 11 697 646 13 689 650 15 677 654 17 666 658 18 658 662 19 650 666 21 638 669 22 630 673 23 623 677 24 615 681 25 607 685 26 599 689 27 591 693 28 583 697 29 576 701 30 568 705 31 560 708 32 552 712 33 544 716 34 537 720 35 529 724 36 521 728 37 513 732 38 505 736 39 498 740 40 490 744 41 482 748 42 474 751 43 466 755 44 458 759 45 451 763 46 443 767 47 435 771 48 427 775 49 419 779 50 412 783 51 404 787 52 396 791 53 388 794 54 380 798 55 373 802 56 365 806 57 357 810 58 349 814 59 341 818 60 333 822 61 326 826 62 318 830 63 310 833 64 302 837 65 294 841 66 287 845 67 279 849 68 271 853 69 263 857 70 255 861 71 248 865 72 240 869 73 232 873 74 224 876 75 216 880 76 208 884 77 201 888 78 193 892 79 185 896 80 177 900 81 169 904 82 162 908 83 154 912 84 146 916 85 138 919 86 130 923 87 123 927 88 115 931 89 107 935 90 99 939 91 91 943 92 83 947 93 76 951 94 68 955 95 60 958 96 52 962 97 44 966 98 37 970 99 29 974 100 21 978 101 13 982 102 5 986 103 1 994 104 13 998 105 5 994 106 1 990 107 1 998 [OBJ] 18 \"blue and orange plastic Therizinosaurus figurine\" 0 26 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 525 4 998 537 6 998 548 8 998 560 10 998 572 11 994 564 13 994 576 15 990 583 17 990 595 18 986 603 20 982 611 22 978 619 24 974 626 25 970 634 27 966 642 29 962 650 31 955 654 33 951 662 35 947 669 37 939 673 39 931 677 41 927 685 43 919 689 45 912 693 47 904 697 49 896 701 51 888 705 53 880 708 55 873 712 57 865 716 59 857 720 61 849 724 63 841 728 65 833 732 67 826 736 69 818 740 71 810 744 73 802 748 75 794 751 77 787 755 79 779 759 81 771 763 83 763 767 85 755 771 87 748 775 89 740 779 91 732 783 93 724 787 95 716 791 97 708 794 99 701 798 101 693 802 103 685 806 105 677 810 107 669 814 109 662 818 111 654 822 113 646 826 115 638 830 117 630 833 119 623 837 [OBJ] 19 \"grey and black basketball shoe\" 0 26 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 986 1 998 974 3 998 951 4 994 986 5 994 970 6 994 958 7 994 947 8 990 974 9 990 958 10 990 947 11 986 970 12 986 958 13 986 947 14 982 970 15 982 958 16 982 947 17 978 966 18 978 955 19 978 943 20 974 962 21 974 951 22 974 939 23 970 958 24 970 947 25 966 962 26 966 951 [OBJ] 20 \"grey and white plastic shark figurine\" 5 26 [VIS] 0 false 5 true [IN-FRAME] 0 false 5 true [PX] 0 null null 5 1 310 6 5 318 8 9 326 9 13 333 11 17 341 12 21 349 13 17 337 14 25 345 15 33 353 16 41 361 17 48 369 18 56 376 19 64 384 20 72 392 21 80 400 22 87 408 23 95 416 24 103 423 25 111 431 26 119 439 27 126 447 28 134 455 29 142 462 30 150 470 31 158 478 32 166 486 33 173 494 34 181 501 35 189 509 36 197 517 37 205 525 38 212 533 39 220 541 40 228 548 41 236 556 42 244 564 43 251 572 44 259 580 45 267 587 46 275 595 47 283 603 48 291 611 49 298 619 50 306 626 51 314 634 52 322 642 53 330 650 54 337 658 55 345 666 56 353 673 57 361 681 58 369 689 59 376 697 60 384 705 61 392 712 62 396 720 63 404 728 64 412 736 65 419 744 66 427 751 67 435 759 68 443 767 69 451 775 70 458 783 71 466 791 72 474 798 73 482 806 74 490 814 75 498 822 76 505 830 77 513 837 78 521 845 79 529 853 80 537 861 81 544 869 82 552 876 83 560 884 84 568 892 85 576 900 86 583 908 87 591 916 88 599 923 89 607 931 90 615 939 91 623 947 92 630 955 93 638 962 94 646 970 95 654 978 96 662 986 97 669 990 101 681 994 102 689 998 103 697 994 104 701 998 105 708 994 106 716 998 107 720 990 108 728 998 109 736 994 110 744 998 111 751 994 112 759 998 113 767 994 114 775 998 115 783 994 116 791 998 117 798 994 118 806 998 119 814 994 [OBJ] 21 \"grey and white plastic shark figurine\" 5 26 [VIS] 0 false 5 true [IN-FRAME] 0 false 5 true [PX] 0 null null 5 1 330 7 1 318 8 5 330 10 5 318 11 9 330 13 9 318 14 13 330 16 13 318 17 17 330 19 21 337 21 21 326 22 25 337 24 25 326 25 29 337 27 33 345 28 33 333 29 37 345 31 37 333 32 41 345 34 44 353 35 44 337 36 48 349 38 52 357 39 52 341 40 56 353 42 60 361 43 60 349 44 64 357 45 64 345 46 68 353 47 68 341 48 72 353 49 72 341 50 76 349 51 76 337 52 80 345 53 80 333 54 83 341 55 83 330 56 87 337 57 87 326 58 91 333 59 91 322 60 95 330 61 95 318 62 99 326 63 99 314 64 103 322 65 103 310 66 107 318 67 107 306 68 111 314 69 111 302 70 115 310 71 115 298 72 119 306 73 119 294 74 123 302 75 123 291 76 126 298 77 126 287 78 130 294 79 130 283 80 134 291 81 134 279 82 138 287 83 138 275 84 142 283 85 142 271 86 146 279 87 146 267 88 150 275 89 150 263 90 154 271 91 154 259 92 158 267 93 158 255 94 162 263 95 162 251 96 166 259 97 166 248 98 169 255 99 169 244 100 173 251 101 173 240 102 177 248 103 177 236 104 181 244 105 181 232 106 185 240 107 189 248 108 189 236 109 193 244 110 193 232 111 197 240 112 197 228 113 201 236 114 201 224 115 205 232 116 208 240 117 208 228 118 212 236 119 212 224 [OBJ] 22 \"grey and white plastic shark figurine\" 5 26 [VIS] 0 false 5 true [IN-FRAME] 0 false 5 true [PX] 0 null null 5 13 322 6 13 337 7 13 353 8 13 365 9 13 380 10 13 392 11 13 404 12 13 416 13 13 427 14 13 439 15 13 451 16 13 462 17 13 474 18 13 486 19 13 498 20 13 509 21 13 521 22 13 533 23 13 544 24 13 556 25 13 568 26 13 580 27 13 591 28 13 603 29 13 615 30 13 626 31 13 638 32 13 650 33 13 662 34 13 673 35 13 685 36 13 697 37 13 708 38 13 720 39 13 732 40 13 744 41 13 755 42 13 767 43 13 779 44 13 791 45 13 802 46 13 814 47 13 826 48 13 837 49 13 849 50 13 861 51 13 873 52 13 884 53 13 896 54 13 908 55 13 919 56 13 931 57 13 943 58 13 955 59 13 966 60 13 978 61 13 990 62 13 998 63 9 986 64 17 994 65 21 990 67 33 990 69 44 990 71 56 990 73 68 990 75 80 990 77 91 990 79 103 990 81 111 990 83 119 986 86 130 986 88 138 982 91 146 978 94 154 974 97 162 970 99 169 966 102 177 962 105 185 958 108 193 955 111 201 947 114 208 943 117 216 939 119 224 935 [OBJ] 23 \"grey and white plastic shark figurine\" 10 26 [VIS] 0 false 10 true [IN-FRAME] 0 false 10 true [PX] 0 null null 10 998 361 12 998 349 14 994 357 16 994 345 18 990 353 20 990 341 22 986 349 24 986 337 25 982 345 27 982 333 28 978 341 30 978 330 31 974 337 33 970 345 34 970 333 35 966 341 37 962 349 38 962 337 39 958 345 41 958 333 42 955 341 44 951 349 45 951 337 46 947 345 48 943 353 49 943 341 50 939 349 52 935 357 53 935 345 54 931 353 56 927 361 57 927 349 58 923 357 60 919 365 61 919 353 62 916 361 64 912 369 65 912 357 66 908 365 68 904 373 70 900 365 71 896 373 72 896 361 73 892 369 75 888 376 76 888 365 77 884 373 79 880 380 80 880 369 81 876 376 83 873 384 84 873 373 85 869 380 87 865 388 88 865 376 90 861 384 91 861 373 92 857 380 94 853 388 96 853 376 97 849 384 99 845 392 100 845 380 101 841 388 103 837 396 105 837 384 106 833 392 108 830 400 110 826 408 112 826 396 113 822 404 115 818 412 117 818 400 118 814 408 119 810 416 [OBJ] 24 \"white and blue athletic shoe\" 17 26 [VIS] 0 false 17 true 21 false [IN-FRAME] 0 false 17 true 21 false [PX] 0 null null 17 1 833 19 1 845 20 1 861 21 null null [OBJ] 25 \"white and light blue athletic shoe\" 17 26 [VIS] 0 false 17 true 20 false [IN-FRAME] 0 false 17 true 20 false [PX] 0 null null 17 1 857 19 1 869 20 null null [OBJ] 26 \"grey and white plastic shark figurine\" 20 26 [VIS] 0 false 20 true [IN-FRAME] 0 false 20 true [PX] 0 null null 20 998 376 22 998 361 24 994 369 25 994 357 27 990 365 29 986 357 31 982 365 33 978 357 34 974 365 36 970 357 37 966 365 39 962 357 40 958 365 41 955 373 42 951 380 43 951 369 44 947 376 45 943 384 46 939 392 47 939 380 48 935 388 49 931 396 50 927 404 51 923 412 52 919 419 53 916 427 54 912 435 55 908 443 56 904 451 57 900 458 58 900 447 59 896 455 60 892 462 61 888 470 62 884 478 63 880 486 64 876 494 65 873 501 66 869 509 67 865 517 68 861 525 69 857 533 70 853 541 71 849 548 72 845 556 73 841 564 74 837 572 75 833 580 76 830 587 77 826 595 78 822 603 79 818 611 80 814 619 81 810 626 82 806 634 83 802 642 84 798 650 85 794 658 86 791 666 87 787 673 88 783 681 89 779 689 90 775 697 91 771 705 92 767 712 93 763 720 94 759 728 95 755 736 96 751 744 97 748 751 98 744 759 99 740 767 100 736 775 101 732 783 102 728 791 103 724 798 104 720 806 105 716 814 106 712 822 107 708 830 108 705 837 109 701 845 110 697 853 111 693 861 112 689 869 113 685 876 114 681 884 115 677 892 116 673 900 117 669 908 118 666 916 119 662 923 [OBJ] 27 \"white and blue track spike\" 24 26 [VIS] 0 false 24 true [IN-FRAME] 0 false 24 true [PX] 0 null null 24 998 759 25 998 748 27 994 755 29 990 763 31 986 771 33 982 779 35 978 787 37 974 794 39 970 802 41 966 810 43 962 818 45 958 826 47 955 833 49 951 841 51 947 849 53 943 857 55 939 865 57 935 873 59 931 880 61 927 888 63 923 896 65 919 904 67 916 912 69 912 919 71 908 927 73 904 935 75 900 943 77 896 951 81 892 958 82 888 966 86 884 974 89 880 982 92 876 990 95 873 994 98 884 994 [OBJ] 28 \"grey and white plastic shark figurine\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 1 451 27 1 435 29 5 447 31 5 431 32 9 447 34 9 431 36 13 447 38 13 435 40 13 423 41 17 439 43 17 427 45 17 419 46 21 435 48 21 423 49 25 439 51 25 427 52 29 439 54 29 427 56 33 439 58 33 427 59 37 439 61 37 427 62 41 439 64 41 427 66 44 435 68 44 423 69 48 435 71 48 423 72 52 435 74 52 423 75 56 435 77 56 423 79 56 412 80 60 423 82 60 412 83 64 423 85 64 412 86 68 423 88 68 412 89 72 423 91 72 412 92 76 423 94 76 412 95 80 423 97 80 412 98 83 423 99 83 412 100 87 423 101 87 412 102 91 423 103 91 412 104 95 423 105 95 412 106 99 423 107 99 412 108 103 423 109 103 412 110 107 423 111 107 412 112 111 423 113 111 412 114 115 423 116 119 416 117 123 423 119 126 416 [OBJ] 29 \"black and white high-top sneaker\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 998 845 27 998 833 29 994 845 31 994 833 33 990 841 35 990 830 36 986 841 38 986 830 39 982 841 41 982 830 42 978 841 44 978 830 45 974 837 47 970 845 48 970 833 49 966 841 51 966 830 52 962 837 54 962 826 55 958 833 57 955 841 58 955 830 59 951 837 61 951 826 62 947 833 64 943 841 65 943 830 66 939 837 68 939 826 69 935 833 71 931 841 72 931 830 73 927 837 75 927 826 76 923 833 78 919 841 79 919 830 80 916 837 82 916 826 83 912 833 85 908 841 86 908 830 87 904 837 90 900 845 91 900 833 92 896 841 95 892 849 97 888 857 99 884 865 101 880 873 103 876 880 105 873 888 107 869 896 109 865 904 111 861 912 113 857 919 116 853 927 119 849 935 [OBJ] 30 \"white and blue athletic shoe\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 998 857 27 998 845 28 994 853 30 994 841 31 990 849 33 990 837 34 986 845 36 986 833 37 982 841 39 978 849 41 978 837 42 974 845 44 970 853 46 970 841 47 966 849 49 962 857 51 962 845 52 958 853 54 955 861 56 955 849 57 951 857 59 947 865 61 947 853 62 943 861 64 939 869 66 935 876 68 935 865 69 931 873 71 927 880 73 923 888 74 919 896 76 916 904 79 912 912 81 908 919 83 904 927 86 900 935 89 896 943 91 892 951 94 888 958 97 884 966 99 880 974 102 876 982 105 873 990 106 880 994 107 873 990 111 876 998 [OBJ] 31 \"grey and white plastic shark figurine\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 998 873 27 998 857 28 994 865 30 994 849 31 990 857 33 986 865 34 986 853 35 982 861 37 978 869 39 978 857 40 974 865 42 970 873 43 970 861 45 966 869 47 962 876 49 962 865 50 958 873 52 955 880 54 955 869 56 951 876 58 951 865 59 947 873 61 943 880 63 939 888 64 939 876 66 935 884 68 931 892 70 931 880 71 927 888 73 923 896 75 919 904 77 916 912 79 912 919 81 908 927 83 904 935 86 900 943 89 896 951 92 892 958 95 888 966 98 884 974 99 876 978 [OBJ] 32 \"grey and white plastic shark figurine\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 994 888 27 994 873 28 990 884 30 990 869 31 986 880 33 986 869 34 982 880 36 978 888 37 978 876 39 974 884 40 974 873 41 970 884 43 966 892 44 966 880 46 962 888 47 962 876 49 958 884 50 958 873 52 955 880 53 955 869 54 951 880 56 947 888 57 947 876 59 943 884 60 943 873 62 939 880 63 939 869 65 935 876 66 935 865 68 931 873 69 931 861 71 927 869 72 927 857 74 923 865 75 923 853 77 919 861 78 919 849 80 916 857 81 916 845 83 912 853 84 912 841 86 908 849 88 904 857 89 904 845 91 900 853 93 896 861 94 896 849 96 892 857 98 888 865 99 888 853 101 884 861 103 880 869 104 880 857 106 876 865 108 873 873 110 869 880 111 869 869 113 865 876 115 861 884 117 861 873 119 857 880 [OBJ] 33 \"white and light blue athletic shoe\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 994 869 27 994 857 28 990 865 30 990 853 31 986 861 33 986 849 34 982 857 36 982 845 37 978 853 39 978 841 40 974 849 42 974 837 44 970 845 46 970 833 47 966 841 49 966 830 51 962 837 53 962 826 54 958 833 56 958 822 57 955 830 59 951 837 61 951 826 62 947 833 64 947 822 65 943 830 67 943 818 68 939 826 70 935 833 72 935 822 73 931 830 75 927 837 77 927 826 79 923 833 81 919 841 83 919 830 84 916 837 86 912 845 88 912 833 90 908 841 92 904 849 94 900 857 96 900 845 98 896 853 99 892 861 101 892 849 103 888 857 105 884 865 107 880 873 109 880 861 111 876 869 113 873 876 116 869 884 119 865 892 [OBJ] 34 \"grey and white plastic shark figurine\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 994 982 27 990 990 28 998 994 29 990 998 31 986 990 33 982 998 34 990 994 35 982 998 36 990 994 37 982 998 41 990 994 42 978 998 43 986 994 44 974 998 45 986 994 46 974 998 47 986 994 48 974 998 52 990 994 53 978 998 54 966 998 55 978 998 56 966 998 57 978 998 58 966 998 59 978 998 60 966 998 61 978 998 62 966 998 63 978 998 64 966 998 65 978 998 66 966 998 67 978 998 68 966 998 69 978 998 70 966 998 71 978 998 72 966 998 73 978 998 74 966 998 75 978 998 76 966 998 77 978 998 78 966 998 79 978 998 80 966 998 81 978 998 82 966 998 83 978 998 84 966 998 85 978 998 86 966 998 87 978 998 88 966 998 89 978 998 90 966 998 91 978 998 92 966 998 93 978 998 94 966 998 95 978 998 96 966 998 97 978 998 98 966 998 99 978 998 [OBJ] 35 \"grey and white plastic shark figurine\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 990 994 27 986 986 29 982 994 31 978 990 33 974 998 34 970 990 36 966 998 37 962 990 39 958 998 40 955 990 42 951 998 43 947 990 45 943 998 46 939 990 48 935 998 49 931 990 51 927 998 52 923 990 54 919 998 55 916 990 57 912 998 58 908 990 60 904 998 61 900 990 63 896 998 64 892 990 66 888 998 67 884 990 69 880 998 70 876 990 72 873 998 73 869 990 75 865 998 76 861 990 78 857 998 79 853 990 81 849 998 82 845 990 84 841 998 85 837 990 87 833 998 88 830 990 90 826 998 91 822 990 93 818 998 94 814 990 96 810 998 97 806 990 99 802 998 [OBJ] 36 \"white and blue athletic shoe\" 25 26 [VIS] 0 false 25 true [IN-FRAME] 0 false 25 true [PX] 0 null null 25 994 990 27 990 998 28 986 990 29 986 978 30 982 986 31 982 974 32 978 982 33 978 966 34 974 974 35 974 962 36 970 970 37 970 958 38 966 966 39 966 955 40 962 962 41 962 951 42 958 958 43 958 947 44 955 955 45 955 943 46 951 951 47 951 939 48 947 947 49 947 935 50 943 943 51 943 931 52 939 939 53 939 927 54 935 935 55 935 923 56 931 931 57 931 919 58 927 927 59 927 916 60 923 923 61 919 931 62 919 919 63 916 927 64 916 916 65 912 923 66 912 912 67 908 919 68 908 908 69 904 916 70 904 904 71 900 912 72 896 919 73 896 908 74 892 916 75 892 904 76 888 912 77 888 900 78 884 908 79 880 916 80 880 904 81 876 912 82 876 900 83 873 908 84 869 916 85 869 904 86 865 912 87 865 900 88 861 908 89 857 916 90 857 904 91 853 912 92 853 900 93 849 908 94 845 916 95 845 904 96 841 912 97 841 896 98 837 904 99 833 912 100 833 896 101 830 904 102 826 912 103 826 896 104 822 904 105 822 8",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow scooter\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 80 738 106 75 732 [OBJ] 1 \"blue mountain\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 383 819 7 386 865 10 383 818 12 382 814 19 387 866 20 385 858 21 382 810 23 383 797 24 382 808 26 383 828 27 381 805 28 381 800 30 380 802 31 383 846 32 380 799 33 380 795 34 380 818 35 380 797 36 379 793 37 382 836 38 380 790 39 380 811 40 380 790 41 380 786 43 380 828 44 379 784 45 379 781 46 377 780 48 378 777 49 380 820 50 376 775 53 378 811 54 376 767 55 376 770 56 375 765 59 375 761 60 376 799 61 374 758 63 376 793 64 373 754 66 375 786 67 373 748 69 373 745 70 372 743 72 373 774 73 371 736 75 369 774 76 369 718 77 364 771 78 361 769 79 359 760 80 358 756 81 363 719 82 352 748 83 351 746 84 352 740 85 359 714 87 355 715 88 348 733 89 354 713 91 352 714 92 356 709 93 352 714 94 355 710 95 354 713 96 340 724 97 348 723 98 354 721 99 329 730 100 326 733 103 326 728 104 326 725 105 325 722 106 324 720 108 307 732 109 304 731 110 305 729 111 299 731 112 298 730 113 297 728 114 294 728 115 293 727 116 330 707 117 284 723 119 282 721 [OBJ] 2 \"cyan suv\" 0 120 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 733 628 1 730 628 3 725 631 4 722 631 5 724 631 6 719 634 7 717 635 9 712 638 11 709 639 12 704 640 14 701 643 15 699 644 16 696 645 17 694 647 18 691 648 19 688 648 20 686 650 22 680 652 23 678 653 24 676 655 25 673 656 26 670 657 27 665 658 29 659 661 30 657 664 31 655 665 32 651 666 33 649 667 34 644 669 36 640 672 37 636 673 38 630 676 39 628 677 40 625 678 41 620 681 42 617 682 43 615 683 44 609 686 45 605 689 46 601 690 47 598 691 48 594 694 49 591 695 50 585 698 51 580 700 52 577 701 53 572 704 54 567 706 55 564 707 56 558 710 57 554 713 58 548 715 59 543 718 60 537 721 61 533 723 62 530 724 63 522 728 64 519 729 65 512 733 66 513 730 67 503 736 68 498 740 69 491 744 70 485 746 71 486 743 72 474 751 73 473 749 74 462 757 75 459 759 77 457 763 78 455 765 79 451 767 80 450 768 81 445 772 82 444 773 83 442 774 84 438 777 85 436 778 86 432 782 87 428 784 88 424 786 89 420 790 90 416 793 91 410 799 92 406 801 93 401 805 94 396 813 95 391 817 96 387 821 97 382 824 98 378 825 99 373 828 100 369 829 101 364 831 102 359 814 103 352 856 104 349 859 105 346 864 106 343 868 107 339 869 108 335 871 109 330 872 110 325 873 111 321 879 112 316 880 113 310 882 114 305 884 115 300 889 116 294 890 117 289 894 118 283 896 119 278 899 [OBJ] 3 \"gray fighter\" 9 120 [VIS] 0 false 9 true [IN-FRAME] 0 false 9 true [PX] 0 null null 9 0 308 10 2 307 11 5 307 13 7 305 14 8 302 15 10 299 16 12 296 17 23 296 18 35 296 19 39 295 20 55 295 21 61 296 22 12 304 23 26 305 24 41 305 25 56 304 26 71 304 27 12 301 28 16 295 29 29 294 30 43 294 31 57 294 32 71 294 33 87 294 34 101 294 35 115 294 36 129 294 37 141 291 38 155 290 39 168 289 40 175 280 41 197 291 42 202 280 43 222 288 44 230 280 45 243 280 46 255 279 47 270 282 48 282 280 49 295 281 50 309 283 51 321 282 52 334 283 53 348 284 54 360 285 55 373 285 56 385 286 57 398 288 58 410 290 59 422 290 60 435 292 61 446 292 62 458 292 63 470 293 64 482 293 65 494 294 66 505 294 67 516 295 68 527 295 69 538 295 70 549 296 71 560 296 72 571 297 73 582 298 74 593 297 75 604 298 76 613 298 77 624 298 78 634 299 79 644 299 80 656 301 81 666 301 82 678 303 83 689 304 84 698 304 85 708 304 86 717 304 87 726 304 88 736 304 89 745 305 90 755 304 91 763 305 92 772 305 93 781 305 94 790 305 95 798 305 96 807 305 97 816 305 98 825 305 99 833 305 100 841 306 101 850 306 102 858 306 103 866 306 104 874 306 105 883 306 106 891 306 107 899 306 108 908 307 109 915 307 110 923 307 111 930 307 112 938 307 113 946 307 114 953 307 115 961 307 116 968 307 117 976 307 118 983 307 119 988 309 [OBJS-END] [INTERACTIONS-START] [INTERACT] 2 \"contact\" 0 97 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 0 \"contact\" 0 119 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 8 \"contact\" 0 119 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 1 \"collision\" 75 75 [IN] 1 2 [OUT] 1 2 [INTERACT] 3 \"collision\" 103 103 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 4 \"collision\" 105 105 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 5 \"contact\" 106 109 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 6 \"collision\" 111 111 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 7 \"contact\" 112 119 [IN] 1 65535 [OUT] 1 65535 [INTERACTIONS-END] [SCENE-END]\n",
"parse_debug": {
"pred_parse_valid": false,
"pred_parse_error": "Could not find [SCENE-END] after [SCENE-START].",
"pred_parse_error_type": "ValueError",
"guided_grammar_sha256": "dac80e533cdc467e6ed7e750f131c895bdb25be06b660e6ed92a7366e99f178e",
"guided_grammar_path": "strive_datasets/prep/kubric/scene_repr/profiles/grammars/selector_sentinel_no_3d.lark",
"debug_dir": "/gscratch/krishna/suryadv/strive_dynsuperclevr/runtime/36996080_inference_full/debug/test_full_shard_002/parse_failures/sample_000005",
"guided_grammar_debug_path": "/gscratch/krishna/suryadv/strive_dynsuperclevr/runtime/36996080_inference_full/debug/test_full_shard_002/parse_failures/sample_000005/grammar.ebnf.gz"
}
}Parsed prediction
{
"error": "ValueError: Could not find [SCENE-END] after [SCENE-START]."
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "yellow scooter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 80,
"y": 738
},
{
"t": 106,
"x": 75,
"y": 732
}
],
"birth": 0,
"death": 120
},
{
"id": 1,
"type": "blue mountain",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 383,
"y": 819
},
{
"t": 7,
"x": 386,
"y": 865
},
{
"t": 10,
"x": 383,
"y": 818
},
{
"t": 12,
"x": 382,
"y": 814
},
{
"t": 19,
"x": 387,
"y": 866
},
{
"t": 20,
"x": 385,
"y": 858
},
{
"t": 21,
"x": 382,
"y": 810
},
{
"t": 23,
"x": 383,
"y": 797
},
{
"t": 24,
"x": 382,
"y": 808
},
{
"t": 26,
"x": 383,
"y": 828
},
{
"t": 27,
"x": 381,
"y": 805
},
{
"t": 28,
"x": 381,
"y": 800
},
{
"t": 30,
"x": 380,
"y": 802
},
{
"t": 31,
"x": 383,
"y": 846
},
{
"t": 32,
"x": 380,
"y": 799
},
{
"t": 33,
"x": 380,
"y": 795
},
{
"t": 34,
"x": 380,
"y": 818
},
{
"t": 35,
"x": 380,
"y": 797
},
{
"t": 36,
"x": 379,
"y": 793
},
{
"t": 37,
"x": 382,
"y": 836
},
{
"t": 38,
"x": 380,
"y": 790
},
{
"t": 39,
"x": 380,
"y": 811
},
{
"t": 40,
"x": 380,
"y": 790
},
{
"t": 41,
"x": 380,
"y": 786
},
{
"t": 43,
"x": 380,
"y": 828
},
{
"t": 44,
"x": 379,
"y": 784
},
{
"t": 45,
"x": 379,
"y": 781
},
{
"t": 46,
"x": 377,
"y": 780
},
{
"t": 48,
"x": 378,
"y": 777
},
{
"t": 49,
"x": 380,
"y": 820
},
{
"t": 50,
"x": 376,
"y": 775
},
{
"t": 53,
"x": 378,
"y": 811
},
{
"t": 54,
"x": 376,
"y": 767
},
{
"t": 55,
"x": 376,
"y": 770
},
{
"t": 56,
"x": 375,
"y": 765
},
{
"t": 59,
"x": 375,
"y": 761
},
{
"t": 60,
"x": 376,
"y": 799
},
{
"t": 61,
"x": 374,
"y": 758
},
{
"t": 63,
"x": 376,
"y": 793
},
{
"t": 64,
"x": 373,
"y": 754
},
{
"t": 66,
"x": 375,
"y": 786
},
{
"t": 67,
"x": 373,
"y": 748
},
{
"t": 69,
"x": 373,
"y": 745
},
{
"t": 70,
"x": 372,
"y": 743
},
{
"t": 72,
"x": 373,
"y": 774
},
{
"t": 73,
"x": 371,
"y": 736
},
{
"t": 75,
"x": 369,
"y": 774
},
{
"t": 76,
"x": 369,
"y": 718
},
{
"t": 77,
"x": 364,
"y": 771
},
{
"t": 78,
"x": 361,
"y": 769
},
{
"t": 79,
"x": 359,
"y": 760
},
{
"t": 80,
"x": 358,
"y": 756
},
{
"t": 81,
"x": 363,
"y": 719
},
{
"t": 82,
"x": 352,
"y": 748
},
{
"t": 83,
"x": 351,
"y": 746
},
{
"t": 84,
"x": 352,
"y": 740
},
{
"t": 85,
"x": 359,
"y": 714
},
{
"t": 87,
"x": 355,
"y": 715
},
{
"t": 88,
"x": 348,
"y": 733
},
{
"t": 89,
"x": 354,
"y": 713
},
{
"t": 91,
"x": 352,
"y": 714
},
{
"t": 92,
"x": 356,
"y": 709
},
{
"t": 93,
"x": 352,
"y": 714
},
{
"t": 94,
"x": 355,
"y": 710
},
{
"t": 95,
"x": 354,
"y": 713
},
{
"t": 96,
"x": 340,
"y": 724
},
{
"t": 97,
"x": 348,
"y": 723
},
{
"t": 98,
"x": 354,
"y": 721
},
{
"t": 99,
"x": 329,
"y": 730
},
{
"t": 100,
"x": 326,
"y": 733
},
{
"t": 103,
"x": 326,
"y": 728
},
{
"t": 104,
"x": 326,
"y": 725
},
{
"t": 105,
"x": 325,
"y": 722
},
{
"t": 106,
"x": 324,
"y": 720
},
{
"t": 108,
"x": 307,
"y": 732
},
{
"t": 109,
"x": 304,
"y": 731
},
{
"t": 110,
"x": 305,
"y": 729
},
{
"t": 111,
"x": 299,
"y": 731
},
{
"t": 112,
"x": 298,
"y": 730
},
{
"t": 113,
"x": 297,
"y": 728
},
{
"t": 114,
"x": 294,
"y": 728
},
{
"t": 115,
"x": 293,
"y": 727
},
{
"t": 116,
"x": 330,
"y": 707
},
{
"t": 117,
"x": 284,
"y": 723
},
{
"t": 119,
"x": 282,
"y": 721
}
],
"birth": 0,
"death": 120
},
{
"id": 2,
"type": "cyan suv",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 733,
"y": 628
},
{
"t": 1,
"x": 730,
"y": 628
},
{
"t": 3,
"x": 725,
"y": 631
},
{
"t": 4,
"x": 722,
"y": 631
},
{
"t": 5,
"x": 724,
"y": 631
},
{
"t": 6,
"x": 719,
"y": 634
},
{
"t": 7,
"x": 717,
"y": 635
},
{
"t": 9,
"x": 712,
"y": 638
},
{
"t": 11,
"x": 709,
"y": 639
},
{
"t": 12,
"x": 704,
"y": 640
},
{
"t": 14,
"x": 701,
"y": 643
},
{
"t": 15,
"x": 699,
"y": 644
},
{
"t": 16,
"x": 696,
"y": 645
},
{
"t": 17,
"x": 694,
"y": 647
},
{
"t": 18,
"x": 691,
"y": 648
},
{
"t": 19,
"x": 688,
"y": 648
},
{
"t": 20,
"x": 686,
"y": 650
},
{
"t": 22,
"x": 680,
"y": 652
},
{
"t": 23,
"x": 678,
"y": 653
},
{
"t": 24,
"x": 676,
"y": 655
},
{
"t": 25,
"x": 673,
"y": 656
},
{
"t": 26,
"x": 670,
"y": 657
},
{
"t": 27,
"x": 665,
"y": 658
},
{
"t": 29,
"x": 659,
"y": 661
},
{
"t": 30,
"x": 657,
"y": 664
},
{
"t": 31,
"x": 655,
"y": 665
},
{
"t": 32,
"x": 651,
"y": 666
},
{
"t": 33,
"x": 649,
"y": 667
},
{
"t": 34,
"x": 644,
"y": 669
},
{
"t": 36,
"x": 640,
"y": 672
},
{
"t": 37,
"x": 636,
"y": 673
},
{
"t": 38,
"x": 630,
"y": 676
},
{
"t": 39,
"x": 628,
"y": 677
},
{
"t": 40,
"x": 625,
"y": 678
},
{
"t": 41,
"x": 620,
"y": 681
},
{
"t": 42,
"x": 617,
"y": 682
},
{
"t": 43,
"x": 615,
"y": 683
},
{
"t": 44,
"x": 609,
"y": 686
},
{
"t": 45,
"x": 605,
"y": 689
},
{
"t": 46,
"x": 601,
"y": 690
},
{
"t": 47,
"x": 598,
"y": 691
},
{
"t": 48,
"x": 594,
"y": 694
},
{
"t": 49,
"x": 591,
"y": 695
},
{
"t": 50,
"x": 585,
"y": 698
},
{
"t": 51,
"x": 580,
"y": 700
},
{
"t": 52,
"x": 577,
"y": 701
},
{
"t": 53,
"x": 572,
"y": 704
},
{
"t": 54,
"x": 567,
"y": 706
},
{
"t": 55,
"x": 564,
"y": 707
},
{
"t": 56,
"x": 558,
"y": 710
},
{
"t": 57,
"x": 554,
"y": 713
},
{
"t": 58,
"x": 548,
"y": 715
},
{
"t": 59,
"x": 543,
"y": 718
},
{
"t": 60,
"x": 537,
"y": 721
},
{
"t": 61,
"x": 533,
"y": 723
},
{
"t": 62,
"x": 530,
"y": 724
},
{
"t": 63,
"x": 522,
"y": 728
},
{
"t": 64,
"x": 519,
"y": 729
},
{
"t": 65,
"x": 512,
"y": 733
},
{
"t": 66,
"x": 513,
"y": 730
},
{
"t": 67,
"x": 503,
"y": 736
},
{
"t": 68,
"x": 498,
"y": 740
},
{
"t": 69,
"x": 491,
"y": 744
},
{
"t": 70,
"x": 485,
"y": 746
},
{
"t": 71,
"x": 486,
"y": 743
},
{
"t": 72,
"x": 474,
"y": 751
},
{
"t": 73,
"x": 473,
"y": 749
},
{
"t": 74,
"x": 462,
"y": 757
},
{
"t": 75,
"x": 459,
"y": 759
},
{
"t": 77,
"x": 457,
"y": 763
},
{
"t": 78,
"x": 455,
"y": 765
},
{
"t": 79,
"x": 451,
"y": 767
},
{
"t": 80,
"x": 450,
"y": 768
},
{
"t": 81,
"x": 445,
"y": 772
},
{
"t": 82,
"x": 444,
"y": 773
},
{
"t": 83,
"x": 442,
"y": 774
},
{
"t": 84,
"x": 438,
"y": 777
},
{
"t": 85,
"x": 436,
"y": 778
},
{
"t": 86,
"x": 432,
"y": 782
},
{
"t": 87,
"x": 428,
"y": 784
},
{
"t": 88,
"x": 424,
"y": 786
},
{
"t": 89,
"x": 420,
"y": 790
},
{
"t": 90,
"x": 416,
"y": 793
},
{
"t": 91,
"x": 410,
"y": 799
},
{
"t": 92,
"x": 406,
"y": 801
},
{
"t": 93,
"x": 401,
"y": 805
},
{
"t": 94,
"x": 396,
"y": 813
},
{
"t": 95,
"x": 391,
"y": 817
},
{
"t": 96,
"x": 387,
"y": 821
},
{
"t": 97,
"x": 382,
"y": 824
},
{
"t": 98,
"x": 378,
"y": 825
},
{
"t": 99,
"x": 373,
"y": 828
},
{
"t": 100,
"x": 369,
"y": 829
},
{
"t": 101,
"x": 364,
"y": 831
},
{
"t": 102,
"x": 359,
"y": 814
},
{
"t": 103,
"x": 352,
"y": 856
},
{
"t": 104,
"x": 349,
"y": 859
},
{
"t": 105,
"x": 346,
"y": 864
},
{
"t": 106,
"x": 343,
"y": 868
},
{
"t": 107,
"x": 339,
"y": 869
},
{
"t": 108,
"x": 335,
"y": 871
},
{
"t": 109,
"x": 330,
"y": 872
},
{
"t": 110,
"x": 325,
"y": 873
},
{
"t": 111,
"x": 321,
"y": 879
},
{
"t": 112,
"x": 316,
"y": 880
},
{
"t": 113,
"x": 310,
"y": 882
},
{
"t": 114,
"x": 305,
"y": 884
},
{
"t": 115,
"x": 300,
"y": 889
},
{
"t": 116,
"x": 294,
"y": 890
},
{
"t": 117,
"x": 289,
"y": 894
},
{
"t": 118,
"x": 283,
"y": 896
},
{
"t": 119,
"x": 278,
"y": 899
}
],
"birth": 0,
"death": 120
},
{
"id": 3,
"type": "gray fighter",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 9,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 9,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 9,
"x": 0,
"y": 308
},
{
"t": 10,
"x": 2,
"y": 307
},
{
"t": 11,
"x": 5,
"y": 307
},
{
"t": 13,
"x": 7,
"y": 305
},
{
"t": 14,
"x": 8,
"y": 302
},
{
"t": 15,
"x": 10,
"y": 299
},
{
"t": 16,
"x": 12,
"y": 296
},
{
"t": 17,
"x": 23,
"y": 296
},
{
"t": 18,
"x": 35,
"y": 296
},
{
"t": 19,
"x": 39,
"y": 295
},
{
"t": 20,
"x": 55,
"y": 295
},
{
"t": 21,
"x": 61,
"y": 296
},
{
"t": 22,
"x": 12,
"y": 304
},
{
"t": 23,
"x": 26,
"y": 305
},
{
"t": 24,
"x": 41,
"y": 305
},
{
"t": 25,
"x": 56,
"y": 304
},
{
"t": 26,
"x": 71,
"y": 304
},
{
"t": 27,
"x": 12,
"y": 301
},
{
"t": 28,
"x": 16,
"y": 295
},
{
"t": 29,
"x": 29,
"y": 294
},
{
"t": 30,
"x": 43,
"y": 294
},
{
"t": 31,
"x": 57,
"y": 294
},
{
"t": 32,
"x": 71,
"y": 294
},
{
"t": 33,
"x": 87,
"y": 294
},
{
"t": 34,
"x": 101,
"y": 294
},
{
"t": 35,
"x": 115,
"y": 294
},
{
"t": 36,
"x": 129,
"y": 294
},
{
"t": 37,
"x": 141,
"y": 291
},
{
"t": 38,
"x": 155,
"y": 290
},
{
"t": 39,
"x": 168,
"y": 289
},
{
"t": 40,
"x": 175,
"y": 280
},
{
"t": 41,
"x": 197,
"y": 291
},
{
"t": 42,
"x": 202,
"y": 280
},
{
"t": 43,
"x": 222,
"y": 288
},
{
"t": 44,
"x": 230,
"y": 280
},
{
"t": 45,
"x": 243,
"y": 280
},
{
"t": 46,
"x": 255,
"y": 279
},
{
"t": 47,
"x": 270,
"y": 282
},
{
"t": 48,
"x": 282,
"y": 280
},
{
"t": 49,
"x": 295,
"y": 281
},
{
"t": 50,
"x": 309,
"y": 283
},
{
"t": 51,
"x": 321,
"y": 282
},
{
"t": 52,
"x": 334,
"y": 283
},
{
"t": 53,
"x": 348,
"y": 284
},
{
"t": 54,
"x": 360,
"y": 285
},
{
"t": 55,
"x": 373,
"y": 285
},
{
"t": 56,
"x": 385,
"y": 286
},
{
"t": 57,
"x": 398,
"y": 288
},
{
"t": 58,
"x": 410,
"y": 290
},
{
"t": 59,
"x": 422,
"y": 290
},
{
"t": 60,
"x": 435,
"y": 292
},
{
"t": 61,
"x": 446,
"y": 292
},
{
"t": 62,
"x": 458,
"y": 292
},
{
"t": 63,
"x": 470,
"y": 293
},
{
"t": 64,
"x": 482,
"y": 293
},
{
"t": 65,
"x": 494,
"y": 294
},
{
"t": 66,
"x": 505,
"y": 294
},
{
"t": 67,
"x": 516,
"y": 295
},
{
"t": 68,
"x": 527,
"y": 295
},
{
"t": 69,
"x": 538,
"y": 295
},
{
"t": 70,
"x": 549,
"y": 296
},
{
"t": 71,
"x": 560,
"y": 296
},
{
"t": 72,
"x": 571,
"y": 297
},
{
"t": 73,
"x": 582,
"y": 298
},
{
"t": 74,
"x": 593,
"y": 297
},
{
"t": 75,
"x": 604,
"y": 298
},
{
"t": 76,
"x": 613,
"y": 298
},
{
"t": 77,
"x": 624,
"y": 298
},
{
"t": 78,
"x": 634,
"y": 299
},
{
"t": 79,
"x": 644,
"y": 299
},
{
"t": 80,
"x": 656,
"y": 301
},
{
"t": 81,
"x": 666,
"y": 301
},
{
"t": 82,
"x": 678,
"y": 303
},
{
"t": 83,
"x": 689,
"y": 304
},
{
"t": 84,
"x": 698,
"y": 304
},
{
"t": 85,
"x": 708,
"y": 304
},
{
"t": 86,
"x": 717,
"y": 304
},
{
"t": 87,
"x": 726,
"y": 304
},
{
"t": 88,
"x": 736,
"y": 304
},
{
"t": 89,
"x": 745,
"y": 305
},
{
"t": 90,
"x": 755,
"y": 304
},
{
"t": 91,
"x": 763,
"y": 305
},
{
"t": 92,
"x": 772,
"y": 305
},
{
"t": 93,
"x": 781,
"y": 305
},
{
"t": 94,
"x": 790,
"y": 305
},
{
"t": 95,
"x": 798,
"y": 305
},
{
"t": 96,
"x": 807,
"y": 305
},
{
"t": 97,
"x": 816,
"y": 305
},
{
"t": 98,
"x": 825,
"y": 305
},
{
"t": 99,
"x": 833,
"y": 305
},
{
"t": 100,
"x": 841,
"y": 306
},
{
"t": 101,
"x": 850,
"y": 306
},
{
"t": 102,
"x": 858,
"y": 306
},
{
"t": 103,
"x": 866,
"y": 306
},
{
"t": 104,
"x": 874,
"y": 306
},
{
"t": 105,
"x": 883,
"y": 306
},
{
"t": 106,
"x": 891,
"y": 306
},
{
"t": 107,
"x": 899,
"y": 306
},
{
"t": 108,
"x": 908,
"y": 307
},
{
"t": 109,
"x": 915,
"y": 307
},
{
"t": 110,
"x": 923,
"y": 307
},
{
"t": 111,
"x": 930,
"y": 307
},
{
"t": 112,
"x": 938,
"y": 307
},
{
"t": 113,
"x": 946,
"y": 307
},
{
"t": 114,
"x": 953,
"y": 307
},
{
"t": 115,
"x": 961,
"y": 307
},
{
"t": 116,
"x": 968,
"y": 307
},
{
"t": 117,
"x": 976,
"y": 307
},
{
"t": 118,
"x": 983,
"y": 307
},
{
"t": 119,
"x": 988,
"y": 309
}
],
"birth": 9,
"death": 120
}
],
"interactions": [
{
"id": 2,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 97,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 8,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 119,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
2
],
"outputs": [
1,
2
],
"begins_at": 75,
"ends_at": 75,
"type": "collision"
},
{
"id": 3,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 103,
"ends_at": 103,
"type": "collision"
},
{
"id": 4,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 105,
"ends_at": 105,
"type": "collision"
},
{
"id": 5,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 106,
"ends_at": 109,
"type": "contact"
},
{
"id": 6,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 111,
"ends_at": 111,
"type": "collision"
},
{
"id": 7,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 112,
"ends_at": 119,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": null,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": null,
"avg_image_position_segmentation_acc": null,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": null,
"avg_image_position_segmentation_acc_large": null,
"avg_image_position_segmentation_acc_first_frame": null,
"avg_image_position_segmentation_visible_acc": null,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": null,
"visibility_true_acc": null,
"visibility_false_acc": null,
"is_in_frame_acc": null,
"is_in_frame_true_acc": null,
"is_in_frame_false_acc": null,
"vis_acc_from_px": null,
"vis_acc_from_px_true_acc": null,
"vis_acc_from_px_false_acc": null,
"num_matches": 0,
"num_unmatched_pred": 0,
"num_unmatched_target": 4
},
"supports": {
"avg_type_similarity": 0,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 0,
"avg_image_position_segmentation_acc": 0,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 0,
"avg_image_position_segmentation_acc_large": 0,
"avg_image_position_segmentation_acc_first_frame": 0,
"avg_image_position_segmentation_visible_acc": 0,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 0,
"visibility_true_acc": 0,
"visibility_false_acc": 0,
"is_in_frame_acc": 0,
"is_in_frame_true_acc": 0,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 0,
"vis_acc_from_px_true_acc": 0,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [],
"unmatched_target_ids": [
0,
1,
2,
3
],
"unmatched_pred_ids": [],
"target_type_counts": {
"yellow scooter": 1,
"blue mountain": 1,
"cyan suv": 1,
"gray fighter": 1
},
"pred_type_counts": {},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 120
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"2.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"3.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"jaccard_by_threshold": {
"0.5": 0.0,
"1.0": 0.0,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"dynamic": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {},
"segmentation_precision_by_threshold": {},
"segmentation_recall_by_threshold": {},
"segmentation_f1_by_threshold": {},
"segmentation_frame_average_precision": null,
"segmentation_frame_average_recall": null,
"segmentation_frame_average_f1": null,
"segmentation_first_frame_counts": {},
"segmentation_first_frame_precision": null,
"segmentation_first_frame_recall": null,
"segmentation_first_frame_f1": null,
"segmentation_by_motion_group": {}
},
"per_match_metrics": [],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 9,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 9
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"interaction_matching": {
"matches": [],
"unmatched_target_ids": [
0,
1,
2,
3,
4,
5,
6,
7,
8
],
"unmatched_pred_ids": [],
"target_type_counts": {
"contact": 5,
"collision": 4
},
"pred_type_counts": {},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 8,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 8
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 1,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 4,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 5,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"contact": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}