STRIVE · 08/28/2026
Dyn generalization · test_24 · super_clevr_1153
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow scooter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 80 738 [OBJ] 1 \"blue mountain\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 383 819 7 386 865 10 383 818 12 382 814 19 387 866 20 385 858 21 382 810 23 383 797 [OBJ] 2 \"cyan suv\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 733 628 1 730 628 3 725 631 4 722 631 5 724 631 6 719 634 7 717 635 9 712 638 11 709 639 12 704 640 14 701 643 15 699 644 16 696 645 17 694 647 18 691 648 19 688 648 20 686 650 22 680 652 23 678 653 [OBJ] 3 \"gray fighter\" 9 24 [VIS] 0 false 9 true [IN-FRAME] 0 false 9 true [PX] 0 null null 9 0 308 10 2 307 11 5 307 13 7 305 14 8 302 15 10 299 16 12 296 17 23 296 18 35 296 19 39 295 20 55 295 21 61 296 22 12 304 23 26 305 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_24/super_clevr_1153/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow cardboard Raisinets box\" 0 10 [VIS] 0 true 3 false 8 true 9 false 10 true [IN-FRAME] 0 true [PX] 0 1 771 3 null null 8 1 771 9 null null 10 1 771 [OBJ] 1 \"yellow plastic candy bar\" 0 10 [VIS] 0 true 3 false 10 true [IN-FRAME] 0 true [PX] 0 13 755 3 null null 10 1 775 11 1 759 12 5 771 13 5 759 14 9 767 15 13 775 16 13 763 17 17 771 18 21 779 19 21 767 20 25 775 21 29 783 22 29 771 23 33 779 [OBJ] 2 \"yellow plastic candy bar\" 0 10 [VIS] 0 true 3 false 9 true [IN-FRAME] 0 true [PX] 0 17 728 3 null null 9 9 728 10 13 736 11 13 724 12 17 732 13 17 720 14 21 728 15 21 716 16 25 724 17 25 712 18 29 720 19 29 708 20 33 716 21 33 705 22 37 712 23 37 697 [OBJ] 3 \"yellow plastic candy bar\" 0 10 [VIS] 0 true 3 false 8 true [IN-FRAME] 0 true [PX] 0 17 822 3 null null 8 9 814 9 13 822 11 13 810 12 17 818 14 17 806 15 21 814 17 21 802 18 25 810 20 25 798 21 29 806 23 29 794 [OBJ] 4 \"yellow plastic candy bar\" 0 10 [VIS] 0 true 3 false 8 true [IN-FRAME] 0 true [PX] 0 25 767 3 null null 8 13 767 9 21 763 13 33 763 14 25 759 16 37 759 17 29 755 19 41 755 20 33 751 22 44 751 23 37 748 [OBJ] 5 \"yellow plastic candy bar\" 0 10 [VIS] 0 true 3 false 8 true [IN-FRAME] 0 true [PX] 0 25 794 3 null null 8 21 787 9 25 794 14 29 787 17 37 783 21 41 775 22 48 779 23 41 771 [OBJ] 6 \"yellow plastic candy bar\" 0 10 [VIS] 0 true 3 false 8 true [IN-FRAME] 0 true [PX] 0 33 787 3 null null 8 29 775 9 33 783 14 41 779 17 48 775 20 52 767 21 56 775 22 52 763 23 56 771 [OBJ] 7 \"yellow cardboard Raisinets box\" 0 10 [VIS] 0 true 3 false 7 true [IN-FRAME] 0 true [PX] 0 37 732 3 null null 7 25 728 8 33 724 13 41 720 16 48 716 20 56 712 22 64 708 [OBJ] 8 \"yellow cardboard curl cream box\" 0 10 [VIS] 0 true 3 false 8 true [IN-FRAME] 0 true [PX] 0 41 791 3 null null 8 41 787 9 48 791 13 56 787 17 64 783 21 76 783 22 68 775 23 76 779 [OBJ] 9 \"yellow plastic candy bar\" 0 10 [VIS] 0 true 3 false 7 true [IN-FRAME] 0 true [PX] 0 44 767 3 null null 7 41 755 8 44 763 13 52 759 17 60 755 21 68 751 22 60 744 23 68 748 [OBJ] 10 \"yellow wooden toy construction vehicle set\" 0 10 [VIS] 0 true 3 false 7 true [IN-FRAME] 0 true [PX] 0 48 771 3 null null 7 48 763 10 56 759 14 64 755 18 72 751 22 80 748 [OBJ] 11 \"yellow wooden toy construction vehicle set\" 0 10 [VIS] 0 true 3 false 8 true [IN-FRAME] 0 true [PX] 0 52 775 3 null null 8 48 767 9 52 775 11 56 767 15 64 763 19 72 759 23 80 755 [OBJ] 12 \"yellow cardboard Raisinets box\" 0 10 [VIS] 0 true 3 false 8 true [IN-FRAME] 0 true [PX] 0 52 787 3 null null 8 52 779 9 56 787 13 60 779 17 68 775 21 76 771 22 68 767 23 76 771 [OBJ] 13 \"yellow cardboard Raisinets box\" 0 10 [VIS] 0 true 3 false 8 true [IN-FRAME] 0 true [PX] 0 60 751 3 null null 8 60 744 11 68 740 15 76 736 19 83 732 23 91 728 [OBJ] 14 \"yellow and blue plastic soccer cleat\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 396 876 1 396 865 4 396 876 5 396 865 9 396 853 10 396 865 11 396 853 15 396 841 16 400 853 17 396 841 20 400 849 21 396 837 22 400 845 23 396 833 [OBJ] 15 \"blue and black plastic DVD set\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 404 802 2 408 794 7 408 783 10 412 775 13 412 787 14 412 775 17 412 763 19 416 771 20 416 759 22 416 748 23 419 755 [OBJ] 16 \"blue and orange plastic Therizinosaurus figurine\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 419 853 4 419 841 7 419 830 10 419 818 13 423 810 16 423 798 19 423 787 22 427 779 23 423 771 [OBJ] 17 \"blue and orange plastic Therizinosaurus figurine\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 427 873 3 427 861 7 427 849 10 427 837 13 427 826 16 431 818 19 431 806 22 431 794 [OBJ] 18 \"black plastic USB flash drive\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 427 884 5 427 873 9 427 861 12 427 849 15 431 841 19 431 830 22 431 818 23 435 826 [OBJ] 19 \"blue and orange plastic Therizinosaurus figurine\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 681 669 3 685 662 4 677 666 5 681 658 7 689 654 10 689 642 11 681 646 12 685 638 13 681 646 14 685 638 15 681 646 17 685 634 18 677 638 19 681 630 20 685 623 21 681 630 22 685 623 23 681 630 [OBJ] 20 \"teal and lime running shoe\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 708 630 3 null null [OBJ] 21 \"blue and black hiking shoe\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 720 642 3 null null [OBJ] 22 \"teal and lime running shoe\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 720 654 3 null null [OBJ] 23 \"teal fabric mermaid backpack\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 724 666 3 null null [OBJ] 24 \"grey and black basketball shoe\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 732 642 3 null null [OBJ] 25 \"teal and lime running shoe\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 751 630 1 744 634 4 740 642 7 728 646 10 720 650 13 712 654 16 705 658 18 701 666 19 693 662 22 685 666 23 681 658 [OBJ] 26 \"black and grey athletic shoe\" 0 10 [VIS] 0 true 3 false [IN-FRAME] 0 true [PX] 0 779 662 3 null null [OBJ] 27 \"white and grey cardboard card and invitation kit\" 4 10 [VIS] 0 false 4 true [IN-FRAME] 0 false 4 true [PX] 0 null null 4 1 314 5 5 322 7 13 318 9 21 314 10 29 318 11 41 318 12 56 318 13 83 318 14 107 318 15 126 318 16 146 318 17 166 318 18 185 318 19 205 318 20 220 318 21 240 318 22 255 318 23 271 318 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 10 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 10 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 20 \"contact\" 0 10 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 21 \"contact\" 0 10 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 22 \"contact\" 0 10 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 23 \"contact\" 0 10 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 24 \"contact\" 0 10 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 25 \"contact\" 0 10 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 26 \"contact\" 0 10 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 27 \"contact\" 0 10 [IN] 9 65535 [OUT] 9 65535 [INTERACT] 2 \"contact\" 0 10 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 3 \"contact\" 0 10 [IN] 11 65535 [OUT] 11 65535 [INTERACT] 4 \"contact\" 0 10 [IN] 12 65535 [OUT] 12 65535 [INTERACT] 5 \"contact\" 0 10 [IN] 13 65535 [OUT] 13 65535 [INTERACT] 6 \"contact\" 0 10 [IN] 14 15 [OUT] 14 15 [INTERACT] 7 \"contact\" 0 10 [IN] 14 65535 [OUT] 14 65535 [INTERACT] 8 \"contact\" 0 10 [IN] 15 16 [OUT] 15 16 [INTERACT] 9 \"contact\" 0 10 [IN] 15 65535 [OUT] 15 65535 [INTERACT] 10 \"contact\" 0 10 [IN] 16 65535 [OUT] 16 65535 [INTERACT] 11 \"contact\" 0 10 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 12 \"contact\" 0 10 [IN] 18 65535 [OUT] 18 65535 [INTERACT] 13 \"contact\" 0 10 [IN] 19 65535 [OUT] 19 65535 [INTERACT] 14 \"contact\" 0 10 [IN] 20 65535 [OUT] 20 65535 [INTERACT] 15 \"contact\" 0 10 [IN] 21 65535 [OUT] 21 65535 [INTERACT] 16 \"contact\" 0 10 [IN] 22 65535 [OUT] 22 65535 [INTERACT] 17 \"contact\" 0 10 [IN] 23 65535 [OUT] 23 65535 [INTERACT] 18 \"contact\" 0 10 [IN] 24 65535 [OUT] 24 65535 [INTERACT] 19 \"contact\" 0 10 [IN] 25 65535 [OUT] 25 65535 [INTERACT] 28 \"collision\" 10 10 [IN] 26 65535 [OUT] 26 65535 [INTERACT] 29 \"contact\" 10 11 [IN] 26 65535 [OUT] 26 65535 [INTERACT] 12 \"collision\" 12 12 [IN] 19 25 [OUT] 19 25 [INTERACT] 13 \"contact\" 13 14 [IN] 19 25 [OUT] 19 25 [INTERACT] 30 \"collision\" 15 15 [IN] 27 65535 [OUT] 27 65535 [INTERACT] 31 \"contact\" 16 19 [IN] 27 65535 [OUT] 27 65535 [INTERACT] 32 \"collision\" 19 19 [IN] 29 65535 [OUT] 29 65535 [INTERACT] 33 \"contact\" 20 21 [IN] 29 65535 [OUT] 29 65535 [INTERACT] 34 \"collision\" 21 21 [IN] 30 65535 [OUT] 30 65535 [INTERACT] 35 \"collision\" 23 23 [IN] 30 65535 [OUT] 30 65535 [INTERACT] 36 \"contact\" 24 25 [IN] 30 65535 [OUT] 30 65535 [INTERACT] 37 \"collision\" 25 25 [IN] 31 65535 [OUT] 31 65535 [INTERACT] 38 \"contact\" 26 27 [IN] 31 65535 [OUT] 31 65535 [INTERACT] 39 \"collision\" 29 29 [IN] 32 65535 [OUT] 32 65535 [INTERACT] 40 \"contact\" 30 210 [IN] 32 65535 [OUT] 32 65535 [INTERACT] 41 \"collision\" 44 44 [IN] 34 65535 [OUT] 34 65535 [INTERACT] 42 \"collision\" 47 47 [IN] 34 65535 [OUT] 34 65535 [INTERACT] 43 \"contact\" 48 52 [IN] 34 65535 [OUT] 34 65535 [INTERACT] 44 \"collision\" 54 54 [IN] 35 65535 [OUT] 35 65535 [INTERACT] 45 \"collision\" 56 56 [IN] 36 65535 [OUT] 36 65535 [INTERACT] 46 \"contact\" 57 60 [IN] 36 65535 [OUT] 36 65535 [INTERACT] 47 \"collision\" 59 59 [IN] 37 65535 [OUT] 37 65535 [INTERACT] 48 \"contact\" 60 61 [IN] 37 65535 [OUT] 37 65535 [INTERACT] 49 \"collision\" 63 63 [IN] 38 65535 [OUT] 38 65535 [INTERACT] 50 \"contact\" 64 65 [IN] 38 65535 [OUT] 38 65535 [INTERACT] 51 \"collision\" 67 67 [IN] 40 65535 [OUT] 40 65535 [INTERACT] 52 \"contact\" 68 72 [IN] 40 65535 [OUT] 40 65535 [INTERACT] 53 \"collision\" 74 74 [IN] 41 65535 [OUT] 41 65535 [INTERACT] 54 \"contact\" 75 83 [IN] 41 65535 [OUT] 41 65535 [INTERACT] 55 \"collision\" 83 83 [IN] 42 65535 [OUT] 42 65535 [INTERACT] 56 \"contact\" 84 88 [IN] 42 65535 [OUT] 42 65535 [INTERACT] 57 \"collision\" 90 90 [IN] 43 65535 [OUT] 43 65535 [INTERACT] 58 \"contact\" 91 99 [IN] 43 65535 [OUT] 43 65535 [INTERACT] 0 \"collision\" 97 97 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 3 \"collision\" 97 97 [IN] 1 7 [OUT] 1 7 [INTERACT] 59 \"collision\" 97 97 [IN] 45 65535 [OUT] 45 65535 [INTERACT] 60 \"contact\" 98 102 [IN] 45 65535 [OUT] 45 65535 [INTERACT] 4 \"collision\" 99 99 [IN] 1 7 [OUT] 1 7 [INTERACT] 61 \"collision\" 99 99 [IN] 46 65535 [OUT] 46 65535 [INTERACT] 62 \"contact\" 99 102 [IN] 46 65535 [OUT] 46 65535 [INTERACT] 63 \"collision\" 100 101 [IN] 48 65535 [OUT] 48 65535 [INTERACT] 64 \"collision\" 102 102 [IN] 49 65535 [OUT] 49 65535 [INTERACT] 65 \"contact\" 103 104 [IN] 49 65535 [OUT] 49 65535 [INTERACT] 66 \"collision\" 106 106 [IN] 50 65535 [OUT] 50 65535 [INTERACT] 67 \"contact\" 107 108 [IN] 50 65535 [OUT] 50 65535 [INTERACT] 68 \"collision\" 110 111 [IN] 51 65535 [OUT] 51 65535 [INTERACT] 69 \"contact\" 111 114 [IN] 51 65535 [OUT] 51 65535 [INTERACT] 6 \"collision\" 113 113 [IN] 1 7 [OUT] 1 7 [INTERACT] 70 \"collision\" 113 113 [IN] 52 65535 [OUT] 52 65535 [INTERACT] 71 \"contact\" 114 118 [IN] 52 65535 [OUT] 52 65535 [INTERACT] 7 \"contact\" 114 119 [IN] 1 7 [OUT] 1 7 [INTERACT] 68 \"collision\" 115 119 [IN] 51 65535 [OUT] 51 65535 [INTERACT] 72 \"collision\" 119 119 [IN] 53 65535 [OUT] 53 65535 [INTERACT] 73 \"contact\" 120 123 [IN] 53 65535 [OUT] 53 65535 [INTERACT] 74 \"collision\" 123 124 [IN] 54 65535 [OUT] 54 65535 [INTERACT] 75 \"contact\" 124 128 [IN] 54 65535 [OUT] 54 65535 [INTERACT] 76 \"collision\" 128 129 [IN] 55 65535 [OUT] 55 65535 [INTERACT] 77 \"contact\" 129 133 [IN] 55 65535 [OUT] 55 65535 [INTERACT] 78 \"collision\" 132 133 [IN] 56 65535 [OUT] 56 65535 [INTERACT] 79 \"contact\" 133 136 [IN] 56 65535 [OUT] 56 65535 [INTERACT] 7 \"collision\" 141 141 [IN] 1 7 [OUT] 1 7 [INTERACT] 80 \"collision\" 145 146 [IN] 57 65535 [OUT] 57 65535 [INTERACT] 81 \"collision\" 152 153 [IN] 58 65535 [OUT] 58 65535 [INTERACT] 82 \"collision\" 156 157 [IN] 59 65535 [OUT] 59 65535 [INTERACT] 83 \"collision\" 162 162 [IN] 60 65535 [OUT] 60 65535 [INTERACT] 84 \"collision\" 166 166 [IN] 61 65535 [OUT] 61 65535 [INTERACT] 85 \"collision\" 169 166 [IN] 62 65535 [OUT] 62 65535 [INTERACT] 86 \"contact\" 170 185 [IN] 62 65535 [OUT] 62 65535 [INTERACT] 87 \"collision\" 181 181 [IN] 63 65535 [OUT] 63 65535 [INTERACT] 88 \"contact\" 182 189 [IN] 63 65535 [OUT] 63 65535 [INTERACT] 89 \"collision\" 189 193 [IN] 64 65535 [OUT] 64 65535 [INTERACT] 90 \"collision\" 193 197 [IN] 65 65535 [OUT] 65 65535 [INTERACT] 91 \"collision\" 201 201 [IN] 66 65535 [OUT] 66 65535 [INTERACT] 92 \"collision\" 205 205 [IN] 67 65535 [OUT] 67 65535 [INTERACT] 93 \"collision\" 212 212 [IN] 68 65535 [OUT] 68 65535 [INTERACT] 94 \"collision\" 216 216 [IN] 69 65535 [OUT] 69 65535 [INTERACT] 95 \"collision\" 224 224 [IN] 70 65535 [OUT] 70 65535 [INTERACT] 96 \"collision\" 232 232 [IN] 71 65535 [OUT] 71 65535 [INTERACT] 97 \"collision\" 236 236 [IN] 72 65535 [OUT] 72 65535 [INTERACT] 98 \"collision\" 244 244 [IN] 73 65535 [OUT] 73 65535 [INTERACT] 99 \"collision\" 251 251 [IN] 74 65535 [OUT] 74 65535 [INTERACT] 100 \"collision\" 259 259 [IN] 75 65535 [OUT] 75 65535 [INTERACT] 101 \"collision\" 267 267 [IN] 76 65535 [OUT] 76 65535 [INTERACT] 102 \"collision\" 275 275 [IN] 77 65535 [OUT] 77 65535 [INTERACT] 103 \"collision\" 283 283 [IN] 78 65535 [OUT] 78 65535 [INTERACT] 104 \"collision\" 291 291 [IN] 79 65535 [OUT] 79 65535 [INTERACT] 105 \"collision\" 298 294 [IN] 80 65535 [OUT] 80 65535 [INTERACT] 106 \"collision\" 306 298 [IN] 81 65535 [OUT] 81 65535 [INTERACT] 107 \"collision\" 314 302 [IN] 82 65535 [OUT] 82 65535 [INTERACT] 108 \"collision\" 322 306 [IN] 83 65535 [OUT] 83 65535 [INTERACT] 109 \"collision\" 330 310 [IN] 84 65535 [OUT] 84 65535 [INTERACT] 110 \"collision\" 337 314 [IN] 85 65535 [OUT] 85 65535 [INTERACT] 111 \"collision\" 345 318 [IN] 86 65535 [OUT] 86 65535 [INTERACT] 112 \"collision\" 353 322 [IN] 87 65535 [OUT] 87 65535 [INTERACT] 113 \"collision\" 361 326 [IN] 88 65535 [OUT] 88 65535 [INTERACT] 114 \"collision\" 369 330 [IN] 89 65535 [OUT] 89 65535 [INTERACT] 115 \"collision\" 376 333 [IN] 90 65535 [OUT] 90 65535 [INTERACT] 116 \"collision\" 384 337 [IN] 91 65535 [OUT] 91 65535 [INTERACT] 117 \"collision\" 392 341 [IN] 92 65535 [OUT] 92 65535 [INTERACT] 118 \"collision\" 396 349 [IN] 93 65535 [OUT] 93 65535 [INTERACT] 119 \"collision\" 404 353 [IN] 94 65535 [OUT] 94 65535 [INTERACT] 120 \"collision\" 412 357 [IN] 95 65535 [OUT] 95 65535 [INTERACT] 121 \"collision\" 419 361 [IN] 96 65535 [OUT] 96 65535 [INTERACT] 122 \"collision\" 427 365 [IN] 97 65535 [OUT] 97 65535 [INTERACT] 123 \"collision\" 435 369 [IN] 98 65535 [OUT] 98 65535 [INTERACT] 124 \"collision\" 443 373 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 5 \"collision\" 98 99 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 0 \"collision\" 99 99 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 100 102 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 125 \"collision\" 106 107 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 2 \"collision\" 107 108 [IN] 1 7 [OUT] 1 7 [INTERACT] 13 \"collision\" 114 115 [IN] 19 25 [OUT] 19 25 [INTERACT] 130 \"collision\" 120 121 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 14 \"contact\" 119 130 [IN] 19 25 [OUT] 19 25 [INTERACT] 131 \"collision\" 127 129 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 132 \"collision\" 134 136 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 133 \"collision\" 142 146 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 134 \"collision\" 149 150 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 135 \"collision\" 156 158 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 136 \"collision\" 162 162 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 126 \"collision\" 122 123 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 137 \"collision\" 169 173 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 138 \"collision\" 177 177 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 139 \"collision\" 185 189 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 140 \"collision\" 193 193 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 141 \"collision\" 197 197 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 142 \"collision\" 205 197 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 143 \"collision\" 212 201 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 144 \"collision\" 220 205 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 145 \"collision\" 228 208 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 146 \"collision\" 236 212 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 147 \"collision\" 244 216 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 148 \"collision\" 251 220 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 149 \"collision\" 259 224 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 150 \"collision\" 267 228 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 151 \"collision\" 275 232 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 152 \"collision\" 283 236 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 153 \"collision\" 291 240 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 154 \"collision\" 298 244 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 155 \"collision\" 306 248 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 156 \"collision\" 314 251 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 157 \"collision\" 322 255 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 158 \"collision\" 330 259 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 159 \"collision\" 337 263 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 160 \"collision\" 345 267 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 161 \"collision\" 353 271 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 162 \"collision\" 361 275 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 163 \"collision\" 369 279 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 164 \"collision\" 376 283 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 165 \"collision\" 384 287 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 166 \"collision\" 392 291 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 167 \"collision\" 400 294 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 168 \"collision\" 408 298 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 169 \"collision\" 416 302 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 170 \"collision\" 423 306 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 171 \"collision\" 431 310 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 172 \"collision\" 439 314 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 173 \"collision\" 447 318 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 174 \"collision\" 455 322 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 175 \"collision\" 462 326 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 176 \"collision\" 470 330 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 177 \"collision\" 478 333 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 178 \"collision\" 486 337 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 179 \"collision\" 494 341 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 180 \"collision\" 501 345 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 181 \"collision\" 509 349 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 182 \"collision\" 517 353 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 183 \"collision\" 525 357 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 184 \"collision\" 533 361 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 185 \"collision\" 541 365 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 186 \"collision\" 548 369 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 187 \"collision\" 556 373 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 188 \"collision\" 564 376 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 189 \"collision\" 572 380 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 190 \"collision\" 580 384 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 191 \"collision\" 587 388 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 192 \"collision\" 595 392 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 193 \"collision\" 603 396 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 194 \"collision\" 611 392 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 195 \"collision\" 619 400 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 196 \"collision\" 626 396 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 197 \"collision\" 634 404 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 198 \"collision\" 642 408 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 12 \"collision\" 646 650 [IN] 19 25 [OUT] 19 25 [INTERACT] 199 \"collision\" 654 384 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 200 \"collision\" 662 392 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 201 \"collision\" 669 396 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 202 \"collision\" 677 404 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 203 \"collision\" 685 408 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 204 \"collision\" 693 412 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 205 \"collision\" 701 416 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 206 \"collision\" 708 419 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 207 \"collision\" 716 423 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 208 \"collision\" 724 427 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 209 \"collision\" 732 431 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 210 \"collision\" 740 435 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 211 \"collision\" 748 439 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 212 \"collision\" 755 443 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 213 \"collision\" 763 447 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 214 \"collision\" 771 451 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 215 \"collision\" 779 455 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 216 \"collision\" 787 458 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 217 \"collision\" 794 462 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 218 \"collision\" 802 466 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 219 \"collision\" 810 470 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 220 \"collision\" 818 474 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 221 \"collision\" 826 478 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 222 \"collision\" 833 482 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 223 \"collision\" 841 486 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 224 \"collision\" 849 490 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 225 \"collision\" 857 494 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 226 \"collision\" 865 498 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 227 \"collision\" 873 501 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 228 \"collision\" 880 505 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 229 \"collision\" 888 509 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 230 \"collision\" 896 513 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 231 \"collision\" 904 517 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 232 \"collision\" 912 521 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 233 \"collision\" 919 525 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 234 \"collision\" 927 529 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 235 \"collision\" 935 533 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 236 \"collision\" 943 537 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 237 \"collision\" 951 541 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 238 \"collision\" 958 544 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 240 \"collision\" 974 548 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 241 \"collision\" 982 552 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 242 \"collision\" 986 556 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 243 \"collision\" 994 560 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 3 \"collision\" 998 998 [IN] 1 7 [OUT] 1 7 [INTERACT] 56 \"collision\" 998 998 [IN] 42 65535 [OUT] 42 65535 [INTERACT] 1 \"collision\" 998 998 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 4 \"collision\" 998 998 [IN] 10 65535 [OUT] 10 65535 [INTERACT] 48 \"collision\" 998 998 [IN] 37 65535 [OUT] 37 65535 [INTERACT] 50 \"collision\" 998 998 [IN] 38 65535 [OUT] 38 65535 [INTERACT] 51 \"collision\" 998 998 [IN] 40 65535 [OUT] 40 65535 [INTERACT] 52 \"collision\" 998 998 [IN] 41 65535 [OUT] 41 65535 [INTERACT] 53 \"collision\" 998 998 [IN] 43 65535 [OUT] 43 65535 [INTERACT] 54 \"collision\" 998 998 [IN] 45 65535 [OUT] 45 65535 [INTERACT] 59 \"collision\" 998 998 [IN] 45 65535 [OUT] 45 65535 [INTERACT] 11 \"collision\" 998 998 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 12 \"contact\" 0 6 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 60 \"collision\" 1 2 [IN] 46 65535 [OUT] 46 65535 [INTERACT] 49 \"collision\" 1 2 [IN] 37 65535 [OUT] 37 65535 [INTERACT] 65 \"collision\" 2 2 [IN] 49 65535 [OUT] 49 65535 [INTERACT] 66 \"contact\" 3 10 [IN] 49 65535 [OUT] 49 65535 [INTERACT] 61 \"collision\" 4 5 [IN] 46 65535 [OUT] 46 65535 [INTERACT] 123 \"collision\" 6 6 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 62 \"collision\" 7 8 [IN] 46 65535 [OUT] 46 65535 [INTERACT] 124 \"collision\" 8 10 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 63 \"collision\" 10 10 [IN] 48 65535 [OUT] 48 65535 [INTERACT] 69 \"collision\" 10 10 [IN] 51 65535 [OUT] 51 65535 [INTERACT] 64 \"contact\" 11 102 [IN] 48 65535 [OUT] 48 65535 [INTERACT] 13 \"collision\" 12 12 [IN] 19 25 [OUT] 19 25 [INTERACT] 70 \"contact\" 11 111 [IN] 51 65535 [OUT] 51 65535 [INTERACT] 67 \"collision\" 12 12 [IN] 50 65535 [OUT] 50 65535 [INTERACT] 15 \"collision\" 12 12 [IN] 2 7 [OUT] 2 7 [INTERACT] 125 \"collision\" 12 12 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 14 \"contact\" 13 14 [IN] 19 25 [OUT] 19 25 [INTERACT] 60 \"collision\" 13 14 [IN] 46 65535 [OUT] 46 65535 [INTERACT] 16 \"contact\" 13 14 [IN] 2 7 [OUT] 2 7 [INTERACT] 10 \"collision\" 14 14 [IN] 17 65535 [OUT] 17 65535 [INTERACT] 17 \"collision\" 14 14 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 126 \"collision\" 14 14 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 104 \"collision\" 14 15 [IN] 74 65535 [OUT] 74 65535 [INTERACT] 13 \"collision\" 15 15 [IN] 19 25 [OUT] 19 25 [INTERACT] 18 \"contact\" 15 16 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 114 \"collision\" 15 15 [IN] 94 65535 [OUT] 94 65535 [INTERACT] 71 \"collision\" 15 15 [IN] 52 65535 [OUT] 52 65535 [INTERACT] 72 \"contact\" 16 17 [IN] 52 65535 [OUT] 52 65535 [INTERACT] 115 \"collision\" 17 17 [IN] 94 65535 [OUT] 94 65535 [INTERACT] 19 \"collision\" 18 18 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 73 \"collision\" 18 18 [IN] 53 65535 [OUT] 53 65535 [INTERACT] 105 \"collision\" 18 18 [IN] 74 65535 [OUT] 74 65535 [INTERACT] 106 \"contact\" 19 20 [IN] 74 65535 [OUT] 74 65535 [INTERACT] 116 \"collision\" 19 19 [IN] 94 65535 [OUT] 94 65535 [INTERACT] 2 \"collision\" 19 20 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 107 \"collision\" 20 20 [IN] 75 65535 [OUT] 75 65535 [INTERACT] 117 \"contact\" 20 23 [IN] 94 65535 [OUT] 94 65535 [INTERACT] 20 \"collision\" 20 23 [IN] 2 7 [OUT] 2 7 [INTERACT] 112 \"collision\" 21 21 [IN] 88 65535 [OUT] 88 65535 [INTERACT] 118 \"collision\" 21 21 [IN] 95 65535 [OUT] 95 65535 [INTERACT] 108 \"contact\" 21 65 [IN] 75 65535 [OUT] 75 65535 [INTERACT] 113 \"contact\" 22 102 [IN] 88 65535 [OUT] 88 65535 [INTERACT] 239 \"collision\" 23 24 [IN] 99 65535 [OUT] 99 65535 [INTERACT] 119 \"contact\" 22 103 [IN] 95 65535 [OUT] 95 65535 [INTERACT] 8 \"collision\" 23 23 [IN] 1 7 [OUT] 1 7",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"yellow scooter\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 80 738 [OBJ] 1 \"blue mountain\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 383 819 7 386 865 10 383 818 12 382 814 19 387 866 20 385 858 21 382 810 23 383 797 [OBJ] 2 \"cyan suv\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 733 628 1 730 628 3 725 631 4 722 631 5 724 631 6 719 634 7 717 635 9 712 638 11 709 639 12 704 640 14 701 643 15 699 644 16 696 645 17 694 647 18 691 648 19 688 648 20 686 650 22 680 652 23 678 653 [OBJ] 3 \"gray fighter\" 9 24 [VIS] 0 false 9 true [IN-FRAME] 0 false 9 true [PX] 0 null null 9 0 308 10 2 307 11 5 307 13 7 305 14 8 302 15 10 299 16 12 296 17 23 296 18 35 296 19 39 295 20 55 295 21 61 296 22 12 304 23 26 305 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACTIONS-END] [SCENE-END]\n",
"parse_debug": {
"pred_parse_valid": false,
"pred_parse_error": "Could not find [SCENE-END] after [SCENE-START].",
"pred_parse_error_type": "ValueError",
"guided_grammar_sha256": "501d31b980c1f9495be86a80917e58d24972e52c4d442eafe69e5c4fc7522dc6",
"guided_grammar_path": "strive_datasets/prep/kubric/scene_repr/profiles/grammars/selector_sentinel_no_3d.lark",
"debug_dir": "/gscratch/krishna/suryadv/strive_dynsuperclevr/runtime/36996079_inference_24/debug/test_24_shard_002/parse_failures/sample_000005",
"guided_grammar_debug_path": "/gscratch/krishna/suryadv/strive_dynsuperclevr/runtime/36996079_inference_24/debug/test_24_shard_002/parse_failures/sample_000005/grammar.ebnf.gz"
}
}Parsed prediction
{
"error": "ValueError: Could not find [SCENE-END] after [SCENE-START]."
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "yellow scooter",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 80,
"y": 738
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "blue mountain",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 383,
"y": 819
},
{
"t": 7,
"x": 386,
"y": 865
},
{
"t": 10,
"x": 383,
"y": 818
},
{
"t": 12,
"x": 382,
"y": 814
},
{
"t": 19,
"x": 387,
"y": 866
},
{
"t": 20,
"x": 385,
"y": 858
},
{
"t": 21,
"x": 382,
"y": 810
},
{
"t": 23,
"x": 383,
"y": 797
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "cyan suv",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 733,
"y": 628
},
{
"t": 1,
"x": 730,
"y": 628
},
{
"t": 3,
"x": 725,
"y": 631
},
{
"t": 4,
"x": 722,
"y": 631
},
{
"t": 5,
"x": 724,
"y": 631
},
{
"t": 6,
"x": 719,
"y": 634
},
{
"t": 7,
"x": 717,
"y": 635
},
{
"t": 9,
"x": 712,
"y": 638
},
{
"t": 11,
"x": 709,
"y": 639
},
{
"t": 12,
"x": 704,
"y": 640
},
{
"t": 14,
"x": 701,
"y": 643
},
{
"t": 15,
"x": 699,
"y": 644
},
{
"t": 16,
"x": 696,
"y": 645
},
{
"t": 17,
"x": 694,
"y": 647
},
{
"t": 18,
"x": 691,
"y": 648
},
{
"t": 19,
"x": 688,
"y": 648
},
{
"t": 20,
"x": 686,
"y": 650
},
{
"t": 22,
"x": 680,
"y": 652
},
{
"t": 23,
"x": 678,
"y": 653
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "gray fighter",
"vis": [
{
"t": 0,
"v": false
},
{
"t": 9,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": false
},
{
"t": 9,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": null,
"y": null
},
{
"t": 9,
"x": 0,
"y": 308
},
{
"t": 10,
"x": 2,
"y": 307
},
{
"t": 11,
"x": 5,
"y": 307
},
{
"t": 13,
"x": 7,
"y": 305
},
{
"t": 14,
"x": 8,
"y": 302
},
{
"t": 15,
"x": 10,
"y": 299
},
{
"t": 16,
"x": 12,
"y": 296
},
{
"t": 17,
"x": 23,
"y": 296
},
{
"t": 18,
"x": 35,
"y": 296
},
{
"t": 19,
"x": 39,
"y": 295
},
{
"t": 20,
"x": 55,
"y": 295
},
{
"t": 21,
"x": 61,
"y": 296
},
{
"t": 22,
"x": 12,
"y": 304
},
{
"t": 23,
"x": 26,
"y": 305
}
],
"birth": 9,
"death": 24
}
],
"interactions": [
{
"id": 0,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 1,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": null,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": null,
"avg_image_position_segmentation_acc": null,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": null,
"avg_image_position_segmentation_acc_large": null,
"avg_image_position_segmentation_acc_first_frame": null,
"avg_image_position_segmentation_visible_acc": null,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": null,
"visibility_true_acc": null,
"visibility_false_acc": null,
"is_in_frame_acc": null,
"is_in_frame_true_acc": null,
"is_in_frame_false_acc": null,
"vis_acc_from_px": null,
"vis_acc_from_px_true_acc": null,
"vis_acc_from_px_false_acc": null,
"num_matches": 0,
"num_unmatched_pred": 0,
"num_unmatched_target": 4
},
"supports": {
"avg_type_similarity": 0,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 0,
"avg_image_position_segmentation_acc": 0,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 0,
"avg_image_position_segmentation_acc_large": 0,
"avg_image_position_segmentation_acc_first_frame": 0,
"avg_image_position_segmentation_visible_acc": 0,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 0,
"visibility_true_acc": 0,
"visibility_false_acc": 0,
"is_in_frame_acc": 0,
"is_in_frame_true_acc": 0,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 0,
"vis_acc_from_px_true_acc": 0,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [],
"unmatched_target_ids": [
0,
1,
2,
3
],
"unmatched_pred_ids": [],
"target_type_counts": {
"yellow scooter": 1,
"blue mountain": 1,
"cyan suv": 1,
"gray fighter": 1
},
"pred_type_counts": {},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"2.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"3.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"jaccard_by_threshold": {
"0.5": 0.0,
"1.0": 0.0,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {},
"segmentation_precision_by_threshold": {},
"segmentation_recall_by_threshold": {},
"segmentation_f1_by_threshold": {},
"segmentation_frame_average_precision": null,
"segmentation_frame_average_recall": null,
"segmentation_frame_average_f1": null,
"segmentation_first_frame_counts": {},
"segmentation_first_frame_precision": null,
"segmentation_first_frame_recall": null,
"segmentation_first_frame_f1": null,
"segmentation_by_motion_group": {}
},
"per_match_metrics": [],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 3,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"interaction_matching": {
"matches": [],
"unmatched_target_ids": [
0,
1,
2
],
"unmatched_pred_ids": [],
"target_type_counts": {
"contact": 3
},
"pred_type_counts": {},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 3,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"contact": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 3,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 3
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_type": {
"contact": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}