STRIVE · 08/28/2026
Dyn generalization · test_24 · super_clevr_1163
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"purple minivan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 100 982 1 116 976 2 100 970 3 117 964 4 118 958 5 133 952 6 121 947 7 121 942 8 126 936 9 139 931 10 133 927 11 135 922 12 141 918 13 133 914 14 128 909 15 127 905 16 130 901 17 128 898 18 127 892 19 126 884 20 125 879 21 123 872 22 121 871 23 120 860 [OBJ] 1 \"purple dirtbike\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 233 772 2 250 743 13 243 782 17 233 769 18 237 765 19 241 760 20 244 756 21 251 752 22 255 748 23 258 746 [OBJ] 2 \"red utility\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 703 670 3 738 703 5 758 650 6 770 703 7 758 650 [OBJ] 3 \"green airliner\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 752 196 1 748 195 2 744 195 3 741 195 4 737 195 5 733 195 6 729 194 8 726 195 10 722 195 11 719 195 12 708 194 13 705 194 14 701 194 15 698 194 16 695 194 17 691 193 18 687 193 19 683 193 20 679 193 21 675 193 22 670 192 23 668 192 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 18 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 5 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 0 \"collision\" 17 17 [IN] 0 1 [OUT] 0 1 [INTERACT] 2 \"collision\" 22 22 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 3 \"contact\" 23 23 [IN] 0 65535 [OUT] 0 65535 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_24/super_clevr_1163/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"purple fabric dollhouse sofa\" 0 1 [VIS] 0 true 4 false [IN-FRAME] 0 true 4 false [PX] 0 5 994 2 1 986 4 null null [OBJ] 1 \"purple fabric dollhouse sofa\" 0 1 [VIS] 0 true 4 false [IN-FRAME] 0 true 4 false [PX] 0 9 990 2 1 986 4 null null [OBJ] 2 \"purple plastic medicine bottle\" 0 1 [VIS] 0 true 6 false [IN-FRAME] 0 true 6 false [PX] 0 13 986 2 5 982 6 null null [OBJ] 3 \"purple and white rubber flip-flop\" 0 10 [VIS] 0 true 4 false [IN-FRAME] 0 true 4 false [PX] 0 13 994 2 5 990 4 null null [OBJ] 4 \"pink plastic file sorter\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 138 978 1 130 970 2 123 962 3 119 955 4 115 943 5 111 931 6 130 927 7 138 919 8 142 908 9 146 896 10 138 888 11 130 880 12 130 869 13 126 861 14 130 853 15 142 849 16 150 841 17 158 837 18 162 830 19 166 822 20 169 814 21 173 806 22 177 798 23 181 791 [OBJ] 5 \"grey and pink running shoe\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 251 779 1 251 791 2 248 779 3 251 791 6 248 775 7 251 787 8 251 798 9 255 791 10 259 775 11 263 767 12 267 759 13 267 771 14 267 783 15 267 794 16 271 779 17 271 791 18 271 802 19 271 814 20 275 798 21 275 810 22 275 822 23 275 833 [OBJ] 6 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 572 673 4 580 677 [OBJ] 7 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 576 669 7 583 673 10 591 677 [OBJ] 8 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 583 666 7 591 669 [OBJ] 9 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 591 669 7 603 669 10 611 673 [OBJ] 10 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 595 666 8 603 662 [OBJ] 11 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 611 673 8 623 673 [OBJ] 12 \"red and black plastic handheld game console\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 615 681 9 626 681 [OBJ] 13 \"dark grey and orange athletic shoe\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 650 677 3 658 681 9 669 681 [OBJ] 14 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 654 669 3 662 673 7 669 669 10 677 673 [OBJ] 15 \"red and black plastic handheld game console\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 689 677 3 697 673 9 705 669 10 701 677 11 705 669 12 708 677 13 705 669 14 708 677 15 705 669 16 708 677 17 705 669 18 708 677 19 705 669 20 708 677 21 705 669 22 708 677 23 705 669 [OBJ] 16 \"red and yellow wooden toy airplane with pilot\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 705 619 1 701 626 2 708 619 4 705 626 5 712 619 7 708 626 8 716 619 9 712 626 10 720 619 11 716 626 12 724 619 13 720 626 14 728 619 15 724 626 16 732 619 17 728 626 18 736 619 19 732 626 20 740 619 21 736 626 22 744 619 23 740 626 [OBJ] 17 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 708 666 5 716 662 10 724 658 13 728 650 14 724 658 16 732 650 17 728 658 20 736 654 [OBJ] 18 \"red and yellow wooden toy airplane with pilot\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 716 623 2 724 619 4 720 626 6 728 623 8 736 619 9 732 626 10 740 619 11 736 626 12 744 619 14 740 626 15 748 619 16 744 626 18 751 619 20 748 626 21 755 619 22 751 626 23 759 619 [OBJ] 19 \"red and green wooden toy light switch\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 720 677 6 728 673 12 736 669 [OBJ] 20 \"red and green wooden toy vegetable cutting set\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 732 654 4 740 650 9 748 646 [OBJ] 21 \"red and green wooden bird rattle\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 736 669 6 744 666 12 751 662 [OBJ] 22 \"red and green wooden toy vegetable cutting set\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 751 654 3 759 650 9 767 646 13 775 642 [OBJ] 23 \"red and green wooden toy vegetable cutting set\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 755 669 5 763 666 9 771 662 [OBJ] 24 \"red and green wooden toy vegetable cutting set\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 775 658 5 783 654 9 791 650 [OBJ] 25 \"green plant pot\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 794 650 6 802 646 9 810 642 [OBJ] 26 \"green plant pot\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 802 646 4 810 642 8 818 638 [OBJ] 27 \"green and white plastic portable speaker\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 810 654 5 818 650 9 826 646 [OBJ] 28 \"green and white plastic portable speaker\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 822 646 6 830 642 10 837 638 [OBJ] 29 \"dark grey and pink running shoe\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 845 638 4 853 634 8 861 630 [OBJ] 30 \"green and white plastic portable speaker\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 849 650 5 857 646 9 865 642 [OBJ] 31 \"green plant pot\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 865 642 4 873 638 9 880 634 [OBJ] 32 \"green plant pot\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 873 638 3 880 634 8 888 630 [OBJ] 33 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 884 638 4 892 634 8 900 630 [OBJ] 34 \"green plant pot\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 900 630 4 908 626 8 919 623 [OBJ] 35 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 916 630 4 927 630 7 935 626 9 943 623 [OBJ] 36 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 939 626 4 951 626 8 962 623 9 958 615 [OBJ] 37 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 951 626 3 962 626 6 970 619 9 978 615 [OBJ] 38 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 970 623 3 978 619 5 982 626 6 982 615 9 990 611 11 994 619 12 994 607 14 998 615 15 998 603 17 994 626 18 994 611 19 994 599 21 998 607 22 998 595 23 998 583 [OBJ] 39 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 619 4 998 611 8 998 623 9 998 634 10 998 646 11 998 658 12 998 669 13 998 681 14 998 693 15 998 705 16 998 716 17 998 728 18 998 740 19 998 751 20 998 763 21 998 775 22 998 787 23 998 798 [OBJ] 40 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 642 4 998 654 6 998 666 8 998 681 9 998 693 10 998 705 11 998 716 12 998 728 13 998 740 14 998 751 15 998 763 16 998 775 17 998 787 18 998 798 19 998 810 20 998 822 21 998 833 22 998 845 23 998 857 [OBJ] 41 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 666 4 998 681 6 998 693 8 998 708 9 998 720 10 998 732 11 998 744 12 998 755 13 998 767 14 998 779 15 998 791 16 998 802 17 998 814 18 998 826 19 998 837 20 998 849 21 998 861 22 998 873 23 998 884 [OBJ] 42 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 685 2 998 697 4 998 708 6 998 724 8 998 736 10 998 748 12 998 759 14 998 771 16 998 783 18 998 794 20 998 806 22 998 818 23 998 830 [OBJ] 43 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 701 2 998 712 4 998 724 6 998 736 8 998 748 10 998 759 12 998 771 14 998 783 16 998 798 18 998 810 20 998 826 22 998 837 23 998 849 [OBJ] 44 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 712 2 998 724 4 998 736 6 998 748 8 998 759 10 998 771 12 998 783 14 998 798 16 998 810 18 998 822 20 998 837 22 998 849 23 998 861 [OBJ] 45 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 720 2 998 732 4 998 744 6 998 755 8 998 767 10 998 779 12 998 791 14 998 802 16 998 814 18 998 826 20 998 837 22 998 849 23 998 861 [OBJ] 46 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 728 1 998 740 3 998 751 5 998 763 7 998 775 9 998 787 11 998 798 13 998 810 15 998 822 17 998 833 19 998 845 21 998 857 23 998 869 [OBJ] 47 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 740 3 998 751 6 998 763 9 998 775 11 998 787 14 998 798 16 998 810 19 998 822 21 998 833 23 998 845 [OBJ] 48 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 748 2 998 759 4 998 771 6 998 783 8 998 794 10 998 806 12 998 818 14 998 830 16 998 841 18 998 853 20 998 865 22 998 876 23 998 888 [OBJ] 49 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 759 2 998 771 4 998 783 6 998 794 8 998 806 10 998 818 12 998 830 14 998 841 16 998 853 18 998 865 20 998 876 22 998 888 23 998 900 [OBJ] 50 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 775 3 998 787 6 998 798 9 998 810 11 998 822 14 998 833 17 998 845 19 998 857 21 998 869 23 998 880 [OBJ] 51 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 791 2 998 802 4 998 814 6 998 826 8 998 837 10 998 849 12 998 861 14 998 873 16 998 884 18 998 896 20 998 908 22 998 919 23 998 931 [OBJ] 52 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 802 2 998 814 4 998 826 6 998 837 8 998 849 10 998 861 12 998 873 14 998 884 16 998 896 18 998 908 20 998 919 22 998 931 23 998 943 [OBJ] 53 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 814 2 998 826 4 998 837 6 998 849 8 998 861 10 998 873 12 998 884 14 998 896 16 998 908 18 998 919 20 998 931 22 998 943 23 998 955 [OBJ] 54 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 830 2 998 841 4 998 853 6 998 865 8 998 876 10 998 888 12 998 900 14 998 912 16 998 923 18 998 935 20 998 947 22 998 958 23 998 970 [OBJ] 55 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 845 2 998 857 4 998 869 6 998 880 8 998 892 10 998 904 12 998 916 14 998 927 16 998 939 18 998 951 20 998 962 22 998 974 23 994 986 [OBJ] 56 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 865 2 998 876 4 998 888 6 998 900 8 998 912 10 998 923 12 998 935 14 998 947 16 998 958 18 998 970 20 998 982 21 994 994 22 998 998 [OBJ] 57 \"green and yellow cardboard whey protein box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 884 2 998 896 4 998 908 6 998 919 8 998 931 10 998 943 12 998 955 14 998 966 16 998 978 17 994 990 20 998 998 21 994 990 22 998 994 23 994 998 [OBJ] 58 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 900 2 998 912 4 998 923 6 998 935 8 998 947 10 998 958 12 998 970 14 998 982 15 994 990 18 998 994 19 994 998 [OBJ] 59 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 916 2 998 927 4 998 939 6 998 951 8 998 962 10 998 974 12 998 986 14 994 994 16 998 998 17 994 994 19 998 990 20 994 998 21 998 994 22 994 998 23 998 994 [OBJ] 60 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 939 2 998 951 4 998 962 6 998 974 8 998 986 10 998 998 11 994 994 14 998 990 15 994 998 17 998 994 18 994 998 20 998 994 21 994 998 23 998 994 [OBJ] 61 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 951 2 998 962 4 998 974 6 998 986 8 998 994 9 994 986 11 994 998 12 990 990 14 990 998 15 986 990 17 986 998 18 982 990 20 982 998 21 978 990 23 978 998 [OBJ] 62 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 970 2 998 982 4 998 990 6 994 998 7 990 990 10 986 998 11 982 990 14 978 998 15 974 990 18 966 994 21 958 998 22 955 990 23 951 998 [OBJ] 63 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 986 2 994 994 3 998 990 5 998 998 6 994 990 8 994 998 9 990 990 11 990 998 12 986 990 14 986 998 15 982 990 17 982 998 18 978 990 20 978 998 21 974 990 23 974 998 [OBJ] 64 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 634 1 994 646 3 994 658 5 994 669 7 994 681 9 994 693 11 994 705 13 994 716 15 994 728 17 994 740 19 994 751 21 994 763 23 994 775 [OBJ] 65 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 650 3 994 666 5 994 677 7 994 689 9 994 701 11 994 712 13 994 724 15 994 736 17 994 748 19 994 759 21 994 771 23 994 783 [OBJ] 66 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 666 2 994 677 4 994 689 6 994 701 8 994 712 10 994 724 12 994 736 14 994 748 16 994 759 18 994 771 20 994 783 22 994 794 23 990 787 [OBJ] 67 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 685 2 994 697 4 994 708 6 994 720 8 994 732 10 994 744 12 994 755 14 994 767 16 994 779 18 994 791 20 994 802 22 994 814 23 994 826 [OBJ] 68 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 701 3 994 712 5 994 724 7 994 736 9 994 748 11 994 759 13 994 771 15 994 783 17 994 794 19 994 806 21 994 818 23 994 830 [OBJ] 69 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 720 2 994 732 4 994 744 6 994 755 8 994 767 10 994 779 12 994 791 14 994 802 16 994 814 18 994 826 20 994 837 22 994 849 23 990 857 [OBJ] 70 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 740 2 994 751 4 994 763 6 994 775 8 994 787 10 994 798 12 994 810 14 994 822 16 994 833 18 994 845 20 994 857 22 994 869 23 990 876 [OBJ] 71 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 759 2 994 771 4 994 783 6 994 794 8 994 806 10 994 818 12 994 830 14 994 841 16 994 853 18 994 865 20 994 876 22 994 888 23 990 896 [OBJ] 72 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 775 2 994 787 4 994 798 6 994 810 8 994 822 10 994 833 12 994 845 14 994 857 16 994 869 18 994 880 20 994 892 22 994 904 23 990 912 [OBJ] 73 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 794 2 994 806 4 994 818 6 994 830 8 994 841 10 994 853 12 994 865 14 994 876 16 994 888 18 994 900 20 994 912 22 994 923 23 990 931 [OBJ] 74 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 810 2 994 822 4 994 833 6 994 845 8 994 857 10 994 869 12 994 880 14 994 892 16 994 904 18 994 916 20 994 927 22 994 939 23 990 947 [OBJ] 75 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 826 2 994 837 4 994 849 6 994 861 8 994 873 10 994 884 12 994 896 14 994 908 16 994 919 18 994 931 20 994 943 22 994 955 23 990 962 [OBJ] 76 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 841 2 994 853 4 994 865 6 994 876 8 994 888 10 994 900 12 994 912 14 994 923 16 994 935 18 994 947 20 994 958 22 994 970 23 990 978 [OBJ] 77 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 857 2 994 869 4 994 880 6 994 892 8 994 904 10 994 916 12 994 927 14 994 939 16 994 951 18 994 962 19 990 970 21 990 982 22 986 990 23 994 994 [OBJ] 78 \"white and blue cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 978 2 990 990 5 986 998 6 990 990 7 986 998 9 982 990 10 982 935 11 982 986 12 978 994 15 974 986 16 970 994 19 966 986 20 962 994 23 955 990 [OBJ] 79 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 990 197 2 982 205 3 974 208 5 962 212 7 955 216 9 943 220 11 931 224 13 923 228 15 912 232 17 904 236 19 896 240 21 888 244 23 880 248 [OBJ] 80 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 990 220 3 982 228 5 978 236 7 970 240 9 966 248 11 958 251 13 951 255 15 943 259 17 935 263 19 927 267 21 919 271 23 912 275 [OBJ] 81 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 990 232 3 986 240 5 978 244 7 970 248 9 962 251 11 955 255 13 947 259 15 939 263 17 931 267 19 923 271 21 916 275 23 908 279 [OBJ] 82 \"white and light blue athletic shoe\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 310 3 978 314 5 970 318 8 958 322 11 947 326 14 935 330 16 927 333 19 916 337 22 904 341 23 912 345 [OBJ] 83 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 240 2 978 244 5 966 248 8 955 251 10 947 255 12 939 259 14 931 263 16 923 267 18 916 271 20 908 275 22 900 279 23 892 283 [OBJ] 84 \"white and blue track spike\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 330 3 978 333 6 970 337 9 958 341 12 947 345 15 935 349 18 923 353 21 912 357 23 904 361 [OBJ] 85 \"grey and orange plastic raccoon action figure\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 345 3 978 349 6 966 353 9 958 357 12 947 361 15 939 365 17 931 369 19 923 373 22 912 376 23 904 380 [OBJ] 86 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 365 2 986 376 4 986 388 6 986 400 8 986 412 10 986 423 12 986 435 14 986 447 16 986 458 18 986 470 20 986 482 22 986 494 23 982 486 [OBJ] 87 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 380 2 986 392 4 986 404 6 986 416 8 986 427 10 986 439 12 986 451 14 986 462 16 986 474 18 986 486 20 986 498 22 986 509 23 982 517 [OBJ] 88 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 400 3 986 416 5 986 427 7 986 439 9 986 451 11 986 462 13 986 474 15 986 486 17 986 498 19 986 509 21 986 521 23 986 533 [OBJ] 89 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 416 3 986 431 6 986 443 9 986 455 12 986 466 15 986 478 17 986 490 20 986 501 22 986 513 23 982 521 [OBJ] 90 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 427 4 986 439 7 986 451 10 986 462 13 986 474 16 986 486 19 986 498 22 986 509 23 982 517 [OBJ] 91 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 439 1 986 451 4 986 462 7 986 474 10 986 486 12 986 498 14 986 509 17 986 521 19 986 533 21 986 544 23 986 556 [OBJ] 92 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 451 2 986 462 4 986 474 6 986 486 8 986 498 10 986 509 12 986 521 14 986 533 16 986 544 18 986 556 20 986 568 22 986 580 23 982 587 [OBJ] 93 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 462 3 986 474 6 986 486 9 986 498 11 986 509 14 986 521 17 986 533 19 986 544 22 986 556 23 982 564 [OBJ] 94 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 470 3 986 482 6 986 494 9 986 505 12 986 517 15 986 529 18 986 541 21 986 552 23 982 560 [OBJ] 95 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 482 2 986 494 5 986 505 8 986 517 11 986 529 14 986 541 17 986 552 20 986 564 22 982 572 [OBJ] 96 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 490 3 986 501 6 986 513 9 986 525 12 986 537 15 986 548 18 986 560 21 986 572 23 982 580 [OBJ] 97 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 501 2 986 513 4 986 525 7 986 537 10 986 548 13 986 560 16 986 572 19 986 583 22 986 595 23 982 587 [OBJ] 98 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 513 2 986 525 5 986 537 8 986 548 11 986 560 14 986 572 17 986 583 20 986 595 23 986 607 [OBJ] 99 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 521 2 986 533 5 986 544 8 986 556 11 986 568 14 986 580 17 986 591 20 986 603 23 986 615 [OBJ] 100 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 529 2 986 541 5 986 552 8 986 564 11 986 576 14 986 587 17 986 599 19 986 611 22 986 623 23 982 630 [OBJ] 101 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 537 2 986 548 5 986 560 8 986 572 11 986 583 14 986 595 17 986 607 19 982 615 22 986 626 23 982 634 [OBJ] 102 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 548 2 986 560 5 986 572 8 986 583 11 986 595 14 986 607 17 986 619 20 986 630 23 986 642 [OBJ] 103 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 556 2 986 568 5 986 580 8 986 591 11 986 603 14 986 615 17 986 626 20 986 638 23 986 650 [OBJ] 104 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 564 2 986 576 5 986 587 8 986 599 11 986 611 14 986 623 17 986 634 20 986 646 23 986 658 [OBJ] 105 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 572 2 986 583 5 986 595 8 986 607 11 986 619 14 986 630 17 986 642 20 986 654 23 986 666 [OBJ] 106 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 580 3 986 591 6 986 603 9 986 615 12 986 626 15 986 638 18 986 650 21 986 662 23 982 669 [OBJ] 107 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 591 3 986 603 6 986 615 9 986 626 12 986 638 15 986 650 18 986 662 21 986 673 23 982 681 [OBJ] 108 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 599 2 986 611 5 986 623 8 986 634 11 986 646 14 986 658 17 986 669 20 986 681 23 986 693 [OBJ] 109 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 611 3 986 623 6 986 634 9 986 646 12 986 658 15 986 669 18 986 681 21 986 693 23 982 701 [OBJ] 110 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 623 2 986 634 5 986 646 8 986 658 11 986 669 14 986 681 17 986 693 20 986 705 23 986 716 [OBJ] 111 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 634 2 986 646 5 986 658 8 986 669 11 986 681 14 986 693 17 986 705 20 986 716 23 986 728 [OBJ] 112 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 646 3 986 658 6 986 669 9 986 681 12 986 693 15 986 705 18 986 716 21 986 728 23 982 736 [OBJ] 113 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 658 2 986 669 5 986 681 8 986 693 11 986 705 14 986 716 17 986 728 20 986 740 23 986 751 [OBJ] 114 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 669 2 986 681 5 986 693 8 986 705 11 986 716 14 986 728 17 986 740 20 986 751 23 986 763 [OBJ] 115 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 681 2 986 693 5 986 705 8 986 716 11 986 728 14 986 740 17 986 751 20 986 763 23 986 775 [OBJ] 116 \"green and yellow cardboard ink cartridge box\" 0 10 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 693 ",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"purple minivan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 100 982 1 116 976 2 100 970 3 117 964 4 118 958 5 133 952 6 121 947 7 121 942 8 126 936 9 139 931 10 133 927 11 135 922 12 141 918 13 133 914 14 128 909 15 127 905 16 130 901 17 128 898 18 127 892 19 126 884 20 125 879 21 123 872 22 121 871 23 120 860 [OBJ] 1 \"purple dirtbike\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 233 772 2 250 743 13 243 782 17 233 769 18 237 765 19 241 760 20 244 756 21 251 752 22 255 748 23 258 746 [OBJ] 2 \"red utility\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 703 670 3 738 703 5 758 650 6 770 703 7 758 650 [OBJ] 3 \"green airliner\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 752 196 1 748 195 2 744 195 3 741 195 4 737 195 5 733 195 6 729 194 8 726 195 10 722 195 11 719 195 12 708 194 13 705 194 14 701 194 15 698 194 16 695 194 17 691 193 18 687 193 19 683 193 20 679 193 21 675 193 22 670 192 23 668 192 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 18 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 5 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 0 \"collision\" 17 17 [IN] 0 1 [OUT] 0 1 [INTERACT] 2 \"collision\" 22 22 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 3 \"contact\" 23 23 [IN] 0 65535 [OUT] 0 65535 [INTERACTIONS-END] [SCENE-END]\n",
"parse_debug": {
"pred_parse_valid": false,
"pred_parse_error": "Could not find [SCENE-END] after [SCENE-START].",
"pred_parse_error_type": "ValueError",
"guided_grammar_sha256": "501d31b980c1f9495be86a80917e58d24972e52c4d442eafe69e5c4fc7522dc6",
"guided_grammar_path": "strive_datasets/prep/kubric/scene_repr/profiles/grammars/selector_sentinel_no_3d.lark",
"debug_dir": "/gscratch/krishna/suryadv/strive_dynsuperclevr/runtime/36996079_inference_24/debug/test_24_shard_002/parse_failures/sample_000015",
"guided_grammar_debug_path": "/gscratch/krishna/suryadv/strive_dynsuperclevr/runtime/36996079_inference_24/debug/test_24_shard_002/parse_failures/sample_000015/grammar.ebnf.gz"
}
}Parsed prediction
{
"error": "ValueError: Could not find [SCENE-END] after [SCENE-START]."
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "purple minivan",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 100,
"y": 982
},
{
"t": 1,
"x": 116,
"y": 976
},
{
"t": 2,
"x": 100,
"y": 970
},
{
"t": 3,
"x": 117,
"y": 964
},
{
"t": 4,
"x": 118,
"y": 958
},
{
"t": 5,
"x": 133,
"y": 952
},
{
"t": 6,
"x": 121,
"y": 947
},
{
"t": 7,
"x": 121,
"y": 942
},
{
"t": 8,
"x": 126,
"y": 936
},
{
"t": 9,
"x": 139,
"y": 931
},
{
"t": 10,
"x": 133,
"y": 927
},
{
"t": 11,
"x": 135,
"y": 922
},
{
"t": 12,
"x": 141,
"y": 918
},
{
"t": 13,
"x": 133,
"y": 914
},
{
"t": 14,
"x": 128,
"y": 909
},
{
"t": 15,
"x": 127,
"y": 905
},
{
"t": 16,
"x": 130,
"y": 901
},
{
"t": 17,
"x": 128,
"y": 898
},
{
"t": 18,
"x": 127,
"y": 892
},
{
"t": 19,
"x": 126,
"y": 884
},
{
"t": 20,
"x": 125,
"y": 879
},
{
"t": 21,
"x": 123,
"y": 872
},
{
"t": 22,
"x": 121,
"y": 871
},
{
"t": 23,
"x": 120,
"y": 860
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "purple dirtbike",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 233,
"y": 772
},
{
"t": 2,
"x": 250,
"y": 743
},
{
"t": 13,
"x": 243,
"y": 782
},
{
"t": 17,
"x": 233,
"y": 769
},
{
"t": 18,
"x": 237,
"y": 765
},
{
"t": 19,
"x": 241,
"y": 760
},
{
"t": 20,
"x": 244,
"y": 756
},
{
"t": 21,
"x": 251,
"y": 752
},
{
"t": 22,
"x": 255,
"y": 748
},
{
"t": 23,
"x": 258,
"y": 746
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "red utility",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 703,
"y": 670
},
{
"t": 3,
"x": 738,
"y": 703
},
{
"t": 5,
"x": 758,
"y": 650
},
{
"t": 6,
"x": 770,
"y": 703
},
{
"t": 7,
"x": 758,
"y": 650
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "green airliner",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 752,
"y": 196
},
{
"t": 1,
"x": 748,
"y": 195
},
{
"t": 2,
"x": 744,
"y": 195
},
{
"t": 3,
"x": 741,
"y": 195
},
{
"t": 4,
"x": 737,
"y": 195
},
{
"t": 5,
"x": 733,
"y": 195
},
{
"t": 6,
"x": 729,
"y": 194
},
{
"t": 8,
"x": 726,
"y": 195
},
{
"t": 10,
"x": 722,
"y": 195
},
{
"t": 11,
"x": 719,
"y": 195
},
{
"t": 12,
"x": 708,
"y": 194
},
{
"t": 13,
"x": 705,
"y": 194
},
{
"t": 14,
"x": 701,
"y": 194
},
{
"t": 15,
"x": 698,
"y": 194
},
{
"t": 16,
"x": 695,
"y": 194
},
{
"t": 17,
"x": 691,
"y": 193
},
{
"t": 18,
"x": 687,
"y": 193
},
{
"t": 19,
"x": 683,
"y": 193
},
{
"t": 20,
"x": 679,
"y": 193
},
{
"t": 21,
"x": 675,
"y": 193
},
{
"t": 22,
"x": 670,
"y": 192
},
{
"t": 23,
"x": 668,
"y": 192
}
],
"birth": 0,
"death": 24
}
],
"interactions": [
{
"id": 1,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 18,
"type": "contact"
},
{
"id": 4,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 5,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 17,
"ends_at": 17,
"type": "collision"
},
{
"id": 2,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 22,
"ends_at": 22,
"type": "collision"
},
{
"id": 3,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 23,
"ends_at": 23,
"type": "contact"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": null,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": null,
"avg_image_position_segmentation_acc": null,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": null,
"avg_image_position_segmentation_acc_large": null,
"avg_image_position_segmentation_acc_first_frame": null,
"avg_image_position_segmentation_visible_acc": null,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": null,
"visibility_true_acc": null,
"visibility_false_acc": null,
"is_in_frame_acc": null,
"is_in_frame_true_acc": null,
"is_in_frame_false_acc": null,
"vis_acc_from_px": null,
"vis_acc_from_px_true_acc": null,
"vis_acc_from_px_false_acc": null,
"num_matches": 0,
"num_unmatched_pred": 0,
"num_unmatched_target": 4
},
"supports": {
"avg_type_similarity": 0,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 0,
"avg_image_position_segmentation_acc": 0,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 0,
"avg_image_position_segmentation_acc_large": 0,
"avg_image_position_segmentation_acc_first_frame": 0,
"avg_image_position_segmentation_visible_acc": 0,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 0,
"visibility_true_acc": 0,
"visibility_false_acc": 0,
"is_in_frame_acc": 0,
"is_in_frame_true_acc": 0,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 0,
"vis_acc_from_px_true_acc": 0,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [],
"unmatched_target_ids": [
0,
1,
2,
3
],
"unmatched_pred_ids": [],
"target_type_counts": {
"purple minivan": 1,
"purple dirtbike": 1,
"red utility": 1,
"green airliner": 1
},
"pred_type_counts": {},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"2.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"3.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"jaccard_by_threshold": {
"0.5": 0.0,
"1.0": 0.0,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"dynamic": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {},
"segmentation_precision_by_threshold": {},
"segmentation_recall_by_threshold": {},
"segmentation_f1_by_threshold": {},
"segmentation_frame_average_precision": null,
"segmentation_frame_average_recall": null,
"segmentation_frame_average_f1": null,
"segmentation_first_frame_counts": {},
"segmentation_first_frame_precision": null,
"segmentation_first_frame_recall": null,
"segmentation_first_frame_f1": null,
"segmentation_by_motion_group": {}
},
"per_match_metrics": [],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 6,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 6
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"interaction_matching": {
"matches": [],
"unmatched_target_ids": [
0,
1,
2,
3,
4,
5
],
"unmatched_pred_ids": [],
"target_type_counts": {
"collision": 2,
"contact": 4
},
"pred_type_counts": {},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 5,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 1,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 2,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 4,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"contact": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}