STRIVE · 08/28/2026
Dyn generalization · test_24 · super_clevr_1121
Everything fed into evaluation and the full per-sample output it produced.
Dataset row: exact video + user prompt + assistant target
{
"messages": [
{
"role": "user",
"content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`."
},
{
"role": "assistant",
"content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"green wagon\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 212 549 2 216 550 4 219 552 7 221 553 8 223 554 9 224 555 11 226 556 13 227 557 15 230 558 18 233 560 20 235 561 22 237 563 [OBJ] 1 \"yellow sedan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 296 568 2 297 570 3 300 571 5 301 573 6 303 574 7 305 575 9 307 577 12 310 577 13 312 579 15 314 582 17 316 584 19 317 586 21 318 589 22 319 590 [OBJ] 2 \"blue dirtbike\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 344 831 1 348 843 2 344 831 3 348 843 6 341 831 9 348 841 10 343 830 11 341 829 12 350 840 13 343 828 17 344 825 18 345 824 22 347 821 [OBJ] 3 \"red airliner\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 386 160 1 400 158 2 391 160 3 403 159 4 405 160 5 406 161 6 396 165 7 409 165 8 411 167 9 404 170 10 414 172 11 416 175 12 417 178 13 419 182 14 422 186 15 424 191 16 425 195 17 426 200 18 429 205 19 430 210 20 433 217 21 434 222 22 438 230 23 441 236 [OBJ] 4 \"green wagon\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 710 942 1 697 939 2 690 935 3 687 932 4 749 930 5 737 926 6 723 923 7 707 920 8 699 917 9 687 914 10 679 910 11 669 905 12 657 898 13 644 891 14 632 883 15 619 876 16 608 869 17 596 863 18 587 857 19 576 851 20 566 845 21 555 839 22 544 832 23 536 825 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 0 \"collision\" 3 3 [IN] 0 1 [OUT] 0 1 [INTERACTIONS-END] [SCENE-END]"
}
],
"videos": [
"/mmfs1/gscratch/krishna/suryadv/strive_dynsuperclevr/data/lf/videos/test_24/super_clevr_1121/frames.max_frames_24.mp4"
]
}Inference record: exact prompt + prediction + label + parse debug
{
"prompt": "user\n\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [VIS] t vis ... [IN-FRAME] t is_in_frame ... [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n `[SCENE-START] [OBJS-START] [OBJ] [VIS] [IN-FRAME] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `VIS` uses repeated groups of 2 values in order:\n `t vis`.\n `vis` is a boolean value.\n- `IN-FRAME` uses repeated groups of 2 values in order:\n `t is_in_frame`.\n `is_in_frame` is a boolean value.\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n- `px` and `py` are normalized pixel coordinates in the range `[0, 1]`,\n scaled by `1000` and then converted to integers by dropping the decimal part.\n- For occluded or out-of-frame frames under this setting, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids.\n- Use lowercase booleans (`true`/`false`) for `vis` and `is_in_frame`.\nassistant\n",
"predict": "[SCENE-START] [OBJS-START] [OBJ] 0 \"plastic fruit bath toy set\" 0 6 [VIS] 0 true 2 false [IN-FRAME] 0 true 2 false [PX] 0 1 568 2 null null [OBJ] 1 \"green and grey plastic Doctor Doom action figure\" 0 6 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 208 560 1 216 564 3 228 568 4 220 564 5 236 572 6 244 576 8 232 572 10 240 576 11 248 580 13 255 583 14 244 576 15 251 580 16 263 587 17 248 576 18 259 583 20 267 587 21 255 580 22 267 587 23 275 591 [OBJ] 2 \"yellow cardboard curl cream box\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 259 525 1 null null [OBJ] 3 \"yellow cardboard curl cream box\" 0 6 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 267 544 1 271 552 4 279 560 5 287 564 6 275 560 7 283 564 11 291 568 14 298 572 16 310 576 17 298 572 19 306 576 22 314 580 [OBJ] 4 \"yellow cardboard curl cream box\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 275 521 2 283 525 5 291 529 6 294 537 7 291 529 8 294 537 9 291 529 10 294 537 11 291 529 13 294 537 15 298 544 17 302 552 19 298 544 20 302 552 22 306 560 [OBJ] 5 \"yellow cardboard curl cream box\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 291 537 1 null null [OBJ] 6 \"yellow cardboard curl cream box\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 310 525 1 null null [OBJ] 7 \"yellow cardboard curl cream box\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 314 541 1 null null [OBJ] 8 \"blue and dark grey plastic screwdriver\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 322 541 1 null null [OBJ] 9 \"blue and dark grey plastic screwdriver\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 330 544 1 null null [OBJ] 10 \"blue and dark grey plastic screwdriver\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 333 548 1 null null [OBJ] 11 \"blue and black plastic handheld gaming console\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 341 552 1 null null [OBJ] 12 \"blue and dark grey plastic screwdriver\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 349 548 1 null null [OBJ] 13 \"blue and black hiking shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 349 556 1 null null [OBJ] 14 \"blue and black plastic handheld gaming console\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 353 552 1 null null [OBJ] 15 \"blue and black hiking shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 361 548 1 null null [OBJ] 16 \"blue and black hiking shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 365 556 1 null null [OBJ] 17 \"blue and black plastic screwdriver\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 365 560 1 null null [OBJ] 18 \"blue and black plastic screwdriver\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 369 564 1 null null [OBJ] 19 \"blue and black hiking shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 373 552 1 null null [OBJ] 20 \"blue and black plastic screwdriver\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 376 560 1 null null [OBJ] 21 \"red and yellow wooden toy airplane with pilot\" 0 6 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 384 162 1 392 166 2 396 173 3 404 177 4 400 185 5 408 189 6 416 197 7 408 205 8 416 212 9 423 220 10 431 228 11 435 236 12 423 244 13 431 251 14 439 259 15 447 267 16 455 275 17 416 279 18 423 287 19 431 294 20 439 302 21 447 310 22 416 314 23 423 322 [OBJ] 22 \"green and orange plastic magnifying glass\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 408 169 1 null null [OBJ] 23 \"wooden stack of game blocks\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 412 173 1 null null [OBJ] 24 \"black and blue athletic shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 416 181 1 null null [OBJ] 25 \"green and grey plastic Doctor Doom action figure\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 423 181 1 null null [OBJ] 26 \"green and grey plastic Doctor Doom action figure\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 427 185 1 null null [OBJ] 27 \"black and blue athletic shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 431 189 1 null null [OBJ] 28 \"black and blue athletic shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 447 193 1 null null [OBJ] 29 \"black plastic gaming mouse\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 462 197 1 null null [OBJ] 30 \"black leather penny loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 470 197 1 null null [OBJ] 31 \"green wooden toy train\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 478 201 1 null null [OBJ] 32 \"black and blue athletic shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 599 470 2 599 458 5 599 474 6 599 462 11 599 474 12 599 462 18 599 474 19 595 462 [OBJ] 33 \"black and blue athletic shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 615 478 2 607 474 3 611 482 4 603 474 5 611 482 6 599 474 7 607 482 10 599 478 11 607 486 12 595 478 13 603 486 14 595 478 15 603 486 16 595 478 17 603 486 18 595 478 19 603 486 20 595 478 21 587 474 22 599 486 23 587 474 [OBJ] 34 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 720 478 1 null null [OBJ] 35 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 720 486 1 null null [OBJ] 36 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 720 494 1 null null [OBJ] 37 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 728 490 1 null null [OBJ] 38 \"green and yellow running shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 728 498 1 null null [OBJ] 39 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 732 490 1 null null [OBJ] 40 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 736 494 1 null null [OBJ] 41 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 744 498 1 null null [OBJ] 42 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 751 494 1 null null [OBJ] 43 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 755 501 1 null null [OBJ] 65 \"yellow hoodie dollhouse boy\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 814 494 1 null null [OBJ] 66 \"yellow plastic Game Boy cartridge\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 826 501 1 null null [OBJ] 67 \"white and red high-top sneaker\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 830 505 1 null null [OBJ] 68 \"grey and black basketball shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 841 509 1 null null [OBJ] 69 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 853 509 1 null null [OBJ] 70 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 861 513 1 null null [OBJ] 61 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 873 505 1 null null [OBJ] 62 \"black and blue athletic shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 884 513 1 null null [OBJ] 63 \"grey and black basketball shoe\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 904 513 1 null null [OBJ] 64 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 912 517 1 null null [OBJ] 71 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 935 517 1 null null [OBJ] 72 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 951 517 1 null null [OBJ] 60 \"green wooden toy train\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 970 513 1 null null [OBJ] 75 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 994 517 1 null null [OBJ] 44 \"dark green fabric loafer\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 994 509 1 null null [OBJ] 52 \"green wooden toy train\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 994 525 1 null null [OBJ] 53 \"dark green plastic coffee container\" 0 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 994 541 1 null null [OBJ] 73 \"green wooden toy train\" 1 6 [VIS] 0 true 5 false [IN-FRAME] 0 true [PX] 0 998 498 5 null null [OBJ] 56 \"green and grey plastic Doctor Doom action figure\" 4 6 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 541 1 998 556 2 998 568 3 998 583 4 998 595 5 998 607 6 998 619 7 994 638 10 998 646 11 990 658 12 994 666 14 990 677 15 994 685 17 986 693 18 990 701 20 994 712 22 990 724 23 994 732 [OBJ] 57 \"green and yellow wooden vegetable garden toy set\" 5 6 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 470 5 994 478 [OBJ] 50 \"green and yellow running shoe\" 5 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 998 521 1 null null [OBJ] 51 \"green and yellow wooden vegetable garden toy set\" 5 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 998 529 1 null null [OBJ] 0 \"black and blue athletic shoe\" 7 6 [VIS] 0 true 7 false [IN-FRAME] 0 true [PX] 0 1 490 7 null null [OBJ] 15 \"green and red plastic ninja turtle action figure\" 8 6 [VIS] 0 true 6 false [IN-FRAME] 0 true [PX] 0 1 560 6 null null [OBJ] 16 \"green and red wooden toy crocodile\" 9 6 [VIS] 0 true 6 false [IN-FRAME] 0 true [PX] 0 1 568 6 null null [OBJ] 54 \"blue and yellow basketball shoe\" 9 6 [VIS] 0 true 6 false [IN-FRAME] 0 true [PX] 0 998 541 6 null null [OBJ] 55 \"white plastic supplement bottle\" 9 6 [VIS] 0 true 6 false [IN-FRAME] 0 true [PX] 0 998 548 6 null null [OBJ] 31 \"green and white plastic portable speaker\" 10 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 998 556 1 null null [OBJ] 32 \"grey and orange athletic shoe\" 10 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 998 564 1 null null [OBJ] 49 \"dark green plastic coffee container\" 10 6 [VIS] 0 true 1 false [IN-FRAME] 0 true [PX] 0 998 580 1 null null [OBJ] 2 \"white cardboard ink cartridge box\" 10 11 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 208 529 1 216 533 5 224 537 8 232 541 11 240 544 14 248 548 16 255 552 18 263 556 21 271 560 23 279 564 [OBJ] 30 \"dark grey cardboard motherboard box\" 10 12 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 587 10 998 599 11 994 611 12 998 619 14 998 630 15 994 638 17 994 650 18 998 658 20 998 669 22 994 681 23 998 689 [OBJ] 46 \"green ink cartridge cardboard box\" 10 13 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 595 10 998 607 12 994 619 14 994 630 16 990 642 18 990 654 20 986 666 22 986 677 23 982 685 [OBJ] 47 \"dark grey cardboard motherboard box\" 10 13 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 603 10 998 615 11 994 623 12 998 630 14 994 638 16 990 646 18 990 658 20 986 666 22 982 673 23 986 681 [OBJ] 48 \"dark grey cardboard motherboard box\" 10 13 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 611 10 998 623 11 994 630 12 998 638 14 994 646 15 998 654 17 994 662 19 990 673 21 990 685 22 986 693 23 990 701 [OBJ] 13 \"blue and yellow fabric sneaker\" 10 14 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 626 10 998 638 12 994 650 13 998 658 15 994 666 16 998 673 18 994 681 20 990 689 21 994 697 23 990 705 [OBJ] 14 \"white and black athletic shoe\" 10 14 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 634 10 998 646 12 994 654 14 990 662 16 986 669 18 982 677 20 978 685 22 974 693 23 970 701 [OBJ] 26 \"black and blue athletic shoe\" 10 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 646 10 998 658 12 994 666 14 990 673 16 986 681 18 982 689 20 978 697 22 974 705 23 970 712 [OBJ] 27 \"white plastic supplement bottle\" 10 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 650 10 998 662 13 994 669 15 990 677 17 986 685 19 982 693 21 978 701 23 974 708 [OBJ] 28 \"dark grey and yellow running shoe\" 10 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 662 10 998 673 14 994 685 17 990 693 20 986 701 23 982 712 [OBJ] 17 \"white and black athletic shoe\" 10 16 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 681 10 998 693 12 994 701 15 990 708 18 986 716 21 982 724 23 978 732 [OBJ] 18 \"white and black athletic shoe\" 10 17 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 689 10 998 701 15 994 708 16 994 720 17 990 728 19 986 736 21 982 744 23 978 751 [OBJ] 19 \"white and black athletic shoe\" 10 18 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 705 10 998 716 12 994 724 14 990 732 16 986 740 18 982 748 20 978 755 22 974 763 23 970 771 [OBJ] 5 \"black and blue fabric boat shoe\" 11 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 1 580 11 1 591 [OBJ] 10 \"dark green plastic coffee container\" 12 13 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 1 591 12 1 603 [OBJ] 35 \"dark red cardboard supplement box\" 12 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 1 595 12 1 607 [OBJ] 45 \"green and orange plastic magnifying glass\" 12 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 1 603 12 1 615 [OBJ] 36 \"black and blue plastic screwdriver\" 12 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 349 841 12 349 853 15 357 861 17 349 865 20 357 869 22 353 876 [OBJ] 37 \"green and grey plastic Doctor Doom action figure\" 12 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 837 943 1 830 947 2 798 943 3 775 939 4 751 935 5 728 931 6 705 927 7 681 923 8 666 919 9 638 912 10 615 908 11 587 900 12 564 892 13 541 888 14 525 884 15 505 880 16 494 876 17 478 873 18 466 869 19 455 865 20 443 861 21 431 857 22 423 853 23 416 849 [OBJ] 38 \"blue and black hiking shoe\" 12 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 837 951 1 830 955 2 814 951 3 802 951 4 791 951 5 775 947 6 763 947 7 751 943 8 736 943 9 724 939 10 708 939 11 697 935 12 685 931 13 673 927 14 658 927 15 646 923 16 634 919 17 626 916 18 615 912 19 603 912 20 595 908 21 583 904 22 576 900 23 568 896 [OBJ] 23 \"dark grey and light grey plastic monster action figure\" 13 14 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 716 13 994 728 14 998 736 15 994 744 17 990 751 19 986 759 21 982 767 23 978 775 [OBJ] 24 \"blue and white athletic shoe\" 13 15 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 724 13 994 736 14 990 744 16 986 751 18 982 759 20 978 767 22 974 775 23 970 783 [OBJ] 11 \"white plastic supplement bottle\" 14 16 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 732 14 994 744 16 990 751 19 986 759 21 982 767 23 978 775 [OBJ] 12 \"black and blue athletic shoe\" 14 18 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 740 14 994 748 15 990 755 17 986 763 19 982 771 21 978 779 23 974 787 [OBJ] 25 \"dark green plastic coffee container\" 15 17 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 748 15 994 759 16 990 767 18 986 775 20 982 783 22 978 791 23 974 798 [OBJ] 21 \"red and yellow wooden toy airplane with pilot\" 17 18 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 380 205 1 384 212 2 392 216 3 396 224 4 404 228 5 408 236 6 416 240 7 419 248 8 427 251 9 431 259 10 439 263 11 443 271 12 451 275 13 458 279 14 466 287 15 474 291 16 478 298 17 486 302 18 494 306 19 501 310 20 509 314 21 517 318 22 525 322 23 533 326 [OBJ] 22 \"blue and black hiking shoe\" 17 18 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 388 216 1 392 224 2 400 228 3 404 236 4 412 240 5 416 248 6 423 251 7 431 255 8 435 263 9 443 267 10 451 271 11 455 279 12 462 283 13 466 291 14 474 294 15 482 298 16 486 306 17 494 310 18 501 314 19 505 322 20 513 326 21 521 330 22 529 333 23 537 337 [OBJ] 33 \"black and white fabric stuffed orca\" 18 20 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 767 18 994 779 20 990 787 22 986 794 23 982 802 [OBJ] 34 \"black and white fabric stuffed orca\" 18 21 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 775 18 994 787 19 990 794 20 986 802 21 982 810 22 978 818 23 974 826 [OBJ] 58 \"white and red high-top sneaker\" 19 20 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 791 19 994 802 20 990 810 21 986 818 22 982 826 23 978 833 [OBJ] 59 \"white plastic supplement bottle\" 19 21 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 798 19 994 810 20 990 818 21 986 826 22 982 833 23 978 841 [OBJ] 3 \"light brown wooden alphabet tile set\" 20 21 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 25 615 20 21 623 [OBJ] 40 \"white and grey cardboard desktop hard drive box\" 20 21 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 1 595 20 1 607 [OBJ] 39 \"blue and black hiking shoe\" 21 22 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 865 994 1 857 990 2 845 990 3 833 990 4 822 990 5 810 990 6 798 990 7 787 986 8 775 986 9 763 986 10 751 986 11 740 986 12 728 986 13 716 986 14 705 986 15 693 986 16 681 986 17 669 986 18 658 986 19 646 986 20 634 986 21 623 986 22 611 986 23 603 982 [OBJ] 54 \"black and grey high-top sneaker\" 21 22 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 822 1 994 830 4 990 841 6 986 849 8 982 857 10 978 865 12 974 873 14 970 880 16 966 888 18 962 896 20 958 904 22 951 912 23 947 919 [OBJ] 50 \"green and yellow running shoe\" 22 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 845 1 994 853 2 990 861 4 986 873 5 982 880 7 978 892 8 974 900 10 970 912 11 966 919 13 962 931 14 958 939 16 955 951 17 951 958 19 947 966 21 943 974 23 935 986 [OBJ] 53 \"blue and yellow rubber sandal\" 22 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 865 22 994 884 [OBJ] 56 \"black cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 880 23 994 896 [OBJ] 30 \"black cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 900 23 994 919 [OBJ] 41 \"grey plastic Nintendo 64 game cartridge\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 919 23 994 935 [OBJ] 43 \"dark grey cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 943 23 994 958 [OBJ] 44 \"dark grey cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 955 23 994 970 [OBJ] 57 \"black and grey athletic shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 974 23 994 986 [OBJ] 1 \"red and black plastic handheld game console\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 99 525 10 95 533 16 99 541 18 91 537 [OBJ] 29 \"black cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 99 544 2 107 548 3 95 548 6 103 552 11 91 552 12 103 556 14 91 556 16 103 560 17 91 560 19 103 564 20 91 564 22 103 568 23 91 568 [OBJ] 20 \"red and yellow wooden toy airplane with pilot\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 365 154 1 369 162 2 376 166 3 380 173 4 388 177 5 392 185 6 396 193 7 404 197 8 408 205 9 412 212 10 416 220 11 423 224 12 427 232 13 435 236 14 439 244 15 443 251 16 451 255 17 455 263 18 462 267 19 466 275 20 474 279 21 482 287 22 486 294 23 494 302 [OBJ] 61 \"blue and white striped fabric dishtowel\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 994 23 994 986 [OBJ] 62 \"white and black athletic shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 998 23 994 990 [OBJ] 63 \"grey and pink sneaker\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 978 23 986 986 [OBJ] 66 \"grey and pink fabric sneaker\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 990 23 990 998 [OBJ] 69 \"blue and black hiking shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 990 974 23 986 986 [OBJ] 60 \"dark green plastic coffee container\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 990 990 23 986 998 [OBJ] 31 \"red plastic hair dryer\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 978 912 23 970 919 [OBJ] 32 \"dark grey cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 978 978 23 970 986 [OBJ] 33 \"dark grey cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 974 982 23 966 990 [OBJ] 6 \"dark grey plastic supplement bottle with orange cap\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 955 990 23 947 994 [OBJ] 67 \"black and blue athletic shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 951 994 23 943 998 [OBJ] 68 \"dark grey and pink running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 943 994 23 935 998 [OBJ] 4 \"white plastic supplement bottle\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 931 990 23 923 994 [OBJ] 61 \"white cardboard ink cartridge box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 919 990 23 912 994 [OBJ] 65 \"dark grey cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 912 994 23 904 998 [OBJ] 42 \"dark grey and pink running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 904 994 23 900 986 [OBJ] 13 \"white cardboard ink cartridge box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 884 994 23 876 998 [OBJ] 15 \"dark green plastic coffee container\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 869 990 23 861 994 [OBJ] 16 \"dark green plastic coffee container\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 857 990 23 849 994 [OBJ] 17 \"dark green plastic coffee container\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 845 990 23 837 994 [OBJ] 18 \"dark grey cardboard motherboard box\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 837 994 23 830 998 [OBJ] 19 \"dark red and black plastic curling wand\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 826 994 23 818 998 [OBJ] 50 \"green and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 974 23 994 982 [OBJ] 51 \"green and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 998 982 23 994 990 [OBJ] 52 \"green and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 994 990 23 990 998 [OBJ] 54 \"green and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 990 994 23 986 998 [OBJ] 55 \"dark green plastic coffee container\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 986 994 23 982 998 [OBJ] 27 \"dark blue fabric chukka boot\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 978 974 23 974 982 [OBJ] 28 \"black and blue athletic shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 974 982 23 970 990 [OBJ] 29 \"black and blue athletic shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 966 990 23 962 998 [OBJ] 39 \"light grey fabric chukka boot\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 958 990 23 951 994 [OBJ] 41 \"black and blue athletic shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 951 994 23 943 998 [OBJ] 45 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 943 994 23 935 998 [OBJ] 46 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 935 994 23 927 998 [OBJ] 47 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 923 994 23 916 998 [OBJ] 48 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 912 994 23 904 998 [OBJ] 50 \"dark red plastic rat sensei action figure\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 900 998 23 892 994 [OBJ] 53 \"black and blue athletic shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 884 998 23 876 994 [OBJ] 49 \"white plastic supplement bottle\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 869 994 23 861 998 [OBJ] 40 \"light grey fabric chukka boot\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 857 994 23 849 998 [OBJ] 26 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 845 994 23 837 998 [OBJ] 27 \"dark blue fabric chukka boot\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 833 994 23 826 998 [OBJ] 28 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 822 994 23 814 998 [OBJ] 29 \"dark blue fabric chukka boot\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 814 994 23 806 998 [OBJ] 23 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 798 994 23 791 998 [OBJ] 24 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 787 994 23 779 998 [OBJ] 25 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 775 994 23 767 998 [OBJ] 26 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 763 994 23 755 998 [OBJ] 27 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 751 994 23 744 998 [OBJ] 28 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 740 994 23 732 998 [OBJ] 29 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 724 994 23 716 998 [OBJ] 30 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 712 994 23 705 998 [OBJ] 31 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 697 994 23 689 998 [OBJ] 32 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 681 994 23 673 998 [OBJ] 33 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 666 994 23 658 998 [OBJ] 34 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 654 994 23 646 998 [OBJ] 36 \"green and grey plastic Doctor Doom action figure\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 623 994 23 611 994 [OBJ] 37 \"green and yellow wooden vegetable garden toy set\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 611 998 23 603 994 [OBJ] 38 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 599 998 23 587 998 [OBJ] 39 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 587 998 23 576 998 [OBJ] 41 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 572 998 23 560 998 [OBJ] 42 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 560 998 23 548 998 [OBJ] 43 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 548 998 23 537 998 [OBJ] 44 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 537 998 23 525 998 [OBJ] 7 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 478 998 23 466 998 [OBJ] 9 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 439 998 23 427 998 [OBJ] 10 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 427 998 23 416 998 [OBJ] 11 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 416 998 23 404 998 [OBJ] 12 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 404 998 23 392 998 [OBJ] 13 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 392 998 23 380 998 [OBJ] 14 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 384 994 23 373 994 [OBJ] 21 \"red and yellow wooden toy airplane with pilot\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 373 232 1 376 240 2 384 244 3 388 251 4 392 259 5 400 263 6 404 271 7 412 275 8 416 283 9 423 287 10 427 294 11 435 298 12 443 302 13 447 310 14 455 314 15 458 322 16 466 326 17 470 333 18 474 341 19 482 345 20 486 353 21 494 357 22 498 365 23 505 369 [OBJ] 22 \"blue and black hiking shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 380 240 1 384 248 3 392 255 4 396 263 5 400 271 6 404 279 7 408 287 8 412 294 9 416 302 10 419 310 11 423 318 12 431 322 13 435 330 14 439 337 15 443 345 16 447 353 17 451 361 18 458 365 19 462 373 20 466 380 21 470 388 22 474 396 23 482 400 [OBJ] 23 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 376 248 4 376 263 5 380 271 7 384 279 9 388 291 11 392 298 13 396 306 15 400 314 17 404 322 19 412 330 21 416 337 22 419 345 [OBJ] 24 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 376 255 2 380 263 4 384 271 6 388 279 8 392 287 10 396 294 12 400 302 14 404 310 16 408 318 18 412 326 20 419 333 22 423 341 [OBJ] 25 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 365 251 1 369 259 3 373 267 5 376 275 7 380 283 9 384 291 11 388 298 13 392 306 15 396 314 17 400 322 19 404 330 21 408 337 23 412 345 [OBJ] 26 \"dark grey and yellow running shoe\" 23 23 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 353 255 4 357 267 8 357 279 11 361 291 14 365 298 17 365 310 20 369 318 22 373 326 [OBJ",
"label": "[SCENE-START] [OBJS-START] [OBJ] 0 \"green wagon\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 212 549 2 216 550 4 219 552 7 221 553 8 223 554 9 224 555 11 226 556 13 227 557 15 230 558 18 233 560 20 235 561 22 237 563 [OBJ] 1 \"yellow sedan\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 296 568 2 297 570 3 300 571 5 301 573 6 303 574 7 305 575 9 307 577 12 310 577 13 312 579 15 314 582 17 316 584 19 317 586 21 318 589 22 319 590 [OBJ] 2 \"blue dirtbike\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 344 831 1 348 843 2 344 831 3 348 843 6 341 831 9 348 841 10 343 830 11 341 829 12 350 840 13 343 828 17 344 825 18 345 824 22 347 821 [OBJ] 3 \"red airliner\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 386 160 1 400 158 2 391 160 3 403 159 4 405 160 5 406 161 6 396 165 7 409 165 8 411 167 9 404 170 10 414 172 11 416 175 12 417 178 13 419 182 14 422 186 15 424 191 16 425 195 17 426 200 18 429 205 19 430 210 20 433 217 21 434 222 22 438 230 23 441 236 [OBJ] 4 \"green wagon\" 0 24 [VIS] 0 true [IN-FRAME] 0 true [PX] 0 710 942 1 697 939 2 690 935 3 687 932 4 749 930 5 737 926 6 723 923 7 707 920 8 699 917 9 687 914 10 679 910 11 669 905 12 657 898 13 644 891 14 632 883 15 619 876 16 608 869 17 596 863 18 587 857 19 576 851 20 566 845 21 555 839 22 544 832 23 536 825 [OBJS-END] [INTERACTIONS-START] [INTERACT] 1 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 0 \"collision\" 3 3 [IN] 0 1 [OUT] 0 1 [INTERACTIONS-END] [SCENE-END]\n",
"parse_debug": {
"pred_parse_valid": false,
"pred_parse_error": "Could not find [SCENE-END] after [SCENE-START].",
"pred_parse_error_type": "ValueError",
"guided_grammar_sha256": "501d31b980c1f9495be86a80917e58d24972e52c4d442eafe69e5c4fc7522dc6",
"guided_grammar_path": "strive_datasets/prep/kubric/scene_repr/profiles/grammars/selector_sentinel_no_3d.lark",
"debug_dir": "/gscratch/krishna/suryadv/strive_dynsuperclevr/runtime/36996079_inference_24/debug/test_24_shard_000/parse_failures/sample_000021",
"guided_grammar_debug_path": "/gscratch/krishna/suryadv/strive_dynsuperclevr/runtime/36996079_inference_24/debug/test_24_shard_000/parse_failures/sample_000021/grammar.ebnf.gz"
}
}Parsed prediction
{
"error": "ValueError: Could not find [SCENE-END] after [SCENE-START]."
}Parsed ground truth
{
"objects": [
{
"id": 0,
"type": "green wagon",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 212,
"y": 549
},
{
"t": 2,
"x": 216,
"y": 550
},
{
"t": 4,
"x": 219,
"y": 552
},
{
"t": 7,
"x": 221,
"y": 553
},
{
"t": 8,
"x": 223,
"y": 554
},
{
"t": 9,
"x": 224,
"y": 555
},
{
"t": 11,
"x": 226,
"y": 556
},
{
"t": 13,
"x": 227,
"y": 557
},
{
"t": 15,
"x": 230,
"y": 558
},
{
"t": 18,
"x": 233,
"y": 560
},
{
"t": 20,
"x": 235,
"y": 561
},
{
"t": 22,
"x": 237,
"y": 563
}
],
"birth": 0,
"death": 24
},
{
"id": 1,
"type": "yellow sedan",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 296,
"y": 568
},
{
"t": 2,
"x": 297,
"y": 570
},
{
"t": 3,
"x": 300,
"y": 571
},
{
"t": 5,
"x": 301,
"y": 573
},
{
"t": 6,
"x": 303,
"y": 574
},
{
"t": 7,
"x": 305,
"y": 575
},
{
"t": 9,
"x": 307,
"y": 577
},
{
"t": 12,
"x": 310,
"y": 577
},
{
"t": 13,
"x": 312,
"y": 579
},
{
"t": 15,
"x": 314,
"y": 582
},
{
"t": 17,
"x": 316,
"y": 584
},
{
"t": 19,
"x": 317,
"y": 586
},
{
"t": 21,
"x": 318,
"y": 589
},
{
"t": 22,
"x": 319,
"y": 590
}
],
"birth": 0,
"death": 24
},
{
"id": 2,
"type": "blue dirtbike",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 344,
"y": 831
},
{
"t": 1,
"x": 348,
"y": 843
},
{
"t": 2,
"x": 344,
"y": 831
},
{
"t": 3,
"x": 348,
"y": 843
},
{
"t": 6,
"x": 341,
"y": 831
},
{
"t": 9,
"x": 348,
"y": 841
},
{
"t": 10,
"x": 343,
"y": 830
},
{
"t": 11,
"x": 341,
"y": 829
},
{
"t": 12,
"x": 350,
"y": 840
},
{
"t": 13,
"x": 343,
"y": 828
},
{
"t": 17,
"x": 344,
"y": 825
},
{
"t": 18,
"x": 345,
"y": 824
},
{
"t": 22,
"x": 347,
"y": 821
}
],
"birth": 0,
"death": 24
},
{
"id": 3,
"type": "red airliner",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 386,
"y": 160
},
{
"t": 1,
"x": 400,
"y": 158
},
{
"t": 2,
"x": 391,
"y": 160
},
{
"t": 3,
"x": 403,
"y": 159
},
{
"t": 4,
"x": 405,
"y": 160
},
{
"t": 5,
"x": 406,
"y": 161
},
{
"t": 6,
"x": 396,
"y": 165
},
{
"t": 7,
"x": 409,
"y": 165
},
{
"t": 8,
"x": 411,
"y": 167
},
{
"t": 9,
"x": 404,
"y": 170
},
{
"t": 10,
"x": 414,
"y": 172
},
{
"t": 11,
"x": 416,
"y": 175
},
{
"t": 12,
"x": 417,
"y": 178
},
{
"t": 13,
"x": 419,
"y": 182
},
{
"t": 14,
"x": 422,
"y": 186
},
{
"t": 15,
"x": 424,
"y": 191
},
{
"t": 16,
"x": 425,
"y": 195
},
{
"t": 17,
"x": 426,
"y": 200
},
{
"t": 18,
"x": 429,
"y": 205
},
{
"t": 19,
"x": 430,
"y": 210
},
{
"t": 20,
"x": 433,
"y": 217
},
{
"t": 21,
"x": 434,
"y": 222
},
{
"t": 22,
"x": 438,
"y": 230
},
{
"t": 23,
"x": 441,
"y": 236
}
],
"birth": 0,
"death": 24
},
{
"id": 4,
"type": "green wagon",
"vis": [
{
"t": 0,
"v": true
}
],
"is_in_frame": [
{
"t": 0,
"v": true
}
],
"image_positions": [
{
"t": 0,
"x": 710,
"y": 942
},
{
"t": 1,
"x": 697,
"y": 939
},
{
"t": 2,
"x": 690,
"y": 935
},
{
"t": 3,
"x": 687,
"y": 932
},
{
"t": 4,
"x": 749,
"y": 930
},
{
"t": 5,
"x": 737,
"y": 926
},
{
"t": 6,
"x": 723,
"y": 923
},
{
"t": 7,
"x": 707,
"y": 920
},
{
"t": 8,
"x": 699,
"y": 917
},
{
"t": 9,
"x": 687,
"y": 914
},
{
"t": 10,
"x": 679,
"y": 910
},
{
"t": 11,
"x": 669,
"y": 905
},
{
"t": 12,
"x": 657,
"y": 898
},
{
"t": 13,
"x": 644,
"y": 891
},
{
"t": 14,
"x": 632,
"y": 883
},
{
"t": 15,
"x": 619,
"y": 876
},
{
"t": 16,
"x": 608,
"y": 869
},
{
"t": 17,
"x": 596,
"y": 863
},
{
"t": 18,
"x": 587,
"y": 857
},
{
"t": 19,
"x": 576,
"y": 851
},
{
"t": 20,
"x": 566,
"y": 845
},
{
"t": 21,
"x": 555,
"y": 839
},
{
"t": 22,
"x": 544,
"y": 832
},
{
"t": 23,
"x": 536,
"y": 825
}
],
"birth": 0,
"death": 24
}
],
"interactions": [
{
"id": 1,
"inputs": [
0,
65535
],
"outputs": [
0,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 2,
"inputs": [
1,
65535
],
"outputs": [
1,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 3,
"inputs": [
2,
65535
],
"outputs": [
2,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 4,
"inputs": [
4,
65535
],
"outputs": [
4,
65535
],
"begins_at": 0,
"ends_at": 23,
"type": "contact"
},
{
"id": 0,
"inputs": [
0,
1
],
"outputs": [
0,
1
],
"begins_at": 3,
"ends_at": 3,
"type": "collision"
}
]
}Complete per-sample evaluator report
{
"metrics": {
"avg_type_similarity": null,
"avg_size_diff": null,
"avg_3d_iou": null,
"avg_image_position_px_diff": null,
"avg_image_position_segmentation_acc": null,
"avg_image_position_segmentation_acc_small": null,
"avg_image_position_segmentation_acc_medium": null,
"avg_image_position_segmentation_acc_large": null,
"avg_image_position_segmentation_acc_first_frame": null,
"avg_image_position_segmentation_visible_acc": null,
"avg_quaternion_geodesic_deg": null,
"visibility_acc": null,
"visibility_true_acc": null,
"visibility_false_acc": null,
"is_in_frame_acc": null,
"is_in_frame_true_acc": null,
"is_in_frame_false_acc": null,
"vis_acc_from_px": null,
"vis_acc_from_px_true_acc": null,
"vis_acc_from_px_false_acc": null,
"num_matches": 0,
"num_unmatched_pred": 0,
"num_unmatched_target": 5
},
"supports": {
"avg_type_similarity": 0,
"avg_size_diff": 0,
"avg_3d_iou": 0,
"avg_image_position_px_diff": 0,
"avg_image_position_segmentation_acc": 0,
"avg_image_position_segmentation_acc_small": 0,
"avg_image_position_segmentation_acc_medium": 0,
"avg_image_position_segmentation_acc_large": 0,
"avg_image_position_segmentation_acc_first_frame": 0,
"avg_image_position_segmentation_visible_acc": 0,
"avg_quaternion_geodesic_deg": 0,
"visibility_acc": 0,
"visibility_true_acc": 0,
"visibility_false_acc": 0,
"is_in_frame_acc": 0,
"is_in_frame_true_acc": 0,
"is_in_frame_false_acc": 0,
"vis_acc_from_px": 0,
"vis_acc_from_px_true_acc": 0,
"vis_acc_from_px_false_acc": 0
},
"matching": {
"matches": [],
"unmatched_target_ids": [
0,
1,
2,
3,
4
],
"unmatched_pred_ids": [],
"target_type_counts": {
"green wagon": 2,
"yellow sedan": 1,
"blue dirtbike": 1,
"red airliner": 1
},
"pred_type_counts": {},
"assignment_backend": "scipy_dense",
"cost_functions": [
"mutual_type",
"same_seg"
],
"cost_weights": {
"mutual_type": 1.0,
"same_seg": 1.0
},
"num_frames": 24
},
"scene_metrics": {
"thresholds": [
0.5,
1.0,
2.0,
3.0
],
"counts_by_threshold": {
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"2.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"3.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"jaccard_by_threshold": {
"0.5": 0.0,
"1.0": 0.0,
"2.0": 0.0,
"3.0": 0.0
},
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_by_motion_group": {
"static": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"dynamic": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 2
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"unknown": {
"visibility_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"visibility_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"visibility_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"segmentation_counts_by_threshold": {},
"segmentation_precision_by_threshold": {},
"segmentation_recall_by_threshold": {},
"segmentation_f1_by_threshold": {},
"segmentation_frame_average_precision": null,
"segmentation_frame_average_recall": null,
"segmentation_frame_average_f1": null,
"segmentation_first_frame_counts": {},
"segmentation_first_frame_precision": null,
"segmentation_first_frame_recall": null,
"segmentation_first_frame_f1": null,
"segmentation_by_motion_group": {}
},
"per_match_metrics": [],
"interaction_metrics": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 5,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 5
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"interaction_supports": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"interaction_matching": {
"matches": [],
"unmatched_target_ids": [
0,
1,
2,
3,
4
],
"unmatched_pred_ids": [],
"target_type_counts": {
"collision": 1,
"contact": 4
},
"pred_type_counts": {},
"assignment_backend": "scipy_dense",
"cost_functions": [
"type_exact",
"temporal_span",
"input_object_set_mutual_jaccard",
"output_object_set_mutual_jaccard"
],
"cost_weights": {
"type_exact": 1.0,
"temporal_span": 1.0,
"input_object_set_mutual_jaccard": 0.5,
"output_object_set_mutual_jaccard": 0.5
}
},
"per_interaction_match_metrics": [],
"interaction_metrics_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 4,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"object_object": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 1,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_participants": {
"object_ground": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"object_object": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
},
"interaction_metrics_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 1,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 1
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
},
"contact": {
"avg_interaction_temporal_span_giou": null,
"avg_interaction_input_object_jaccard": null,
"avg_interaction_output_object_jaccard": null,
"avg_interaction_type_exact_match_acc": null,
"num_interaction_matches": 0,
"num_unmatched_pred_interactions": 0,
"num_unmatched_target_interactions": 4,
"interaction_temporal_iou_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_object_jaccard_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"interaction_temporal_iou_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_temporal_iou_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_temporal_iou_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_input_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_input_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_counts_by_threshold": {
"0.25": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.5": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"0.75": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
},
"1.0": {
"true_positives": 0,
"false_positives": 0,
"false_negatives": 4
}
},
"interaction_output_object_jaccard_precision_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_recall_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
},
"interaction_output_object_jaccard_f1_by_threshold": {
"0.25": 0.0,
"0.5": 0.0,
"0.75": 0.0,
"1.0": 0.0
}
}
},
"interaction_supports_by_type": {
"collision": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
},
"contact": {
"avg_interaction_temporal_span_giou": 0,
"avg_interaction_input_object_jaccard": 0,
"avg_interaction_output_object_jaccard": 0,
"avg_interaction_type_exact_match_acc": 0
}
}
}Exact evaluation configuration
{
"predictions_jsonl": "../inference_output.jsonl",
"dataset_json": "../test_data/movi_dump_sentinel_sharegpt.json",
"scene_root": "../test_data/movi_dump",
"output_json": "../evaluation_report.json",
"max_samples": null,
"num_workers": 1,
"show_progress": true,
"log_level": "INFO",
"visibility_filter_mode": "any_frame",
"pos2d_mode": "scaled_normalized",
"pos2d_scale": 1000.0,
"image_width": null,
"image_height": null,
"type_embed_model_name": "all-MiniLM-L6-v2",
"scene_similarity_thresholds": [
0.5,
1.0,
2.0,
3.0
],
"scene_segmentation_thresholds": [
0.25,
0.5,
0.75,
1.0
],
"matching": {
"costs": [
{
"name": "mutual_type",
"weight": 1.0
},
{
"name": "same_seg",
"weight": 1.0,
"missing_value": 1.0,
"kwargs": {
"only_use_first_frame": false
}
}
]
},
"interaction_matching": {
"costs": [
{
"name": "type_exact",
"weight": 1.0
},
{
"name": "temporal_span",
"weight": 1.0
},
{
"name": "input_object_set_mutual_jaccard",
"weight": 0.5
},
{
"name": "output_object_set_mutual_jaccard",
"weight": 0.5
}
]
},
"return_per_sample": true
}