Standard MOVi-E · 2162

{
  "messages": [
    {
      "role": "user",
      "content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n  `[SCENE-START] [OBJS-START] [OBJ] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n  - `t` is the timestamp (frame number).\n  - `px` and `py` are the pixel coordinates of the object in that frame, normalized to [0,1] relative to the frame dimensions, scaled by 1000 and rounded to the nearest integer. For example, `px` of 0.1234 would be represented as 123.\n- When an object is occluded or not visible in a frame, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids."
    },
    {
      "role": "assistant",
      "content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"caramel suede bootie\" 0 24 [PX] 0 29 533 3 29 521 5 29 412 15 37 416 23 41 423 [OBJ] 1 \"red and green wooden bird rattle\" 0 24 [PX] 0 99 48 1 138 52 2 169 72 3 208 107 4 236 134 5 271 181 6 298 220 7 326 267 8 353 310 9 357 302 10 380 302 11 380 357 13 361 361 14 337 376 15 322 376 16 279 361 17 267 373 18 244 388 19 228 388 20 205 435 22 181 435 23 154 435 [OBJ] 2 \"light and dark blue running shoe\" 0 24 [PX] 0 103 798 1 99 771 6 99 759 7 95 728 11 95 716 14 95 701 17 95 689 19 91 642 20 95 669 21 91 630 23 91 619 [OBJ] 3 \"wooden pull toy train\" 0 24 [PX] 0 310 205 1 318 212 4 322 224 5 333 232 6 373 224 7 275 216 8 283 240 12 279 259 13 244 244 14 232 251 15 212 251 16 283 224 17 291 232 18 408 259 19 400 271 20 392 283 21 365 287 22 353 287 [OBJ] 4 \"grey plastic Super Mario Kart game cartridge\" 0 24 [PX] 0 470 876 4 466 865 7 474 861 8 501 857 9 568 857 10 666 861 11 525 841 12 591 841 13 486 826 14 552 826 15 650 830 16 517 810 17 615 814 18 486 794 19 583 798 20 451 779 21 548 783 22 650 787 23 513 767 [OBJ] 5 \"glittery rose gold leather ballet flat\" 0 24 [PX] 0 787 521 14 794 517 17 783 513 20 759 505 [OBJ] 6 \"white and purple leather sneaker\" 0 24 [PX] 0 837 392 2 833 400 3 841 392 5 837 400 9 833 408 12 841 404 13 833 416 [OBJ] 7 \"Wii Party U game box with controller\" 0 24 [PX] 0 849 33 1 818 41 2 798 56 3 775 76 4 740 103 5 708 166 6 677 232 7 646 310 8 623 392 9 595 470 10 611 482 11 595 486 12 544 494 13 529 505 14 517 521 15 498 541 16 517 548 17 482 556 19 470 568 20 490 568 21 458 572 23 447 572 [OBJ] 8 \"white and red cardboard facial hair dye box\" 0 24 [PX] 0 904 126 1 853 146 2 771 158 3 728 189 4 689 228 5 630 267 6 685 119 7 591 162 8 552 169 9 501 177 10 458 177 11 416 169 12 376 177 13 474 275 14 447 279 15 419 283 16 396 291 17 373 287 18 353 283 19 330 279 20 169 330 21 150 330 22 134 341 23 123 353 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 5 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 6 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 7 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 8 \"contact\" 0 23 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 9 \"contact\" 0 23 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 18 \"collision\" 7 7 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 1 \"collision\" 8 8 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 19 \"collision\" 9 9 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 12 \"collision\" 9 10 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 2 \"contact\" 9 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 3 \"collision\" 11 11 [IN] 1 8 [OUT] 1 8 [INTERACT] 13 \"collision\" 11 12 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 4 \"contact\" 12 13 [IN] 1 8 [OUT] 1 8 [INTERACT] 10 \"collision\" 14 14 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 14 \"collision\" 14 14 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 15 \"contact\" 15 17 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 11 \"contact\" 15 23 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 16 \"collision\" 22 22 [IN] 8 65535 [OUT] 8 65535 [INTERACT] 17 \"contact\" 23 23 [IN] 8 65535 [OUT] 8 65535 [INTERACTIONS-END] [SCENE-END]"
    }
  ],
  "videos": [
    "/mmfs1/gscratch/krishna/suryadv/strive_fullvideo_reference_v2/data/kubric_strive/movi_e_drop_never_visible_scaled_no_3d_vis_in_frame_occ_null/videos/2162/frames.mp4"
  ]
}