Standard MOVi-E · 7577

{
  "messages": [
    {
      "role": "user",
      "content": "<video>\nAnalyze the provided video and return ONLY a valid sentinel sequence in this format:\n[SCENE-START] [OBJS-START]\n[OBJ] id \"type\" birth death [PX] t px py ...\n[OBJ] ...\n[OBJS-END]\n[INTERACTIONS-START]\n[INTERACT] id \"type\" begins_at ends_at [IN] in0 in1 ... [OUT] out0 out1 ...\n[INTERACT] ...\n[INTERACTIONS-END] [SCENE-END]\n\nConstraints:\n- Return only the sentinel text (no JSON, XML, markdown, comments, or extra prose).\n- Keep tags exactly as written:\n  `[SCENE-START] [OBJS-START] [OBJ] [PX] [OBJS-END] [INTERACTIONS-START] [INTERACT] [IN] [OUT] [INTERACTIONS-END] [SCENE-END]`\n- Use quoted strings for `type` (double quotes).\n- `PX` uses repeated groups of 3 values in order: `t px py`.\n  - `t` is the timestamp (frame number).\n  - `px` and `py` are the pixel coordinates of the object in that frame, normalized to [0,1] relative to the frame dimensions, scaled by 1000 and rounded to the nearest integer. For example, `px` of 0.1234 would be represented as 123.\n- When an object is occluded or not visible in a frame, `PX` may use `t null null`.\n- `IN` and `OUT` contain whitespace-separated integer ids."
    },
    {
      "role": "assistant",
      "content": "[SCENE-START] [OBJS-START] [OBJ] 0 \"black helmet with green and silver designs\" 0 24 [PX] 0 37 365 2 41 357 5 44 345 7 48 337 9 52 330 11 56 322 14 64 314 17 68 302 19 72 294 21 80 283 22 83 271 23 91 267 [OBJ] 1 \"light beige cardboard foundation box\" 0 24 [PX] 0 201 498 1 189 501 2 177 505 3 185 498 4 162 505 7 142 505 8 212 466 9 189 474 11 173 474 13 138 482 14 240 435 16 205 443 17 193 443 18 150 455 19 283 404 21 240 412 22 228 412 23 201 416 [OBJ] 2 \"orange and dark grey striped high-top athletic shoe\" 0 24 [PX] 0 244 263 2 248 255 3 259 244 6 263 236 7 275 228 9 279 220 10 287 216 12 298 205 14 310 197 15 318 193 16 326 189 17 333 185 18 341 177 19 349 173 20 357 169 21 365 166 22 376 162 [OBJ] 3 \"brown leather ankle boot\" 0 24 [PX] 0 298 154 1 314 154 2 326 150 3 322 142 4 333 138 6 341 126 7 345 119 8 353 115 9 357 107 10 365 103 11 376 99 12 384 91 13 392 87 14 400 83 15 423 83 17 431 72 18 443 68 19 455 64 20 462 60 21 478 60 22 490 56 23 498 52 [OBJ] 4 \"white ceramic bowl\" 0 24 [PX] 0 330 806 4 302 810 7 291 810 9 275 810 10 263 810 11 283 806 12 263 806 13 248 806 14 263 806 15 236 806 16 244 802 17 216 802 20 197 798 22 193 787 23 189 779 [OBJ] 5 \"white and red cardboard Netgear router box\" 0 24 [PX] 0 607 212 2 615 216 4 623 212 6 634 212 8 642 216 10 650 220 12 662 220 13 669 216 16 693 216 17 705 216 19 716 220 20 728 220 22 732 228 23 751 228 [OBJ] 6 \"colorful wooden toy xylophone\" 0 24 [PX] 0 935 166 1 931 189 2 931 212 3 919 255 4 900 306 5 923 341 6 900 388 7 869 423 8 896 439 9 748 462 10 876 517 11 837 552 12 748 568 13 748 619 14 708 654 15 701 716 16 689 724 17 666 744 18 646 759 19 623 779 20 599 787 21 576 751 22 560 759 23 541 814 [OBJ] 7 \"red fabric dishtowel\" 15 24 [PX] 0 null null 15 1 392 16 1 380 18 5 369 19 5 357 21 13 345 22 13 333 23 17 326 [OBJS-END] [INTERACTIONS-START] [INTERACT] 0 \"contact\" 0 23 [IN] 0 65535 [OUT] 0 65535 [INTERACT] 1 \"contact\" 0 23 [IN] 1 65535 [OUT] 1 65535 [INTERACT] 2 \"contact\" 0 23 [IN] 2 65535 [OUT] 2 65535 [INTERACT] 3 \"contact\" 0 23 [IN] 3 65535 [OUT] 3 65535 [INTERACT] 4 \"contact\" 0 23 [IN] 4 65535 [OUT] 4 65535 [INTERACT] 5 \"contact\" 0 23 [IN] 5 65535 [OUT] 5 65535 [INTERACT] 11 \"contact\" 0 23 [IN] 7 65535 [OUT] 7 65535 [INTERACT] 10 \"collision\" 6 6 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 6 \"collision\" 11 11 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 7 \"contact\" 12 13 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 8 \"collision\" 15 15 [IN] 6 65535 [OUT] 6 65535 [INTERACT] 9 \"contact\" 16 23 [IN] 6 65535 [OUT] 6 65535 [INTERACTIONS-END] [SCENE-END]"
    }
  ],
  "videos": [
    "/mmfs1/gscratch/krishna/suryadv/strive_fullvideo_reference_v2/data/kubric_strive/movi_e_drop_never_visible_scaled_no_3d_vis_in_frame_occ_null/videos/7577/frames.mp4"
  ]
}