4DAnyone Turns One Video Into a Moving 3D Person

How It Works

4DAnyone first estimates a moving 3D skeleton from the source video. It renders that skeleton from target angles, giving its video model a guide for where the person’s body should appear in each new view.

The model then generates video from 16 viewpoints around the person. Two new techniques help those views stay consistent. One compresses earlier views into a fixed reference, while the other lets groups of views exchange information as the video takes shape.

Those generated videos become the input for 4D Gaussian splatting. This represents the person with many tiny colored points that move over time, so a viewer can watch the performance from a freely chosen angle.

Results

The researchers tested 4DAnyone on 10 scenes from the DNA-Rendering benchmark and three from DyMVHumans. It led every reported image-quality and consistency measure across both datasets when compared with MV-Performer, TrajectoryCrafter, and a fine-tuned version of ReCamMaster.

The official demonstrations also include ordinary portrait videos recorded outside a capture studio. The paper presents those examples as evidence of broader use, but they do not have the ground-truth camera views needed for the same numerical comparison.

Release Details

The public code currently expects a portrait video showing one full or upper body, at least 121 frames, and only mild camera movement. The team lists support for lower-memory inference below 32 GB and a complete open-source 4D reconstruction path as future work.

The full research pipeline is still compute-heavy. The paper reports about two minutes to prepare a clip on an RTX 4090, about seven minutes to generate each four-video group on one H20 GPU, and about 30 minutes to train the final 4D scene on an RTX 4090.

Loose clothing can change across views because a body skeleton does not describe flowing fabric. An incorrect pose estimate also carries into every generated angle. The authors separately warn that realistic human synthesis can enable deepfakes and call for consent and clear disclosure.

For creators, the released model makes the first part available now. One simple video can become a ring of consistent moving views, with a more accessible end-to-end 4D workflow as the next step.