index.html

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width,initial-scale=1" />
  <meta name="theme-color" content="#000000" />
  <link rel="stylesheet" href="./static/css/bulma.min.css">
  <link rel="stylesheet" href="Open-Sans.css">
  <link rel="stylesheet" href="index.css">
  <title></title>
  <script defer="defer" src="./static/js/main.cb41f6a5.js"></script>
  <link href="./static/css/main.4017e162.css" rel="stylesheet">
  <meta name="description"
        content="OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models">
  <title>OmniHuman-1 Project</title>
</head>

<body>
  <nav class="navbar" role="navigation" aria-label="main navigation">
    <div class="navbar-brand">
        <a role="button" class="navbar-burger" aria-label="menu" aria-expanded="false">
            <span aria-hidden="true"></span>
            <span aria-hidden="true"></span>
            <span aria-hidden="true"></span>
        </a>
    </div>
    <div class="navbar-menu">
        <div class="navbar-start" style="flex-grow: 1; justify-content: center;">

            <div class="navbar-item has-dropdown is-hoverable">
                <a class="navbar-link">
                    More Research
                </a>
                <div class="navbar-dropdown">
                    <a class="navbar-item" href="https://loopyavatar.github.io/">
                        Loopy
                    </a>
                    <a class="navbar-item" href="https://cyberhost.github.io/">
                        CyberHost
                    </a>
                    <a class="navbar-item" href="https://omnihuman-lab.github.io/">
                      OmniHuman
                  </a>
                </div>
            </div>
        </div>

    </div>
  </nav>

  <div id="root" class="column-flex">
    <div id="title-flex" class="column-flex">
      <h1> OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models </h1>
      <span>
        <a target="_blank" href="" onclick="return false;">Gaojie&nbsp;Lin</a><sup>*</sup>,
        <a href="https://jianwenjiang.github.io">Jianwen&nbsp;Jiang</a><sup>*†</sup>,
        <a target="_blank" href="" onclick="return false;">Jiaqi&nbsp;Yang</a><sup>*</sup>,
        <a href="https://zhengzerong.github.io/">Zerong&nbsp;Zheng</a><sup>*</sup>,
        <a target="_blank" href="" onclick="return false;">Chao&nbsp;Liang</a><sup></sup>
        
        <br />
      </span>
      <span>Bytedance</span>
      <span><sup>*</sup>Equal contribution,<sup>†</sup>Project lead
        <!-- ,<sup>‡</sup>Internship at Bytedance -->
      </span>
      <div class="flex flex-gap" style="margin-bottom:0.5em;">
        <a target="_blank" href="http://arxiv.org/abs/2502.01061" ><button>Paper</button></a>
<!-- 	      <a target="_blank" href="" onclick="alert('Coming Soon!');return false;"><button>Paper</button></a> -->
        <a target="_blank" href="https://omnihuman-lab.github.io"><button>Page</button></a>
      </div>
      <small><span><b>TL;DR</b>: We propose an end-to-end multimodality-conditioned human video generation framework named OmniHuman, which can generate human videos based on a single human image and motion signals (e.g., audio only, video only, or a combination of audio and video). In OmniHuman, we introduce a multimodality motion conditioning mixed training strategy, allowing the model to benefit from data scaling up of mixed conditioning. This overcomes the issue that previous end-to-end approaches faced due to the scarcity of high-quality data. OmniHuman significantly outperforms existing methods, generating extremely realistic human videos based on weak signal inputs, especially audio. It supports image inputs of any aspect ratio, whether they are portraits, half-body, or full-body images, delivering more lifelike and high-quality results across various scenarios.</span></small>
      <small><span><b>Currently, we do not offer services/downloads anywhere, nor do we have any SNS accounts for the project.</b></span></small>
      <small><span><b>Please be cautious of fraudulent information. We will provide timely updates on future developments.</b></span></small>
      <div class='responsive-image-container'>
        <img src='image/overall.png' alt='' />
      </div>
    </div>

    <div id="sections" class="column-flex">
      <h3>Generated Videos</h3>
        <p>
          OmniHuman supports various visual and audio styles. It can generate realistic human videos <strong>at any aspect ratio and body proportion (portrait, half-body, full-body all in one)</strong>, with realism stemming from comprehensive aspects including motion, lighting, and texture details.<br/>
          
        </p>
        <p class="styled-text">
          <b>*</b> Note that to generate all results on this page, <strong>only any single image and audio are required</strong>, except for the demo showcasing video and combined driving signals. For the sake of a clean layout, we have omitted the display of reference images, which are the first frame of the generated video in most cases. If you need comparisons or further information, please do not hesitate to contact us.
        </p>
        <div class="video-slider">
          <video src="video/main1.mp4"></video>
          <video src="video/main2.mp4"></video>
          <video src="video/main3.mp4"></video>
        </div>
        <div class="video-slider">
          <video src="video/talk2.mp4"></video>
          <video src="video/talk3.mp4"></video>
        </div>
        
        <!-- 
      <h3>Talking</h3>
        <p>OmniHuman can support input of any aspect ratio in terms of speech. It significantly improves the handling of gestures, which is a challenge for existing methods, and produces highly realistic results. The audio and images for some of the test cases are sourced from <a href="https://www.youtube.com/watch?v=5Jk8qITsqdM&t=127s&ab_channel=TEDxTalks">link1</a>, <a href="https://www.youtube.com/watch?v=ITxWUu6UcWQ&t=251s&ab_channel=TEDxTalks">link2</a>, <a href="https://www.youtube.com/watch?v=oO8w6XcXJUs&ab_channel=RealTimewithBillMaher">link3</a>, <a href="https://www.youtube.com/watch?v=oO8w6XcXJUs&ab_channel=RealTimewithBillMaher">link4</a>.</p>
        <div class="video-slider">
          <video src="video/talk2.mp4"></video>
          <video src="video/talk3.mp4"></video>
        </div>
        
 -->


      <h3>Diversity</h3>
        <p>In terms of input diversity, OmniHuman supports cartoons, artificial objects, animals, and challenging poses, ensuring motion characteristics match each style's unique features.</p>
        <div class="video-slider">
          <video src="video/div1.mp4"></video>
          <video src="video/div2.mp4"></video>
          <video src="video/div3.mp4"></video>
        </div>
        <div class="video-slider">
          <video src="video/div4.mp4"></video>
          <video src="video/div5.mp4"></video>
          <video src="video/div6.mp4"></video>
          <video src="video/div7.mp4"></video>
        </div>


      <h3>More Portrait Cases</h3>
        <p>Here, we also include a section dedicated to portrait aspect ratio results, which are derived from test samples in the <a href="https://celebv-hq.github.io/">CelebV-HQ</a>datasets.</p>
        <div class="video-slider">
          <video src="video/head1.mp4"></video>
          <video src="video/head2.mp4"></video>
          <video src="video/head3.mp4"></video>
        </div>
        <div class="video-slider">
          <video src="video/head4.mp4"></video>
          <video src="video/head5.mp4"></video>
          <video src="video/head6.mp4"></video>
        </div>

      
      <h3>Singing</h3>
        <p>OmniHuman can support various music styles and accommodate multiple body poses and singing forms. It can handle high-pitched songs and display different motion styles for different types of music. Please remember to select the highest video quality. The generated video quality also highly depends on the quality of the reference image.

        <div class="iframe-container">
          <iframe src="https://www.youtube.com/embed/XF5vOR7Bpzs?si=8xdu-gtd9AzUd615" allowfullscreen></iframe>
          <iframe src="https://www.youtube.com/embed/0cwvT-J7PcQ?si=tfnipoBtxDU0gcNH" allowfullscreen></iframe>
          <iframe src="https://www.youtube.com/embed/4w2AeGHy31Q?si=DQaeCWRj0V9SRm52" allowfullscreen></iframe>
          <iframe src="https://www.youtube.com/embed/1NU8NzvAxEg?si=tu3zNPGjUtdL1ag9" allowfullscreen></iframe>
        </div>


        <div class="iframe-container">
          <iframe src="https://www.youtube.com/embed/ns8kn5NGL44?si=5-uMWr431xZamSlY" allowfullscreen></iframe>
          <iframe src="https://www.youtube.com/embed/GGTcVOb2S9k?si=tLMOPd_1rOXXeO5x" allowfullscreen></iframe>
          <iframe src="https://www.youtube.com/embed/CVVNK06LVac?si=VVJTt1nHcKq_xq7n" allowfullscreen></iframe>
          <iframe src="https://www.youtube.com/embed/XV4E0-TVsvg?si=Q_936m0uHLW86cQk" allowfullscreen></iframe>
        </div>


      <h3>Compatibility with Video Driving</h3>
        <p>Due to OmniHuman's mixed condition training characteristics, it can support not only audio driving but also video driving to mimic specific video actions, as well as combined audio and video driving to control specific body parts like recent methods.


      <h3>Ethics Concerns</h3>
        <p>
          The images and audios used in these demos are from public sources or generated by models, and are solely used to demonstrate the capabilities of this research work. If there are any concerns, please contact us (jianwen.alan@gmail.com) and we will delete it in time. The template of this webpage is based on the one from <a href="https://www.microsoft.com/en-us/research/project/vasa-1/">VASA-1</a>,  and some test audios are from <a href="https://www.microsoft.com/en-us/research/project/vasa-1">VASA-1</a>,<a href="https://loopyavatar.github.io">Loopy</a>,<a href="https://cyberhost.github.io/">CyberHost</a>.
        </p>
      
      <h3>BibTeX</h3>
        <p>If you find this project useful for your research, you can cite us and check out our other related works:</p>
        <pre><code>
          @article{lin2025omnihuman1,
            title={OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models}, 
            author={Gaojie Lin and Jianwen Jiang and Jiaqi Yang and Zerong Zheng and Chao Liang},
            journal={arXiv preprint arXiv:2502.01061},
            year={2025}
          }
          
          @article{jiang2024loopy,
            title={Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency},
            author={Jiang, Jianwen and Liang, Chao and Yang, Jiaqi and Lin, Gaojie and Zhong, Tianyun and Zheng, Yanbo},
            journal={arXiv preprint arXiv:2409.02634},
            year={2024}
          }

          @article{lin2024cyberhost,
            title={CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention},
            author={Lin, Gaojie and Jiang, Jianwen and Liang, Chao and Zhong, Tianyun and Yang, Jiaqi and Zheng, Yanbo},
            journal={arXiv preprint arXiv:2409.01876},
            year={2024}
          }
        </code></pre> 

      <br/>
      <br/>
      <br/>
    </div>
  </div>
  <script src="index.js"></script>
  <script>
    function comming_soon_click() {
      alert('Comming soon!');
    }
    function TBD_click() {
      alert('TBD');
    }
  </script>
</body>


</html>