Meta has launched 3D Gen, a new AI system that generates high-quality 3D assets from detailed text in less than 60 seconds. The technology marks a significant advance in AI-powered 3D content creation, potentially changing how developers and artists create virtual environments for games, VR and other applications.
The system is built on two main research papers: Meta 3D AssetGen and Meta 3D TextureGen. These studies detail novel approaches to 3D generation and texture refinement from text, which address long-standing challenges in the field.
Meta 3D Gen uses a two-step process to create 3D assets. The first step, powered by AssetGen, generates a 3D mesh with Physically Based Rendering (PBR) content from a text prompt. This step overcomes the difficult task of translating 2D concepts into 3D structures.
One Meta researcher explained, “AssetGen generates both shadow and albedo views of objects. This dual approach allows our system to more accurately estimate 3D texture and material properties, resulting in higher-quality renderings.” There are assets.”
The second step, run by TextureGen, refines the texture created in the first step. This step is critical to achieving the level of detail required for professional-grade 3D assets. TextureGen uses a combination of viewspace and UVspace generation techniques to create detailed textures while maintaining consistency in the 3D model.
Meta 3D Gen stands out not only for its speed, but for its ability to produce high-quality results across a wide range of prompts. The system can handle simple objects, complex characters, and complex scenes with remarkable fidelity to user instructions.
In tests against other industry-leading solutions, Meta 3D Gen consistently outperformed its competitors. Professional 3D artists and designers showed a clear preference for Meta 3D Gen's outputs, praising its instant fidelity and visual quality, especially for complex text detail.
The potential applications for this technology are vast. Game developers can use 3D Gen to rapidly prototype environments and characters, significantly speeding up the development process. Architectural visualization firms can create detailed 3D models of buildings and interiors from text descriptions, streamlining the design process. In the field of virtual and augmented reality, 3D Gen can enable the rapid creation of immersive environments and objects, potentially accelerating the development of metaverse applications.