A garment is not a picture of a garment. It is a material, with weight, stretch and sheen.
Fabric has a weave or a knit, a sheen, a weight, some stretch and some stiffness. Those properties decide how it looks under light and how it hangs on a body, and they are only partly visible in a product photo.
Graphics has long modelled them explicitly. Reflectance models describe appearance, and cloth simulation describes motion and drape. Learning-based methods now estimate appearance from photographs and learn garment dynamics. We are interested in bringing that physical understanding into generative try-on, so that a heavy wool coat and a light silk blouse behave differently even when their product photos look alike.
Questions we are exploring
- Can we infer how a material behaves (weight, stiffness, stretch) from a product photo well enough to guide drape?
- How should texture be represented so it survives warping and generation intact?
- How do we measure texture fidelity in a way that matches what people perceive?
Further reading
- Deschaintre et al. Single-Image SVBRDF Capture with a Rendering-Aware Deep Network. ACM Transactions on Graphics (SIGGRAPH), 2018.
- Baraff & Witkin Large Steps in Cloth Simulation. SIGGRAPH, 1998.
- Santesteban, Otaduy & Casas SNUG: Self-Supervised Neural Dynamic Garments. CVPR, 2022.
- Grigorev et al. HOOD: Hierarchical Graphs for Generalized Modelling of Clothing Dynamics. CVPR, 2023.
- Ding et al. Image Quality Assessment: Unifying Structure and Texture Similarity. IEEE TPAMI, 2022.
