Vida Adeli, Soroush Mehraban, Jacob Rommann +3cs.CV
Generating co-speech gestures that are temporally coherent, semantically aligned with speech, and grounded with surrounding objects remains challenging. Prior speech-driven gesture models emphasize audio-gesture alignment but do not explicitly account for posture constraints or surrounding objects, failing to capture the inherent correlation between body gestures and the physical space. We present Puppeteer, a posture-aware, object-grounded co-speech gesture diffusion model operating in a causal latent space. We decompose long gestures into structured primitives and learn a causal variational autoencoder that encodes them into temporally ordered latent tokens, each depending only on the past. We then perform conditional diffusion directly in the causal latent space, conditioning on speech signals, motion history, an initial posture reference, and object geometry to synthesize physically consistent gestures. This temporally ordered latent formulation enables explicit temporal control and supports tasks such as gesture in-betweening and gesture completion. To better assess co-speech gesture synthesis beyond existing measures, we introduce new evaluation metrics tailored to this task. We also created SceneGes, the first curated synthetic 3D dataset of embodied co-speech gestures and corresponding 3D objects, enabling object-grounded gesture generation. Experiments show that Puppeteer generates more diverse and temporally synchronized gestures than prior methods, while enabling object-grounded gesture synthesis.
Quality control in smart manufacturing increasingly relies on data-driven methods, particularly deep learning, to automate the inspection of manufactured parts. Recent advances in three-dimensional (3D) metrology have enabled fine-scale assessment of dimensional accuracy, surface quality, and shape conformity. However, deep learning methods for point-cloud-based inspection require large volumes of labeled data covering part designs and defect types, which are costly and time-consuming to obtain. Moreover, defective parts are intrinsically rare in mass production, and the resulting class imbalance can degrade model performance and make rare defect types difficult to detect. Synthetic data generation (SDG) offers a promising approach to address these challenges by producing large, balanced, and fully annotated datasets. Yet, applying SDG to precision components requires representing part geometry and defect morphology parametrically, so that design and quality can be co-varied. This article describes MFGNet-Gear, a publicly available synthetic 3D dataset comprising 24,000 paired polygon meshes and point clouds across 12 gear designs and 4 quality classes, with 500 instances per design-quality combination. Gear geometries are generated with parametric computer-aided design software, with dimensional parameters perturbed by $\pm$0.0254 mm and defect parameters sampled from distributions representing defect morphologies. For each mesh, 100,000 points are uniformly sampled using Open3D and stored as N $\times$ 3 coordinate text files. Metadata labels identify the gear design and quality class, supporting part design classification, geometric defect detection, representation learning, and dataset benchmarking. MFGNet-Gear provides an open-source dataset for deep learning-based 3D metrology, with a reproducible generation pipeline extensible to additional part designs.