Researchers from RAI receive recognition at the CVPR

Researchers from the planned Cluster of Excellence “Reasonable Artificial Intelligence” (RAI) and hessian.AI took part in the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), held in Seattle (USA) from June 17 to 21. As the premier computer vision event, CVPR attracts more than 10,000 participants annually.

RAI research papers received awards

Several workshop papers presented by researchers from the RAI project received special recognition:

The publication “Is Synthetic Data All We Need? Benchmarking the Robustness of Models Trained with Synthetic Images”, won a Best Paper Award at the SyntaGen – Harnessing Generative Models for SyntheticVisual Datasets workshop. The paper, which examines the robustness of models trained with real images compared to models trained with synthetic images, was presented by Krishnakant Singh, PhD candidate at the Visual Inference Lab (VISINF) at TU Darmstadt. Singh was accompanied by Independent Research Group Leader Simone Schaub-Meyer and Professor Stefan Roth.

A Best Student Paper Award was awarded to Christoph Reich, who presented the paper “A Perspective on Deep Vision Performance with Standard Image and Video Codecs” at the AIS: Vision, Graphics and AI for Streaming Workshop.

At the GenAI Media Generation Challenge Workshop, the paper “LEDITS++: Limitless Image Editing using Text-to-Image Models” achieved an outstanding second place. LEDITS++ presents an efficient yet versatile and precise textual image manipulation technique. The LEDITS++ research team, represented by Manuel Brack, Felix Friedrich, and Patrick Schramowski, was rewarded with a Meta Quest 3 headset.

Scientists from the RAI cluster project shared their expertise

Professor Justus Thies delivered two keynote speeches: “Digital Humans: Recent Advances in Capturing and Controlling Digital Humans – from 2D DeepFakes to volumetric avatars that can be used in immersive telepresence,” was presented at the Workshop on Media Forensics. The Populating Empty Cities – Virtual Humans for Robotics and Autonomous Driving workshop featured his keynote “Capturing and Controlling Digital Humans: Text-based appearance and motion control of volumetric human avatars using diffusion-based generative models.”

Professor Stefan Roth gave an Invited Talk on accelerating and evaluating explanation methods in computer vision during the SAIAD 2024 – Safe Artificial Intelligence for All Domains workshop.

Professor Kristian Kersting joined the conference remotely to present on Reasonable Artificial Intelligence (RAI) during the 3rd Explainable AI for Computer Vision (XAI4CV) Workshop. Kersting’s talk emphasized the need for both explainable and interpretable AI, illustrated by several examples.

Manuel Brack, researcher at the German Research Center for Artificial Intelligence (DFKI) and PhD candidate at the Artificial Intelligence & Machine Learning Lab at TU Darmstadt, presented an Invited Talk on “Enhancing Visual Content Safety: Multimodal Approaches for Dataset Curation and Model Safeguarding” as part of the Workshop on Multimodal Content Moderation (MMCM).

With six conference papers and five workshop papers, the researchers from the RAI project and hessian.AI presented a total of 11 papers at CVPR 2024. An outstanding achievement, considering that from 11,532 paper submissions, only 2,719 (23.6%) were accepted. Topics ranged from human face and body modeling and motion synthesis to text-driven image editing. “Our various contributions to CVPR 2024 show that we are part of the rapid development in computer vision. The enormous potential of this technology has been demonstrated by the diversity of the research presented,” remarks Simone Schaub-Meyer, Independent Research Group Leader, TU Darmstadt.

List of papers published at the CVPR 2024 main conference:

Visual Inference Lab (VISINF), led by Prof. Stefan Roth

Adapters Strike Back, Jan-Martin O. Steitz, Stefan Roth. Learn more

3D Graphics & Vision Group (GraVis), led by Prof. Justus Thies

DPHMs: Diffusion Parametric Head Models for Depth-based Tracking, Jiapeng Tang, Angela Dai, Yinyu Nie, Lev Markhasin, Justus Thies, Matthias Nießner. Learn more

SCULPT: Shape-Conditioned Unpaired Learning of Posedependent Clothed and Textured Human Meshes, Soubhik Sanyal, Partha Ghosh, Jinlong Yang, Michael J. Black, Justus Thies, Timo Bolkart . Learn more

HAAR: Text-Conditioned Generative Model of 3D Strand-based Human Hairstyles, Vanessa Sklyarova, Egor Zakharov, Otmar Hilliges, Michael J. Black, Justus Thies. Learn more

DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis, Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, Matthias Nießner. Learn more

FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models, Shivangi Aneja, Justus Thies, Angela Dai, Matthias Nießner. Learn more

List of papers presented at CVPR 2024 workshops:

Artificial Intelligence and Machine Learning Lab, led by Prof. Kristian Kersting

LEDITS++: Limitless Image Editing using Text-to-Image Models, Manuel Brack, Felix Friedrich, Katharina Kornmeier, Linoy Tsaban, Patrick Schramowski, Kristian Kersting, Apolinário Passos. Learn more
LLAVAGUARD: VLM-based Safeguard for Vision Dataset Curation and Safety Assessment, Lukas Helff, Felix Friedrich, Manuel Brack, Kristian Kersting, Patrick Schramowski (presented only). Learn more

Visual Inference Lab (VISINF), led by Prof. Stefan Roth

Boosting Unsupervised Semantic Segmentation with Principal Mask Proposals, Oliver Hahn, Nikita Araslanov, Simone Schaub-Meyer, Stefan Roth. Learn more
A Perspective on Deep Vision Performance with Standard Image and Video Codecs, Christoph Reich, Oliver Hahn, Daniel Cremers, Stefan Roth, Biplob Debnath. Learn more
Is Synthetic Data All We Need? Benchmarking the Robustness of Models Trained with Synthetic Images, Krishnakant Singh, Thanush Navaratnam, Jannik Holmer, Simone Schaub-Meyer, Stefan Roth. Learn more

About the CVPR

Sponsored by the IEEE Computer Society (CS) and the Computer Vision Foundation (CVF), the Conference on Computer Vision and Pattern Recognition (CVPR) is the preeminent annual computer vision conference for students, academics, and industry researchers. It showcases significant advancements in all areas of computer vision and pattern recognition, as well as their impact on different fields and industries. This year’s conference attracted more than 12,000 scientists and engineers from 76 countries and regions. Featuring the main conference, an expo, numerous co-located workshops, and a variety of short courses, CVPR provides ample opportunity for networking, recruiting, inspiration, and motivation. Learn more