Research

Excellent and broad
AI expertise.

Machine Learning

(Probabilistic) Deep Learning

Statistical Relational AI

Computer Vision

Natural Language Processing

Robotics

Models of Higher Cognition

Psychology of Information Processing

Database Systems

Software Engineering

Distributed Systems

Hardware

Bioinformatics

Semantic Web

Sustainability

Medicine

Finance


620+ Publications

Artificial Intelligence Publications from our team.


170.000+ Citations

A growing number of citations in other publications.


AI Conferences

Program Chair at ACL, CVPR, UAI, CoRL, ECML PKDD among others.


Fellows

EurAI, IEEE, ELLIS among others.


Projects

hessian.AI initiates and participates in a lot of projects to drive cutting-edge research, promote interdisciplinarity, and foster the transfer of AI research to the broader community.

HMWK Cluster projects

LOEWE projects

A selection of projects in which hessian.AI is involved:


DFG


BMBF


Federal and State

  • ATHENE AVSV Automatic Vulnerability Scanning and Verification
  • ATHENE SenPai Security and Privacy in Artificial Intelligence

EU

Graduate School

With the hessian.AI Graduate School, that is currently being established, we combine the goal of promoting young scientists in the field of artificial intelligence in the best possible way, ensuring optimal conditions for doctoral studies, and continuously strengthening the international appeal of hessian.AI.

For this purpose, hessian.AI has established its own supervision concept as well as a qualification program. We see the following aspects as central to the doctoral phase in hessian.AI:

Connectom Networking Fund

The Connectom Networking Fund offers seed funding for collaborative research by hessian.AI members and other colleagues at and between the participating universities. In a competitive process, time-limited projects in the entire spectrum of research, teaching, education or application are funded with a maximum of 40,000 EUR per project.  The Connectom Networking Fund is announced once a year. The evaluation and selection of the project outlines to be funded is carried out by a selection committee consisting of representatives of all universities participating in hessian.AI. The submitted outlines will be evaluated on the basis of the following criteria:

Eligible to apply are employees of the universities participating in hessian.AI who can prove that they have at least completed a doctorate. A founding member of hessian.AI must be part of the applying consortium. External researchers can be involved in the project as cooperation partners, but cannot receive funding from the Connectom Networking Fund.

Funded projects in the third round of calls (2022):


Enabling multimodal and multilingual argument mining in court hearings

Dr. Ivan Habernal, TUDa, FB Computer Sciences
Prof. Dr. Christoph Burchard, Goethe University Frankfurt, Faculty of Law

This purposefully interdisciplinary project is designed to provide research for understanding legal reasoning in the hearings of the European Court of Human Rights. The ECtHR hearings are only available as video recordings with mixed languages, including native and non-native speakers. The goal is to create a basic dataset that will 1) serve NLP researchers as a basis for building and evaluating NLP legal models and 2) allow legal scholars to answer empirical legal questions related to the arguments presented before the Court.


Reliable Anonymization Methods for Enabling Language Understanding Models toAssist with Air Traffic Controller Communication

Charles Welch, Ph.D., PU Marburg, FB Computer Sciences
Dr. Ivan Habernal, TUDa, FB Computer Sciences

The goal of the project is to unlock highly sensitive operational speech data from air traffic controllers for research and development purposes, with the underlying goal of improving DFS service performance and safety via effective speech recognition-based controller assistance systems. The planned follow-up project in 2023-2025 aims to develop robust domain-specific speech recognition and semantic annotation models to automate the communication logging tasks of the traffic controllers.


KIDeR – KI basierte Detention von Rissbildung in Bahnschwellen | AI based detection of cracking in railroad sleepers

Prof. Dr. Michael Guckert, TH Mittelhessen, Department of Mathematics, Natural Sciences and Data Processing, FG Business Informatics – Artificial Intelligence
Prof. Dr. Gerd Manthei, TH Mittelhessen, Department of Mechanical and Power Engineering

The project is intended to investigate cracking in prestressed concrete sleepers in service using SE testing (acoustic emission measurements) in order to gain a better insight into the damage behavior. The analysis of the measurement rakes and their classification is a typical use case for Artificial Intelligence approaches (such as Deep Learning (DL) models). The aim of KIDeR is to develop a prototype of an efficient, non-destructive testing method for the detection of cracking in railroad sleepers in service. A particular challenge here is the differentiation of real acoustic emissions and ambient noise.


AI4BirdsDemo: A Demonstrator for Robust Bird Species Recognition in Arbitrary Sound Environments

Dr. Markus Mühling PU Marburg, FB Mathematics & Computer Science
Prof. Dr. Nina Farwig, PU Marburg, FB Biology
Prof. Dr. Bernd Freisleben, PU Marburg, Dept. of Mathematics & Computer Science, Distributed Systems and Intelligent Computing

The project relies on the preliminary work to extend the project AI4Birds by a software application that increases the public visibility of the work and demonstrates the quality of the bird species recognition model. The team will build a real-world bird species recognition model that tackles the previously mentioned challenges, such as out-of-distribution data, to improve the robustness of their bird species recognition approach.


Funded projects in the second round of calls (2022):


Accelerating Cardinality Estimation (ACE)

Prof. Dr.-Ing. Andreas Koch, TUDa, Department of Embedded Systems and Applications (ESA)
Prof. Dr. Carsten Binnig, TUDa, Data Management

Sum-Product-Networks (SPNs) belong to the class of graphical probabilistic models and allow the compact representation of multivariate probability distributions. While the ESA group has mainly investigated the acceleration possibilities of SPNs, the DM group has dealt with for which applications in the field of databases SPNs can be used. This includes, for example, cardinality estimation. It can be used to predict the result sizes of database queries and thus optimize the query processing of database management systems (DBMS). The overall goal of the project is to generally accelerate Cardinality Estimation using RSPNs (Relational SPNs), to automate the development and training process of RSPNs, and furthermore to investigate the potential usability in the context of large databases. The extension of the SPNC, as well as the provision of corresponding training processes, promise in combination highly interesting, practically relevant research results that can also be incorporated into the other projects in the two participating research areas.


AI4Birds: Bird Species Recognition in Complex Soundscape Recordings

Dr. Markus Mühling PU Marburg, FB Mathematics & Computer Science
Prof. Dr. Nina Farbig, PU Marburg, FB Biology
Prof. Dr. Bernd Freisleben, PU Marburg, Dept. of Mathematics & Computer Science, Distributed Systems and Intelligent Computing

In this project we focus on automatically recognizing bird species in audio recordings. To improve current biodiversity monitoring schemes, AI4Birds will use audio recordings across a forest ecosystem to develop novel transformer models based on self-attention for recognizing bird species in soundscapes. Thus, sustainability regarding biodiversity is at the heart of the project. Sustainability with respect to continuing AI4Birds by acquiring additional financial funding is very likely; it is planned to use AI4Birds to explore the funding opportunities within the federal biodiversity sustainability programs. Furthermore, we plan to contribute our results to Microsoft’s “AI for Earth” initiative.


AIQTHmed | AI Quality and Testing Hub in Healthcare

Prof. Dr. Martin Hirsch, Artificial Intelligence in Medicine, UMR and Director of the Institute for Artificial Intelligence at UKGM Marburg
Prof. Dr. Thomas Nauss, Department of Mathematics, Natural Sciences and Data Processing, FG Business Informatics – Artificial Intelligence, PU Marburg

In May 2021, the Hessian Minister for Digital Strategy and Development and the VDE agreed on the establishment of a first nationwide “AI Quality & Testing Hub” (AIQTH1). In the environment of hessian.AI and the Center for Responsible Digitalization (ZEVEDI), this is intended to promote the quality and trustworthiness of AI systems through standardization and certification in the model topic areas of “Mobility”, “Finance” and “Health”, making them verifiable and credible to the population. The aim of the project is to use the EU program DIGITAL EUROPE to strengthen the model topic area “Health” of the AIQTH of hessian.AI and thus the chance to establish this institution in Hesse.


Memristors – a central Hardware for AI

Prof. Dr. Lambert Alff, TUDa, FB Material Sciences
Prof. Dr.-Ing. Andreas Koch, TUDa, Department of Embedded Systems and Applications (ESA)
Prof. Dr. Christian Hochburgen, TUDa, FB ETIT

Artificial Intelligence will find its way almost ubiquitously into the most diverse areas of life. At the same time, however, this also means that the energy efficiency aspect of the associated computing effort will become increasingly important. Therefore, the development of the computer architectures used for AI is an important field of research. A major new component for AI-adapted computer architectures is the so-called memristor. There are several material science approaches that can be used to realize (highly energy-efficient) memristors, but these result in different device behavior or realistic application scenarios have not even been explored. This project aims to bring together the chain necessary for memristors from the material to the component to the circuit for specific applications of AI and to promote joint research projects in the sense of this interdisciplinary and holistic approach.


Mind the gap! Huddle between Materials and Computer Sciences

Prof. Dr. Leopoldo Molina-Luna, TUDa, FB Material Sciences
Prof. Dr. Kristian Kersting, TUDa, FB Computer Sciences

Many of the designers for AI algorithms don’t have the enough background knowledge to keep up with the state-of-the-art research in a natural science field as e.g. the materials sciences. On the other hand, the materials science researchers usually rely on an “educated guess” fashion for determining the parameters for developing AI algorithms and tolls, they pay little to none. There exists a knowledge gap between the computer science and materials science communities and more cross-talk on a fundamental level is needed. The project builds up a seeding platform for implementing and consolidating an inclusive regular exchange between all interested parties. It strengthens the preparation activities for an IRTG application in the field of on operando TEM for Memristors and ML-based data analysis routines.


Innovative UX for User-Centered AI Systems

Prof. Dr. Bernhard Humm, h_da, FB Computer Sciences
Prof. Dr. Andrea Krajewski, h_da, FB Media

Human-centered AI includes, among other things, the appropriate explanation of decisions or recommendations made by the AI system, e.g., by means of Machine Learning (keyword “Explainable AI”). User Experience (UX), on the other hand, is concerned with the development of products, especially IT systems, that are intended to provide the best possible user experience. In this project, innovative UX concepts will be designed, tuned, implemented and evaluated for three different prototype AI systems that are being developed within the BMBF-funded project “Competence Center for Work and Artificial Intelligence (KompAKI)”. One of the AI systems deals with the provision of Machine Learning (ML) for broad user groups with and without programming skills. Two AI systems are intended for operational use in the manufacturing industry (Industry 4.0). This project ideally complements other AI initiatives and promotes networking between hessian.AI partners and different disciplines.


Funded projects in the first round of calls (2021):


SpeedTram

Dr. Florian Stock, TUDa, Department of Mechanical Engineering, Department of Automotive Engineering (FZD)
Prof. Dr. Andreas Koch, TUDa, Department of Embedded Systems and Applications (ESA)

The focus of research on autonomous driving has so far clearly been on cars, but only a few projects have looked at other means of transport. To remedy this, Hessian.AI is funding innovative interdisciplinary research with the SpeedTram project, which focuses on autonomous/assisted driving of streetcars. In it, the Department of Automotive Engineering (FZD) and the Department of Embedded Systems and Applications (ESA) at TU Darmstadt are investigating the accelerated execution of machine learning algorithms required for automation in and of assistance systems for streetcars. Real data recorded during operation on a test vehicle of the local public transport company HEAG are processed. The evaluation of this growing data set, which now exceeds 140 TB, was no longer reasonably possible with existing methods. The work in SpeedTram made it possible to accelerate the two most time-consuming steps of data analysis, namely object recognition based on neural networks and the processing of LIDAR sensor data, by factors of three and 24, respectively. SpeedTram makes an important contribution to raising the innovation potential of automated streetcar guidance and making it usable for future applications.


AI4Bats: Recognizing Bat Species and Bat Behavior in Audio Recordings of Bat Echolocation Calls

Dr. Nicolas Frieß, PU Marburg, FB Geography, Environmental Informatics
Prof. Dr. Bernd Freisegen, PU Marburg, Dept. of Mathematics & Computer Science, Distributed Systems and Intelligent Computing
Prof. Dr. Thomas Nauss, PU Marburg, FB Geography, Environmental Informatics

Biodiversity is important for various ecosystem services that form the basis of human life. The current decline in biodiversity requires a transformation from manual periodic biodiversity assessment to automated real-time monitoring. Bats are one of the most widespread terrestrial mammal species and serve as important bioindicators of ecosystem health. Typically, bats are monitored by recording and analyzing their echolocation calls. In this project, AI4Bats, we present a novel AI-based approach to bat echolocation call detection, bat species recognition, and bat behavior detection in audio spectrograms. It is based on a neural transformer architecture and relies on self-attention mechanisms. Our experiments show that our approach outperforms current approaches for detecting bat echolocation calls and recognizing bat species in several publicly available datasets. While our model for detecting bat echolocation calls achieves an average precision of up to 90.2%, our model for detecting bat species achieves an accuracy of up to 88.7% for 14 bat species found in Germany, some of which are difficult to distinguish even for human experts. AI4Bats lays the foundation for breakthroughs in automated bat monitoring in the field of biodiversity, the potential loss of which is likely to be one of the most significant challenges facing humanity in the near future.


AI@School

Dr. Joachim Bille, TH Mittelhessen, Head of Department FTN
Prof. Dr. Michael Guckert, TH Mittelhessen, Department of Mathematics, Natural Sciences and Data Processing, FG Business Informatics – Artificial Intelligence
Prof. Holger Rohn, TH Mittelhessen, Department of Industrial Engineering and Management, FG Life Cycle Management & Quality Management, Makerspace Friedberg

The goal of the AI@School project was the development of a demonstrator for the vivid communication of basic knowledge of artificial intelligence, which should provide pupils with an early and low-threshold access to AI topics. On the one hand, the demonstrator should contain suitable examples and exhibits for the descriptive transfer of knowledge; on the other hand, an interactive introductory course for the transfer of knowledge should be developed using the exhibits and examples. Based on these offers, a prototypical teaching unit at the advanced course level will also be developed. The project results are to be implemented permanently at hessian.AI; in addition, a Hessian-wide transfer of the concept to suitable institutions in the other parts of the state is planned in the medium to long term.


Robot Learning of Long-Horizon Manipulation bridging Object-centric Representations to Knowledge Graphs

Prof. Dr. Georgia Chalvatzaki, TUDa, FB Informatik, iROSA: Robot Learning of Mobile Manipulation for Intelligent Assistance
Prof. Dr. Iryna Gurevych, TUDa, FB Computer Science, Ubiquitous Knowledge Processing Lab

The goal of this project was to investigate the links between high-level natural language commands and robot manipulation. Humans are able to effectively abstract and decompose natural language commands, e.g. “Make me a coffee”, but such an action is not detailed enough for a robot to execute. The task execution problem in robotics is usually approached as a task and motion planning problem, where a task planner decomposes the abstract goal into a set of logical actions that must be translated into actual actions in the world by a motion generator. The connection between abstract logical action and real-world description (e.g., in terms of the exact position of objects in the scene) makes task and motion planning a very challenging problem. In this project, we approached this problem from three different directions, looking at sub-problems of the topic with respect to our ultimate goal of learning long time horizon manipulation plans using human commonsense and scene graphs:

  1. The association of the object scene with robot manipulation plans using graph neural networks (GNNs) and RL,
  2. Using voice instructions and vision in transformer networks to output subgoals for a low-level planner, and
  3. Translating human instructions into robot plans.

Project results from 2. and 3. are scheduled to be published at a major machine learning conference in the near future. Work from iii will continue as part of a current collaboration between iROSA and UKP.