Research

• SENSORITHM Rhein-Main

• ATHENE AVSV Automatic Vulnerability Scanning and Verification
• ATHENE SenPai Security and Privacy in Artificial Intelligence

• ERC Advanced Grant | Mira Mezini
• ERC Advanced Grant | Iryna Gurevych
• ERC Consolidator Grant | Stefan Roth
• ERC Consolidator Grant | Heinz Koeppl
• ERC Consolidator Grant | Constantin Rothkopf
• ERC Proof-of-Concept Grant | Mira Mezini
• ERC Proof-of-Concept Grant | Jan Peters
• EIC Transition Grant | Jan Peters

Dr. Ivan Habernal, TUDa, FB Computer Sciences
Prof. Dr. Christoph Burchard, Goethe University Frankfurt, Faculty of Law

This purposefully interdisciplinary project is designed to provide research for understanding legal reasoning in the hearings of the European Court of Human Rights. The ECtHR hearings are only available as video recordings with mixed languages, including native and non-native speakers. The goal is to create a basic dataset that will 1) serve NLP researchers as a basis for building and evaluating NLP legal models and 2) allow legal scholars to answer empirical legal questions related to the arguments presented before the Court.

Charles Welch, Ph.D., PU Marburg, FB Computer Sciences
Dr. Ivan Habernal, TUDa, FB Computer Sciences

The goal of the project is to unlock highly sensitive operational speech data from air traffic controllers for research and development purposes, with the underlying goal of improving DFS service performance and safety via effective speech recognition-based controller assistance systems. The planned follow-up project in 2023-2025 aims to develop robust domain-specific speech recognition and semantic annotation models to automate the communication logging tasks of the traffic controllers.

Prof. Dr.-Ing. Andreas Koch, TUDa, Department of Embedded Systems and Applications (ESA)
Prof. Dr. Carsten Binnig, TUDa, Data Management

Sum-Product-Networks (SPNs) belong to the class of graphical probabilistic models and allow the compact representation of multivariate probability distributions. While the ESA group has mainly investigated the acceleration possibilities of SPNs, the DM group has dealt with for which applications in the field of databases SPNs can be used. This includes, for example, cardinality estimation. It can be used to predict the result sizes of database queries and thus optimize the query processing of database management systems (DBMS). The overall goal of the project is to generally accelerate Cardinality Estimation using RSPNs (Relational SPNs), to automate the development and training process of RSPNs, and furthermore to investigate the potential usability in the context of large databases. The extension of the SPNC, as well as the provision of corresponding training processes, promise in combination highly interesting, practically relevant research results that can also be incorporated into the other projects in the two participating research areas.

Dr. Markus Mühling PU Marburg, FB Mathematics & Computer Science
Prof. Dr. Nina Farbig, PU Marburg, FB Biology
Prof. Dr. Bernd Freisleben, PU Marburg, Dept. of Mathematics & Computer Science, Distributed Systems and Intelligent Computing

In this project we focus on automatically recognizing bird species in audio recordings. To improve current biodiversity monitoring schemes, AI4Birds will use audio recordings across a forest ecosystem to develop novel transformer models based on self-attention for recognizing bird species in soundscapes. Thus, sustainability regarding biodiversity is at the heart of the project. Sustainability with respect to continuing AI4Birds by acquiring additional financial funding is very likely; it is planned to use AI4Birds to explore the funding opportunities within the federal biodiversity sustainability programs. Furthermore, we plan to contribute our results to Microsoft’s “AI for Earth” initiative.

Prof. Dr. Martin Hirsch, Artificial Intelligence in Medicine, UMR and Director of the Institute for Artificial Intelligence at UKGM Marburg
Prof. Dr. Thomas Nauss, Department of Mathematics, Natural Sciences and Data Processing, FG Business Informatics – Artificial Intelligence, PU Marburg

In May 2021, the Hessian Minister for Digital Strategy and Development and the VDE agreed on the establishment of a first nationwide “AI Quality & Testing Hub” (AIQTH1). In the environment of hessian.AI and the Center for Responsible Digitalization (ZEVEDI), this is intended to promote the quality and trustworthiness of AI systems through standardization and certification in the model topic areas of “Mobility”, “Finance” and “Health”, making them verifiable and credible to the population. The aim of the project is to use the EU program DIGITAL EUROPE to strengthen the model topic area “Health” of the AIQTH of hessian.AI and thus the chance to establish this institution in Hesse.

Prof. Dr. Lambert Alff, TUDa, FB Material Sciences
Prof. Dr.-Ing. Andreas Koch, TUDa, Department of Embedded Systems and Applications (ESA)
Prof. Dr. Christian Hochburgen, TUDa, FB ETIT

Artificial Intelligence will find its way almost ubiquitously into the most diverse areas of life. At the same time, however, this also means that the energy efficiency aspect of the associated computing effort will become increasingly important. Therefore, the development of the computer architectures used for AI is an important field of research. A major new component for AI-adapted computer architectures is the so-called memristor. There are several material science approaches that can be used to realize (highly energy-efficient) memristors, but these result in different device behavior or realistic application scenarios have not even been explored. This project aims to bring together the chain necessary for memristors from the material to the component to the circuit for specific applications of AI and to promote joint research projects in the sense of this interdisciplinary and holistic approach.

Prof. Dr. Leopoldo Molina-Luna, TUDa, FB Material Sciences
Prof. Dr. Kristian Kersting, TUDa, FB Computer Sciences

Many of the designers for AI algorithms don’t have the enough background knowledge to keep up with the state-of-the-art research in a natural science field as e.g. the materials sciences. On the other hand, the materials science researchers usually rely on an “educated guess” fashion for determining the parameters for developing AI algorithms and tolls, they pay little to none. There exists a knowledge gap between the computer science and materials science communities and more cross-talk on a fundamental level is needed. The project builds up a seeding platform for implementing and consolidating an inclusive regular exchange between all interested parties. It strengthens the preparation activities for an IRTG application in the field of on operando TEM for Memristors and ML-based data analysis routines.

Prof. Dr. Bernhard Humm, h_da, FB Computer Sciences
Prof. Dr. Andrea Krajewski, h_da, FB Media

Human-centered AI includes, among other things, the appropriate explanation of decisions or recommendations made by the AI system, e.g., by means of Machine Learning (keyword “Explainable AI”). User Experience (UX), on the other hand, is concerned with the development of products, especially IT systems, that are intended to provide the best possible user experience. In this project, innovative UX concepts will be designed, tuned, implemented and evaluated for three different prototype AI systems that are being developed within the BMBF-funded project “Competence Center for Work and Artificial Intelligence (KompAKI)”. One of the AI systems deals with the provision of Machine Learning (ML) for broad user groups with and without programming skills. Two AI systems are intended for operational use in the manufacturing industry (Industry 4.0). This project ideally complements other AI initiatives and promotes networking between hessian.AI partners and different disciplines.

Dr. Florian Stock, TUDa, Department of Mechanical Engineering, Department of Automotive Engineering (FZD)
Prof. Dr. Andreas Koch, TUDa, Department of Embedded Systems and Applications (ESA)

The focus of research on autonomous driving has so far clearly been on cars, but only a few projects have looked at other means of transport. To remedy this, Hessian.AI is funding innovative interdisciplinary research with the SpeedTram project, which focuses on autonomous/assisted driving of streetcars. In it, the Department of Automotive Engineering (FZD) and the Department of Embedded Systems and Applications (ESA) at TU Darmstadt are investigating the accelerated execution of machine learning algorithms required for automation in and of assistance systems for streetcars. Real data recorded during operation on a test vehicle of the local public transport company HEAG are processed. The evaluation of this growing data set, which now exceeds 140 TB, was no longer reasonably possible with existing methods. The work in SpeedTram made it possible to accelerate the two most time-consuming steps of data analysis, namely object recognition based on neural networks and the processing of LIDAR sensor data, by factors of three and 24, respectively. SpeedTram makes an important contribution to raising the innovation potential of automated streetcar guidance and making it usable for future applications.

Dr. Nicolas Frieß, PU Marburg, FB Geography, Environmental Informatics
Prof. Dr. Bernd Freisegen, PU Marburg, Dept. of Mathematics & Computer Science, Distributed Systems and Intelligent Computing
Prof. Dr. Thomas Nauss, PU Marburg, FB Geography, Environmental Informatics

Biodiversity is important for various ecosystem services that form the basis of human life. The current decline in biodiversity requires a transformation from manual periodic biodiversity assessment to automated real-time monitoring. Bats are one of the most widespread terrestrial mammal species and serve as important bioindicators of ecosystem health. Typically, bats are monitored by recording and analyzing their echolocation calls. In this project, AI4Bats, we present a novel AI-based approach to bat echolocation call detection, bat species recognition, and bat behavior detection in audio spectrograms. It is based on a neural transformer architecture and relies on self-attention mechanisms. Our experiments show that our approach outperforms current approaches for detecting bat echolocation calls and recognizing bat species in several publicly available datasets. While our model for detecting bat echolocation calls achieves an average precision of up to 90.2%, our model for detecting bat species achieves an accuracy of up to 88.7% for 14 bat species found in Germany, some of which are difficult to distinguish even for human experts. AI4Bats lays the foundation for breakthroughs in automated bat monitoring in the field of biodiversity, the potential loss of which is likely to be one of the most significant challenges facing humanity in the near future.

Dr. Joachim Bille, TH Mittelhessen, Head of Department FTN
Prof. Dr. Michael Guckert, TH Mittelhessen, Department of Mathematics, Natural Sciences and Data Processing, FG Business Informatics – Artificial Intelligence
Prof. Holger Rohn, TH Mittelhessen, Department of Industrial Engineering and Management, FG Life Cycle Management & Quality Management, Makerspace Friedberg

The goal of the AI@School project was the development of a demonstrator for the vivid communication of basic knowledge of artificial intelligence, which should provide pupils with an early and low-threshold access to AI topics. On the one hand, the demonstrator should contain suitable examples and exhibits for the descriptive transfer of knowledge; on the other hand, an interactive introductory course for the transfer of knowledge should be developed using the exhibits and examples. Based on these offers, a prototypical teaching unit at the advanced course level will also be developed. The project results are to be implemented permanently at hessian.AI; in addition, a Hessian-wide transfer of the concept to suitable institutions in the other parts of the state is planned in the medium to long term.

Prof. Dr. Georgia Chalvatzaki, TUDa, FB Informatik, iROSA: Robot Learning of Mobile Manipulation for Intelligent Assistance
Prof. Dr. Iryna Gurevych, TUDa, FB Computer Science, Ubiquitous Knowledge Processing Lab

The goal of this project was to investigate the links between high-level natural language commands and robot manipulation. Humans are able to effectively abstract and decompose natural language commands, e.g. “Make me a coffee”, but such an action is not detailed enough for a robot to execute. The task execution problem in robotics is usually approached as a task and motion planning problem, where a task planner decomposes the abstract goal into a set of logical actions that must be translated into actual actions in the world by a motion generator. The connection between abstract logical action and real-world description (e.g., in terms of the exact position of objects in the scene) makes task and motion planning a very challenging problem. In this project, we approached this problem from three different directions, looking at sub-problems of the topic with respect to our ultimate goal of learning long time horizon manipulation plans using human commonsense and scene graphs:

1. The association of the object scene with robot manipulation plans using graph neural networks (GNNs) and RL,
2. Using voice instructions and vision in transformer networks to output subgoals for a low-level planner, and
3. Translating human instructions into robot plans.

Project results from 2. and 3. are scheduled to be published at a major machine learning conference in the near future. Work from iii will continue as part of a current collaboration between iROSA and UKP.