The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
The Hessian AICon | July 4–6 | SAVE THE DATE
Department receives two new LOEWE professorships
2023/02/21 by HMWK/Michaela Hütig
The Technical University of Darmstadt and hessian.AI continue to expand its leading international position in artificial intelligence (AI) research: hessian.AI and TU Darmstadt receive two new LOEWE professorships for multimodal learning. One LOEWE Top Professorship is awarded to Dr Marcus Rohrbach, who is at the same time taking up his Humboldt Professorship at TU Darmstadt, and one LOEWE Start Professorship to Dr Anna Rohrbach. Both LOEWE professorships are funded with funds from the LOEWE research programme of the State of Hesse totalling five million euros.
The new LOEWE professorships will be established at hessian.AI and expand its AI research expertise. The appointment of the AI researchers effective 1 September is a great success for the TU Darmstadt and hessian.AI. With their professorships in “Multimodal Grounded Learning” and “Multimodal Reliable Artificial Intelligence”, Marcus and Anna Rohrbach are strengthening a research field in which the Hessian Center at TU already plays a leading international role.
The two researchers will pioneer the research goal of developing intelligent systems in harmony with the European value system at the hessian.AI and the Department of Computer Science – both at university level and across the country and Europe.
Research at the interface of image recognition and language
Marcus Rohrbach is moving from AI research at Facebook parent company Meta in California to Darmstadt and will be conducting basic research in the field of multimodal learning at the interface of image recognition and language as part of his LOEWE Excellence Professorship. He will advance AI models that learn from uni- and multi-modal data and have a wide range of functions. The aim is to develop a dependably acting and thus trustworthy AI that can communicate with humans and support them in all tasks and scenarios. Marcus Rohrbach had already been awarded a Humboldt Professorship at TU Darmstadt by the Alexander von Humboldt Foundation for his research in November last year. He now accepted this appointment.
TU President Professor Tanja Brühl remarked that with Marcus Rohrbach, the TU Darmstadt would welcome a pioneering colleague to a Multimodal Reliable Artificial Intelligence professorship. “As an internationally recognized top researcher, he has a remarkable profile at the interfaces of various sub-fields of AI”, she emphasized. “He will be an outstanding addition to the AI expertise at TU Darmstadt and in hessian.AI and will significantly promote cooperation in a wide range of networks. After being awarded an Alexander von Humboldt Professorship, Marcus Rohrbach now receives a LOEWE Excellence Professorship – an outstanding success!”
Develop AI models with human-like abilities
Anna Rohrbach moves from the University of California (UC) Berkeley to Darmstadt and will provide innovative contributions at the interface of image recognition (computer vision) and natural language processing (NLP) as part of her LOEWE Start Professorship. She will develop and research AI models that are supposed to possess abilities similar to those of humans. This multimodal AI should be able to communicate with humans, be anchored in reality and learn from language.
TU President Brühl explained: “With Anna Rohrbach, we are gaining an internationally visible and up-and-coming researcher for a professorship in Multimodal Grounded Learning at TU Darmstadt and in hessian.AI. With her innovative contributions at the interface of image recognition and Natural Language Processing, Anna Rohrbach’s work perfectly complements our Information and Intelligence research field in the area of Third Wave AI research. I am very much looking forward to working with an inspiring colleague!”
“Two outstanding researchers for the department”
Marcus Rohrbach’s LOEWE Top Professorship runs for five years and is endowed with a total of around 4.5 million euros. The share of LOEWE funding is three million euros, with the TU Darmstadt bearing the remaining part. Anna Rohrbach’s professorship is worth a total of around 4.1 million euros for the six-year period. LOEWE funding comprises two million euros, with the TU Darmstadt contributing the remaining share.
The Dean of the Department of Computer Science at TU Darmstadt, Felix Wolf, said: “We are delighted to attract two outstanding researchers to the department in Anna and Marcus Rohrbach. Close collaboration on existing and new projects gives our strong AI research focus further tailwind to realise our vision of reliable AI. Our students will also benefit from more hands-on AI courses.”
Marcus Rohrbach
After completing his doctorate at the Max Planck Institute for Computer Science in Saarbrücken, Marcus Rohrbach first conducted research at UC Berkeley and from 2017 at Facebook AI Research (FAIR). His research results are published in several AI fields at major conferences and by the most important journals for computer vision, computational linguistics, machine learning, and artificial intelligence. He also won various scientific competitions.
Anna Rohrbach
Anna Rohrbach received her PhD from Saarland University in 2017 and then moved to UC Berkeley. Her research focuses on artificial intelligence, machine learning, computer vision and natural language processing multimodal learning. She won various scientific prizes and awards and published her research results in renowned journals and at major conferences.
More information about LOEWE
More information about TU Darmstadt
About Prof. Dr. Lucie Flek
Prof. Dr. Lucie Flek is a professor in the Department of Mathematics and Computer Science at Philipps University Marburg, where she leads the Conversational AI and Social Analytics (CAISA) Lab.
Prior to her appointment in Marburg, Flek studied computer science at the Faculty of Nuclear Physics in Prague and worked at the Large Hadron Collider in CERN. Her expertise in big data processing then took her to Google, where the scientist turned to machine understanding of language.
From there, it was back to science: she received her PhD in 2016 at TU Darmstadt. Flek was a Visiting Researcher at the University of Pennsylvania, USA, and a Research Fellow at University College London, England.
She then worked for three years as a Research Program Manager in Amazon’s Alexa AI program, then as an Associate Professor of AI in Mainz, Germany. Flek has been in Marburg since 2021.
Research at the interface of psychology and AI
Prof. Dr. Lucie Flek has an impressive resume: she has lived, studied, worked and researched in seven countries, been at tech giants such as Google or Amazon, and combines expertise on nuclear physics, psychology, computer science and artificial intelligence. Now, the scientist researches artificial intelligence, language and social behavior.
“Language is even more fun than physics because it depends on how people behave. It’s often very irrational and not everything follows rules like in physics,” Flek says. That’s why she moved from analyzing big data at the massive CERN particle accelerator to Big Data processing at Google.
Working on language is interdisciplinary, according to Flek: “To really understand what a sentence means, you need additional information about human behavior and the social context of a statement.
Flek left her jobs at Google and Amazon to pursue her passion for why questions in academia. Tech companies offer access to interesting data and plenty of computing power, but there is little time for details and in-depth questions.
In Marburg, Flek is now combining all the knowledge she has absorbed so far in her varied career path. She works at the intersection of machine learning, classical computer science, language, the psychology and sociology of social media, and human-computer interaction.
hessian.AI has been a great help to Flek in building a research network and in outreach. For example, one of her post-docs is directly supported by the center. In addition, hessian.AI creates access to industry, where Flek’s AI agents will help improve communication between stakeholders.
When universities need to compete with industry
hessian.AI could also provide computing resources. That’s key because natural language machine processing has changed radically in recent years: Big companies like Google and OpenAI are dominating the increasingly commercial AI industry with their models. Big-tech models are getting bigger almost by the month and increasingly opaque as companies withhold details about training processes.
Flek sees this development as a major challenge, saying that as a university, as a nation, and even as Europe, it is difficult to keep up with the infrastructure of the U.S. industry.
At the same time, she says, it is also the role of science to challenge these large industrial applications. There are numerous ethical issues, such as in what areas large models can be usefully applied and where risks lie. Research needs to ensure that large AI models become better and useful for society.
Empathic computer and sarcasm analysis
In her CAISA lab, the scientist is researching two major topics: Better conversational agents and AI systems that better understand what people write on social media.
Specifically, her team is developing conversational agents, for example, that understand people more empathetically and serve as AI tutors for people giving advice to others. Such systems could help people in teaching, counseling or medicine, for example, better predict student or patient reactions.
Medical students could interact with an AI patient and learn how to talk to sufferers about a serious diagnosis in an emergency.
In social media analysis, Flek focuses on better understanding people, “People don’t always behave the same way. You have to know more about a person, and who they’re talking to, to interpret a sentence correctly.”
The same sentence can mean very different things, making it difficult to evaluate a statement. Context helps here: sarcasm, for example, occurs more often among friends or in political affinity groups.
AI systems that understand us better contribute to more transparent and natural interactions between humans and computers, Flek says. But also between humans and humans.
About Prof. Dr. Bernhard Humm
Prof. Dr. Bernhard Humm researches and teaches at the Department of Computer Science at Darmstadt University of Applied Sciences. He studied at the University of Kaiserslautern and earned his doctorate at the University of Wollongong in Australia.
After working in industry, Humm was appointed professor at Darmstadt University of Applied Sciences, where he still teaches and researches software development, architecture and artificial intelligence.
With holistic research and development to better AI systems
Industry, culture, economy: Humm speaks of a mix of sectors when he describes his application-oriented research, for which he was awarded the Wissenschaftspreis of Darmstadt University of Applied Sciences in 2019 and the Hessischer Forschungspreis in 2014.
The computer scientist develops software applications for various areas of society, such as an AI application for doctors treating cancer patients, a system that identifies faults early on in smart factories, or a platform for art education in the digital space.
For Humm, the benefit for users of AI systems plays the central role: “It is not important to use the most modern technology, but to provide real benefits.” A particular concern for Humm is to keep an eye on the effects of AI systems on society.
Humm therefore teaches and conducts interdisciplinary research, for example with scientists from the fields of philosophy, social sciences and law. He wants to think and develop AI systems holistically, understand them better, explain them and – as a result – use them more effectively.
Humm sees various technical and social dimensions in AI systems, which he would like to reconcile. In 2021, he and an interdisciplinary team published a book on technology assessment and artificial intelligence that offers plenty of food for thought in this regard: “The more Data Science enters research, the more the search for causality is replaced by the search for correlations. This is changing the way science is done.”
The focus is always on people
In his research and teaching on AI systems, the computer scientist always sensitizes people to their ability to act. AI systems can reveal correlations that cannot be recognized independently by specialists. This requires transparent explanations of how recommendations for action are arrived at.
Humm develops software that facilitates the diagnosis of illnesses in hospitals, for example, or accompanies patients in psychotherapies. For Humm, it is clear that AI systems can support therapists, doctors and air traffic controllers in their decision-making, but they cannot and must not replace human expertise and final judgment.
Application-oriented research is so demanding because many dimensions have to be served simultaneously: high benefit for users and society, economic feasibility, technological quality, and scientific depth.
“Uncertainty is the biggest challenge” – How robot-human interactions can work
About Prof. Dr. Georgia Chalvatzaki
Prof. Dr. Georgia Chalvatzaki has been an Assistant Professor at TU Darmstadt since early 2022, where she leads the iROSA group for “Robot Learning of Mobile Manipulation for Assistive Robotics”.
Chalvatzaki completed her PhD studies in 2019 at the “Intelligent Robotics and Automation Lab” at the Electrical and Computer Engineering School of the National Technical University of Athens in Greece and then came to Darmstadt for a postdoctoral position in the “Intelligent Autonomous Systems” group.
In 2021, Chalvatzaki received funding from the prestigious Emmy Noether Program (ENP) of the German Research Foundation (DFG) and started as an independent research group leader.
Robotic visions and their reality
For decades, robotics research has wanted to bring robots into our everyday lives. In the process, it has made enormous progress in recent years. But despite all the progress, there is a huge gap between the visionary goals of robotics and current reality: many tasks that are commonplace for humans are still difficult or impossible for robots to perform.
Chalvatzaki wants to close this gap: robots should use learning algorithms to solve complex tasks in real-world environments while coming to grips with the uncertainty that comes with working with humans.
The scientist is conducting research at the interface between machine learning and classical robotics. Her goal is AI assistants embodied in robots. To this end, Chalvatzaki is working with mobile manipulator robots, which are robotic arms on wheels equipped with numerous sensors.
Intelligent robot assistants could help in nursing care, for example: Germany alone has a shortage of nearly 100,000 caregivers, says Chalvatzaki. Many elderly people are cared for on an outpatient basis. Fewer and fewer young people want to go into care. “If we really want to take care of the aging population, we need robots that can take over some everyday tasks.”
But Chalvatzaki says current robots are quickly overwhelmed by tasks that seem simple to us humans. They might not reliably bring the tablet tray, for example.
Robots must learn to cope with uncertainty
The researcher sees the numerous uncertainties involved in human-robot interaction as a central challenge: Incomplete sensor information captures only part of the environment or generates noise that must be processed by the systems.
The robots also interact in an environment where humans also move – and their behavior is difficult to predict. This can lead to errors or even be dangerous.
Intelligent robot assistants therefore need good coordination, must be able to recognize errors independently and learn from them. This is a central research topic of Chalvatzaki and her colleagues. At the IROS 2022 robotics conference, Chalvatzaki received a best paper award for her research on this topic.
Together with the iROSA Lab, Chalvatzaki also developed an algorithm that enables a safe response to unanticipated human movements. This improves safety for people and the environment when dealing with robots.
Multimodal robotics and powerful AI
Intelligent robot assistants are embodied AI agents, according to Chalvatzaki. They would have to be able to move, perceive and convert comprehensive instructions into small subtasks. Corresponding systems would have to be multimodal in design, i.e., able to process various data such as images, text and audio, and would also need functions that operate at the interface of natural language, logic, geometry and motor control.
Beyond the potential benefits to an aging society, therefore, research on advanced robotic systems also contributes to broader AI research.
This is precisely where Chalvatzaki sees hessian.AI’s major advantage. The center offers cooperation and knowledge sharing between different fields, such as robotics and natural language machine processing. She also appreciates the center’s public relations work, which presents AI research to a broad public and introduces young women to STEM subjects through targeted events, for example.
In 2023, »Kultur einer Digitalstadt«, in cooperation with hessian.AI, repeatedly invites applications for three Artist-in-Science-Residencies for artists of all disciplines.
The studio residencies on the Rosenhöhe in Darmstadt are each linked to one of the renowned Darmstadt research institutes. Cooperation partners are The Hessian Center for Artificial Intelligence (hessian.AI), the GSI Helmholtzzentrum für Schwerionenforschung and the European Space Operations Centre (ESOC).
A total of three 6-week fellowships will be conducted between June and October 2023. Each fellowship is related to one of the three collaborating research institutions. Applications can be submitted from 11.01.2023. Detailed info about the program and the application can be found in the PDF:
hessian.AI is happy to be part of the program again this year and is looking forward to the applicants.
Want to learn more about the program and our first Artist in Science? Find out more 👉 here.
We have big news for 2023! In July, from 4th to 6th, the first hessian.AI conference “The hessian AICon” will take place.
Three days full of discussing the latest AI research trends, networking, a startup fair and getting to know us and the AI ecosystem better. You better save the date to not miss any information!
Date: July 4-6, 2023
Location: darmstadtium – Science and Conference Center, Darmstadt, Germany
Time slot: all day everyday. A more detailed schedule will be published during the first quarter of 2023.
Save the date and sign up for our newsletter! Follow us on our channels to not miss anything: Twitter, LinkedIn, Instagram.
You don’t want to miss out? 👉 Suscribe to the hessian.AI Newsletter!
About the organizer
hessian.AI is a unique centre for AI research excellence in Europe bundling forces from 22 (soon 44) AI Principal Investigator in Hesse, spanning all relevant AI research disciplines from robotics, NLP, Deep Learning over Cognitive Science and Causal Reasoning. It has a high focus on sustainable, trustworthy and explainable AI applications that foster a democratic society and its sustainable transformation. hessian.AI is part of the leading European AI initiatives CLAIRE and ELLIS as well as in the close network with AI application industry experts. This builds a more than solid foundation to truly leverage the potential of AI for a sustainable digital transformation – on a regional, national and European level.
Imagine: Can machines compose? A team of AI experts from hessian.AI and Aleph Alpha as well as musicians from the Singakademie Dresden have dared the experiment: on 20 November 2022, the piece “Meistersinger Reloaded”, composed by artificial intelligence, was premiered.
Roughly a year ago, the conductor Michael Käppler from Dresden approached technical universities in Germany with an unusual request: He had in mind an AI that composes music. Kristian Kersting, the founding co-director of the Hessian Research Centre for AI (hessian.AI) and professor of computer science at TU Darmstadt, was enthusiastic about the idea. He put Käppler in touch with researchers and students from his field, who started to work together with the musician. With their project, they have now reached a wider audience in Dresden and beyond.
“Meistersinger reloaded” was performed on 20 November 2022 in the Lukaskirche Dresden by the Elbland Philharmonie and the Great Choir of the Singakademie under the direction of choirmaster Käppler with a total of around 130 musicians. The AI composition was embedded in a program of pieces by Franz Schubert, Felix Draeseke, Richard Wagner and Alexander von Zemlinsky. Participants were surprised at how harmoniously and pleasingly the composition blended into the concert as a whole. Since there was no signal like an acoustic beep and the order of the announced pieces changed shortly before the performance, only a few managed to solve the acoustic riddle. None of the pieces were out of the ordinary or sounded surprisingly different, which was perhaps the most surprising thing about this evening that brought the project to its artistic and scientific climax.
Cooperation with Aleph Alpha and the Dresden Singakademie
Over the course of a year, a team of computer scientists from the AI R&D company Aleph Alpha at Heidelberg, from the TU Darmstadt and musicians from the Singakademie Dresden created an AI composition about Wagner’s music: In the AI data center of the Heidelberg AI researchers, MIDI files on piano music of the German Romantic period were used to train a large AI transformer model, which independently learned basic rules of composition on the large data set and then received special training on Wagner’s works to be able to imitate his style. Koen Oostermijer and the research team from Aleph Alpha, Michael Lang and Wolfgang Stammer from the TU Darmstadt, as well as artistic director Käppler were in close exchange over many experiments, jointly evaluating different experiments, approaches and iterations until their Wagner AI seemed perfect. It was helpful that large language models, such as Aleph Alpha is developing, have already internalised a similar pattern recognition: Music has a temporal structure like language, but some details of the modelling had to be adapted. After successful training, the AI now continues to write notes for any piece, using the learned structure and style of music in general and Wagner in particular.
“The first piece from the fifth generation was special,” Käppler revealed. It had taken hold of him. The researchers continued to work with this piece, which had something like a motif and most closely resembled a composition, until in the end a piece emerged that Käppler arranged for the orchestra and he adapted the text for the choir to it. Choral music requires longer notes to carry the language: subtleties that the researchers had to take into account when providing input to the AI. For the AI researchers, the project was an exciting balancing act between influencing the system to produce a desired output and leaving it free to produce something creative, playable and worthwhile to listen to that would also please human ears. The three-minute piece contains over 3,000 hours of music that were used for training, as well as many hours of human listening samples. Käppler has not altered the musical substance of the AI composition, but has taken it over for the orchestra as faithfully as possible.
A Feast for the Ears: Can Machines be Creative?
A discussion panel before the concert provided information about the background of the project and its practical foundations. Many concert guests took the opportunity to get in the mood and came to the community centre to learn about AI and art creation. Cultural curiosity united the audience, some guests had a technical background. Michael Käppler, Kristian Kersting, the mathematician and emeritus music professor Christfried Brödel and the Darmstadt AI master’s student and classical musician Matthias Lang discussed with the musicologist Miriam Akkermann (TU Dresden).
The starting point for Kersting and the machine learning researchers from Aleph Alpha and the TU Darmstadt was the question of how music can be formalised in order to produce it by machine. Composition with artistic pretensions is the creation of something new according to existing rules – recognisable, but not too close to the existing, i.e. not mere “parroting”, as Käppler explained. Can that work without emotion? After all, machines presumably don’t have feelings. Ultimately, art is “the encounter of people, made for people”, as Christfried Brödel put it. People remain the recipients of machine-made art, and it should speak to them.
AI is an Emotionally Charged Field
The project had moved in the field of tension between technology and human emotions. The Dresden concert audience reacted with curiosity and openness: obviously no one found anything strange about the performance, and at the end there was applause and flowers for the computer scientists as well. The audience treated them like part-composers. Perhaps that is precisely what is original about it and an answer to some of the open questions: Wagner had considered himself an unsurpassable genius, his musical legacy is overshadowed by anti-Semitism and the closeness of some of his descendants to the Nazi regime as well as the reception of his music during that time. AI “knows” nothing of this and can “concentrate” on the formal aspects of Wagner’s music.
AI is an emotionally charged field: People react to machine learning with fears, hopes and fascination. Fears, because people feel redundant when machines can do something better. Hopes, because practical assistance systems already accompany us in everyday life. The big problems of humanity will probably only be solvable with superhuman intelligence. But can machines also create art, can creativity be attributed to machine production? And if so, what does that mean for us? Some fears, insofar as they existed, were certainly dissolved by the evening and transformed into informed curiosity. The concert was well received, the church was crowded.
Art and Computer Science Appeal – Together
On the technical side, the joint project created a complex AI system optimized for musical art – similar to the one at the base of Aleph Alpha’s multimodal models. On the human side, it brought and continues to bring art and research together for a sensual form of interaction between man and machine. Wagner’s music was not disenchanted at the concert, but renovated for today. In the Meistersingers, an unshakeable belief in art as the basis for meaning and cohesion is expressed, says Käppler. We can all do with some of this spirit (purged of its nationalistic overtones). AI makes a contribution here to contemporary Wagner reception.
hessian.AI – the Hessian Center for Artificial Intelligence together with the CAISA Lab of Philipps-University Marburg hosted the IT Summer School Women4Women from August 22 to September 3. Lucie Flek, hessian.AI member, launched the initiative to get young women excited about the field of artificial intelligence and to show them future career paths.
The Women4Women IT Summer School was aimed at high school girls who already have an initial interest in STEM subjects. The field of computer science was introduced to the female students in a confidential and relaxed atmosphere at Philipps-University Marburg. The aim was to provide initial insights into career prospects in computer science, both in research and in technology and commercial companies in the Marburg area and throughout Hesse. The primary goal is the practical teaching of basic elements of programming, working together in a team and especially having fun while working on the versatile tasks.
With five participants, the event took place in a personal and confidential atmosphere. The students were able to talk with like-minded people about difficulties regarding their professional future: They were alone in their circles of friends with their interest in computer science, furthermore it was known that sexism/discrimination in the field of IT and gaming was a current problem and women were still “underestimated in these fields, role models were missing, to name just a few of the concerns of the female students”.
Hosting the Summer School serves to try to address these concerns: Early exposure to new technologies is intended to foster young women’s interest and create understanding of the areas of application. The hands-on sessions at the event helped to remove initial inhibitions.
The schoolgirls were accommodated in Marburg for the entire event. They were introduced to programming in a playful manner and by creating concrete applications. Already on the second day, they learned to develop a Pacman game. The highlight was the programming of the humanoid robot “Pepper”. This robot is programmed to analyze people and their facial expressions and gestures and to react accordingly to their emotional states.
In addition to the content of the program, the students got to know Marburg better. Together with their tutors, the participants went pedal boating on the Lahn River, took part in a city tour entitled “Famous Women in Marburg” and visited the Botanical Garden.
The percentage of women in technical professions, including those in the AI environment, has increased in recent years, but the gender ratio is still unbalanced. Only just over 20 percent of people working in AI-related fields are women, according to business magazine Forbes. A recent FAZ article highlights misalignments in scientific institutions that make it difficult for women to pursue careers in the system.
That makes it all the more important to show young women the way to take away their fears – it’s worth it:
It was a nice experience to know that there are more girls who share the same interests after all. It encouraged me to continue on the science path, as I had my reservations about it, too.
Feedback from a participant
To encourage the growth of women in the AI industry, hessian.AI aims to promote female talent in the field, make career direction more accessible to them and raise awareness of the need for gender balance.
The complete tutorials with code and slides are posted on Github.