Send email Copy Email Address

2025-06-12
 

ELLIOT: Flagship Initiative to Research and Develop Open Multi-Modal Foundation Models for Robust AI Operation

A new chapter in European Artificial Intelligence (AI) research begins with the launch of ELLIOT – European Large Open Multi-Modal Foundation Models For Robust Generalization On Arbitrary Data Streams. Funded under the Horizon Europe program with a €25 million grant, this four-year Research and Innovation Action will bring together 30 leading organizations from 12 European countries to pioneer the next generation of trustworthy, general-purpose AI models with strong generalization and reasoning, built for real-world, data-rich applications, adhering to open-source research and development. CISPA-researcher Professor Dr. Mario Fritz is a Principal Investigator in ELLIOT. With his AI-expertise, he will contribute to the entire development cycle of the models, from training and algorithmic fairness to security aspects and application.

At the heart of ELLIOT is the development of open Multimodal Generalist Foundation Models: AI systems designed to learn general knowledge and patterns from massive amounts of data of various types — from videos, images, and text to sensor signals, industrial time series, and satellite feeds —  and efficiently transfer the generic knowledge learned in generalist manner to a wide variety of downstream tasks. Unlike current foundation models that face significant challenges in terms of generalisation capabilities and support for multimodal data, ELLIOT’s models will be capable of robust generalisation across conditions not seen during the training, coping well with dynamic, noisy, and temporally-evolving multimodal data streams.

“ELLIOT will significantly strengthen European sovereignty and independence in Artificial Intelligence, particularly in the rapidly evolving area of Multimodal Foundation Models. By leveraging Europe’s scientific excellence, computational infrastructure, and commitment to open and trustworthy innovation, ELLIOT is a major step toward ensuring that Europe actively shapes its own future in AI”, says Dr Yiannis Kompatsiaris, ELLIOT’s Coordinator at the Information Technologies Institute (CERTH-ITI). Mario Fritz, CISPA-researcher and AI-expert, adds: "In ELLIOT, we’re not only pushing the technical boundaries of AI foundation models — we’re fundamentally rethinking how to embed trustworthiness at the core of their design. Our goal is to ensure that these models are robust, transparent, and accountable from the ground up, so they can be safely deployed in critical real-world environments."

ELLIOT will closely cooperate with European and international open-source and open science communities like LAION and open-sci, benefiting from the strong expertise on open foundation models and datasets necessary for their creation established in those networks. These tight links will ensure staying on the frontiers of fast progressing research and making the whole pipeline for foundation model research and development — dataset composition, training, fine-tuning and evaluation — fully open and reproducible, such that it can be validated and easily adapted for specific industry and public sector needs.

Leveraging Europe’s world-class supercomputing infrastructure — including EuroHPC supercomputers like JUPITER, Leonardo, MareNostrum, and LUMI as well the Alps supercomputing infrastructure of the Swiss National Supercomputing Centre — ELLIOT will combine real and synthetic data from public internet and trusted sources, including European Data Spaces, to train and deliver a family of open trustworthy multimodal generalist foundation models along with open datasets and methods. This will enable the AI community in Europe and worldwide to build, study, deploy, extend and evaluate such models and the whole pipeline necessary for their creation in a fully reproducible manner at a level and scale that is currently not possible in open public domain. In that way, ELLIOT will act as a catalyst for open-source and open science in AI, and substantially strengthen Europe’s Sovereign AI vision.

Adopting a multidisciplinary approach that promotes ethical and trustworthy AI research and development, ELLIOT will also investigate technical, legal & social challenges of multimodal foundation models, including EU regulation, societal and ethical impact, fairness and bias mitigation, privacy concerns, diversification, explainability, and security. Moreover, ELLIOT’s results will empower new breakthrough applications and strengthen Europe’s position in the domains of media, earth modelling, robotic perception, autonomous driving, computer engineering and workflow automation.

“Through the development of a Virtual Innovation Lab, ELLIOT will not only ensure the widespread adoption of its results by the AI research community but also the uptake by end users in various application domains”, says Dr Kompatsiaris. “For example, multimedia content editors and journalists in media organisations will be able to use ELLIOT models for new media production and live fact-checking. Other end users targeted by ELLIOT’s use cases include researchers and scientists working on Earth observation and climate modelling, workers in robot-assisted industrial processes, manufacturers of self-driving cars, engineers dealing with software development and hardware design, researchers that analyse extensive collections of rich media documents and more. This uptake of the fine-tuned domain-specific ELLIOT models by the public and industry will ensure a two-way engagement with society”.

Key activities around the proposed research include community development and training of the next generation of European machine learning and AI researchers by leveraging the highly successful and internationally recognized ELLIS (European Laboratory for Learning and Intelligent Systems) PhD program. This will result in a complete ecosystem of European AI: built in Europe, for Europe, and adhering to European values.

The consortium is coordinated by the Information Technologies Institute of the Centre for Research and Technology Hellas and includes powerhouse academic and research institutions, alongside innovative SMEs, public stakeholders and non-profit organisations with diverse expertise. With ELLIOT, Europe positions itself at the frontier of open, trustworthy, and socially beneficial artificial intelligence — capable of driving innovation in critical areas such as media, Earth observation, autonomous systems, and public services.
 

About ELLIOT

Grant agreement no: 101214398
Project duration: July 2025 – June 2029
Funded under: Horizon Europe – HORIZON-CL4-2024-HUMAN-03
Coordinator: Information Technologies Institute (ITI) - Centre for Research and Technology Hellas (CERTH), Greece
Contact person: Dr. Yiannis Kompatsiaris, CERTH-ITI

List of ELLIOT partners:

  1. Information Technologies Institute - Centre for Research and Technology Hellas (Coordinator) (Greece)
  2. Tübingen AI Center / University of Tübingen (Germany)
  3. Research Center Juelich (FZJ) / Juelich Supercomputing Centre (JSC) (Germany)
  4. University of Amsterdam (Netherlands)
  5. Eindhoven University of Technology (Netherlands)
  6. University of Trento (Italy)
  7. Computer Vision Center (Spain)
  8. Jožef Stefan Institute (Slovenia)
  9. LOBA (Portugal)
  10. Barcelona Supercomputing Center (Spain)
  11. CSC - IT Center for Science (Finland)
  12. CINECA (Italy)
  13. Ludwig Maximilian University of Munich (Germany)
  14. University of Valencia (Spain)
  15. University of Modena and Reggio Emilia (Italy)
  16. Aalto University (Finland)
  17. ELLIS Alicante (Spain)
  18. ELLIS Institute Tübingen (Germany)
  19. Czech Technical University in Prague (Czech Republic)
  20. CISPA Helmholtz Center for Information Security (Germany)
  21. KU Leuven (Belgium)
  22. Voxist (France)
  23. Valeo (France)
  24. RoboTwin (Czech Republic)
  25. Openchip (Spain)
  26. Deimos Engineering and Systems (Spain)
  27. Department of Territory, Housing and Ecological Transition of the Catalan Government (Spain)
  28. Flemish Radio and Television Network Organisation  (VRT) (Belgium)
  29. EPFL (Switzerland)
  30. ETH Zurich (Switzerland)