Generative AI models such as GPT and Llama are trained on vast amounts of data, including material collected from the internet. This scale and diversity are what make generative models so powerful and enable them to generate everything from complex text to photorealistic images. However, the training datasets can contain copyrighted materials by artists, writers, and other creators. In addition, generative models can also inadvertently memorize specific individual training examples and later reproduce them almost exactly, sometimes word for word in text or pixel for pixel in images. As AI-generated content is becoming increasingly difficult to distinguish from human-created work, it can be used to produce misinformation and is increasingly re-used to train new models. This creates data loops that can reinforce existing biases, replicate harmful patterns, and degrade model performance. Dziedzic’s project aims to address these problems together by developing tools that identify where data comes from and how it is used.
Combining Weak Signals to Build Strong Evidence
To perform their tasks, generative AI models must learn general patterns from their training data. Problems arise when they memorize individual training examples so precisely that they can later reproduce them almost unchanged. Dziedzic therefore aims to develop new mathematical definitions and measurement methods to determine the extent to which models memorize such information. This can provide clues as to whether those examples were used during training. Researchers can investigate this, for example, by examining the probabilities a model assigns to its outputs or how accurately it can reconstruct individual data examples. However, existing methods face limitations as generative AI models and their training datasets grow larger. “Our central insight is that many weak clues can add up to strong evidence,” explains Dziedzic. “A single image or passage may reveal very little about whether it was used in training. By combining different signals across multiple data points and adding robust digital watermarks where possible, we aim to obtain reliable evidence with statistical guarantees.”
Digital Marks That AI Models Do Not Forget
Another component of the project is a concept known as “data radioactivity.” The idea draws on a technique used in medicine, where radioactive tracers help researchers track how certain substances move through the body. Following a similar principle, the research team aims to develop digital marks. “Of course, this does not involve actual radioactivity, but rather subtle patterns embedded in content,” explains Dziedzic. Unlike conventional digital watermarks, these marks are intended to remain detectable even after the marked content has been used to train new AI models. Traces of the marks should then be identifiable in the models' generated outputs. This could help determine which data was used for training and detect AI-generated content. Dziedzic has already demonstrated that such “radioactive” watermark patterns can be developed for a specific class of image-generating AI models. In the new project, he aims to advance this approach and combine it with other methods for identifying training data.
ERC Grant Provides Freedom for Fundamental Research
“For me personally, receiving an ERC Grant is a great honor and recognition of more than a decade of research. It is also a vote of confidence in an ambitious idea and in my ability to build a team capable of pursuing it,” says Dziedzic. The funding will give him the time and freedom to investigate fundamental scientific questions in depth and translate the resulting insights into practical tools for trustworthy AI. “I hope our research will help translate Europe's commitment to transparent AI into concrete technical capabilities that both people and AI companies can actually use.”
About the ERC
The ERC, set up by the European Union in 2007, is the premier European funding organisation for excellent frontier research. It funds creative researchers of any nationality and age, to run projects based across Europe. The ERC offers four core grant schemes: Starting Grants, Consolidator Grants, Advanced Grants and Synergy Grants. With its additional Proof of Concept Grant scheme, the ERC helps grantees to bridge the gap between their pioneering research and early phases of its commercialisation. The new ERC Plus Grants scheme is open for applications since June 2026. The ERC is led by an independent governing body, the Scientific Council. Maria Leptin has been the President of the ERC since November 2021. The overall ERC budget from 2021 to 2027 is more than €16 billion, as part of the Horizon Europe programme, which is under the responsibility of Ekaterina Zaharieva, European Commissioner for Startups, Research and Innovation.
