Hello Giancarlo. The most important question up front: What is the vision of your project “Testable”?
With “TESTABLE”, we basically keep doing what we have always been doing: We look for vulnerabilities in computer programs. There are already many tools and techniques to analyze code and program functions and to detect errors and vulnerabilities. But these tools have limitations and keep coming up against obstacles given the growing complexity of software programs. This is why we are pursuing a new approach with “TESTABLE”. We have been asking ourselves why testing tools are still overlooking so many vulnerabilities or output false alarms. We were able to identify one big problem: The fashion in which developers write code. In the project, we are building up a database to collect these problematic code patterns for testing tools.
Now I have to know more. What exactly are obstacles for the tools that might be hidden in the code?
Ok, I'll try to explain this using an example: JavaScript – a programming language that is mainly used to build websites – has, for example, an instruction that tells the computer to treat a string as a piece of code, not as data. These instructions make it very difficult for security testing tools to examine the source code and identify where the vulnerability lies. Developers often use these instructions also to fix vulnerabilities in their programs. In this project, we explore alternative ways to write code, helping developers avoid using problematic instructions (we call them testability patterns) and use alternatives instead.
For your work to bear fruit, you would have to ask the developers to write their code differently than they do now. That might be tricky.
(Laughs) Yes, that would be the first step. But for this to happen, there would first have to be alternatives for the code parts that are problematic.
As far as I know, almost nobody writes a program from scratch these days. I've also heard of code-generative language models that can write program code at the push of a button, much in the same way that ChatGPT spits out natural language. Couldn't you just train them with your data so that they produce "better" code?
That would be an idea if we had the relevant data and as much of it as possible. At the moment, however, we are building up a database that contains a list of signatures that we can search for. Then we can say: If the code contains this signature here, then there could be complications when using a tool, say, XYZ. This step shows us where the problems are with the existing code. For such training, we would first and foremost need data on how the code can be rewritten in a better way. Unfortunately, this is something we cannot completely deal with in this project. We have already reached the end of the second year. We have one year left; in August 2024, “TESTABLE” will be completed for the time being.
What do you think where will you be at then?
Our main goal is to push the database as far as possible. We have a repository on GitHub where everyone can participate and contribute. Another goal is to build a community of developers and security testing experts so that we have this self-sustaining mechanism for the testability pattern project, which is also an official project within the OWASP community. But we also continue to work on advancing the state-of-the-art with regard to testing tools. And we are remarkably efficient at it – especially at CISPA. My PhD candidate Soheil Khodayari does his research in the exact same area in which CISPA is supporting “TESTABLE”, and he is achieving amazing results. Only recently, he received a Best-Paper-Award for one of his papers.
Many European institutions from both industry and academe are collaborating in this project. Does the international dimension of the project play a role in its success?
European projects give you the opportunity to work together and they are also good for networking and exchange. Within this framework, we have many collaborations going on, for example by CISPA, SAP and TU Braunschweig, that eventually result in scientific publications. The project meetings are the core element that enables us to build up collaborations. We meet, have workshops, present our results, introduce fresh ideas and constantly look for opportunities to collaborate.
When researchers collaborate with industry partners such as Norton or SAP, how does this cooperation work?
We have a shared plan of what we want to do to make programs more secure. Every partner contributes in their own way to this overall goal. CISPA, for example, drives forward the state-of-the-art when it comes to automated security testing. UC3M does the same, but in the field of privacy and data protection. Eurecom leads the creation of the testability patterns dataset. Pluribus One takes care of machine learning. SAP, Shiftleft, IMQ Minded Security and Norton are our industry partners – they provide us with case studies and also conduct research themselves, all with top-notch researchers. Shiftleft is responsible for identifying potential industrial use cases for the technologies we have developed. Norton concentrates on user-end privacy, and SAP on security testing. Shiftleft produces testing tools which is important to test our newly developed approaches. IMQ Minded Security is in a consulting role. They are the most likely to tell us how our concepts can be implemented in the industry. Every partner makes a contribution of their own.
You seem to be very happy with the project and how it is going.
There are two reasons why it is going so well: First, all of us are closely connected and second, all of us are top researchers. This means that the people working together in this project are striving for success on their own initiative. These are two ingredients that make my work as a scientific coordinator much easier.
I wanted to ask you about the challenges of such a cooperation, but what you have been telling me so far actually sounds very pleasant.
Fortunately, there are not many. If I have to cherry pick one… I can tell you about the biggest on: deciding the location for the next project meeting! We have partners from many beautiful places in Europe, like Sophia Antipolis, in the South of France, Madrid in Spain, Cagliari and Milan in Italy, and Berlin in Germany. We always decide strategically based on what the weather will be like. Our last meeting was in Madrid, fantastic city!
Giancarlo, thank you for this interview.
This project has received funding from the European Union's H2020-SU-DS-2020 Grant Agreement No. 101019206.
