WHAT IS S3 ABOUT?

What if we had software bots that tirelessly test, debug, and monitor our software systems?

IT workers are expensive and scarce. So why can’t we further automate boring, repetitive activities such as testing and debugging? The problem is that we lack computer-readable specifications (so-called oracles) for what the system should do or not do. For decades, this oracle problem has been a roadblock to automated test generation, trusted software repairs, and accurate monitoring of software.
Building on groundbreaking research to infer input languages of systems, S3 introduces a unified approach to learning oracles automatically. It takes a given software system; infers and decodes its inputs and outputs; and runs experiments to extract models of how the system behaves, capturing its semantics by predicting output features for given input features. These models, named system invariants, allow to fully automate critical software development activities:

Testing

System invariants encode languages for automatically generating test inputs and provide oracles for checking test results: “In the TLS server, the <payload> in the <heartbeat-response> must be the same as in the <heartbeat-request>.”

Debugging

System invariants allow narrowing down causes of software behavior (“The X.509 public key certificate is not recognized if <subject-name> contains a zero byte”). Generated tests and oracles ensure reliable automated repair.

Monitoring

System invariants enable detecting abnormal behavior at runtime (“In log4j, logging a <user-agent> containing "${jndi:<url>}" opens <url>”). Problematic queries can be isolated and investigated until the problem is fixed.

In the future, testing, debugging, and monitoring would thus be taken over by software bots who would autonomously explore software behavior, report issues, and suggest actions to their human co-workers, boosting developer productivity and software reliability.