SBFT’25 Competition Report — Fuzzing Track

Zusammenfassung

Measuring fuzzer performance by aggregate coverage measurement loses a significant amount of information regarding a fuzzer’s ability to explore a program. While historically fuzzer performance based on coverage has been measured as a single figure (e.g., "number of edges"), this value is necessarily misleading; two fuzzers can theoretically achieve the same number of edges but nearly mutually exclusive sets of coverage edges. In a previous work, we introduced the concept of relcov, an asymmetric similarity metric based on the Tversky index, that allowed us to identify the relative coverage-finding ability of one fuzzer in terms of another. Seeing that this metric uncovered far greater detail regarding the relationship between fuzzers, we designed a second metric, relscore, suitable for ranking a large number of fuzzers based on the novelty of the edges they discover and the reliability with which they do so. We used relscore as the basis for evaluating the rank of fuzzers during the SBFT’25 competition.