SBFT Tool Competition 2024 --- Fuzzing Track

Summary

This report outlines the objectives, methodology, challenges, and results of the second Fuzzing Competition held at SBFT 2024. This year's competition introduced a novel metric, mutation score, from mutation analysis. To facilitate this, the mutation score measure was integrated into the Fuzzbench platform. In addition to mutation score, we also report the coverage of mutants by fuzzers as a secondary criterion. We had five participating fuzzers, for which we evaluated the effectiveness by running each for 24 hours. Benchmarks from Fuzzbench were injected with faults and used as target applications. The competition was organized into three phases. In the first phase, participants were asked to integrate their fuzzers into Fuzzbench and were allowed to run experiments. During the second phase, we publicly ran all submitted fuzzers on a preliminary set of benchmarks and provided the results to the participants. In the third phase, we publicly ran all submitted fuzzers along with our baseline fuzzers on the complete set of benchmark programs to establish the final results.