Continuous Fuzzing: A Study of the Effectiveness and Scalability of Fuzzing in CI/CD Pipelines
conference paper
While fuzzing can be very costly, it has proven to
be a fundamental technique in uncovering bugs (often security
related) in many applications. A recent study on bug reports from
OSS-Fuzz observed that recent code changes are responsible for
77% of all reported bugs, stressing the importance of continuous
testing. With the increased adoption of CI/CD practices in
software development, it is only natural to look for effective
ways of incorporating fuzzing into continuous security testing.
In this paper, we study the effectiveness of fuzz testing in
CI/CD pipelines with a focus on security related bugs and
seek optimization opportunities to triage commits that do not
require fuzzing. Through experimental analysis, we found that
the fuzzing effort can be reduced by 63% in three of the
nine libraries we analyzed (55% on average). Additionally, we
investigate the correlation between fuzzing campaign duration
and the effectiveness of fuzzers in vulnerability discovery: a
significantly shorter fuzzing campaign facilitates a faster pipeline
for developers, while it can still uncover important bugs. Our
findings suggest that continuous fuzzing is indeed beneficial for
secure software development processes, and that there are many
opportunities to improve its effectiveness.
be a fundamental technique in uncovering bugs (often security
related) in many applications. A recent study on bug reports from
OSS-Fuzz observed that recent code changes are responsible for
77% of all reported bugs, stressing the importance of continuous
testing. With the increased adoption of CI/CD practices in
software development, it is only natural to look for effective
ways of incorporating fuzzing into continuous security testing.
In this paper, we study the effectiveness of fuzz testing in
CI/CD pipelines with a focus on security related bugs and
seek optimization opportunities to triage commits that do not
require fuzzing. Through experimental analysis, we found that
the fuzzing effort can be reduced by 63% in three of the
nine libraries we analyzed (55% on average). Additionally, we
investigate the correlation between fuzzing campaign duration
and the effectiveness of fuzzers in vulnerability discovery: a
significantly shorter fuzzing campaign facilitates a faster pipeline
for developers, while it can still uncover important bugs. Our
findings suggest that continuous fuzzing is indeed beneficial for
secure software development processes, and that there are many
opportunities to improve its effectiveness.
TNO Identifier
986440
Publisher
IEEE
Source title
IEEE/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)
Files
To receive the publication files, please send an e-mail request to TNO Repository.