Marius Smytzek

Teaching by Marius Smytzek

Winter 2021/22

Security Testing

Software has bugs, and catching bugs can involve lots of effort. This course addresses this problem by automating software testing, specifically by generating tests automatically. Recent years have seen the development of novel techniques that lead to dramatic improvements in test generation and software testing.  In this course, we explore these techniques – in theory and in code.

Course Organization

Every week, you will be provided with Jupyter Notebooks that teach a particular topic and illustrate it using plenty of runnable Python code.  These notebooks come from The Fuzzing Book, a textbook on how to generate software tests written by yours truly.

In the notebook, you can edit the code as you like, run your own experiments, and re-use and extend the code to your liking.  Your task will be to use these techniques (and their code) to build a series of fuzzers (i.e. test generators) that find bugs in a number of challenging settings.

Meetings

This course uses the "inverted classroom" principle – you learn at home, and discuss issues with your instructor.  In our weekly meeting, we use the gathering in the lecture hall to

• discuss the assignment of last week
• discuss the assignment of next week
• discuss ongoing projects as well as general questions.

These meetings come with live coding, so we can explore ideas right on the go.

Projects

During this course, you apply the techniques learned in weekly exercises and two projects which form your coursework. Projects are graded for effectiveness, efficiency, elegance, and creativity. Projects offer special challenges which allow you to gain bonus points.

Exercises

Every week, you get a simple exercise assignment covering the material of the last lecture. Performance in these exercises will make 33% of the final grade.  Note that there is no final exam.

Requirements

Advanced programming skills (such as obtained after two years of successfully studying CS) are required. Knowledge in Python is useful, but can easily be acquired along the course.

Passing Criteria and Grading

To pass this course, you need to have

• at least 50% of the points in each of the projects, and 
• at least 50% of the total exercise points.

Your final grade is determined by 66% projects and 33% exercises (see above).

Lecture Plan

The course is organized as "inverted classroom": Every week, we discuss a chapter of the book, which will be supplied with an introduction video; we meet once a week to discuss the material, the associated exercises, and the ongoing projects.

• Introduction to the course; Introduction to software testing; Introduction to fuzzing
• Code Coverage; Mutation-based fuzzing
• Greybox fuzzing
• Fuzzing with Grammars; Efficient Grammar Fuzzing
• Parsing input grammars (usage only); Greybox fuzzing with grammars
• Probabilistic Grammar Fuzzing
• Fuzzing with Generators; Grammar Coverage
• Mining Input Grammars
• Tracking information Flow; Concolic Fuzzing
• Symbolic Fuzzing
• Fuzzing Configurations; Fuzzing APIs
• Testing Web Applications; Testing Graphical User Interfaces
• Reducing Failure-Inducing Inputs
• When to Stop Fuzzing + Current Trends in Fuzzing Research

The lecture plan may be subject to changes; these will be announced in time.

Date, Time, Location

• 14 lectures
• 6 Credit Points
• In-presence tutorial and discussion every Tuesday 16:15–17:45, CISPA, Lecture Hall 0.05
• Optional attendance via Zoom: https://cispa-de.zoom.us/j/96427716796?pwd=RnNTQTlQMGtvMDJ5MWFKYTBpM1RrUT09
• Start: Tuesday October 19