RESEARCH PROJECT
TITHONUS
TITHONUS
TITHONUS WAS A UNIVERSITY PROJECT THAT KEPT GETTING MORE SERIOUS THE
LONGER WE WORKED ON IT. WE BUILT A REVIEW TOOL, TESTED IT WITH REAL
DEVELOPERS, WROTE THE PAPER, AND WON BEST PAPER AT ICSSP 2023.
THE PROBLEM
THE PROBLEM
MOST PULL REQUESTS GET APPROVED WITHOUT ANYONE KNOWING HOW CAREFULLY THEY
WERE ACTUALLY READ. FROM THE OUTSIDE THE PROCESS LOOKS COMPLETE, COMMENTS
RESOLVED AND APPROVAL GIVEN, BUT AN APPROVAL DOES NOT PROVE THE REVIEWER
CAUGHT WHAT MATTERED.
THAT GAP IS WHAT WE WANTED TO MEASURE. A REVIEW CAN SAY "LOOKS GOOD TO ME"
WHILE THE REAL QUESTION STAYS UNANSWERED: DID THE REVIEWER PAY ENOUGH
ATTENTION TO CATCH A KNOWN PROBLEM?
THE IDEA WE BUILT AROUND
THE IDEA WE BUILT AROUND
WE PUT MUTATION TESTING INSIDE THE REVIEW FLOW. THE AUTHOR SEEDS
INTENTIONAL BUGS INTO THE CHANGED CODE, THE REVIEWER GOES THROUGH THE PULL
REQUEST AS USUAL, AND THE SCORE COMES FROM HOW MANY OF THOSE SEEDED BUGS
GET CAUGHT.
IT DELIBERATELY MEASURES ONE NARROW THING, ATTENTION TO KNOWN ISSUES
INSIDE THE CODE BEING REVIEWED, INSTEAD OF PRETENDING TO SCORE REVIEW
QUALITY AS A WHOLE.
WHAT MADE IT MORE THAN COURSEWORK
WHAT MADE IT MORE THAN COURSEWORK
WE DID NOT STOP AT THE PROTOTYPE. THE PROJECT BECAME THE PAPER "TOWARDS
BETTER CODE REVIEWS: USING MUTATION TESTING TO IMPROVE REVIEWER
ATTENTION," CO-AUTHORED WITH COLLABORATORS AT BILKENT UNIVERSITY.
WE VALIDATED THE TOOL WITH EIGHT DEVELOPERS, WON THE UX AWARD AT CSFAIR
2022, AND WENT ON TO WIN BEST PAPER AT ICSSP 2023.
ARTIFACTS
ARTIFACTS