Repairnator bot finds software bugs, successfully submits patches
Can a bot create valid, high-quality fixes for software bugs more rapidly than a human can, and get them accepted by human developers and permanently merged in the code base?
A group of researchers from the University of Lille & Inria Lille (France) and the KTH Royal Institute of Technology (Sweden) have been researching the possibility for a while now, and their Repairnator bot, which is specialized to repair build failures happening during continuous integration, has had some successes.
The research
“Repairnator is specialized to repair build failures happening during continuous integration. It constantly monitors thousands of commits being pushed to the GitHub code hosting platform, and analyzes their corresponding builds. Every minute, it launches new repair attempts in order to fix bugs before the human developer,” Martin Monperrus, Professor of Software Technology at the KTH Royal Institute of Technology, explained.
“Due to our expertise with Java in past research projects, the prototype implementation of Repairnator focuses on repairing software written in the Java programming language, built with the Maven toolchain, in open-source projects hosted on GitHub, which use the Travis CI continuous integration platform.”
They’ve been operating the bot for the last two years and this year it finally succeeded in developing patches that were human-competitive (produced before the human ones and accepted as valid contributions).
The bot created five patches (detailed in this blog post) that were ultimately merged in the code base.
Monperrus did not say how many repair attempts and “misses” were made by the bot during that same period, but in a previous and longer lasting “expedition,” the bot made 3,551 repair attempts and none of those turned out to be human-competitive.
The future
Initially, the developers who accepted the requests did not know that they were made by a bot. In order to remove any bias that developers might have against automated program repair bots, the actual submissions were made to look like they were coming from a (fictitious) software engineer and GitHub user called Luc Esape.
“Now, for sake of ethics, the real identity of Luc has been disclosed on each of his pull-requests,” Monperrus shared.
The researchers consider the Repairnator project a success, and not just for showing that automated program repair can be human-competitive. It has also pointed out some current shortcomings of program repair research, namely that some patches created by bots can be rejected because a bot cannot legally sign a contributor license agreement when needed.
Nevertheless, they believe that “Repairnator prefigures a certain future of software development, where bots and humans will smoothly collaborate and even cooperate on software artifacts.”