Their research is certainly cool and novel. On the surface, this is a straight-forward application of genetic programming (evolving abstract syntax trees):
1. Take function which fails some test case(s)
2. Parse AST
3. Find a bunch of other lines of code in the program and use those as possible mutations
4. Evolve until test performance is improved
The trick is #3. They are going on the hypothesis that the solution to bugs are often found in other parts of a program. For instance, you pass in a variable and forget to check for it being null. It's likely that you have a check for that somewhere else in the program, and if so, then you can add that (templated) line of code into your buggy function and it will now pass that test case.
It's certainly not a panacea, but it does work remarkably well for many bug cases.
Actually, I tried to implement something like this as an undergrad and I found that the search space was simply intractable and the fitness function too coarse grained (I.e binary). What I really like about this approach is that they do a trace of the program execution as a heuristic to guide the evolutionary strategy. That's awesome.
As a side note, a few months ago I reached out to this team to ask them a few questions. They are super nice.
1. Take function which fails some test case(s)
2. Parse AST
3. Find a bunch of other lines of code in the program and use those as possible mutations
4. Evolve until test performance is improved
The trick is #3. They are going on the hypothesis that the solution to bugs are often found in other parts of a program. For instance, you pass in a variable and forget to check for it being null. It's likely that you have a check for that somewhere else in the program, and if so, then you can add that (templated) line of code into your buggy function and it will now pass that test case.
It's certainly not a panacea, but it does work remarkably well for many bug cases.