Patch Analysis

    This page summarizes the analysis of previous patch generation systems, including GenProg, AE and RSRepair. It also contains patches from Kali, a generate-and-validate patch generation system with only modifications that simply delete functionality. It also contains patches from ClearView, a binary generate-and-validate patch generation system.

    The following table summarizes all defects which have plausible patches generated by at least one system. The meaning of each cell is as follows:

                Correct: There is at least one correct patch generated by the system for the defect.
                Yes: There is at least one plausible patch but no correct patches generated by the system for the defect.
                No: There is no plausible patch generated by the system for the defect.

    Please click the corresponding link for each defect for more details, including
a. The new testcase to expose any defects in the reported plausible patches and new patches generated from re-running GenProg(if available).
b. The source code patched by Kali. To get the Kali patch, search for the keyword prophet in the source file.
c. Semantic Analysis. This analysis contains the unpatched source code (preprocessed by GenProg), the patched source code, and the sematic analysis of the GenProg/AE patches. To obtain the GenProg patch, you need to use a code comparison tool such as diff on the unpatched file and the GenProg patched file to see the difference between the files and locate the patch. To obtain the AE patch, search for the keyword __genprog_mutant in the source file (The value of __genprog_mutant will be specified in the semantic page).
d. On these web pages the official patch is relevant code from the version of the application identified by the GenProg researchers as the version without the defect.


Defects
GenProg
AE
RSRepair
Kali

    The next table shows the number of Reported Patches/Defects, Plausible Patches/Defects and Correct Patches/Defects for GenProg, AE, RSRepair and Kali on each application.

GenProg
AE
RSRepair
Kali
Application
Reported Patches/Defects
Plausible Patches/Defects
Correct Patches/Defects
Reported Patches/Defects
Plausible Patches/Defects
Correct Patches/Defects
Reported Patches/Defects
Plausible Patches/Defects
Correct Patches/Defects
Reported Patches/Defects
Plausible Patches/Defects
Correct Patches/Defects
fbc
1/1
1/1
0/0
1/1
1/1
0/0
0/0
0/0
0/0
1/1
1/1
0/0
gmp
2/1
2/1
0/0
1/1
1/1
0/0
5/1
5/1
0/0
1/1
1/1
0/0
gzip
8/1
8/1
0/0
2/2
2/2
0/0
5/1
2/1
0/0
1/1
1/1
0/0
libtiff
155/17
18/3
0/0
17/17
5/5
0/0
75/15
5/1
0/0
5/5
5/5
0/0
lighttpd
36/5
36/5
0/0
4/4
4/4
0/0
20/4
20/4
0/0
5/5
5/5
0/0
php
196/28
29/5
2/1
22/22
7/7
2/2
5/1
2/1
2/1
8/8
8/8
2/2
python
6/1
6/1
3/1
3/3
3/3
1/1
5/1
5/1
2/1
2/2
2/2
1/1
wireshark
10/1
10/1
0/0
4/4
4/4
0/0
5/1
5/1
0/0
4/4
4/4
0/0
Total
414/55
110/18
5/2
54/54
27/27
3/3
120/24
44/10
4/2
27/27
27/27
3/3


Resource

GenProg/AE: Homepage, GenProg Patches, AE Patches, Benchmarks and the Virtual machine.
RSRepair: Homepage, RSRepair Patches.