A battle of three descriptors: SURF, FREAK and BRISK
I think developers and research guys who works with object recognition, image registration and other areas that uses keypoint extraction can find this post useful. Recently (from 2.4.2) a new feature descriptor algorithm was added to OpenCV library. FREAK descriptor is claimed to be superior to ORB and SURF descriptors, yet it’s very fast (comparable to ORB). Also people in comments on my blog mentioned BRISK descriptor which is also new and more efficient than SURF. Well, finally i find a time to compare them and publish my research results.
This post will be very similar to OpenCV comparison reports i made in past. Although those reports were published years ago, they are still somewhat actual. For this test i decided to rewrite the whole testing framework from scratch. The source code will be available soon. But for now, let me explain what i did to find a best of three algorithms. What is main goal of converting image to descriptors? Move from pixel domain to more compact form of representation the same data. In addition we would like our representation be rotation and scale invariant (e.g representation remains the same or changes slightly when source image rotated or scaled). SURF, FREAK and BRISK descriptors claims they are rotation and scale invariant.
Like in the OpenCV comparison report, test application works with test pattern image. And we have four basic transformations: rotation, scale, blur and brightness adjustment. Here how the rotation transformation class looks like:
Other types of transformations looks similar. But it shows the idea.
As you may know we need three components when dealing with descriptors:
- Feature detector - class derived from cv::FeatureDetector class that implements particular detection algorithm. For example, cv::SurfFeatureDetector implements detection algorithm described in SURF paper.
- Descriptor extractor - class derived from cv::DescriptorExtractor. It computes descriptors from passed keypoints. cv::SurfDescriptorExtractor will computer SURF descriptors.
- Descriptor matcher - An instance of cv::BFMatcher of cv::FlannBasedMatcher classes is used to match two sets of descriptors. We store these three objects in FeatureAlgorithm class:
The main test function takes FeatureAlgorithm, Transformation and test image. As output we return list of matching statistics for each run. Here is a brief sequence:
- Convert input image to grayscale
- Detect keypoints and extract descriptors from input grayscale image
- Generate all transformed images using passed transformation algorithm
- For each of the transformed image:
- Detect keypoints and extract descriptors
- Match train descriptors and query
- Split matches to inliers and outliers using homography estimation
- Compute statistics (consumed time, total percent of matches, percent of correct matches, etc) The main cycle is paralleled using OpenMP, on my Quad Core i5 it loads all cores for 100% while doing test. Feature Algoritms:
The following metrics are calculated:
- Percent of matches - quotient of dividing matches count on the minimum of keypoints count on two frames in percents.
- Percent of correct matches - quotient of dividing correct matches count on total matches count in percents.
- Matching ratio - percent of matches * percent of correct matches. In all charts i will use “Matching ratio” ( in percents) value for Y-axis.
After running all tests we collect statistics for each transformation and algorithm. The report table for particular transformation algorithm looks like this:
To make a graphic representation i used Google Spreadsheets to import CSV tables and generate charts. You can find this spreadsheet here: OpenCV 2.4.9 Features Comparison Report.
: chart_6.png (SURF vs FREAK vs BRISK descriptors comparison (rotation)) : chart_6-1.png (SURF vs FREAK vs BRISK descriptors comparison (scale))