After objects have been segmented, their features must be characterized to assist in classification. Handwritten characters vary widely: letters can lean left or right; some are curvy, straight, or scrawled; some 7s have bars through the middle, and some 9s curve underneath. As a result, many features must be available for distinct and sure classification.
Moments provide much useful information. Moments are sums across probability
distribution functions, multiplied by the x and y order:
Mjk = sumx,y xjykfxy(x,y)
Moments correspond to statistical tendencies but are hard to explain intuitively.
For our problem, the pdfs are simply the black-and-white object bitmaps. The
moments are scaled to the mean, normalized to the variance, and calculated
such that they are rotationally invariant. This helps keep consistency in the
moments despite the size or angularity of the object. For our project we
found that various 3rd, 4th, and 5th order moments provide good information.
The first plot above shows moments on the x axis for 100 characters (10 each of 0-9); the second plot is the same image, zoomed in on the x-axis. In the first plot, the moments for each set group close together, conveying a consistent 'tendency' for each set; however, moments in 1, 6 and 9 have some outliers (high variance). In the second plot we notice that in general, 6 moments are not similar to 7,8, or 9 moments; nor do 0, 1 and 5 moments overlap with moments for 7 in general. These patterns are useful for classification.
For more information on moments, see
Frank L. Alt, 'Digital Pattern Recognition by Moments', JACM, 1961,
pp. 241-258
Michael Reed Teague, 'Image Analysis Via the General Theory of Moments',
Journal of Optical Society of America, 1980, pp. 920-930.
Other tests we incorporated in our project were circularity, median y, and hole count. Circularity is a hack test which approximately matches curviness to P^2/A (P=perimeter, A=area). While [0,2-9] are relatively indistinct by this test, this test distinguishes 1 due to its lack of curve.
Median y was found to be useful in distinguishing 7 from other numbers. While the mean y value finds the average y location, median y finds the y location in the middle of the set of all y locs. Ex if an object has points y valued [1 1 1 1 1 2 3 4 5 6 7], its y median is 2. The median for 7 tends to be distinctly 'higher' than medians of numbers such as 1 or 8. The 1/4 y median was also found to be helpful.
Hole count was found to be useful in distinguishing 1,3,4,5,7 from 0,6,8,9. Holes are found by region-filling empty spaces in the object bitmap; regions which fill to the sides are not holes. This test can fail occasionally, however, if the holes in 0, 6, 8 or 9 are not completely closed, if a 2 is drawn with a loop, or if a hole inadvertently exists for some other reason.
Other tests attempted which were not successfully implemented:
Source available for moments, holes, median y, and 1/4 median y.