An Application of Image Processing in Optical Mark Recognition

The Optical Mark Recognition (OMR) is very popular with universities for the reading of multiple-choice questions. In this article, we presented a software system for processing surveys at the Vietnam National University of Agriculture based on digital image processing. This software was built using MATLAB and easy to use. The surveys were digitized using a scanner and sent to the software tool. In this study, we tested more than 170 surveys of nine different types. The software tool correctly detected all the valid answers. It was also able to detect all questions with no or multiple marks.


Introduction
Optical mark recognition (OMR) is a form of automated data processing. Questions with multiple choices are printed on paper. Respondents then mark their answers using pens. In the next step, the sheets are scanned and sent to a computer for processing. There are many applications of OMR including multiple-choice examinations (for students and pupils) and feedback collection (from customers, students, and users, etc.). In universities (i.e., Vietnam National University of Agriculture), collecting feedback from students plays an important role in evaluating and improving the quality of education.
Nowadays, many commercial solutions for OMR are available (e.g., OpScan Series Product from SCANTRON). In common, these products require a dedicated scanner and answer sheets, which motivates the finding of cheaper solutions. Hong Duc University created a software named TickREC for this purpose (Hong-Duc University, 2014). The Vietnam Forestry University also has its software solutions (Mai Ha An, 2014). Increasingly more methods for mark detection have been published. Gaikwad (2015) applied a template matching algorithm after finding the region of interest to find the answers marked (Gaikwad, 2015). Loke et al. (2018)et al. proposed a method based on pixel counting and simple thresholding that can be used under a variety of conditions . Another method by Belag et al. was developed based on the creation of template answer sheets and key points detection algorithms (Belag et al., 2018). Each of these methods (and corresponding software tools) has its own advantages and disadvantages. For example, Belag's tool used a dedicated sheet for answers, this sheet also had checkmarks that helped in case the scanned image was rotated. This kind of sheet is suitable for tests but is not good for surveys. In cases of TickREC and the tool of Mai Ha An (2014), they could process the sheets that contained both questions and answers (Mai Ha An, 2014). Because each software works with a certain type of answer sheet, which was designed as needed by the authors, it is not possible to apply these softwares instantly for the surveys at the Vietnam National University of Agriculture.
Hence, in this work, we created a software for processing surveys at the Vietnam National University of Agriculture. The surveys were scanned by an ordinary scanner and sent to the software to process. This software was designed in such a manner that it was easy to use and no special training was required. This system was cost-effective because no dedicated machine or answer sheets were required.

Materials
In this project, we used nine different types of questionnaires. All of these were used by the Center for Quality Assurance, Vietnam National University of Agriculture: For each type of questionnaire, there were more than 30 sheets that were randomly filled. All of the sheets were scanned with an HP scanner (ScanJet Pro 3000 s3). The output file format was normally JPEG but could also be PNG, BMP, or some other formats supported by MATLAB (see method section for more details). The width and the height of the images were 1655 and 2338 pixels, respectively (these dimensions of images could be slightly different depending on the scanner). The examples of surveys are shown in Figures 1 and 2.

MATLAB -Environment for software development
MATLAB (short name for matrix laboratory) was developed in the 1970s by Cleve Moler (Haigh, 2008). Most of the codes of MATLAB was written by Cleve Moler using FORTRAN. Jack Little and Steve Bangert then reprogrammed MATLAB in C. Together with Cleve Moler, three of them founded the MathWorks in California in 1984. MathWorks then develops, maintains, and distributes MATLAB as a commercial product (Sandeep, 2017). Nowadays, MATLAB supports various platforms such as LINUX, Windows, and MacOS. With MATLAB, users write a few lines of code to acquire instant results without involving a compiler. MATLAB is used for data analysis and visualization. It supports multiple types of data (audios, images, videos, CSV, and Vietnam Journal of Agricultural Sciences . MATLAB also provides App Designer tool which allows the users to different databases). MATLAB also provides App Designer tool which allows the users to build GUI (Graphical User Interface) for their programs (Educba, 2020). For these reasons, we used MATLAB to develop our software tool for data processing. Processing workflow Figure 3 shows the basic steps needed for the processing of one scanned page of questionnaires. For the first step, the selected machine (ScanJet Pro 3000 s3) scanned multiple pages in a single run. After that, our software tool then came into play.
Because our questionnaires were printed in monochrome and then filled using black or blue (the colors of most ballpoint pens), converting images to binary would save us memory and time for processing. With the support from MATLAB, converting images to binary was straightforward. We only needed to call the im2bw function with the original image as a parameter, the function then returned a binary image.
To extract the region of interest (ROI), the region in which people filled in the options, we used a special image called a mask. As shown in Figure 4a, a mask contained only filled options. Our program would then find the ROI. The position and size of ROI (the region inside the red rectangle, Figure 4b) was then used to crop the other scanned images.
With the function imfindcircles from MATLAB, we were able to locate all the options on the cropped images. The number of black pixels in each circle helped us to indicate the selected one.
Our software tool then outputted the selected options for every question on the sheet. The output was eventually stored in a plain text file.

Results and Discussion
The software tool Figure 5 shows the main graphical user interface (GUI) of the program. The user first needed to specify the directory of scanned images by clicking Select image folder button  (area 1). All images in the selected directory would be listed in the area below the button (area 2). The user then selected the mask file by clicking Select mask button (area 3). Depend on the type of questionnaire, we might need to select two masks if the questionnaire contained two pages. To start processing images, the user clicked on Start button (area 4). The result would be displayed at the bottom right of the window (area 5). Table 1 shows a summary of the analysis of 179 questionnaires belonging to nine different types. Our tool correctly detected all valid questions (questions having one option filled). It correctly identified all questions that were not filled (not evaluated by students, as shown in Figure 6a). The tool could also detect the question that had multiple options filled (the students changed their mind and chose another option) (Figure 6b). Because the number of black pixels in each option was used to identify which options were filled, our tool might not work correctly in some cases as follows: Instead of filling in the option, the user used a checkmark (tick) or x a mark (cross) to mark the selected option (Figure 6c). The number of black pixels inside a checked option might not be enough for a valid filled option.

Processing questionnaires
Options were not completely filled (Figure  6d). Similar to the previous case, the option might not be bold enough to be a marked one.
The user used light colors to mark the selected option. In this case, filled areas might become unfilled because of the conversion from color images to binary images.
Apparently, our tool marked this question as NULL in the result area. The user could easily see this and check the answer sheet manually by selecting the corresponding image from the list of images. After checking the images, the user was able to make direct modifications in the result area before exporting the final result to the output file.
If the scanned images were rotated, our tool might encounter a problem due to the scanning or copying process. Especially, when the crop area did not contain all the options, the program could not obtain enough data for analysis (Figure 6e). In the future update, we will give a warning for this kind of sheet. One possible solution to this problem is using checkmarks. Checkmarks are black-filled rectangles or squares located at the corners and the margins of the sheet. By first detecting checkmarks, it is possible to identify whether the sheet is rotated too much if one or more checkmarks at the corners are absent. If all of the checkmarks at four corners are detected, then we can calculate the rotate angle of the sheet. We can eventually rotate the scanned sheet in the reverse angle before detecting the options.

Conclusions
In this study, we have proposed a solution for optical mark recognition problems that do not require a dedicated machine or answer sheet. Instead, we used ordinary scanners and printers with A4 paper. We have built a software program that works with different image formats. It can detects filled options and questions with no/multiple filled options. The output of the program is in plain text and can be easily opened in various softwares, including Microsoft Excel. While other tools only work with one-page questionnaires, our tool can work with surveys that contain two pages. The first result looks promising, but still has room for improvement. Most of the questionnaires contain an area for other ideas (and comments) which may contain handwriting text. In the next version, it is our intention that our software tool will utilize the latest achievements of artificial intelligence to solve this problem or at least give users a warning about having handwriting text on questionnaires. We also want to solve the problem with rotated images. This can be done by detecting rectangles on the questionnaires. The problem now becomes selecting the right one (the rectangle that has options inside), but there are multiple and overlapping rectangles on a single sheet. Another solution for the rotating problem that we plan to apply is using checkmarks (bold rectangles located at the corners and the margins of the questionnaires).