So you’ve got some exams to mark and wondering what the process is like? Here’s a quick video to give you an idea
As an Academic doing marking with this system, you don’t need any software other than a PDF reader ( that can speak acroforms). Then you will just a file formatted something like this (preview below), which you then edit, save and return.
The design is flexible so it can be changed to suit feedback or different procedures in different unis.
I learned that a lot of PDF viewers have patchy implementations, so you get a nicer experience if you go with
- Adobe Reader
- Drawboard PDF (for pen display)
- (your suggestions here)
Things that DON’T work, or have quirks
- Microsoft Teams (you have to download to your desktop and use something else)
- Chrome (can edit but not save, known chrome bug)
- Edge (can edit, but does uses small letters so not as satisfying)
Here’s an original submission, with no pdf.gradex™ treatment. You can delete the annotations. Oops!
If you fancy seeing what happens to your work next, send me your edited form (and let me know if I can post here ….)
Work were kind enough to give me a blog spot to talk about this project:
That also provided the impetus to put up a quick demo release. I do my dev on linux, but it seems pretty standard across the people I’ve sent demos to (Canada, Scotland) that teaching offices are using Windows 10. You can get the demo code from github. If you want to experiment with your own layouts, then there is information on the editing process in the demo README.
Even though I know it is a good thing in projects like this, the curse of YAGNI is always in your mind when you are sinking time into automation processes with their usual little wrinkles that (hopefully) no-one else need ever be burdened with. It started to pay off today, already. I had some user feedback on the grid design, and within 10min of the call ending, I had the design revised and working. I’d split the question boxes to nudge freehand markers to spread their characters out so I could do OCR bounding box estimation better. Of course, I don’t use half marks, and had assumed no one else did either. Oops. I like it better this way, and we can handle the OCR by either (a) begging and pleading with markers to put a space between their handwritten characters, (b) not doing it, (c) giving in an using an external image service (some issues there). I estimate it would have been about an hour’s work to do the geometry change by writing coordinates into golang structs, and it would not have been done with such good alignment either (or the with the bordeline relief-smugness hybrid of knowing it wasn’t YAGNI). So thanks self-from-last-week.
Here’s an example of a page that has been marked by keyboard, and the moderator has found an adding error, which they’ve corrected. The checker re-keys the numbers as part of the checking procedure.
Here’s a couple of the steps shown on their own, showing the moderator has only to check the page if all is ok, but can handle minor issues and communicate them back to the office. If there needed to be additional stages, we’ve got some other coloured “query” and “resolve” sidebars waiting in the wings.
Since we’re going for robustness and safety, we’re rendering all the annotations at each step, exporting to an image file, reimporting that into a new page that has the marking forms added, then exporting that again (phew!) – the stuff in blue all happens in a single command.
The process in averages around 1.4sec/page/stage at our current quality setting of 90 JPEG quality at 175dpi.
mark 1.3s/page moderate-active 1.3s/page check 1.4s/page
The round trip to collect scripts from our exam halls easily comes in at over 45 minutes, or enough time to process about 2,000 pages.