Announcing Perspec 1.0 - Adrian Sieber's Website
I'm very excited to announce the 1.0 release of Perspec!
Perspec is a desktop app for correcting the perspective of images.<br>This is primarily useful for photos of documents and receipts,<br>but it can be used for any kind of image.
This has finally become the app I envisioned<br>when I started working on the project 9 years ago.<br>I didn't think it would take me this long to get here,<br>but I'm very happy with the result and I hope you'll like it too!
Initial Motivation
You're probably familiar with the scanner apps available for mobile phones, like<br>Adobe Scan, vFlat,<br>SwiftScan, … and numerous others.<br>Scanning functionality is also integrated into<br>Dropbox,<br>and these days even natively into iOS itself.
However, I don't like working on my phone and I'd rather just take photos of the documents and receipts<br>and deal with cleaning them up and organizing them on my computer another day.<br>There, I have a big screen, a keyboard, and a precise mouse,<br>which makes editing faster and more accurate.
Also, the mobile apps make some annoying technical decisions<br>in the name of giving users something they're familiar with.
For example:<br>If you store a document as a grayscale PNG,<br>you can get small file sizes without introducing any compression artifacts.<br>However, all the popular apps will give you a grayscale JPEG image<br>with a much bigger file size and worse image quality,<br>just because JPEG is what people are familiar with.
Or maybe I'm giving them too much credit<br>and they actually don't know that PNGs can<br>be smaller than JPEGs if the image contains large areas of uniform color,<br>whereas for normal photos, JPEGs are smaller than PNGs.<br>And no, converting it to PNG afterwards is not an option,<br>as by then the image already contains all the JPEG compression artifacts.
For example, let's compare the results of scanning the following document:
The other apps produce bigger files, and you can clearly see<br>the compression artifacts that degrade the result.
App<br>Result<br>Preview<br>Notes
Perspec
~110 kB, PNG
View result
Scanner Pro
~190 kB, JPEG
View result
Extracted JPEG from exported PDF
iOS
~300 kB, JPEG
View result
Extracted JPEG from exported PDF
Another thing that annoys me more than it should is the<br>ridiculous detection previews that seemingly every app includes these days:
While you're taking a photo, the app shows you a live overlay<br>of where it detects the document.<br>This, however, doesn't help you at all.<br>Just because it can detect the document correctly in the preview video feed<br>doesn't mean it will detect it correctly in the final photo.<br>Due to the higher resolution, different lighting (exposure times, flash, …),<br>and different contrast,<br>the detection will often be quite different in the final photo.
So all the preview is telling you<br>is that there is indeed a document in front of your camera,<br>which you already know since you placed it there. 🤦♂️
Lastly, and most importantly, I knew I could build<br>a better document detection algorithm for the kind of photos I was taking.<br>The detection in existing apps would often be slightly off,<br>even if you had a good picture<br>with good contrast between the document and the background.
Most apps use some kind of edge detection step in their pipeline,<br>as Dropbox explains here.<br>But I knew that documents and receipts often don't have straight edges<br>but rather wrinkled or curved ones.<br>When you try to match even just a slight curve with a straight line,<br>the endpoints will be quite far off.<br>So instead, the app should try to detect the corners and build up the document from there.<br>There is a detailed explanation of the computer vision techniques later in the post.
The Long Road to 1.0
I was still a student when I started working on Perspec<br>and had to scan a lot of stuff for my studies,<br>so I had plenty of motivation to build something like this.
Sure, you could also fix the perspective with Photoshop, Affinity Photo, or GIMP.<br>But the overhead is substantial:<br>Open each photo, find the perspective tool, drag the corners,<br>pick the right export settings, repeat for the next photo, and so on.<br>These tools are built to do everything with any image<br>and not to churn through 50 receipts as quickly as possible.<br>I wanted an app that's focused on this one task,<br>with a workflow that's as streamlined as possible.
My first iteration was a fully automatic CLI app called Perspectra,<br>implemented with Python and scikit-image.<br>You'd pass your image and it would try to detect and extract the document for you.<br>Simple as that.
Although I actually liked scikit-image —<br>feature-rich, yet more straightforward than OpenCV —<br>I quickly realized that I absolutely do not like Python.<br>But more importantly, I realized that I also needed a GUI<br>to fix incorrectly detected document boundaries,<br>as the fully automatic CV pipeline would never get all documents 100% right.
And how do you build a desktop app with a GUI?<br>Obviously with Haskell. 😝<br>Joking aside, I had recently...