To a computer a PDF isn't a document with an invoice number and a taxable amount inside it, it's a sheet with marks on it. You see the invoice number because you know where to look, the program has to be taught. The whole difference between a reading that works and one that wastes your time sits right here, and knowing it helps you understand what to ask for, even if in the end you build it yourself or have nobody build it at all.
First question, does your PDF have the text inside it
It's the thing to check before anything else, and it takes ten seconds. Open the document and try to select the invoice number with the mouse. If it highlights and you can copy it, the text is there. In that case the program reads exactly the same letters you see, with no margin of error on the characters, and the reading starts with a big head start.
If the cursor selects nothing, that PDF is a photograph. It happens with everything that goes through a scanner or arrives as a photo from a driver's phone. There you need character recognition, an OCR, which guesses the letters by looking at the shape of the pixels. It works, but it guesses, and now and then it gets it wrong: a 0 read as an O, an 8 read as a B, a lost comma that moves the figure. With straight, clean scans the errors are few, with a crooked, crumpled sheet photographed in half-light there are plenty.
And the quality of the scan depends on whoever sends you the document, not on you. So if the supplier changes scanner or the warehouseman changes habits, the reading gets worse without anyone having touched a thing.
How the program works out which number is which
There are two ways, and they change things quite a bit.
The first is by position: you tell the program that the invoice number sits at the top right, in that rectangle, and it goes and reads there. Fast, precise, free to maintain as long as the document stays the same. But all it takes is the supplier redoing the template and moving the logo two centimeters, and from that day on you're reading the wrong area.
The second way is to search by context: the program looks for the words “Document total” wherever they have ended up on the page, and takes the number sitting next to them. It stands up better to changes in layout, but it has to be tuned to the variants, because one supplier writes “Total”, another “Taxable amount”, and another puts three of them one under the other.
In practice you use both together, with your own department rules on top: if this supplier always has the item code in that column, you make use of it. It's dull work to set up once, but it's what decides whether the thing works afterwards.
The fields you pull out of an invoice
It depends on what you record in your business software, but the usual round is this.
- Who sent you the document, company name and VAT number, which is the most useful piece because it's what hooks the document to the right supplier record
- Document number and date, which together with the supplier tell you whether you have already recorded that invoice
- Taxable amount, rate, tax and total
- The line items, if you need them, so item code, description, quantity, price
- The references that tie the document to something else, the order number or the delivery note
The line items are the hard part, much harder than the header. A table that breaks across two pages, a long description that takes up three lines, a discount written at the bottom, they are all cases that have to be handled by hand in the rules. If all you want is to record the header and the accounting side, you're often better off leaving the line items alone and saving yourself half the work.
The checks, the part that counts more than the reading
A program that reads almost every document well is worth nothing if you don't know which are the ones it got wrong. A value read badly and slipped into the archive without anyone noticing does more damage than ten documents sitting on hold, because you find out two months later, when a balance doesn't add up.
That's why the checks go before the data goes in, not after.
- The math has to add up, taxable amount plus tax equals total, and the sum of the line items has to give the taxable amount, if it doesn't match to the cent there was a reading error somewhere
- The VAT number has to have the right count of digits and pass the check on its final digit, it's a verification you write in two lines of code and it catches almost every OCR error
- The date has to be a plausible date, not 2019 on an invoice that arrived today
- The supplier and the items have to exist in your business software already, otherwise there isn't much to enter
- That document number from that supplier must not already be recorded, so you avoid the duplicate
The check on the totals is the most valuable of the lot, because it doesn't verify one field at a time, it verifies whether the document makes sense as a whole. If the OCR read 1.180 instead of 1.180,50, the sum doesn't add up and it shows straight away.
What happens once the data is in order
By this point the data already exists in a clean state, and getting it into your business software is the least interesting part. If your program has a file import or a direct integration, you use that and it ends there. If it doesn't, and older business software often doesn't, the system opens the screens and fills in the fields the way a person would, only without getting distracted.
Keep a log of what was done and on which document, because the day something doesn't add up you want to get back to the original PDF in a minute, instead of going by memory.
Doubtful cases are not to be guessed
Sooner or later the document cut in half arrives, the one with the total covered by a stamp, the one from a new supplier with a template nobody has seen. The temptation is to let the program take a guess, and that's the worst mistake of all.
The sound rule is a simple one, if a check doesn't pass the document stops and ends up in a queue, where a person looks at it and decides. Those are the cases that deserve a human head, and there are few of them. The rest goes through on its own. If instead the queue fills up every day it means the rules are badly tuned, and that's a useful signal, not a nuisance.
When it isn't worth doing
I'll spare you the part where I tell you it always works out, because it isn't true.
If you get five invoices a month you'll never win back the time it takes to tune the rules, better to carry on by hand. If every document comes from a different supplier in a different format and never repeats, the reading turns into a permanent chase. If your scans really are terrible, crooked and faded, the shortest road is to ask for the original PDF by email instead of photographing the sheet, and sometimes that alone closes the problem without writing a line of code.
The case where the game is worth the candle is the opposite one, many similar documents that come back every month, from the same suppliers, with a person spending hours retyping them.
How to tell whether it makes sense for you
You don't need a long analysis, very little will do. Look at how many documents really go through in a month, try selecting the text on a few of those PDFs to see whether you're in the easy case or the OCR case, and count the real minutes it takes a person to record one, from opening the file to saving.
Let's do a made-up sum, just to show the method, it isn't anybody's real data: say two hundred documents a month and six minutes each, that's twenty hours a month, meaning a person who does nothing else for two and a half days. With your real numbers the answer will be a different one, maybe much smaller, but at least it's a number and not a feeling.
If the sum tells you the time is there, then look into it, and if it doesn't you've saved yourself a pointless project, which is a good result all the same.