The Business Application of Automated Document Classification

Automated document classification solves the problem of recognizing the category of a document based on its content and appearance.

State-of-the-art document classification models can recognize documents thanks to deep learning principles. In this article, we’ll explain the following aspects of this technology:

• The problem of manual vs. automated document recognition
• What are the main challenges of automated document classification models
• How Alphamoon’s solution delivers a leading technology of AI document classification

Let’s get right to it.

What is Document Classification?

Document classification problem refers to the recognition of the category of a document based solely on its content and appearance.

While any company deals with a mix of paperwork, most document categorization problems are case-specific. Classification of documents means that people recognize documents based on their features and then assort them accordingly. Documents can be classified into folders based on their characteristics (invoices grouped), role in the workflow (corrected invoice is the updated version of a corresponding invoice), or validity (scans of IDs sent to a bank from local branches).

While this job seems easy on paper, humans tend to mix records erroneously, mainly when they are very similar.

For example, Account Payable teams process, for the most part, invoices, corrected invoices, receipts, purchase orders, and other documents within the domain of transactional paperwork. Workers in the field of healthcare process personal records – medical certificates, health records, ID scans, etc. These two separate groups conduct the same operations, including categorization, extraction of information, and archiving the paperwork. However, the class of extracted information differs, as well as the sort of documents used in their line of work.

Manual vs. Automated Document Classification

The above relates to the problem of manual classification. While many employees aren’t grasping at straws while determining the type of document – it’s still one of the most manageable parts of processing paperwork – there are a few tricky parts of the workflow.

When employees digitize documents, they usually scan them first. A nasty pile of paper, comprising hundreds or even thousands of pages, becomes one chunky PDF file. Essential document processing tools can help split pages, but that’s often the end of the road. Labeling documents is, imaginably, a tedious task to perform. And automation can help avoid wasting time.

Another application is email automation and processing emails with attachments in particular. To speed things up, the AI model recognizes what types of documents are attached to process them further according to the specific business rules in a given business process. By integrating the inbox with an intelligent document processing platform, you can quickly know how many offers, invoices, or CVs end up in your mail. With the addition of a data extraction feature, these documents turn out to be data sources – such as contact information, amounts to be paid, and so on.

Challenges of automated document classification

The task of recognizing documents is relatively easy for humans, and it may seem to be a similarly easy-peasy job for robots. However, most software is unable to conduct document recognition with high precision.

Here’s why.

Documents that belong to the same category may vary significantly in terms of visual appearance, content, layout, etc. From the programming perspective, preparing the code to handle any specific document type is intractable.

Furthermore, the number of different categories may be significant and vary over time every time business processes change due to internal or external reasons. As the range of document classes grows, artificial intelligence may have difficulty determining the differences. The issue deepens when types may be very similar in terms of document appearance and content (e.g., Invoice and Corrective Invoice), which is confusing even for humans.

So, whenever the order of classification changes, the software needs to be agile and adaptable.

Then, there’s the layout and formatting of the documents themselves.

They may appear in different formats; they can be editable (e.g., DOCX, PDF), non-editable scans (PDF, TIFF), photos (PNG, JPEG), etc. A machine has to pre-process them into a unified format before classifying them. That’s the responsibility of the OCR part of the model (Optical Character Recognition), which turns images into text. OCR enables the model to perform other tasks relevant to the categorization task – text classification and object detection.

Documents may be long too and contain a lot of text and other components like tables (more on data extraction from tables here) and images based on which the classification has to be performed, and this is challenging from the processing point of view.

Finally, all intelligent document processing platforms struggle when the quality of scanned documents is poor. This is particularly difficult for OCR software, which turns images into text.

Note: You can read about the most common document-related issues in our article.

Alphamoon’s State-of-the-art Document Classification technology

Intelligent document processing combines several machine learning techniques, including NLP (natural language processing) and computer vision. It’s the latter that mimics the human ability to see things. Since algorithms can’t take a glance at a paper and determine its type like humans, ML engineers came up with ways to teach AI models to translate images into non-abstract entities.

In Alphamoon, we use technology based on deep neural networks for document classification. We use a multimodal deep learning neural network that can handle three different modalities:

• Textual context
• Position and size of the words
• Visual appearance of the document

This approach is similar to how humans look when they are determining the category of documents.

Now onto the nitty gritty.

Multimodal deep learning models are very effective in document classification and achieve outstanding results – usually +95% of accuracy on average. Older technologies rely heavily on templates, where various rules are used to conduct categorization. Such a legacy tool would struggle in any case that steers away from the beaten path.

AI-based classification models can also be easily applied to different document classification tasks (by a different task, we mean a different set of category labels). Their component models can be pre-trained in an unsupervised/self-supervised manner and then fine-tuned to a specific task using transfer learning. In consequence, they require relatively small portions of data to learn new tasks effectively. They can generalize to new and previously unseen types of formats, content, and layouts that belong to the same category.

Alphamoon’s automatic classification software tackles most of the common challenges we’ve explained earlier.

• The model can deal with long documents by using a unique structure of network components.
• By using different modalities, Alphamoon can partially handle the problem of low-quality documents and documents that contain objects like tables and images. This includes recognizing invoices and corrected invoices.
• Our engine handles a large number of different categories swiftly and adapts to new classes and new document types by using dedicated continual learning algorithms.
• Continual learning as part of the model leads to improved document classification accuracy over time, thanks to new training examples. Furthermore, this helps in future cases of distinguishing similar categories of documents and tackling the problem of an unknown/unseen sort by using a particular type of loss function for training the model.

Get started with document classification

Automatic document classification can help your team assort documents faster with the assistance of AI. On top of that, Alphamoon’s platform means:

• Quick & easy deployment. Our team helps you onboard so that you can reap the benefits of intelligent document processing fast.
• Intuitive UI. Alphamoon’s platform is designed to support business users who don’t possess dev skills.
• Integrations. Alphamoon can be integrated with Google Drive, One Drive, and Dropbox to source documents from your existing folders.
• Continual learning. The feature of continual learning guarantees that the model learns as it goes, hence providing improved accuracy that is specific to your use cases.
• Complementary features. To gain the most out of Alphamoon, you can consider creating an entire workflow with OCR, Data Extraction, and Full-Text Search features.

Sounds like a plan? Talk to our sales.

Complementary reading

• The sustainable aspect of intelligent document processing
• What is intelligent data extraction?
• Intelligent Document Processing 101: Where to begin with IDP