Laptop Displaying the GigaOm Research Portal

Get your Free GigaOm account today.

Access complimentary GigaOm content by signing up for a FREE GigaOm account today — or upgrade to premium for full access to the GigaOm research catalog. Join now and uncover what you’ve been missing!

GigaOm Performance Benchmark for Intelligent Document Processing (IDP) Solutionsv1.0

Performance Evaluation: Automation Hero Hero Platform and ABBYY FlexiCapture

Table of Contents

  1. Executive Summary
  2. Drivers for IDP Data Extraction Accuracy and Handwriting Recognition Tests
  3. Comparing Vendor Offerings
  4. Test Results
  5. Conclusion
  6. Appendix
  7. Disclaimer
  8. About Automation Hero

1. Executive Summary

This benchmark report aims to compare the performance of two intelligent document processing (IDP) solutions offered by Automation Hero and ABBYY in terms of accuracy of data extraction and handwriting recognition. The first aspect of this performance benchmark test is extracting data from different invoices (in this case, hotel receipts). Both products were deployed in the cloud, and testing was done on the same set of invoices. The use case is IDP-enabled invoice processing for accounts payable (AP) automation. The other aspect is comparing performance regarding the accuracy of data extracted from handwritten documents (specifically, snippets of handwritten text from actual medical forms).

The document structures of these invoices were identical to transparently identify how each IDP solution would treat an unfamiliar invoice layout. Both IDP products processed the same document sets. This provision ensures a neutral collection of test documents that neither IDP product is pre-trained to recognize.

We found that the Automation Hero Hero Platform delivered 68% greater accuracy than ABBYY FlexiCapture in terms of global average (“headers” and “line items” combined) for invoice processing. In terms of headers (e.g., invoice number, invoice date, amount, customer name, customer address, etc.), the Hero Platform delivered 67% greater accuracy than ABBYY FlexiCapture IDP product. And for line items in invoices, the Hero Platform delivered 69% greater accuracy than ABBYY FlexiCapture.

For the other use case, handwriting recognition for snippets from actual medical forms, Automation Hero context-aware optical character recognition (OCR) delivered 281% greater full-field accuracy when compared to ABBYY FlexiCapture.