An essential first step to processing mixed batches with many types of documents is classification. Document Classification methods quickly sort documents by type using key content and layout attributes to identify them.
The most popular document classification systems are advanced AI-based machine learning algorithms that automatically learn how to classify documents based on samples and user feedback. These systems are very powerful but also very expensive. Only large organizations processing millions of pages each year can afford these enterprise solutions.
SimpleIndex naturally has a simpler way to do classification based on keyword patterns in the document text. Simply create a list of document types and assign one or more unique keywords or phrases that will only appear in that document type to each. Logical operators for AND, OR and NOT prevent false matches by requiring multiple keywords for matching or excluding documents that contain certain phrases.
Keyword-based classification works for the vast majority of applications at a fraction of the cost of AI classification.
After classification, SimpleIndex can automatically launch separate document indexing workflows for each document type found in the classified batch. This is especially useful when documents have different metadata requirements or business workflows associated with them.
Check and Repair All PDF Files
You can set SimpleIndex to assume that it needs to check every PDF file and fix it.
Go to this location in the Windows Registry:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\SimpleIndex\Misc
Create a New String Value called “FixAllPDF” and set the value to 1
Keep Pages in Original Order when Bookmarking
If you want to keep all the pages in the same order that they were imported, even though they all go with different bookmarks then do the following.
1. Open the configuration in Notepad.
2. Search for <BOOKMARK_PAGE_ORDER>
3. Change this line from “false” to “true”: <BOOKMARK_PAGE_ORDER>true</BOOKMARK_PAGE_ORDER>
4. Save and close.
Document Classification
An essential first step to processing mixed batches with many types of documents is classification. Document Classification methods quickly sort documents by type using key content and layout attributes to identify them.
The most popular document classification systems are advanced AI-based machine learning algorithms that automatically learn how to classify documents based on samples and user feedback. These systems are very powerful but also very expensive. Only large organizations processing millions of pages each year can afford these enterprise solutions.
SimpleIndex naturally has a simpler way to do classification based on keyword patterns in the document text. Simply create a list of document types and assign one or more unique keywords or phrases that will only appear in that document type to each. Logical operators for AND, OR and NOT prevent false matches by requiring multiple keywords for matching or excluding documents that contain certain phrases.
Keyword-based classification works for the vast majority of applications at a fraction of the cost of AI classification.
After classification, SimpleIndex can automatically launch separate document indexing workflows for each document type found in the classified batch. This is especially useful when documents have different metadata requirements or business workflows associated with them.
Is it possible to search for and retrieve documents with Windows desktop search?
Windows Search works great with SimpleIndex because all index data can be saved to the folder and file names as well as the file properties, and OCR text can be saved to hidden layers in PDF files. Windows Search will read all of these elements when building its index and will return any matching files when you search.
Using Windows Search on a file server allows for instantaneous searching across terabytes of documents and text for all of the users on your network.
IFilters allow Windows Search to search within file contents.
Here are three popular PDF IFilters that will enable text searching for PDF files:
- Foxit PDF IFilter (commercial)
- TET PDF IFilter (free/commercial)
- Adobe PDF IFilter (32-bit / 64-bit) (free)
If you have issues with PDF text searching in Windows 10, this article has detailed instructions for resolving PDF IFilter issues:
https://fixedit.itxpress.biz/2018/07/05/searching-pdfs-in-windows-10/
- Published in Database & Retrieval, Export, Office PDF Text Processing
I’m using full page OCR. The information is all appearing in the txt file but it is losing format about half way through. Data to the right is ending up at the end of the txt doc. Can this be fixed?
SimpleIndex version 7 solves this problem with the incorporation of the FineReader OCR engine. Full text in PDFs will now flow with the formatting of the PDF.
Legacy Versions: SimpleIndex can also be used with other OCR applications and servers to improve accuracy, formatting and performance. Use the OCR applications to convert the scanned images to text or searchable PDF, and SimpleIndex can extract index values from the text and automatically sort and organize the files.
- Published in OCR
How do you configure full text searching in Retrieval mode?
On the Database tab there dropdown in the lower portion of the panel for Full Text OCR Field. Put the name of the field that will store the full-text data there. This must be configured both for Insert and Retrieval mode configurations. The database field needs to be sufficient length to store the entire text of your document.
Of course, the Insert Mode configuration must have “Enable Full Page OCR” checked to generate full text data from images. Text from MS Office documents, PDF files and existing OCR text files can be used without setting this option.
When designing your Retrieval Mode configuration, create a Text field to use for full text search queries. On the Database tab, set the corresponding “Database Field Name” to the full text database field.
When searching on your full text field, SimpleIndex finds the text you enter no matter where it appears in the document. It is able to match partial words. It does not perform boolean or natural language searches. The text entered must match the document text exactly.
- Published in Database & Retrieval, OCR
How do you configure OCR to read index information from MS Office or PDF documents?
MS Office and PDF files generated by software or PDF printer drivers already have the text you need to recognize in the file. Scanned documents need to use OCR to read text from an image of the page. With Office and PDF files, SimpleIndex can just read the text, which is much faster and accurate than image OCR.
To recognize index fields from the document text, first create OCR fields on the Index tab as you would normally. Next, on the Zones & OCR options tab, check the “Use Full Page OCR for this Field” option for each OCR field. This tells SimpleIndex to process the existing file text.
If the index value is a unique pattern of digits or list of possible values, use Template or Dictionary matching to locate the value within the text. Please see the manual for details on Template and Dictionary matching.
If the value appears in a specific location in each file, coordinates can be used to locate it. When processing text, the X, Y, Width and Height settings correspond to line and column numbers within the file text. This is explained in greater depth in the manual.
SimpleIndex will assume that any TXT file with the same name as a file being processed is the OCR text for that file, so this method can work with any type of file.
Find out more about Optical Character Recognition on the SimpleOCR Guide.
- Published in OCR, Office PDF Text Processing
Can OCR text be saved to MS Word or HTML formats?
Yes. On the Zones & OCR tab of the Job Options, there is a dropdown list for “Full-page OCR file type”. By default it is set to TEXT, but can be changed to WORD, HTML or PDF.
If the output file type is set to PDF, OCR text will be embedded as hidden text in the PDF file.
Find out more about Optical Character Recognition on the SimpleOCR Guide.
- Published in Licensing & Installation, OCR
Can SimpleIndex create searchable PDF Image+Text files with hidden text?
If you enable full-page OCR and output to PDF, the full-page OCR text will be inserted as invisible text on each page.
With the addition of the FineReader Engine in version 7, SimpleIndex now creates PDF files with fully searchable text formatted to flow with the image of the document.
Find out more about Optical Character Recognition on the SimpleOCR Guide.
- Published in Export, OCR, Office PDF Text Processing
Your All-In-One Scanning & Sorting Tool
SimpleIndex® has the ability to perform a wide variety of scanning and document organization tasks quickly and automatically. This makes it a must-have tool for IT departments and consultants who often need to:
- Scan and store various documents on network shares
- Organize existing MS Office, PDF and other files on your network
- Attach files to records in a custom database
- Integrate scanning into custom business applications
- Automate data entry from paper or electronic documents
- Reduce click charges for centralized scanning departments
The wide variety of scanning and indexing tasks you can do with SimpleIndex make it an incredibly useful tool to have in your arsenal. Even if your company already has an enterprise document scanning system, the low cost, fast setup and automated processing features of SimpleIndex can be put to use in many places where your existing system either doesn’t work, is too complex for the task or costs too much in click charges to be feasible.
KB Articles for Document Management
- Oracle database is slow to respond
- What is Document Imaging?
- Using alternate database schemas
- Multiple Sort Fields on Search
- Access Database Connection String
- How do I delete an image and it's database entry?
- Is it possible to search for and retrieve documents with Windows desktop search?
- Will your SimpleQB allow me to scan in old invoices or bank statements directly into QuickBooks?
- How do I use the Media Wizard to create searchable DVDs or thumb drives?
- How do I export index data to a database?
- When I click Media Wizard, an error comes up saying "Installer Not detected in C:\Program Files\SimpleIndex\CDFiles\setup.exe" How do I fix this?
- Is the document Search/Retrieval and View functions available in SimpleIndex or available only with the SimpleSearch add-on module?
- Is it possible to have the scanned image itself added to a database and not just the image path?
- I have a scanner/copier that creates PDF and TIFF files and saves them to my file server. Can I use SimpleIndex to create a searchable CD/DVD from these files?
- When exporting to a database, I get the error "Multi-step operation generated errors"
- Is it possible to restrict user access to certain documents in Retrieval Mode?
- How do you configure full text searching in Retrieval mode?
- How do you create searchable CDs or DVDs that you can distribute royalty-free?
- What is "Match & Attach" mode?
- How do you configure the Autofill feature?
Instant Integration With Any Application
SimpleIndex® provides a great alternative to “reinventing the wheel” using a document imaging SDK to implement scanning in your custom application. SimpleIndex uses a command line interface to enable developers to execute customized scanning workflows from another application. This lets you take advantage of SimpleIndex‘s TWAIN/ISIS scanning, zone OCR, barcode recognition and database export features without having to re-code them all yourself.
Whether you are an IT consultant creating an application for internal company use or a software developer looking to add scanning features to your application, SimpleIndex provides the easiest solution available.
To use SimpleIndex as a scanning front-end for your application, simply:
- Create a SimpleIndex job file configured to scan, index and save files the way you want
- (Optional) configure the database export to create/update document records in your DB
- Use a “Shell” command to execute the job file from your program
When you execute the job file from your program, SimpleIndex will scan, read barcodes, OCR zones, etc. and organize the documents into the folders and filenames your program requires. The database export can be configured to add or update records in your database with information captured from the documents as well as the image file path or binary data. Index values can be passed from your program to SimpleIndex via the command line. The whole process can be configured to run minimized so the user never even sees the SimpleIndex screen.
KB Articles for Integration
- Exclude Index Field from Index Log
- Turn Off Prompts and Pop Ups on Job Configurations
- Change the Font Size of Index Fields
- Large documents (>500 pg) Slow to Process - Workaround
- Regular Expression (RegEx) - Syntax or Type
- Index With Non-Latin Character Sets
- Skip to Blank Index on Save Index
- Stop Autorun When Double Clicking Configuration
- Autonumber Increment Value
- Overlap of SimpleView Viewer in SimpleIndex Display
Sort and Index MS Office Documents
This video shows the Sort My Documents sample configuration. Word documents, Excel spreadsheets and PowerPoint presentations are automatically sorted using the SimpleIndex template and dictionary matching algorithms.
The files are reorganized using the Sales Rep, Customer, Document Type and Date extracted from the text. SimpleSearch is then used to search and view the sorted files.
Streamlined Interface
The Fastest, Most Accurate Scanning Solution
Simple
KB Articles for Streamlined Interface
- Command Line to Auto-close SimpleIndex
- Command Line Sample
- Network Scanners & Copiers
- Instant Integration With Any Application
- Automatic Indexing Using Existing Data
- Automated Processing & 1-Click Interface
- Invoice Processing with Full Page OCR
- Indexing Solutions with Barcode Recognition
- Reduce Click Charges for Data Capture
- Your All-In-One Scanning & Sorting Tool
PDF Invoice OCR Demo
This demonstrates the PDF OCR text processing capabilities of SimpleIndex by extracting the Document Number, Date, Document Type, Customer and Total from a number of Estimates and Invoices.
All of this information is read automatically using the existing text layer of a computer generated PDF, such as those created using PDF printer drivers. Template and dictionary matching algorithms are used to locate and extract the correct data values from the text.
Since the existing text is being used, OCR is not performed. This makes processing much faster and 100% accurate. OCR can be used to get text from scanned PDF files with no existing text.
FAQ Related to PDF Invoice
Coming SoonZone OCR and Dynamic OCR
Many document scanning solutions use Zone OCR to obtain index data from the page, while SimpleIndex improves upon this time-tested but ultimately limited model with its unique Dynamic OCR feature. Let’s look at the difference between the two methods:
Zone OCR
Zone OCR is used to read document indexes or tags from text on the page. It is a great way to automate the data entry associated with scanning documents.
However, there are several limitations to zone OCR that must be overcome:
- Index information must be in the exact same place on every page
- Documents shift and skew during scanning, causing the zones to not line up
- If surrounding lines or text on the document are too close, they can encroach on the zone
Dynamic OCR
SimpleIndex overcomes these limitations by using Dynamic OCR technology to locate the desired text even when it moves around on the page. Our simplified version of Dynamic OCR works great for many types of documents at a fraction of the cost of other solutions.
- Index information can appear anywhere on any page
- Unwanted characters are automatically ignored
- Find unique patterns of letters and numbers using Template Matching
(Social Security #, Date, etc.) - Use Dictionary Matching to find a value from a list of possible values
(Vendor Name, Document Type, etc.)
Dynamic OCR Examples
In the video we see how SimpleIndex approaches a typical Zone OCR example. With SimpleIndex you can use large zones that give a wide margin for error. Template and Dictionary matching are then used to extract the 7-digit Account Number, 6-digit Order Number and Company Name. SimpleIndex discards the surrounding text and keeps the correct value.
Another common example is finding a unique identifier, for example a social security number, that could appear anywhere on the page. Simply enter the template ###-##-#### and SimpleIndex will search the full OCR text until it finds a match. Since only one social security number is likely to appear on the page, a match on this pattern is almost certainly the required value.
With dictionary matching, you can give SimpleIndex a list of possible values and it will automatically search the zone or page for each possible value until it finds a match.
Many dynamic forms processing applications can be implemented using these simple algorithms. This makes SimpleIndex far more versatile than other zone OCR solutions that require the index value to be in the exact same location on every page. Yet SimpleIndex costs only a fraction of the price!
SimpleIndex‘s dynamic forms processing can greatly speed up data entry by eliminating a good percentage of indexing work. For many this can put the labor cost of scanning within their reach.
Dynamic OCR can also be applied to MS Office and PDF files, creating a fully automated process for intelligently indexing and reorganizing electronic documents.
Support for Regular Expressions
SimpleIndex OCR has a simple built-in template format, as well as support for Regular Expressions. Regular Expressions (RegEx for short) let you define complex search patterns to extract matching values from the text. This greatly enhances the functionality of the dynamic OCR in SimpleIndex, making it capable of finding variable-length fields with no distinct pattern.
Regular Expressions are a commonly used in text parsing applications. The Perl programming language makes extensive use of RegEx, as do UNIX utilities like “grep”. Many programmers and IT personnel are already familiar with RegEx and can create complex expressions without specific training.
Click here for a reference guide to Regular Expressions
New OCR Features in Version 9
- OCR language pack now includes all available Tesseract languages including Hindi, Tamil, Arabic, Chinese, Thai, Vietnamese, Japanese, Korean, Indonesian, Hebrew and many more.
New OCR Features in Version 8
Versions 7 and above include the industry leading ABBYY FineReader® OCR engine for dramatically improved OCR accuracy and speed. Other OCR enhancements in version 8+ include:
- Point & Click OCR
- Tesseract OCR Engine now included with all license versions
- Match OCR index fields against other index fields
- Skip OCR processing on imported files that already include text, such as PDF + Text files for faster batch times
How to Configure SimpleIndex OCR
Watch this Simple Software University training video to see how to configure and run an OCR job with SimpleIndex.
KB Articles for Optical Character Recognition (OCR)
- Languages Supported in SimpleSoftware OCR Engines
- What is Document Imaging?
- Change the Dictionary Separator Value
- Change the OCR Font or Type
- Regular Expression (RegEx) - Syntax or Type
- Autonumber Increment Value
- I'm using full page OCR. The information is all appearing in the txt file but it is losing format about half way through. Data to the right is ending up at the end of the txt doc. Can this be fixed?
- Is there a way to just use part of a bar code or OCR value? For example, extract "50" from the value "124450"
- If I have a form which is filled manually by hand, can SimpleIndex read the data from it?
- How do you train the OCR engine for better accuracy?
Video Demos
These videos demonstrate several ways SimpleIndex® can automatically index different types of documents. If you are new to SimpleIndex, watching these videos is the easiest way to see what it can do. You can follow along using the sample files included in the SimpleIndex Trial.
- Zone OCR with template matching
- Document barcode recognition
- PDF OCR text parsing
- Sort and index MS Office documents
- Indexing with full-text OCR
- Running jobs from an icon
The sample files are copied to your Configuration Folder when you run the SimpleIndex Trial for the first time. If you can’t find the samples, copy them with the Global Settings Wizard in the File menu.
Compare Major Scanning Solutions
Integrated Solutions Built with SimpleIndex
Zone OCR with Template Matching
Document Barcode Recognition
PDF OCR Text Parsing
Sort and Index MS Office Documents
Full Page OCR Invoice Processing
KB Articles for Optical Character Recognition
- Languages Supported in SimpleSoftware OCR Engines
- What is Document Imaging?
- Change the Dictionary Separator Value
- Change the OCR Font or Type
- Regular Expression (RegEx) - Syntax or Type
- Autonumber Increment Value
- I'm using full page OCR. The information is all appearing in the txt file but it is losing format about half way through. Data to the right is ending up at the end of the txt doc. Can this be fixed?
- Is there a way to just use part of a bar code or OCR value? For example, extract "50" from the value "124450"
- If I have a form which is filled manually by hand, can SimpleIndex read the data from it?
- How do you train the OCR engine for better accuracy?
MS Office & PDF Text Parsing
Office Videos | PDF Video
The template and dictionary matching capabilities of SimpleIndex‘s OCR function can be used to extract index information from the text of existing MS Office and PDF files, or any file with an accompanying TXT file. SimpleIndex® will search the document for matches on unique patterns and value lists, then index the document with the matching data. Zone coordinates can be set to limit the search area to pre-defined regions on standard forms. The result is a fully automated indexing and renaming process for all your electronic documents!
Using existing text, SimpleIndex can index and rename hundreds of files each minute and achieve perfect accuracy. These files can then be quickly searched with SimpleIndex Retrieval, SharePoint and Google search engines, or uploaded into your company’s document/content management system or custom business applications.
Enhanced Text Parsing & PDF Support
MS Office and PDF text parsing features are now included in the Basic version of SimpleIndex, making it much more affordable to enable automatic document sorting on the desktop. Additional Office and PDF features include:
- Convert any MS Office, HTML, XML and image files to PDF before processing
- Read and write password protected PDF file
- Searchable PDF output (Image + Hidden Text)
- Interactive template builder and tester
- Easily select PDF or PDF/A output format
- Native PDF viewer and auto-repair of problematic PDFs
- Read data from PDF forms
- Populate blank PDF forms with index data
Batch Convert Office Documents to PDF
If you have Microsoft Office or OpenOffice installed, you can use SimpleIndex to automatically convert MS Office documents to PDF files for archival. PDF files are better for archival than editable formats like Word and Excel. They can be annotated, encrypted, searched and viewed with free PDF readers.
There are many free applications that let you convert documents to PDF one at a time. SimpleIndex lets you convert thousands of files at once while it also extracts data from the text for indexing or data entry automation. This feature is ideal for migrating or archiving Office documents to SharePoint, document management systems and custom web applications.
Quickly Organize Any File on Your Computer
SimpleIndex lets you process any type of file on your computer. If an OLE-enabled viewer is installed, SimpleIndex will display the document on the screen. Other documents can be opened automatically in their default application when they are indexed. Quickly type index field data that can be used to reorganize the files into subfolders and structured filenames for browsing and searching on your network, or uploaded to your document/content management system or custom business application.
If the file has an accompanying text file (*.TXT) with the same name, the text in that file can be used for index field extraction, fully automating the process.
Viewing & Indexing MS Office Documents
SimpleIndex features full support for viewing and editing MS Office documents (Word, PowerPoint and Excel) on computers with or without MS Office installed. The full application interface is displayed within the SimpleIndex viewer, letting users view the full content of the documents, edit them with all the features of MS Office and save the changes. Modify privileges can be denied using Windows file security or by the SimpleIndex administration wizard to keep out unauthorized changes.
If MS Office is not installed, SimpleIndex can open and display them in the built-in viewer in read-only mode.
KB Articles for MS Office & PDF Text Parsing
- Change the Dictionary Separator Value
- Regular Expression (RegEx) - Syntax or Type
- Check and Repair All PDF Files
- Keep Pages in Original Order when Bookmarking
- Do Not Combine Pages to 1 Bookmark
- Can I split a PDF based on bookmark values?
- Is it possible to search for and retrieve documents with Windows desktop search?
- Can SimpleIndex read bar codes from existing PDF files?
- Is there a way to just use part of a bar code or OCR value? For example, extract "50" from the value "124450"
- How do you configure OCR to read index information from MS Office or PDF documents?