Open database integration is a powerful feature of SimpleIndex® and one that furthers its interoperability with custom programs. Instead of using a proprietary database, SimpleIndex allows you to map its index fields to cells in any database table. It can be configured to create new records, update existing ones or retrieve them for viewing. Using these three basic database functions, SimpleIndex is able to interface with and operate on any database.
SimpleIndex can connect to any ODBC-compliant data source in order to match existing data to documents or export data extracted from documents.
Database Integration
Unprecedented Control of Database Interactions
Open database integration is a powerful feature of SimpleIndex® that furthers interoperability with custom programs. Instead of using a proprietary database, SimpleIndex allows you to map index fields to cells in any database table. It can be configured to create new records, update existing ones, or retrieve them for viewing. Using these three basic database functions, SimpleIndex is able to interface with and operate on any database.
Automatic Indexing Using Existing Data
SimpleIndex’s Autofill feature is an easy way to associate many index fields with one document without retyping data that already exists in another database. Autofill uses a database lookup to retrieve records that match key values entered by the user. Then blank index fields are automatically filled with the lookup data. After entering only one field, data autopopulates, resulting in a document database.
The key field may be typed by the user or read from the document automatically using barcode recognition or OCR. The lookup is performed either when the user changes this field or when the index values are saved. If the lookup finds multiple matching records, the user will be notified and the first set of values will be used by default.
When used with pre-index batches, key information can be read automatically from barcodes or OCR and matched to database records with a single click.
Custom Code Extensions for Autofill and Export
Customized Autofill and Export functions extend this functionality to use APIs and web services to populate or export index data when a direct database connection is not possible. This allows you to directly interface with any application to populate and export document data.
An example is our ChatGPT document analysis service, which uses the same extension interface to provide AI document analysis results that can be used for classification and natural language data extraction.
Match and Attach Mode
SimpleIndex features “Match and Attach” mode, which combines the power of the Autofill feature with the ability to update existing records in your database. This lets you “attach” images to existing database records by “matching” them with values from SimpleIndex.
XML and JSON Data Sources
SimpleIndex also has the ability to read data directly from XML and JSON files. Many document management and cloud storage platforms provide document metadata in XML and JSON files whenever you attempt to migrate data, along with corresponding files that may or may not have logical filenames. Use SimpleIndex to extract the data from these files, organize the documents, and migrate the metadata to a database or new document management platform.
Output to XML or Any Text Format
SimpleIndex offers direct export to pre-formatted XML files, as well as the SimpleExport module that can output any XML schema or text file format.
SimpleExport makes it possible to integrate with any third party database, application, or document management systems (DMS) that accept an XML or text file import.
SimpleExport converts the index values from a SimpleIndex job to any output format through XSL Transformations (xslt). You may use one of the built-in scripts to export to a pre-defined system or design your own. SimpleExport runs automatically after each batch is exported, or it can run as an unattended Windows service.
SimpleExport can also be used as a standalone application for automating XSLT transformations, converting CSV or TSV (delimited text files) to XML, converting images to base64, or integrating other scanning applications with third party software.
Find Out More
KB Articles for Database Integration
- Oracle database is slow to respond
- What is Document Imaging?
- Using alternate database schemas
- Multiple Sort Fields on Search
- Access Database Connection String
- How do I delete an image and it's database entry?
- Is it possible to search for and retrieve documents with Windows desktop search?
- Will your SimpleQB allow me to scan in old invoices or bank statements directly into QuickBooks?
- How do I use the Media Wizard to create searchable DVDs or thumb drives?
- How do I export index data to a database?
Using alternate database schemas
Please refer to the Wiki Documentation for the complete Database reference.
In SQL Server, Oracle and MySQL (among others) user logons are associated with a default “schema” that determines which tables and views they can see.
Users may have access to other tables and views that aren’t in their schema. When connecting to the database these tables will show up in the list, but when you try to get the list of fields you get an “Invalid object reference” error.
To reference these tables, use [Schema].[Table] for the table name to explicitly reference the table.
You can also change the user’s default schema on the server to the one matching the table.
A third option is to create a view under the user’s default schema that selects the necessary records.
At the present time you cannot enter [Schema].[Table] in the wizard, so if you need to use this workaround configure the job in the wizard and modify the table name in the old job options screen.
Access Database Connection String
Please refer to the Wiki Documentation for the complete Access reference.
If you would like to create a database string for an Access database that includes a password use this format:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=path\database.mdb;Jet OLEDB:Database Password=db_password;
How do I delete an image and it’s database entry?
Please refer to the Wiki Documentation for the complete Global Settings Wizard
reference.
If your Job Settings Wizard configuration’s of Database mode setting is set to “Update Existing Records”, the database record and the image will be deleted when you delete an image from a batch. The Table Name must be an updateable query or table and you must have sufficient permissions for both the database record and the file.
You cannot delete files with SimpleSearch. To enable retrieval users to change indexes, rename or delete files, use a SimpleIndex Standard license.
- Published in Database & Retrieval
How do I export index data to a database?
Please refer to the Wiki Documentation for the complete Export reference.
There are a variety of ways to connect to your database. Detailed instructions are provided in the Manual (check the Help menu). Here is a brief overview of the steps involved:
-Create a job configuration to scan and index files
-On the database tab, set the “Database Mode” to “Insert New Records”
-To use ODBC, enter the data source name or file in Data Source
-To connect directly, select your database type under “Select a Data Source” and click Start. A series of dialogs will prompt you for database connection information.
-Select destination Table or View and click Reload
-For each index field, select the corresponding database field that will receive that field value
-The “Output File Field” will receive the path to the exported file
Once you have created records in your database in “Insert” mode, you can change to “Retrieve and View Records” and use SimpleIndex or SimpleSearch to search and view the files.
- Published in Database & Retrieval, Export
Is it possible to have the scanned image itself added to a database and not just the image path?
Please refer to the Wiki Documentation for the complete Database reference.
Yes. Image files can be inserted into binary fields in Access, SQL Server, Oracle, MySQL and other databases.
Check the “Store files as binary objects” option on the Database tab and the “Output File Field” setting can be mapped to a binary field.
If using PDF, MS Office or other non-image files, use the File Type Field to store the file extension of the stored file.
SimpleSearch mode will let you view files stored using this method as well.
- Published in Database & Retrieval, Export
When exporting to a database, I get the error “Multi-step operation generated errors”
Please refer to the Wiki Documentation for the complete Database reference.
This error is usually caused when the data you are trying to insert does not match the data type of the database field. This could be a text value being inserted into a number or date field, or a text value that is longer than the maximum length of the field.
To avoid this error, be sure to use the correct Field Type and/or Template on the Index tab to force the user to enter data in the correct format. You may also need to adjust the data types or lengths for your database fields.
- Published in Database & Retrieval
How do you configure a field to select from a list of possible values?
Please refer to the Wiki Documentation for the complete Index & Batch Logging reference.
List fields are populated using a text file or database field containing the values for that list. The text file scenario will be described first.
To configure a list field, go to the Index tab in the Job Options. Create a list field by selecting “List” for the field type and give it a name. In the “List File/Field” setting for this field, put the full path or UNC to the text file containing the list or click “Set” to browse to the file. If you have not created a file, you can put the path where you want the file stored here and click the “Edit” button. This will open the file in Notepad; new files will prompt you to create them.
On each line of the text file you will put one of the possible values that you want to be able to select from while you are indexing with SimpleIndex. You can copy this information from another source and paste it into the text file.
Save the file in Notepad and close it. The List field is now configured!
To use a database field, your configuration must be connected to a database using the settings on the Database tab. Any fields defined as “List” on the Index tab that have a corresponding field mapped on the Database tab will use the unique values from that field to populate the list.
You can check “Only allow values in list” to disallow users from entering a value not in the list.
Once the List index field is configured properly you can select the values from a drop down after scanning that appear in the main SimpleIndex window while indexing. It also makes the field autofill the closest match from the list based on the characters that you type in that field.
- Published in Indexing & UI
Is it possible to restrict user access to certain documents in Retrieval Mode?
Please refer to the Wiki Documentation for the complete Database Mode reference.
If the database you use to store index information has security access features, you can use these to manage access to documents with SimpleIndex retrieval mode.
In your database, create user accounts and groups and assign them the appropriate permissions to records in your document database.
In SimpleIndex, check the “Require Logon” option on the Logging tab to force the user to logon to the database using the accounts you created. When they search and view documents, they will only be able to access records that the corresponding database user has been assigned access to.
- Published in Database & Retrieval
How do you configure full text searching in Retrieval mode?
Please refer to the Wiki Documentation for the complete Database Settings reference.
On the Database tab there dropdown in the lower portion of the panel for Full Text OCR Field. Put the name of the field that will store the full-text data there. This must be configured both for Insert and Retrieval mode configurations. The database field needs to be sufficient length to store the entire text of your document.
Of course, the Insert Mode configuration must have “Enable Full Page OCR” checked to generate full text data from images. Text from MS Office documents, PDF files and existing OCR text files can be used without setting this option.
When designing your Retrieval Mode configuration, create a Text field to use for full text search queries. On the Database tab, set the corresponding “Database Field Name” to the full text database field.
When searching on your full text field, SimpleIndex finds the text you enter no matter where it appears in the document. It is able to match partial words. It does not perform boolean or natural language searches. The text entered must match the document text exactly.
- Published in Database & Retrieval, OCR
What is “Match & Attach” mode?
Please refer to the Wiki Documentation for the complete Database Mode reference.
Match & Attach mode lets you batch update multiple records in a database using the index data from your SimpleIndex job. For example, if you have a large backfile of documents that you want to scan and link to records in an existing database, you can use Match & Attach to find the corresponding record and set the Image Path field to the newly scanned file.
This allows documents to be indexed with a variety of information and then have it find a particular record based on up to three different key indexes in a data source. It can then fill in additional data columns with indexed information along with the full text information, page count, batch ID and image path.
The Match & Attach uses the key field in the “Autofill Settings…” in the Indexing & File Naming step of the Job Settings Wizard in the File menu. Then fills the data into any blank columns for the record in the database and also changes any fields that are different.
Related Links
- Published in Database & Retrieval, Export
How do you configure the Autofill feature?
The Autofill feature of SimpleIndex is an easy way to associate many index fields with one document without retyping data that already exists in another application.
Autofill uses a database lookup to retrieve records that match a key value or values entered by the user or extracted with SimpleIndex. Blank index fields are then filled in automatically with the data from this lookup. The result is a document database with many different possible search fields, of which only one needed to be entered during scanning.
You can use one, two or three index fields as the key values to extract data from your data source. These index fields can be any of the various types of manual and automatic fields. Once the fields are matched the lookup information will get filled in to the Autofill Index fields.
Instructions
- Navigate to the Indexing & File Naming step in the Job Options Wizard under the File menu.
- Create an Index field(s) in SimpleIndex that will be used for the key matching value(s), which will pull the information from the database. This Index field can be any Type other than Autofill.
- Click the “Autofill Settings…” button on the top right.
- Select the “Datasource:” configured in the Database step in the drop down or click the “New Data Source…” button to create a new connection.
- Click the Load button, which loads the data from the database.
- Select the Table or View that the data is in in the “Table or View:” drop down.
- Select the key field from the database that will be used to match against to pull the other data from the other columns in the selected Table or View from the “Database Key Field 1:” drop down. If other index fields are required for matching the correct records select these in “Database Key Field 2:” and “Database Key Field 3:”
- Select the key index field from SimpleIndex that will be entered or extracted and used to match against the database in the “Index Key Field 1:” drop down. If other index fields are required for matching the correct records select these in the “Index Key Field 2:” and “Index Key Field 2:”, making sure that the correct SimpleIndex fields are linked to the correct database columns.
- Click Finish
- Create new Autofill index fields for each piece of additional data needed to be filled in based on the key field(s). The Template for each of the Autofill index fields should be the exact name of the column from the data source that should be filled.
By checking the “Allow manual input in Autofill fields” the users can edit the filled data manually. By checking “Automatically select first matching record” it will select the first record in the table that matches instead of asking which record should be used, when multiple records in the data source have the same data. By checking “Match on single Key Field” it will match on just one of the key fields, instead of requiring all key fields be match, when multiple key matching fields are used. By checking “Cascading List Mode” each autofill field is displayed as a drop down list populated by the records that match the key field(s) and each selection filters the next list for matching values.
Related Links
- Published in Database & Retrieval, Indexing & UI
How do I connect to an existing Access database?
SimpleIndex has the ability to export, query or autofill from an Access database. This allows you to upload data that is indexed with SimpleIndex to a table or query, search against the database or pull information from the database based on a key value.
Insert New Data
- Open the Job Settings Wizard and go to the Database step
- Set the Database Mode to Insert New Records
- Click the “New Data Source…: button
- Select the Microsoft Access radio button and click Next
- Click “Browse…” button under “Access Database:”, browse to the database file, select it and click Open
- Click the “Load” button under “Table or View:” and then select the Table or Query that needs to be Inserted to
- Link all of the fields in SimpleIndex to the columns in the database that they should Insert to
Search Existing Data
- Open the Job Settings Wizard and go to the Database step
- Set the Database Mode to Retrieve and View Records
- Click the “New Data Source…: button
- Select the Microsoft Access radio button and click Next
- Click “Browse…” button under “Access Database:”, browse to the database file, select it and click Open
- Click the “Load” button under “Table or View:” and then select the Table or Query that needs to be Inserted to
- Link all of the fields in SimpleIndex to the columns in the database that they should Insert to
Autofill Based on Key Values
- Open the Job Settings Wizard and go to the Indexing & File Naming Step
- Click the “Autofill Settings…” button
- Click the “New Data Source…: button
- Select the Microsoft Access radio button and click Next
- Click “Browse…” button under “Access Database:”, browse to the database file, select it and click Open
- Click the “Load” button under “Table or View:” and then select the Table or Query that is being filled from
- Select the key field for the database column in the “Database Key Field 1:”
- Select the key field for the SimpleIndex index field in the “Index Key Field 1:”
- Set-up the Autofill fields that need to be filled in based on the key field
Related Links
- Published in Database & Retrieval
I know nothing about databases. Can I still use the database and Retrieval Mode features?
Yes! We’ve made it simple to create a database that you can use to store index information and perform searches with SimpleIndex Retrieval Mode.
Once you have configured a SimpleIndex job to scan and store documents on your network, use these steps to create a database to store index information for searching:
- Go to the Database step in the Job Settings Wizard
- Click “New Data Source…”
- Select Default and then click Next
- Click Yes when asked to “Copy default database to config file folder and configure settings?”
- The database will be copied to the folder that the Job Configuration is saved in and the name of the database will be the same name as the Job Configuration with all of the fields in SimpleIndex
This creates a database and automatically links it to your SimpleIndex Job Configuration for uploading to or searching from.
Related Links
- Published in Database & Retrieval