Overview
The following sections summarize the document types supported by the Advanced Text Extraction Module.
View incoming links.
Archive Formats
Format | Versions | Notes | AIE Version Added In |
|---|---|---|---|
LZA Self Extracting Compress | All |
|
|
LZH Compress | All |
|
|
Microsoft Office Binder | 95, 97 |
|
|
RAR | 1.5, 2.0, 2.9 |
|
|
Self-extracting .exe | All |
|
|
UNIX Compress (.Z) | All |
|
|
UNIX GZip | All |
|
|
UNIX tar | All |
|
|
Uuencode | All |
|
|
Zip | PKZip, WinZip |
|
|
JAR | All |
|
|
BZip2 | All |
|
|
7z |
| BZIP2 and split archives not supported | AIE 3.5 |
7z Self Extracting exe |
| BZIP2 and split archives not supported | AIE 3.5 |
Microsoft Cabinet (CAB) |
|
| AIE 3.5 |
Database Formats
Format | Versions | Notes |
|---|---|---|
DataEase | 4.x | |
DBase | III, IV, V | |
First Choice DB | Through 3.0 | |
Framework DB | 3.0 | |
Microsoft Access | 1.0, 2.0, 95-2010 | text only |
Microsoft Works DB for DOS | 2.0 | |
Microsoft Works DB for Macintosh | 2.0 | |
Microsoft Works DB for Windows | 3.0, 4.0 | |
Microsoft Works DB for DOS | 1.0 | |
Paradox for DOS | 2.0 - 4.0 | |
Paradox for Windows | 1.0 | |
Q&A Database | Through 2.0 | |
R:Base, RBASE (rbf) | R:Base 5000, R:Base System V | |
Reflex | 2.0 | |
SmartWare II DB | 1.02 |
Email/MIME Formats
Format | Versions | AIE Version Added In |
|---|---|---|
Apple Mail Message (EMLX) | 2.0 | AIE 3.5 |
Encoded mail messages | MHT, Multi Part Alternative, Multi Part Digest, Multi Part Mixed, Multi Part News Group, Multi Part Signed, TNEF |
|
IBM Lotus Notes Domino XML Language DXL | 8.5 | AIE 3.0 |
IBM Lotus Notes NSF | 7.x, 8.x | AIE 3.0 |
MBOX Mailbox |
| AIE 3.5 |
Microsoft Outlook (MSG) | 97 - 2013 |
|
Microsoft Outlook Express (EML) | All |
|
Microsoft Outlook Forms Template (OFT) | 97 - 2013 |
|
Microsoft Outlook OST | 97 - 2013 |
|
Microsoft Outlook PST | 97 - 2013 |
|
Microsoft Outlook PST (Mac) | 2001 |
|
| MSG with Digital Signature (SMIME) | AIE 3.6 | |
RFC 822 |
| AIE 3.5 |
| EML with Digital Signature (SMIME) | AIE 3.6 |
Multimedia Formats
Format | Versions | Notes | AIE Version Added In |
|---|---|---|---|
3G2 | All | ATEM performs File ID only. |
|
3GP | All | ATEM performs File ID only. |
|
AIFF | All | ATEM performs File ID only. |
|
ASF(subformats: WMA/WMV/DVR-MS) | All | File ID and Metadata extraction only | AIE 3.0 |
AU | All | ATEM performs File ID only. |
|
AVI | All | Metadata extraction only |
|
| DICOM | All | ATEM performs File ID only. | AIE 3.6 |
FLAC | All | ATEM performs File ID only. |
|
Flash | 6.x, 7.x, Lite | text extraction only |
|
Flash | 9, 10 | ATEM performs File ID only. | AIE 3.5 |
IVF | All | ATEM performs File ID only. |
|
MID | All | ATEM performs File ID only. |
|
MP2 | All | ATEM performs File ID only. |
|
MP3 | All | ID3 metadata only |
|
MPEG-1 Audio layer 3 V ID3 v1 | All | Metadata extraction only |
|
MPEG-1 Audio layer 3 V ID3 v2 | All | Metadata extraction only |
|
MPEG-1 Video V 2 (File ID only) | All |
|
|
MPEG-1 Video V 3 (File ID only) | All |
|
|
MPEG-2 Audio | All | File ID only |
|
MPEG-4 | All | File ID and metadata extraction only | AIE 3.0 |
MPEG-7 | All | File ID and metadata extraction only | AIE 3.0 |
Microsoft Access Report Snapshot (.snp) | All | File ID only | AIE 3.0 |
Microsoft Windows Media Playlist (.wpl) | All | File ID only | AIE 3.0 |
OGG | All | ATEM performs File ID only. |
|
QuickTime | All | Metadata extraction only |
|
Real Media | All | File ID only |
|
RIFF | All | Metadata extraction only |
|
RPIX | All | File ID support only | AIE 3.0 |
Windows Media ASF | All | Metadata extraction only |
|
Windows Media DVR-MS | All | Metadata extraction only |
|
Windows Media Audio WMA | All | Metadata extraction only |
|
Windows Media Video WMV | All | Metadata extraction only |
|
WAV | All | Metadata extraction only |
|
Other Formats
Format | Versions | Notes | AIE Version Added In |
|---|---|---|---|
AOL Messenger | 7.3 | File ID only | AIE 3.5 |
| COM DOS executable | AIE 5.0 | ||
Microsoft InfoPath 2007 | 2007 | File ID only | AIE 3.0 |
Microsoft Live Messenger | 10.0 | via XML filter | AIE 3.5 |
| Microsoft Office Theme Files | 2007-2013 | File ID only | AIE 3.6 |
Microsoft OneNote | 2007, 2010 | text only. Single file packages only (*.onepkg). Section files and Table of content files are not identified. | AIE 3.0 |
Microsoft Project | 98 – 2003 | table view only | AIE 3.5 |
Microsoft Project | 2007, 2010 | table view only | AIE 3.5 |
Microsoft Windows Compiled Help .chm |
| File ID only | AIE 3.5 |
Microsoft Windows DLL | All |
|
|
Microsoft Windows Executable | All |
|
|
Microsoft Windows Explorer Command .scf |
| File ID only | AIE 3.5 |
Microsoft Windows Help .hlp |
| File ID only | AIE 3.5 |
Microsoft Windows Shortcut .lnk |
| File ID only | AIE 3.5 |
| Thumbs.db Windows Image Cache | AIE 5.0 | ||
Trillian Text Log File | 4.2 | via text filter | AIE 3.5 |
Trillian XML Log File | 4.2 | File ID only | AIE 3.5 |
TrueType Font ttf, ttc |
| File ID only | AIE 3.5 |
vCalendar | 2.1 |
|
|
vCard | 2.1 |
|
|
Yahoo! Messenger | 6.x - 8 |
|
|
Presentation Formats
Format | Versions | Notes | AIE Version Added In |
|---|---|---|---|
| Apple iWork Keynote | 09 | AIE 3.6 | |
Corel Presentations | 6.0 - X3 |
|
|
Harvard Graphics Presentation DOS | 3.0 |
|
|
IBM Lotus Symphony Presentations | 1.x | Support for 1.2 added in AIE 3.0 |
|
KingSoft WPS Presentations | Office 2010 | The Presentation format is identified and processed as MS PowerPoint because the formats are the same | AIE 3.0 |
Lotus Freelance | 1.0-Millennium 9.8 |
|
|
Lotus Freelance for OS/3 | 2 |
|
|
Lotus Freelance for Windows | 95, 97, SmartSuite 9.8 |
|
|
Microsoft PowerPoint for Macintosh | 4.0 - 2011 |
|
|
Microsoft PowerPoint for Windows | 3.0 - 2013 |
|
|
Microsoft PowerPoint for Windows Slideshow | 2007-2013 |
|
|
| Microsoft PowerPoint for Windows Template | 2007-2013 | ||
Novell Presentations | 3.0, 7.0 |
|
|
OpenOffice Impress | 1.1, 2.0 |
|
|
OpenOffice Impress | 3.x |
| AIE 3.0 |
StarOffice Impress | 5.2 - 8.0 |
|
|
WordPerfect Presentations | 5.1-X5 |
|
|
Raster Image Formats
Format | Versions | Notes |
|---|---|---|
| Adobe Photoshop | 4.0, CS1-CS6 | |
| Adobe Photoshop PSD | All | File ID only |
CALS Raster (GP4) | Type I, Type II |
|
Computer Graphics Metafile | ANSI, CALS, NIST |
|
Encapsulated PostScript (EPS) | TIFF header Only |
|
GEM Image (Bitmap) | All |
|
Graphics Interchange Format (GIF) | All |
|
IBM Graphics Data Format (GDF) | 1.0 |
|
IBM Picture Interchange Format | 1.0 |
|
JBIG2 | All |
|
Graphic Embeddings in PDF | All |
|
JFIF (JPEG not in TIFF format) | All |
|
JPEG | All |
|
JPEG 2000 | JP2 |
|
Kodak Flash Pix | All |
|
Kodak Photo CD | 1.0 |
|
Lotus PIC | All |
|
Lotus Snapshot | All |
|
Macintosh PICT | BMP only |
|
Macintosh PICT2 | BMP only |
|
MacPaint | All |
|
Microsoft Windows Bitmap | All |
|
Microsoft Windows Cursor | All |
|
Microsoft Windows Icon | All |
|
OS/2 Bitmap | All |
|
OS/2 Warp Bitmap | All |
|
Paint Shop Pro (Win32 only) | 5.0, 6.0 |
|
PC Paintbrush (PCX) | All |
|
PC Paintbrush DCX (multi-page PCX) | All |
|
Portable Bitmap (PBM) | All |
|
Portable Graymap PGM | All |
|
Portable Network Graphics (PNG) | All |
|
Portable Pixmap (PPM) | All |
|
Progressive JPEG | All |
|
StarOffice Draw | 6.x - 8.0 |
|
Sun Raster | All |
|
TIFF | Group 5 & 6 |
|
TIFF CCITT | Group 3 & 4 |
|
TruVision TGA (Targa) | 2.0 |
|
WBMP wireless graphics format | All |
|
Word Perfect Graphics | 1.0 |
|
X-Windows Bitmap | x10 compatible |
|
X-Windows Dump | x10 compatible |
|
X-Windows Pixmap | x10 compatible |
|
WordPerfect Graphics | 2.0 - 10.0 | Office X4 support added in AIE 3.0 |
Spreadsheet Formats
Format | Versions | Notes | AIE Version Added In |
|---|---|---|---|
| Apple iWork Numbers | 09 | AIE 3.6 | |
Enable Spreadsheet | 3.0 - 4.5 |
|
|
First Choice SS | Through 3.0 |
|
|
Framework SS | 3.0 |
|
|
IBM Lotus Symphony Spreadsheets | 1.x | Support for 1.2 added in AIE 3.0 |
|
KingSoft WPS ?Spreadsheets | Office 2010 |
| AIE 3.0 |
Lotus 1-2-3 | Through Millennium 9.8 |
|
|
Lotus 1-2-3 Charts (DOS and Windows) | Through 5.0 |
|
|
Lotus 1-2-3 for OS/2 | 2.0 |
|
|
Microsoft Excel Charts | 2.x - 2007 |
|
|
Microsoft Excel for Macintosh | 98 - 2011 |
|
|
Microsoft Excel for Windows | 3.0 - 2013 |
|
|
Microsoft Excel for Windows 2003 XML | 2003 | text only | AIE 3.5 |
Microsoft Excel 2007/2010 Binary (.xlsb) | 2007-2013 |
| AIE 3.0 |
Microsoft Works SS for DOS | 2.0 |
|
|
Microsoft Works SS for Macintosh | 2.0 |
|
|
Microsoft Works SS for Windows | 3.0, 4.0 |
|
|
Multiplan | 4.0 |
|
|
Novell PerfectWorks Spreadsheet | 2.0 |
|
|
OpenOffice Calc | 1.1 - 2.0 |
|
|
OpenOffice Calc | 3.x |
| AIE 3.0 |
PFS: Plan | 1.0 |
|
|
QuattroPro for DOS | Through 5.0 |
|
|
QuattroPro for Windows | Through X5 |
|
|
SmartWare Spreadsheet | All |
|
|
SmartWare II SS | 1.02 |
|
|
StarOffice Calc | 5.2 - 8.0 |
|
|
SuperCalc | 5.0 |
|
|
Symphony | Through 2.0 |
|
|
VP-Planner | 1.0 |
|
|
WordPerfect Office X4 spreadsheets |
|
| AIE 3.0 |
Text/Markup Formats
Format | Versions | AIE Version Added In |
|---|---|---|
ANSI Text | 7 & 8 bit |
|
ASCII Text | 7 & 8 bit |
|
DOS character set | All |
|
EBCDIC | All |
|
HTML (CSS rendering not supported) | 1.0 - 4.0 |
|
IBM DCA/RFT | All |
|
IBM Lotus Notes DXL | All | AIE 3.0 |
Macintosh character set | All |
|
Rich Text Format (RTF) | All |
|
Unicode Text | 3.0, 4.0 |
|
UTF-8 | All |
|
Wireless Markup Language | All |
|
XML (text only) | All |
|
XHTML | 1.0 |
|
Processing of Plain Text Files
- During the processing of plain text files, ATEM ascertains their encoding.
- ATEM provides extra, fine-grained file type recognition for plain text documents based on the doctype definitions in the <install_dir>conf\textextraction\doc-types-config.xml file. Refer to Processing of Plain Text Files with Text Extraction for more information.
Processing of HTML Files
ATEM offers several features with regard to HTML file processing:
- Extensive character encoding detection and recognition
- Optional boilerplate removal using advanced heuristic techniques
- Configurable HTML element to IngestDocument field extraction using a selector syntax
Refer to Processing of HTML Files with Text Extraction for more information.
Advanced Processing of Markup Types
Note that ATEM ships with transformers, such as ExtractText, that provide more advanced functionality for the processing of markup languages such as XML.
This advanced functionality includes such features as:
- options to extract (or not) values, attributes, and comments
- more fine-grained type recognition for XML which includes such subtypes as BPEL, OWL, RDF, XSD, SOAP, and WSDL.
Vector Image Formats
Format | Versions | Notes | AIE Version Added In |
|---|---|---|---|
Adobe Illustrator | 4.0 - 7.0 |
|
|
Adobe Illustrator XMP | CS1-CS6 |
|
|
Adobe InDesign XMP | CS1-CS6 |
|
|
Adobe InDesign Interchange (XMP only) | All |
|
|
Adobe PDF | 1.0 - 1.7 (Acrobat 1 - 10) |
|
|
Adobe PDF Package | 1.7 (Acrobat 8 - 10) |
| AIE 3.0 |
Adobe PDF Portfolio | 1.7 (Acrobat 8 - 10) |
|
|
Ami Draw | SDW |
|
|
AutoCAD Drawing | 3.0, 2.6 |
|
|
AutoCAD Drawing | 9.0 - 14.0 |
|
|
AutoCAD Drawing | 2000i - 2012 |
|
|
| AutoDesk DWF Archive File | AIE 5.0 | ||
AutoShade Rendering | 2 |
|
|
Corel Draw | 2.0 - 9.0 |
|
|
Corel Draw Clipart | 5.0, 7.0 |
|
|
Enhanced Metafile (EMF) | All |
|
|
Escher graphics | All |
|
|
FrameMaker Graphics (FMV) | 3.0 - 5.0 |
|
|
Gem File (Vector) | All |
|
|
Harvard Graphics Chart DOS | 2.0 - 3.0 |
|
|
Harvard Graphics for Windows | All |
|
|
HP Graphics Language | 2.0 |
|
|
IGES Drawing | 5.1 - 5.3 |
|
|
Micrografx Designer | Through 3.1 |
|
|
Micrografx Designer | 6.0 |
|
|
Micrografx Draw | Through 4.0 |
|
|
Microsoft XPS | All |
|
|
Novell PerfectWorks Draw | 2 |
|
|
OpenOffice Draw | 1.1 - 3.0 |
|
|
OpenOffice Draw | 3.x |
| AIE 3.0 |
| Scalable Vector Graphics File | AIE 5.0 | ||
| SVG | All | processed as XML, not rendered | AIE 3.6 |
Visio (Page Preview mode WMF/EMF) | 4.0 |
|
|
Visio | 5.0 - 2007 | .vdx, .vsx, .vtx are not supported |
|
Windows Metafile | All |
|
|
Word Processing Formats
Format | Versions | Notes | AIE Version Added In |
|---|---|---|---|
Adobe FrameMaker | 3.0 - 6.0 | MIF only |
|
Adobe Illustrator Postscript | Level 2 |
|
|
Ami | All |
|
|
Ami Pro for OS2 | All |
|
|
Ami Pro for Windows | 2.0, 3.0 |
|
|
| Apple iWork Pages | 09 | AIE 3.6 | |
DEC DX | Through 4.0 |
|
|
DEC DX Plus | 4.0, 4.1 |
|
|
Enable Word Processor | 3.0 - 4.5 |
|
|
First Choice WP | 1.0, 3.0 |
|
|
Framework WP | 3.0 |
|
|
Hangul | 97 - 2010 |
|
|
IBM DCA/FFT | All |
|
|
IBM DisplayWrite | 2.0 - 5.0 |
|
|
IBM Lotus Symphony | 1.2 |
| AIE 3.0 |
IBM Writing Assistant | 1.01 |
|
|
Ichitaro | 5.0, 6.0, 8.0 - 13.0, 2004 |
|
|
Ichitaro | 2010 |
| AIE 3.5 |
JustWrite | Through 3.0 |
|
|
KingSoft WPS Writer | Office 2010 |
| AIE 3.0 |
Legacy | 1.1 |
|
|
Lotus Manuscript | Through 2.0 |
|
|
Lotus WordPro | 9.7, 96 - Millennium 9.8 |
|
|
MacWrite II | 1.1 |
|
|
Mass 11 | Through 8.0 |
|
|
Microsoft Publisher | 2003 - 2007 |
| AIE 3.0 |
Microsoft Word for DOS | 4.0 - 6.0 |
|
|
Microsoft Word for Macintosh | 4.0 - 6.0, 98 - 2011 |
|
|
Microsoft Word for Windows | 1.0 - 2013 |
|
|
Microsoft Word for Windows 2003 XML | 2003 | text only | AIE 3.5 |
Microsoft Word 2010 | 2010 |
| AIE 3.0 |
Microsoft WordPad | All |
|
|
Microsoft Works WP for DOS | 2.0 |
|
|
Microsoft Works WP for Macintosh | 2.0 |
|
|
Microsoft Works WP for Windows | 3.0, 4.0 |
|
|
Microsoft Write for Windows | 1.0 - 3.0 |
|
|
MultiMate | Through 4.0 |
|
|
MultiMate Advantage | 2.0 |
|
|
Navy DIF | All | A.k.a. Navy/Lotus Data Interchange Format (.dif). | AIE 3.0 |
Nota Bene | 3.0 |
|
|
Novell PerfectWorks Word Processor | 2.0 |
|
|
OfficeWriter | 4.0 - 6.0 |
|
|
OpenOffice Writer | 1.1 - 2.0 |
|
|
OpenOffice Writer | 3.x |
| AIE 3.0 |
PC File Doc | 5.0 |
|
|
PFS: Write | A, B |
|
|
Professional Write for DOS | 1.0, 2.0 |
|
|
Professional Write Plus for Windows | 1.0 |
|
|
Q&A Write | 2.0, 3.0 |
|
|
Samna Word IV | 1.0 - 3.0 |
|
|
Samna Word IV+ | All |
|
|
Signature | 1.0 |
|
|
SmartWare II WP | 1.02 |
|
|
Sprint | 1.0 |
|
|
StarOffice Writer | 5.2 - 8.0 |
|
|
Total Word | 1.2 |
|
|
Wang IWP | Through 2.6 |
|
|
WordMarc Composer | All |
|
|
WordMarc Composer+ | All |
|
|
WordMarc Word Processor | All |
|
|
WordPerfect for DOS | 4.2 |
|
|
WordPerfect for Macintosh | 1.02 - 3.1 |
|
|
WordPerfect for Windows | 5.1 - X5 |
|
|
Wordstar 2000 for DOS | 1.0- 3.0 |
|
|
Wordstar 2000 for DOS | 2.0 , 3.0 |
|
|
Wordstar for DOS | 3.0 - 7.0 |
|
|
Wordstar for Windows | 1.0 |
|
|
XyWrite | Through III+ |
|
|