Overview

The Attivio Intelligence Engine (AIE) provides simple tools for ingesting XML content.  You can configure the tools for the structure and size of the XML source, as well as for metadata-extraction requirements. How you load XML files into AIE depends on whether the XML files contain multiple documents, and on the size of the XML files being ingested.

• If the XML file contains a single document, use a File Connector and the standard xmlIngest workflow. This approach is described on the One Doc per XML File page.

• If the XML file contains multiple documents, and the file is less than 10 MB in size, using the File Connector with a customized workflow that includes a splitXml transformer is the most efficient method. This procedure is described on the Multiple Docs per Small XML File page.

• If the XML file contains multiple documents and is larger than 10 MB, it is more efficient to use the XML Connector (which incorporates the XmlScanner) with the xmlIngest standard workflow. See the Multiple Docs per Large XML File page.

View incoming links.

• Activating Incremental Updating ,
• Content Ingestion - Concepts and Tools ,
• Custom Index Security With ACLs ,
• Field Properties ,
• Loading Factbook XML ,
• Multiple Docs per Large XML File ,
• Multiple Docs per Small XML File ,
• One Doc per XML File ,
• Scope Search

XML Processing Workflow Components

AIE contains several components specifically designed to support XML ingestion. Most of them are part of the standard xmlIngest workflow.

ParseXml

The ParseXml stage is designed to parse XML from ContentPointers , strings, or byte arrays into a DOM object. This is typically the first stage in any workflow designed to handle XML, as later stages all depend on the parsed XML. In addition, the ParseXml stage supports adding namespace prefixes to URI mappings to the DOM. This ensures that later stages can properly select nodes based on namespace-aware XPath expressions.

Content Encoding

The ParseXml stage uses the encoding scheme (character set) declared in the XML header: it defaults to UTF-8 encoding if no character set is specified in the header.

SplitXml

The SplitXmlstage is designed to split a single IngestDocument  containing XML into multiple IngestDocuments containing subsections of the XML. For example, the SplitXml stage could take a catalog of books in a single XML record, and create a separate IngestDocument for each book in the catalog. The SplitXml stage can also copy parent field values to the child records. By default, the parent XML IngestDocument drops from ingestion after the stage completes.

ExtractXPaths

The ExtractXpaths stage uses a series of XPath-to-field-name mappings to copy content from XML elements into IngestDocument fields. 

DropDom

It is considered best practice to include a dropDom stage at the end of most XML workflows. Once XML parsing and extraction is complete, deleting the DOM object from the IngestDocument saves on memory footprint and network bandwidth across the system.

Incremental Updating

This connector supports the Activating Incremental Updating features. There is a tutorial example of incremental updating here.

Rowset XML

AIE contains a default workflow to handle RowSet XML formatted content. Generally, rowset XML files are large, so using the XMLScanner is recommended. The default rowsetIngest workflow can ingest the resulting IngestDocuments with each rowset XML field mapping to a new AttiviDocument field as expected.