Release Date: 29 June 2019
This page summarizes the modules, new features, enhancements, and bug fixes included in the Attivio Platform 5.6.1 release, as well as listing any known issues.
Attivio Platform 5.6.1 installs the following module versions when the Full installation type (the default) is selected:
|cloudsupport||1.0.6||Adds support for interacting with Cloud services|
|connectoradmin||1.0.1||Used to host the Connector Admin UI application from within Attivio nodes|
|module-alm||1.0.6.aie56||Provides English language advanced linguistics capabilities, including language detection and entity extraction|
|searchanalytics||1.1.0||Analyze how users are querying your Attivio index.|
|searchui||1.0.6||Used to host the Search UI application from within Attivio nodes|
|upgradeproject||1.0.0||Upgrades an existing 5.5.1 project to be compatible with 5.6.1 and generates a change report of an existing 5.5.1 project against a default 5.5.1 project|
|webcrawler||1.4.5||Configure connectors to crawl web content and process the pages as ingest documents.|
These modules are not installed if the Core installation type is selected. Most of the Attivio documentation assumes that you have selected the Full option, with the core platform and external modules. Some options mentioned in the documentation may be unavailable if you select the Core option and omit the external modules.
Most of the Attivio documentation assumes that you have selected the Full option, with the core platform and external modules. Some options mentioned in the documentation may be unavailable if you select the Core option and omit the external modules.
Attivio Platform 5.6.1 adds support for multi-level faceting. This feature supports visualization of a hierarchy for the documents matching a query, and enables filtering documents using arbitrary levels of this hierarchy. As an example, documents with location data can be faceted using a country > state or province > city hierarchy, enabling users to easily refine their results (to show only documents from France or California or London, say) from a single hierarchical facet.
See the Multi-Level Facet Guide for more information.
Connector Admin UI
The new Connector Admin UI simplifies creation, editing, and management of connectors and provides insights into ingested data.
The Attivio Java SDK now supports the implementation of custom connector scanner classes to deliver content into the Attivio workflow from any data source. The SDK specifies the interfaces and annotations the scanner should implement so that it can be configured and controlled from the Connector Admin UI or the legacy Administrator UI. The SDK also specifies Attivio exposed interfaces which the scanner can use to feed content and access Attivio services.
See the public SDK's Scanners page for more information on custom scanner development.
Customizable JRE for Attivio processes ("Bring Your Own JRE")
Users can now configure their own Java Runtime Environment (JRE) to use for running Attivio processes. Any Java SE 11 JRE can be specified.
See Updating Attivio JREs for more information.
- PLAT-43014: The Attivio installer now includes OpenJDK 11.0.2 as the default Java Runtime Environment (JRE) for Attivio processes.
Improved Hadoop cluster stability and user experience
- PLAT-42943: Distribution of index containers across YARN nodes now includes logic to ensure High Availability.
pmemenforcement changed to be
falseby default for Attivio Hadoop clusters.
- PLAT-43027: The CLI now validates project configurations to ensure that all YARN containers will be allocable when the index
sliderPlacementPolicyproperty is set to its default value of 32 (anti-affinity).
PLAT-43027: When deploying an index, the CLI now displays progress while adding the index's
.tarfile to the Slider
- PLAT-43027: The CLI no longer builds the Slider
.ziparchive twice (once for abc-index, once for index), instead updating the existing archive with relevant index configuration settings.
PLAT-43027: The CLI displays status STOPPING for Hadoop services when a stop (or restart) has been requested but the service has not timed out yet in ZooKeeper.
- PLAT-43027: The CLI
statuscommand's output now sorts services on the same host alphabetically instead of randomly to improve readability.
- PLAT-43027: The CLI
stop hadoopcommands now use case-insensitive regular expressions, allow for multiple spaces between
<serviceRegex>instead of just one, and print out a message reporting whether any entries match the regular expression or not.
PLAT-43027: Added resiliency to the HdfsFailoverController and HdfsNameNode services.
- PLAT-43027: Increased timeout periods for writes to HDFS journal log and start of new segments from 20 seconds to 40 seconds to better handle stress situations.
PLAT-43269: The minimum timeout period for the ZooKeeper service was raised from 4 seconds to 20 seconds to provide more resiliency against SAN slowness.
PLAT-43269: Users can now specify
dataLogDirand other properties for ZooKeeper service.
- PLAT-43277: Enabled user configuration of abc-index memory usage, partition count, and row count.
Smaller memory footprint for clustered projects
PLAT-43027: Reduced memory allocated for Slider app master to 512 MB for indexes of 16 or fewer partitions, or 1024 MB (1 GB) for indexes of more than 16 partitions
Slider app master's memory allocations are controlled by these properties in
# slider app master memory, default is 512, unit are megabytes slider.am.memory=512 # ... for larger indices (> 16 indexwriter/indexsearcher processes) slider.am.memory-large=1024
PLAT-43027: Changed Attivio Hadoop cluster default
yarn.scheduler.minimum-allocation-mbproperty value to 512 to permit smaller configurations.
PLAT-43027: Changed Attivio Hadoop cluster default
yarn.scheduler.maximum-allocation-mbproperty value from 3072 to 4096.
PLAT-43027: Reduced default heap memory allocations for many services in Attivio Hadoop cluster:
HdfsFailoverController from 1024 MB to 256 MB.
HdfsNameNode from 5376 MB to 2048 MB.
YarnResourceManager from 2048 MB to 1024 MB.
YarnNodeManager from 4096 MB to 1024 MB.
HBaseMaster from 4096 MB to 1024 MB.
HBaseRegion from 3093 MB to 3072 MB.
PLAT-43027: Reduced default HDFS replication factor from 3 to 2.
- PLAT-40938: Added payload support to trigger dictionaries.
- PLAT-41380: Optimized
textschema field type.
- PLAT-41808: Cleaned up REST APIs to reduce use of custom deserializers.
- PLAT-41811: Added user-selected FacetFilters to REST query response.
- PLAT-41912: Added multi-threaded facet execution.
- PLAT-41816: Facet interchanges between engine and query dispatcher now use JSON format to reduce I/O
- PLAT-42022: Autowarm queries are now executed in parallel.
- PLAT-42038: Static teaser field is not displayed in query results unless explicitly requested.
- PLAT-42058: Java 8 features are now used to simplify
- PLAT-42068: Query requests now use binary serialization to reduce memory overhead (and improve performance) for reads and writes of query objects across network.
- PLAT-42085: Objects read from the query dispatcher are now buffered to streamline read operations.
- PLAT-42116: Added Ignore Scopes blacklist property to
AddEntitySentimenttransformer class. By default, this blacklist includes the
- PLAT-40376: Resolved an issue where REST Perfmon API returned "NaN" metrics values.
- PLAT-41495: Resolved an issue where
NormalizeUnicodeingest transformer only processed surface tokens.
- PLAT-41562: Dictionary Administration web application no longer allows addition of terms to Derived-mode Entity dictionaries.
- PLAT-41755: Resolved an issue where the unclustered (HSQL) Attivio store could throw a
NullPointerExceptionwhen updating sent-documents count for a completed connector run.
- PLAT-41925: Resolved an issue where not all Attivio Hadoop services appeared in CLI when all were stopped.
- PLAT-41956: Resolved an issue which could cause the
ExtractHashTagstransformer class to throw a
- PLAT-41988: Resolved an issue where connector history's sent-documents count for current execution record was incorrect for unclustered projects.
- PLAT-42006: Resolved an issue where CLI
help zkcommand output pointed to non-existent URL.
- PLAT-42186: Removed deprecated
removerowCLI commands. These functions are now accessed via options for the
- PLAT-43340: Resolved security vulnerabilities by upgrading to
- PLAT-43509: Resolved an issue where the Publish Dictionary button was not properly disabled in the Rules Dictionary Administration web application.
PLAT-43620: Resolved an issue which could cause deadlocks in the unclustered (HSQL) Attivio store.
PLAT-43815: Removed deprecated sail module.
- PLAT-43930: Resolved an issue where the Ingest Audit Viewer's Show Details function could fail with a "Premature EOF"
- PLAT-44000: If you update the Attivio JRE to point to any version other than Java SE 11, Attivio processes may display inconsistent startup failure messages.
- Solution: Ensure that the Attivio JRE points to a Java SE 11 installation.
- PLAT-44001: The Excel Connector cannot read files from Excel versions prior to Excel 97/2000/XP/2003.
- Solution: Use the
excelcnv.exetool shipped with recent versions of Microsoft Office to convert older Excel files to the modern
.xlsxformat prior to ingestion.
- Solution: Use the