Overview

The Advanced Query Language provides sophisticated tools for use from a client program where the query can be assembled by software. Unlike the Simple Query Language, it is not intended for end users. However, a project developer can use Search UI or the Debug Search page to cut-and-paste advanced queries to test syntax.

The Advanced Query Language is a prefix notation query language. In other words, the operator is specified first, followed by the parameters to be used by that operator.

View incoming links.

• Access Control JOIN ,
• Attivio Glossary ,
• Attivio Platform Overview ,
• Dates and Date Formats ,
• Debug Search ,
• Deleting Content ,
• Dictionary Entity Extraction ,
• Non-English Queries ,
• Query and Response - Concepts and Tools ,
• Query Languages ,
• Query Modification ,
• Querying of Unstructured Data ,
• Relational Querying JOIN ,
• Reloading Documents ,
• Scope Facets ,
• Scope Search ,
• Search Application Example ,
• Security Debugging ,
• Sorting Results ,
• SUIT and Search UI ,
• Updating Documents that Match a Query ,
• Using the Query Builder ,
• Working with Rules Dictionaries ,
• XML REST API ,
• Zoned Indexing

Syntactic Elements

Expressions

Each expression in Advanced Query Language is structured as follows:

• Each expression has an operator, 1 or more arguments, and 0 or more parameters.
• Operator names are case insensitive. For clarity and as a best practice, all operators will appear in UPPER CASE on this page.
• Some operators ("unary" operators) take only a single argument.

• Other operators ("n-ary" operators) may take one or more arguments.

  The Advanced Query Language inherits Simple Query Language constructs for Data Types, Terms, Fields, and Ranges.

Special Characters

The following characters have special meanings in Advanced Query Language:

The above list includes the space character. To treat any of these characters literally, you must escape it with a backslash \ . You need not escape any characters inside double-quotes, except for the double-quote character itself.

Query Language Parameters

In the following expression, each PARAM and PARAMVALUE can have an arbitrary value (except that PARAM cannot be an operator name).

Certain parameters have specific meaning to certain operators; these are listed below in each operator's description section. Note that parameters specified in an expression will be inherited by those sub-expressions that do not define the parameter. In other words, the expression:

...is equivalent to the expression...

Shared Query Parameters

Some parameters that are shared among several different operators include:

Scope Operator

Under certain circumstances, various features of AIE permit searching for, or within, a "scope."  A scope is a bounded region in a token list that represents a sentence, a person, a location, a company, a keyphrase, a date, an entity-sentiment value, or is based on one of the XML elements found in the source document. 

See Scope Search for details and examples.

Date Operator

Advanced query languages provides a DATE operator for specifying date values in custom formats. The syntax is as follows:

If not specified, date_format is the standard ISO-8601 format ("yyyy-MM-dd'T'HH:mm:ss"). If not specified, date_timezone is assumed to be UTC.

Examples:

Term Operator

The TERM operator is a unary operator that provides the ability to search for terms or phrases.
The argument for the TERM operator must be a String.

TERM specific parameters

Term Score Mode

The TERM operator supports 3 scoring modes.

• DEFAULT - TF/IDF Scoring calculation
• POSITION - score calculated based on first match's position
• CONSTANT - score will be constant for all documents that match the term

Examples

Fuzzy Operator

The FUZZY operator provides the ability to match documents with terms that are similar to a search term. Similarity between terms is computed using Levenshtein distance formula.

FUZZY Specific Parameters

Examples

Regex Operator

The REGEX operator provides the ability to match documents with terms that match a specified regular expression. The full list of REGEX operators appears on the Lucene RegExp page.

Anchoring Operators

Term anchoring is provided by the STARTSWITH, ENDSWITH and EQUALS operators.
These operators are unary operators that follow the same semantics as the TERM operator.

Field Operators

The following operators apply to field names:

ISNULL Operator

The ISNULL operator can be applied to a field name. It will match documents that contain no value for the specified field.

Special Internal Fields

There is one internal field (not in the AIE Schema) that can be used in queries.

Document ID Field

The AIE document ID field is .id.  You can write Simple Query Language queries that directly match this field.

Proximity Query Operators

Proximity searching is provided by the NEAR, ONEAR and PHRASE operators.

The NEAR operator searches for terms that are within a specified distance of each other. NEAR is an n-ary operator. Its arguments can be Strings, a special form of the OR operator or the SCOPE operator. If using an OR operator inside a NEAR, its arguments must be Strings. The OR operator inside a NEAR provides the ability to "stack" terms in the same position.

The ONEAR operator is similar to the NEAR operator, except it requires matching documents have terms in the same order as specified in the ONEAR expression.

The PHRASE operator is an ordered near with a default distance of 0 between terms.

NEAR/ONEAR/PHRASE Specific Parameters

Examples

Two Query Terms

Given the content "the big dogs went running to the market"...

More Than Two Query Terms

Proximity operators can take an arbitrary number of query terms. The query NEAR(word1, word2, ... wordn, distance=d) will match documents that have at least one block of n+x words that includes at least one instance of each word in the query. The corresponding ONEAR query works similarly and requires that the query terms appear in order. For example:

Boolean Query Operators

AND

The AND operator is an n-ary operator whose arguments can be any other expression. It will only match documents when all sub-expressions match.

Examples

OR

The OR operator is an n-ary operator whose arguments can be any other expression. It will match any document where one or more of the sub-expressions matches.

OR Specific Parameters

Examples

Score Mode

AND and OR queries support specifying a scoring mode using the score query parameter.

Examples

NOT

The NOT operator is a unary operator whose argument is another expression. It will match the inverse of the sub-expression.

Examples

Boost Query Operator

The BOOST operator can be used to apply a boost query and/or score functions to a search query.  The search query determines which documents will be returned. The boost query determines which of the returned documents will be sorted to the top of the list.

Syntax

The <query> specifies the search query for matching documents, and can be any valid advanced query language query. The <boost-query> specifies a query that will be used for boosting the score of documents matching the search query and can be any valid advanced query langauge query. The <boost-query> will not impact which documents match the <query> in any way. The <score-function> is a field expression.

BOOST Specific Parameters

Examples

Subquery Operator

The SUBQUERY operator allows annotating sections of a query. It does not modify what documents match the query in any way. It takes any legal advanced query language query as an argument.

This operator is especially useful in annotating sections of a query with additional parameters. For example, it allows annotating the "user" component of a query. These annotations can be then used by custom query transformers to perform different transformations based on parameters set on the SUBQUERY operator.

Embedded Query Operator

The QUERY operator can be used to embed a user query string in an Advanced Query Language query.
This is very useful for providing a user input text box for providing a Simple Query Language query and wrapping that query string with advanced query syntax.

QUERY Specific Parameters

Examples

Join Query Operator

The JOIN operator can be used to perform relational queries inside an index.

Syntax

The <query> can be any valid advanced query language query. The JOIN query must also specify one or more <clause>. A <clause> defines the criteria for joining. Mulitple types of join clause are supported and provide different functionality.

INNER and OUTER JOIN Clauses

Syntax

The <clause-query> can be any advanced query language query (including JOIN queries). If it is a JOIN query, this will result in a multi-level join.

INNER/OUTER Specific Parameters

On Parameter

The on parameter is used for specifying the primary/foreign key relationship. The format for this parameter is: primaryKey[ <OP> foreignKey] The operator and foreign key are both optional. If not specified the operator will be = and the foreignKey will be the same as the primaryKey.

Supported join operators are:

INNER/OUTER Clause Examples

Composite Join Query

The COMPOSITEJOIN operator can be used to perform a search across a document that is represented as multiple IngestDocuments. This differs from typical JOIN queries as it allows for the evaluation of complex boolean queries against a composite document consisting of multiple attiviDocuments. This is the most useful when a document is indexed as two separate documents: a metadata record containing metadata fields, and a content record containing large extracted text field.

Syntax

The <query> can be any valid advanced query language query. The COMPOSITEJOIN query must also specify one or more <clause>. A <clause> defines the criteria for selecting child documents to join to documents matching <rootquery>.

Parameters

Clause Syntax

Clause Parameters

Example

Filters

Filter expressions can be added to any part of the query tree, or can be specified separately from the query.

To apply a filter in a query, use the FILTER operator:

• <search-query> - any advanced query language query expression
• <filter> - The filter to apply to <search-query>. Must be one of the filters described below.

You can also specify a filter expression alongside a query, using the appropriate parameter defined for your query API.

{
  "workflow": "search",
  "query" : "*:*",
  "filters" : [ "language:English", "author:Austen" ]

Query Filters

Query Filters are the simplest type of filters. They can be any other Advanced Query Language query. The query used for filtering will not be scored and will only be used for filtering the results from the search query.

Syntax:

The <filter-query> can be any valid Advanced Query Language query.

Example:

Geographic Distance Filtering

The DISTANCE filter expression limits a result set to include only documents with position field values (of type POINT) that lie inside a great circle defined by the specified center longitude and latitude and radius. See Geographic Searching for more information.

DISTANCE Syntax:

The <position-field> parameter value should be the name of an Attivio schema field of type POINT. The <center-longitude> and <center-latitude> parameter values are expressed in decimal degrees, with positive values representing northern latitudes and eastern longitudes. The <maximum-distance> parameter value is specified in decimal kilometers by default (but see discussion of the units parameter, below).

DISTANCE Syntax Variations

You can omit the <center-longitude> and <center-latitude> arguments if your query request specifies these values via the HTTP REST APIs' geo.longitude and geo.latitude query parameters:

Similarly, you can omit the <position-field> prefix if your query request specifies the desired field via a geo.field query parameter:

If you set all three query parameters, you can use this syntax:

DISTANCE Specific Parameters

Examples:

Return the set of documents with position field values within 500 miles of Vienna:

All of the example queries below should return the set of documents with position field values within 1,600 kilometers of the point at 41° N., 93° W.:

Shape Intersection Filtering

The RECTANGLE, POLYGON, CIRCLE, and ELLIPSE shape filter expressions limit a result set to include only documents with position field values (of type POINT) that lie within a specified shape of the given type, or those with shape field values (of type SHAPE) that intersect with a specified shape of the listed type. See Shape Intersection Filtering for more information.

Syntax:

Examples:

Saved Filters

Saved filters allow external systems to provide arbitrarily large sets of document IDs to be used as a filter against a result set from the index. The SAVEDFILTER operator takes a single argument, which must be a URI that all engine partitions can access. Each partition will request the URI and expects a newline separated list of document IDs to be returned. Note that each partition will fetch its own copy of the URI via an HTTP GET; caching (etc.) of the document IDs is left up to the web service implementing the back end.

Syntax:

SAVEDFILTER(<URI>)

Examples:

// search for the word 'dog' in the index but return documents whose document IDs are in the file hosted at http://myhost/someListOfDocumentIds.txt
AND(dog, SAVEDFILTER("http://myhost/someListOfDocumentIds.txt")
 
// same as above but use a URI that takes CGI arguments to generate document IDs
AND(dog, SAVEDFILTER("http://myhost/myService?arg1=val1&arg2=val2")

Sample return content from URI

documentId1
documentId2
documentId3

Query Builder User Interface

In some places in the Attivio user interfaces where an Advanced Query Language query may be entered, it is possible to use the Query Builder dialog box to construct the text of the query using graphical building blocks. There will be an Edit icon (like the pencil shown in the Create Rule dialog, below) next to the query field; clicking it opens the Query Builder dialog box. Edit the query using the controls in the dialog box and click "Save" when you are done to update the text field with your new query.