Page tree
Skip to end of metadata
Go to start of metadata

Rules dictionaries allow you to find documents in your AIE index and tag them with additional information regarding how well they match a series of queries. Using them, you can, for example, mark documents that contain suspicious content with a score, and then use those scores to find and act on documents which are above a certain threshold.

Rules dictionaries can be incorporated into the system's ingest workflows so all new documents are automatically tagged in this way upon ingestion. There is a component, ApplyTriggers, which can be inserted into your workflows and configured to apply the rules from a specific dictionary to each document as it is processed, and your workflows can have multiple instances of this component configured, subjecting your documents to the scrutiny of multiple rules dictionaries. The component and its use are described in the class' Javadoc documentation.

The remainder of this document describes using the user interface to create and manipulate the rules dictionaries available in your AIE system.

"Rules" and "rules dictionaries" are also sometimes referred to as "alerts", "triggers" or "trigger dictionaries."

The Rules Dictionary Administration Application

There is an sub-application built into the AIE Administration application which allows you to manage rules dictionaries in your system. To access this application, log into the AIE Admin UI in the standard way and click the Rules Dictionary Administration link under the Business Center group on the left panel. The application will open in a separate tab or window, depending on how your web browser is configured. To close it, simply close the window or tab that contains it.

Required Modules

These features require the inclusion of the triggers module when you run createproject to create the project directories. The module must be properly licensed or Attivio will fail to start.

Working with the Rules Dictionaries in Your AIE System

The Rules Dictionary Administration application has two main pages, the All Rules Dictionaries page, which lets you add and delete whole dictionaries as well as view and edit dictionary-level properties, and the individual Rules Dictionary page, which lets you add and delete individual rules, as well as view and edit rule-level properties. These pages, as well as the various dialog boxes and wizards in the application, are described in the sections of this document that follow, in the context of the various actions you will need to perform in order to create, edit, and validate your rules dictionaries.

Viewing Existing Rules Dictionaries

The All Rules Dictionaries page shows a list of all rules dictionaries in your AIE system. This page is the default page that you see when clicking the Rules Dictionary Administration link or when clicking the Attivio logo from elsewhere within the Rules Dictionary Administration application.

The list of dictionaries can be narrowed down by typing the beginning of a dictionary name in the Search for dictionaries... field. As you type, the list of dictionaries is limited to those the first characters of whose names match what you have typed. For example, you might type "an" to limit the list to dictionaries named "Anti-Money Laundering," "Anonymity," and "Anger Management."

When you select an individual dictionary in the list, the inspector on the right side of the page provides details about how that dictionary is configured. In the inspector, in addition to the list of detailed information for the particular dictionary, you can see controls at the top: the pencil-shaped button icon lets you edit the dictionary, and the gear-shaped button opens a pop-up list of commands that let you copy, compile, publish, export, or delete the dictionary. This functionality is all described in the functional sections below.

Viewing the Rules in a Dictionary

The individual dictionary page shows a list of all rules in a specific rules dictionary. To open the individual dictionary page, select a dictionary in the list on the All Rules Dictionaries page and click the dictionary's name  in the list.

The list of rules can be narrowed down by typing the beginning of a rule name in the "Search for rules..." field. As you type, the list of rules is limited to those the first characters of whose names match what you have typed. For example, you can type "sn" to limit the list to rules named "Sneaky," "SnakeInTheGrass," and "Snoop."

You can also limit the rules shown in dictionaries whose rules use multiple locales by choosing one of the locale (language) filters on the left side of the screen. Each locale filter shows how many rules use it and clicking one will narrow the list to only those rules. Clicking another locale's name will switch the display to that locale's rules. Clicking the X button next to the currently selected locale will return the display to not be filtered by locale.

When you select an individual rule in the list, the inspector on the right side of the page provides details about how that rule is configured. In the inspector, in addition to the list of detailed information for the particular rule, you can see controls at the top: the pencil-shaped button icon lets you edit the dictionary, and the gear-shaped button opens a pop-up list of commands that let you delete the rule. This functionality is all described in the functional sections below.

Adding a New Rules Dictionary

To add a brand new dictionary to your AIE system, click the Add a Dictionary button in the page's header. This opens the Create Rules Dictionary wizard which asks for the required parameters for creating a dictionary. The wizard has two pages, the first asks for general information and the second for advanced properties.

Create Rules Dictionary Wizard: Describe

The Describe page of the wizard asks for the name of the dictionary and the synonym dictionary to use.

ParameterDescription
NameThis will be the name of your rules dictionary. Note that you won't be able to change the name once the dictionary has been created.
Synonym DictionaryThis is the synonym dictionary that will be used for query expansion when your dictionary is used. (For example, if your query uses the term "America," the query dictionary could be configured to know to also look for "US" or "USA."

Create Rules Dictionary Wizard: Advanced Properties

The Describe page of the wizard asks which AIE schema to use and information about how to store the rule, policy, and version information in the documents being processed. Unless you fully understand the implications, you should leave this page's fields set to their default values.

ParameterDescription
Schema NameThis is a pop-up menu showing the names of the schemas in your AIE system. Generally, the default schema, called "default," is what will make sense.
Rule Name FieldThis is the name of the field that will be used for storing the name of rules which matched your document. If you use a field other than the default "trigger.names" field, you will need to make sure it is defined in your index.
Rule Score FieldThis is the name of the field that will be used for storing the total score of rules which matched your document. If you use a field other than the default "trigger.score" field, you will need to make sure it is defined in your index.
Rule Name ScopesIf this is set, then names of rules will be included in scope tagging of the document pieces that were matched.

Policy Name Field

This is the name of the field that will be used for storing the name of policies whose rules matched your document. If you use a field other than the default "trigger.entities" field, you will need to make sure it is defined in your index.
Policy Score FieldThis is the name of the field that will be used for storing the total score of policies whose rules which matched your document. This should be a dynamic field — the name of each policy will be substituted for the initial asterisk in the resulting documents. If you use a field other than the default "*_score" field, you will need to make sure it is defined in your index. For example, the field mypolicy_score in the document will contain the sum of the scores of all rules whose policy is "mypolicy" and whose queries matched that document.

Policy Name Scopes

If this is set, then names of policies will be included in scope tagging of the document pieces that were matched by the rules in those policies.
Version FieldThis is the name of the field where the version of the rules dictionary is stored for any documents which are matched. If you use a field other than the default "trigger.version" field, you will need to make sure it is defined in your index.

Copying an Existing Rules Dictionary

To copy a rules dictionary, select it in the list so it appears in the inspector and then click the gear button and choose Copy from the pop-up menu that opens. The Copy Rules Dictionary dialog box appears, asking you to provide a name for the copy. Once you type a name and click the Copy button, the dictionary is duplicated.

Editing a Rules Dictionary

To edit a rules dictionary, select it in the list so it appears in the inspector and then click the pencil button to open the Edit Rules Dictionary wizard. This wizard behaves exactly the same as the Create Rules Dictionary wizard, with the exception that the name of the dictionary cannot be changed.

Deleting a Rules Dictionary

To delete a rules dictionary, select it in the list on the All Rules Dictionaries page so it appears in the inspector and then click the gear button and choose Delete from the pop-up menu that opens. Once you confirm that you really want to delete the dictionary, it is removed.

NOTE:

Be aware that deleting a dictionary cannot be undone. If you think you might need to recreate the same dictionary at a later date, you may want to export it before deleting it.

Compiling a Rules Dictionary

In order for dictionaries to be usable by AIE, they must be both compiled and published. Compiling a dictionary stores it in a form which is easier for the system to use when processing documents. This includes parsing the query, tokenization and lemmatization, expanding synonyms using a synonym dictionary, and validation against the types of fields as defined in the schema. Tokenization, lemmatization, and synonym expansion are be applied based on the locale specified for each rule in the dictionary.

Dictionaries which need to be compiled show up in the list with the Needs to be compiled icon in the top-right corner of their listings. On the individual dictionary page, you can see which individual rules have changed since the previous time the dictionary was compiled.

Compiling from the All Rules Dictionary Page

To compile a rules dictionary, select it in the list so it appears in the inspector and then click the gear button and choose Compile from the pop-up menu that opens. A notification will show up in the bottom-right corner if the window indicating whether the compilation has succeeded or failed (and if it failed, why).

Compiling from the Individual Dictionary Page

To compile a rules dictionary, simply click the Compile Dictionary button at the top of the page. A notification will show up in the bottom-right corner if the window indicating whether the compilation has succeeded or failed (and if it failed, why).

Publishing a Rules Dictionary

In order for dictionaries to be usable by AIE, they must be both compiled and published. Publishing a dictionary marks it as available to the components in the workflow that use it to process incoming documents. Most changes that you make to a dictionary, including compiling it, will cause it to need to be published again. Dictionaries which need to be published show up in the list with the Needs to be published icon in the top-right corner of their listings. On the individual dictionary page, you can see which individual rules have changed since the previous time the dictionary was published.

Publishing from the All Rules Dictionary Page

To publish a rules dictionary, select it in the list so it appears in the inspector and then click the gear button and choose Publish from the pop-up menu that opens. A notification will show up in the bottom-right corner if the window indicating whether the publication has succeeded or failed (and if it failed, why).

Publishing from the Individual Dictionary Page

To publish a rules dictionary, simply click the Publish Dictionary button at the top of the page. A notification will show up in the bottom-right corner if the window indicating whether the publication has succeeded or failed (and if it failed, why).

NOTE:

If a dictionary doesn't currently need to be published, then the Publish command will not show up in the pop-up menu and the publish button will be disabled.

NOTE:

When you publish a dictionary, the last-compiled version is published. If there are changes that haven't yet been compiled, they will not be included in the published dictionary. If you want these changes in your published dictionary, you should compile it first. (If the dictionary has never been compiled, you will get an error if you try to publish it without compiling it first.)

Testing a Rules Dictionary

You can validate that the rules in your dictionary are behaving as you expect them to by doing a test run of the dictionary against an actual document in your AIE index. You will be able to see the values in the fields of the document before and after processing with the dictionary to ensure that the changes you expect are there.

To test a rules dictionary, navigate to the individual dictionary page for that dictionary and click the Test Dictionary button. In the dialog that appears, enter the ID of the document in your index you want to test against. When you click the Test button, you will see a table containing the fields in the test document both before and after the test was run. Fields whose values have changed will be highlighted in yellow for easy identification.

[screenshot]

NOTE:

To use the Test Dictionary feature, you will need to know the ID of the document in the index you want to test against. You can obtain this by doing a query against your index using a tool such as SAIL and then using the value of the .id field from the document you find. You may need assistance from your system administrator to obtain this.

Importing Rules Dictionaries

You can import rules dictionaries from XML-formatted files that were previously exported. To do so, click the Import Rules Dictionary button at the top of the All Rules Dictionaries page. The Import Rules Dictionary dialog appears. Browse to the XML file that you want to import and click the Import button.

If a dictionary already exists with the same name as the one defined in the file, you can either choose to use a different name for the imported copy by typing in the Rename field or check the Overwrite checkbox to replace the existing dictionary with the imported copy.

Exporting Rules Dictionaries

You can export rules dictionaries to XML-formatted files that can later be re-imported. To export a rules dictionary, select it in the list so it appears in the inspector and then click the gear button and choose Export as XML from the pop-up menu that opens. Your browser will begin downloading the XML file to its usual download directory.

 

NOTE:

Exporting rules dictionaries is a good way to preserve changes in a source control system or other external location. It can also be useful when moving rules dictionaries between AIE systems.

Adding a new Rule

To add a new rule to a dictionary, go to that dictionary's page by clicking its name in the listing on the All Rules Dictionaries page. Then  click the Add a Rule button in the page's header.

The Create Rule dialog asks you to specify information about the rule you wish to add.

ParameterDescription
NameThe name of the rule. The name must start with a number, letter, or underscore and contain only numbers, letters, underscores, dashes, and periods. Note that you won't be able to change the name once the rule has been created.
Query String

This is a string in the Attivio Advanced Query Language, or AQL, which will be used to determine if the rule should be applied to a particular document. Documentation for AQL is available here. If the query string you enter is not syntactically correct, you will see an error when you try to create the rule.

Use of the NOW keyword with RANGE expressions on date fields is not supported within trigger rules as it will be converted to the current time during compilation instead of query run time, likely not producing the desired result. Instead, use the wildcard character to match any date in the upper or lower bounds.

RANGE("2019-02-15T23:29:33.288", *, lower=exclusive, upper=exclusive) 


If you do not want to edit the query text directly, you can click the pencil icon next to this field to use the AQL Query Builder dialog to create or edit a query. Within the Query Builder you will see descriptions of how to use the various query operators and what their parameters are used for as well as see links to the full documentation online.

PolicyThe name of the policy into which the rule is grouped. As you type the policy name, you will see a list of existing policies whose names match what you are typing, to help you choose an existing name. You may also, however, enter an entirely new policy name. Like the rule's name, the policy name must start with a number, letter, or underscore and contain only numbers, letters, underscores, dashes, and periods.
ScoreThis is the value that will be added to the documents score if this rule matches it. This must be an integer.
LocaleThis is a pop-up list of the available locales (languages). The locale is used when evaluating the query against the documents.

Editing a Rule

To edit a rule, select it in the list on the individual dictionary page and click the pencil button in the inspector. The Edit Rule dialog appears letting you update the rule's properties. The dialog behaves exactly like the Create Rule dialog except that you cannot edit the name of the rule.

Deleting a Rule

To delete a rule, select it in the list on the individual dictionary page so it appears in the inspector and then click the gear button and choose Delete from the pop-up menu that opens. Once you confirm that you really want to delete the rule, it is removed.

NOTE:

Be aware that deleting a rule cannot be undone.

  • No labels