Page tree
Skip to end of metadata
Go to start of metadata

Overview

AIE records and tracks System Events  on the Diagnostics > System Events page of the AIE Administrator. System events (INFO, WARN, ERROR) are also recorded in the <data-agent>\projects\<project-name>\<environment>\logs\logs-<node-name>\aie-node.error.log file, and in <data-agent>\projects\<project-name>\<environment>\logs\logs-<node-name>\aie-node.log

If you are using the Java Server API you might want to issue custom system events to flag error conditions in your code. This page demonstrates how to issue and view these events.

AIE Logging Configuration

View incoming links.

Sample Data

This example uses a simple ingest transformer (a "document" transformer) to create the environment for issuing system events. The transformer watches for a document with ID = 1, and then issues a INFO, WARN and FATAL events.

The incoming document is drawn from a brief CSV file containing this data:

C:\docs.csv
"id","title"
"1","Sample document"

System events are frequently issued from conditional statements that test some field value in a document. This simple document gives us the opportunity to set up that architecture in the body of the example.

Create a Custom Document Transformer in Java

An "ingest" or "document" transformer is a stage in an AIE workflow that examines ingested documents and (usually) modifies them on their way to the index. In this case, our example will test the ID value of the document. If the ID is "1" the transformer will issue three system events and then exit without modifying the document.

To create a simple ingest transformer, do the following:

  1. In AIE Designer, in the Package Explorer, right-click the appropriate project (AIE - plusjava in this case), and click the New -> Class menu item. The New Java Class dialog box appears.
  2. Specify the Source folder as the /src folder of your project, in this case AIE - plusjava/src.
  3. Select (or invent) an appropriate Java Package for the new transformer. In this case we chose com.acme.example.
  4. Enter the transformer's name. In this case it is GenerateSystemEvents.
  5. Accept the remaining defaults and click Finish. This opens an editor for GenerateSystemEvents .java.

The GenerateSystemEvents.java file is attached to this page. You can paste it into the edit buffer and proceed with the exercise.

Imported Classes

The following AIE classes are required for this example.

GenerateSystemEvents.java
package com.acme.example;

import java.util.HashMap;
import java.util.Map;
import com.attivio.sdk.AttivioException;
import com.attivio.sdk.ingest.IngestDocument;
import com.attivio.sdk.server.component.configuration.SystemEventPublisherAware;
import com.attivio.sdk.server.component.ingest.DocumentModifyingTransformer;
import com.attivio.sdk.server.util.SystemEventPublisher;
import com.attivio.sdk.server.annotation.ConfigurationOptionInfo;

Class Declaration

The class implements the DocumentModifyingTransformer and SystemEventPublisherAware interfaces.

GenerateSystemEvents.java
public class GenerateSystemEvents implements DocumentModifyingTransformer,
        SystemEventPublisherAware {

    private SystemEventPublisher pub;
    private Map<String, String> infoEventMetadata;
    private Map<String, String> warnEventMetadata;
    private Map<String, String> fatalEventMetadata;

When the SystemEventPublisherAware interface is invoked, AIE automatically creates a SystemEventPublisher  for the use of the new class. In this code snippet we have declared a SystemEventPublisher object, pub, which will be initialized by the setSystemEventPublisher() method (see below).

Then we declared three Map arrays that will be used to attach metadata to the System Events.

setSystemEventPublisher()

The setSystemEventPublisher() method is called automatically by the AIE system to attach a SystemEventPublisher to this transformer component. We created the SystemEventPublisher named pub in the previous step. This method must be present.

    @Override
    public void setSystemEventPublisher(SystemEventPublisher pub) {
        this.pub = pub;
    }

processDocument()

Every DocumentModifyingTransformer must implement a processDocument() method. AIE invokes this method once for every incoming document that arrives at this transformer.

GenerateSystemEvents.java
    public boolean processDocument(IngestDocument doc) throws AttivioException {
        
        // Generate three System Events for document with ID "1"
        if  (doc.getId().equals("1")) { 

The conditional test recognizes documents where the ID value is "1". In that case, the logic generates three system events.

INFO Event

To create an INFO-severity system event, invoke the SystemEventPublisher's infoEvent() method. This method takes three arguments: a subsystem string, a description string, and a metadata map.

In our example class, we create an infoEventMetadata HashMap to store optional comments describing the circumstances that caused the SystemEvent. Each row of the array consists of a key and a value. These metadata entries will not be displayed in the main System Events screen, but will be visible in the dialog box which appears when a user selects an event and chooses "Event Details".

GenerateSystemEvents.java
            /* Populate map of metadata for INFO-level event */
            infoEventMetadata = new HashMap<String, String>();
            infoEventMetadata.put("INFO event key 1", "INFO event value 1");
            infoEventMetadata.put("INFO event key 2", "INFO event value 2");

            /* Generate INFO-level System Event */
            pub.infoEvent("INFO event subsystem", "INFO event description",
                    infoEventMetadata);

The subsystem string tells viewers where the event occurred. The description explains what occurred. Then the infoEventMetadata strings provide additional information if necessary.

Note that infoEvent() requires a metadata map (infoEventMetadata in this case) but the map can be empty.

WARN Event

Issuing a WARN event is almost identical to issuing an INFO event.

GenerateSystemEvents.java
            /* Populate map of metadata for WARN-level event */
            warnEventMetadata = new HashMap<String, String>();
            warnEventMetadata.put("WARN event key 1", "WARN event value 1");
            warnEventMetadata.put("WARN event key 2", "WARN event value 2");

            /* Generate WARN-level System Event */
            pub.warnEvent("WARN event subsystem", "WARN event description",
                    warnEventMetadata);

FATAL Event

And issuing a FATAL event is very similar to the previous two events:

GenerateSystemEvents.java
            /* Populate map of metadata for FATAL-level event */
            fatalEventMetadata = new HashMap<String, String>();
            fatalEventMetadata.put("FATAL event key 1", "FATAL event value 1");
            fatalEventMetadata.put("FATAL event key 2", "FATAL event value 2");

            /* Generate FATAL-level System Event */
            pub.fatalEvent("FATAL event subsystem", "FATAL event description",
                    fatalEventMetadata);
        }

Return Value

The processDocument() method must return either true or false. If true, the document is passed to the next stage of the workflow. If false, the document is dropped.

GenerateSystemEvents.java
        /* Continue document processing */
        return true;
    }

Configure a Component and Workflow

We have defined a new class of transformer. Now we have to instantiate it as a named workflow component, and insert it into an AIE workflow.

Update and Deploy Events

Once the java coding is done, start the project running in AIE.  If this is the first time the project has run, the project .jar file will be automatically deployed to the configuration servers. The second and subsequent times it will be necessary to deploy the project manually from the Designer or the AIE-CLI interface, with the "force" option turned on.  If you have already created a component (or modified a workflow) dynamically in the AIE Administrator, you'll have to update the project and then deploy it. Do this each time you make a change in the java file.

See Developing in AIE - Concepts and Tools for a tutorial on update and deploy events in AIE development.

Create the Component

Start the project running and then open the AIE Administrator. Navigate to System Management > Palette. Open the New dialog box and navigate Platform Components > Document Transformers > GenerateSystemEvents.  Click the OK button to open the Component Editor.

Give the new component a name. We used MyGenerateSystemEvents.

Click Save.

Add to Workflow

We'd like to insert the new component in a workflow where it will be guaranteed to see the one special document we're going to load for it.  The easy way to achieve this is to insert the component in the ingest workflow just before the indexer subflow. All documents pass that point on their way to the index.

In the AIE Administrator, navigate to System Management > Workflows > Ingest > ingest.  This opens a Workflow Editor.  Use this editor to add the existing MyGenerateSystemEvents transformer just before the indexer, as shown in this image:

Save the workflow.

Create the CSV Connector

In order to load the single-document CSV file, we'll need a CSV Connector.

In the AIE Administrator, navigate to System Management > Connectors. Click the New link and select Connectors > CSV Files. This opens a Connector Editor.

Give the connector a name. We used MyCSVConnector.

Tell the connector where to find the target CSV file. 

Set an ID field to the label "id" (the first column in the target file).  Be sure that the ID Field Format is "%s." 

Save the connector.

Run the Example

To test this example, follow this procedure:

  1. In the AIE Administrator, navigate to System Management > Connectors.  Run the MyCSVConnector. It will take a few seconds to open the target CSV file and load its single document. Wait for the connector to finish before proceeding.
  2. If desired, you can navigate to Query > Debug Search or SAIL and use the Simple Query Language to search for *:*. You should see your minimal new document in the results list.
  3. Navigate to Diagnostics > System Events.  You should see three new System Events in the list.
  4. Edit the node log file, <data-agent>\projects\<project-name>\<environment>\logs\logs-<node-name>\aie-node.log. Look for new systems events in the log, similar to this one:

    <data-agent>\projects\<project-name>\<environment>\logs\logs-<node-name>\aie-node.log
    2016-05-03 10:08:58,033 ERROR LoggingSystemEventPublisher [AieReceiver-avm://ingest.MyGenerateSystemEvents.worker.0] - ATTIVIO-SystemEvent-1 : [ingest.MyGenerateSystemEvents] SystemEvent [id=-1, severity=SYS_FATAL, date=1462295338033, correlationState=NONE, getNodeset()=local, getSubsystem()=FATAL event subsystem, getMetadata()={FATAL event key 1=FATAL event value 1, FATAL event key 2=FATAL event value 2}, getDescription()=FATAL event description] 
    

 

 

  • No labels