Page tree
Skip to end of metadata
Go to start of metadata

Overview

The Attivio Platform uses the Quartz Enterprise Job Scheduler to manage and trigger scheduled events within the system. The scheduler module comes with a service component, the SchedulerService, that encapsulates the integration of the Quartz scheduler. It also ships with a Java client class, SchedulerClient , you can use to interact with the SchedulerService, providing methods for managing scheduled tasks, as well as pausing and resuming the scheduler.


Required Modules

These features require the inclusion of the scheduler module when you run createproject to create the project directories.

View incoming links.

 

Configuring the Scheduler Service

You can activate the scheduler in any Attivio project using the createproject tool. Including the scheduler module in a custom configuration configures the scheduler for the new project. When using a multi-node topology you must add a service location for the scheduler as follows. Only a single instance of the scheduler across all nodes is necessary so you can set the mySchedulerNodeSet below with a single node/nodeset name. 

<service-location name="scheduler" nodeset="mySchedulerNodeSet" />

The default scheduler configuration registers an in-process receiver called scheduler and a remote input with URI:

http://:/scheduler>http://<servername>:<Attivio node baseport + 1>/scheduler>

Under the default configuration the baseport is 17000. When using https via for all endpoint communication, you must add 2 to the baseport. With the default port configuration and the scheduler configuration shown above, the URL looks like this:

If using https:

Managing Scheduled Tasks

Create a SchedulerClientFactory and using the factory, create a SchedulerClient pointing to the external HTTP endpoint for the scheduler service as follows:

import com.attivio.sdk.modules.scheduler.*;

SchedulerClientFactory clientFactory = new DefaultSchedulerClientFactory();

SchedulerClient client = clientFactory.createRemoteSchedulerClient("http://localhost:17001/scheduler")

Scheduling a new task by invoking the 'saveTask' method on the client, passing in the name of the new task, owner and schedule, as well as the job implementation class that handles the triggered job (for details, see Cron-based Schedule Formats and Implementing a Custom Job).

import com.attivio.sdk.modules.scheduler.*;

ScheduledTask newTask = new ScheduledTask();

newTask.setName("my-new-task");
newTask.setOwner("me");
newTask.setJobClassName("org.acme.jobs.SimpleJob");
newTask.setCronExpression("0 2 * * * ?");

client.saveTask(newTask, true);

Together, the owner and name form the unique identifier for a task.  Updating the task require passing true as the second argument to the saveTask() method to indicate that you want to overwrite the existing task with the same name and owner.

// Update our existing task to run at 3 AM


import com.attivio.sdk.modules.scheduler.*;

ScheduledTask updatedTask = new ScheduledTask();

updatedTask.setName("my-new-task");
updatedTask.setOwner("me");
updatedTask.setJobClassName("org.acme.jobs.SimpleJob");
updatedTask.setCronExpression("0 0 3 * * ?");

client.saveTask(updatedTask, true);

To update the task without overwriting an existing one, pass false as the second argument to the saveTask() method.  You can check to see if the task was overwritten by looking at the return value.

// Update our existing task to run at 3 AM


import com.attivio.sdk.modules.scheduler.*;

ScheduledTask updatedTask = new ScheduledTask();

updatedTask.setName("my-new-task");
updatedTask.setOwner("me");
updatedTask.setJobClassName("org.acme.jobs.SimpleJob");
updatedTask.setCronExpression("0 0 3 * * ?");

// saveStatus is false if a task with the same name and owner already exists
//            is true if the task was successfully created
boolean saveStatus = client.saveTask(updatedTask, false);     

To deleting a task, do the following:

// Delete an existing task by supplying just the name and owner
client.deleteTask("my-new-task", "me");

Additionally, the client supports operations to pause the scheduler engine. This stops the firing of scheduled tasks, and resume operation again:

client.pauseScheduler();

client.resumeScheduler();

Below is a test program that uses the code samples above:

import com.attivio.sdk.modules.scheduler.*;
import com.attivio.model.AttivioException;

public class SchedulerTest {

  public static void main( String[] args ) {

  try {

      SchedulerClientFactory clientFactory = new DefaultSchedulerClientFactory();

      SchedulerClient client = clientFactory.createRemoteSchedulerClient("http://localhost:17001/scheduler")

      ScheduledTask newTask = new ScheduledTask();
      newTask.setName("my-new-task");
      newTask.setOwner("me");
      newTask.setJobClassName("org.acme.jobs.SimpleJob");
      newTask.setCronExpression("0 2 * * * ?");
      client.saveTask(newTask, true);

      // Update our existing task to run at 3 AM
      ScheduledTask updatedTask = new ScheduledTask();
      updatedTask.setName("my-new-task");
      updatedTask.setOwner("me");
      updatedTask.setJobClassName("org.acme.jobs.SimpleJob");
      updatedTask.setCronExpression("0 0 3 * * ?");
      client.saveTask(updatedTask, true);

      // Delete an existing task by supplying just the name and owner
      client.deleteTask("my-new-task", "me");

      client.pauseScheduler();

      client.resumeScheduler();

    } catch( AttivioException e ) {
      e.printStackTrace();
    }

  }

}

Cron-Based Schedule Formats

Attivio uses Quartz's CronTrigger implementation – a modified UNIX cron-style string format to represent a recurring schedule – for all scheduling. 

At its simplest, the cron format supplies the second, minute, hour, day (either date or day of the week), month, and year when a task should be triggered, supporting wildcards for each value for schedule recurrence. This is represented by six values (with an optional seventh), separated by spaces, for example, "* * * * * ?". The order of the values within the expression is illustrated in the table below, along with some typical examples.

Seconds

Minutes

Hours

Day of
the Month

Month

Day of
the Week

Year
(Optional)

Description

*

*

*

*

*

?

 

Fire every second of every hour of every day

0

0

3

*

*

?

 

Fire every day at 3AM

30

*

*

*

*

?

 

Fire every minute, on the 30th second

0

0

9

1

*

?

 

Fire once a month, at 9AM on the 1st

0

0

0

?

*

SAT

 

Fire every Saturday at midnight

0

0

0

*

*

7

 

Fire every Saturday at midnight (alternative)

0

15

14

4

7

?

2076

Fire on July 4th, 2076 at 2:15PM

These are the most basic usages of the cron format. Each value can also contain lists ("15,30" in "days of month" fires on the 15th and 30th), ranges ("9-17" in hours only fires between 9AM and 5PM), base + offset ("2/5" in minutes fires every 5 minutes, starting at 2 minutes after the hour, i.e. :02, :07, :12, etc), "n-th" values ("6#3" in "day of week" fires on the third Friday of every month, where Friday is the 6th day of the week).

See Quartz's CronTrigger Tutorial for a complete explanation.

Be aware that the SchedulerClient.scheduleTask and SchedulerClient.updateTask methods throw exceptions if invalid cron expressions are supplied.

Implementing a Custom Job

After scheduling a task to trigger, implement a job that does something useful once is triggered. To do this, create a new class that implements the com.attivio.model.AttivioScheduledJob interface. This interface defines one method:

void execute(Map<?, ?> context) throws AttivioException;

When the scheduler determines that it is time to trigger a task, it does the following:

  • Loads the jobClassName argument supplied when the task was scheduled/updated.
  • Creates a new instance of the job class.
  • Packages the task name, owner, and any other relevant metadata into a Java Map.

  • Invokes the execute  method of the job implementation, passing in the metadata Map as the argument.

 

 

The scheduler may invoke the execute method of several task instances concurrently, so the implementation must perform appropriate synchronization if shared resources are used.

Because Attivio loads the job implementation class, you must put the class in the Attivio classpath.

 

Note that the SchedulerService validates the supplied jobClassName by verifying that: the class exists, and can be loaded. It then implements the AttivioScheduledJob interface. This means that Attivio requires access to the class both at task scheduling time and at task execution time.

Also note that the job class need NOT be accessible to whatever process is using the SchedulerClient. The client deals only with the string representation of the class name. This is useful if a scheduler UI is running on a separate machine from Attivio.

The following is a basic example of a custom job implementation class.

package org.acme.jobs;

import com.attivio.model.AttivioScheduledJob;
import com.attivio.model.AttivioException;
import java.util.Map;

/** Simple scheduled job implementation class that sends a message to System.out when it is triggered */
public class SimpleJob implements AttivioScheduledJob {

  @Override
  public void execute(Map<?,?> context) throws AttivioException {
    // Get the scheduled job's name and owner from the context
    String jobName = (String)context.get(AttivioScheduledJob.JOB_NAME_PROP);
    String jobOwner = (String)context.get(AttivioScheduledJob.JOB_OWNER_PROP);

    // Send a message to System.out, illustrating that the job was triggered
    System.out.println("My SimpleJob was triggered by " + jobOwner + "." + jobName);
  }
}

When creating a new job implementation, put the jar containing the job class into the <install_dir> /lib or <project_dir> /lib directory and restart the Attivio node before scheduling the job.

Scheduling Connectors

You can schedule connectors to run automatically at predetermined times or intervals.

Scheduling a Connector from the Command Line

The following command runs the connector named "dbConnector" every 30 minutes.
It assumes that "dbConnector" connector is already configured on the system.

bin/aie-exec.exe schedule -s http://localhost:17001/scheduler --cron "0 0,30 * * * ?" --connector-uri dbConnector

Scheduling a Connector from the Connector Editor

You can schedule a connector from the Use the Attivio Administrator. Navigate to the System Management > Connectors page. Click on a connector to open the editor, then click the Scheduler tab.

ScheduleTab2

The advantage of using the connector editor is that the UI helps you work out the cron parameters.

Scheduling Common Index Tasks

Attivio provides functionality to support periodic commits, refreshes, optimizations, and backups of an index. 

Sample Configuration File

To schedule index tasks, define a bean for each task in a configuration XML file in your <project-dir>/conf/bean directory, along these lines:

<project-dir>/conf/bean/scheduledIndexTasks.xml
<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns="http://www.springframework.org/schema/beans"
       xmlns:util="http://www.springframework.org/schema/util"
       xmlns:sec="http://www.springframework.org/schema/security"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/util
                           http://www.springframework.org/schema/util/spring-util.xsd
                           http://www.springframework.org/schema/security
                           http://www.springframework.org/schema/security/spring-security-4.1.xsd">

  <!-- commit the index every 30 minutes (at 0 and 30 minutes after the hour) -->
  <bean name="scheduledCommit" class="com.attivio.platform.service.ScheduledIndexMessageDescriptor">
    <property name="messageUri" value="indexer" />
    <property name="cronExpression" value="0 0/30 * * * ? *" />
    <property name="indexMessageType" value="COMMIT" />
  </bean>  
  <!-- refresh the index every 15 minutes (at 5, 20, 35, and 50 minutes after the hour) -->
  <bean name="scheduledCommit" class="com.attivio.platform.service.ScheduledIndexMessageDescriptor">
    <property name="messageUri" value="indexer" />
    <property name="cronExpression" value="0 5/15 * * * ? *" />
    <property name="indexMessageType" value="REFRESH" />
  </bean>  
  <!-- optimize the index at midnight every Saturday -->
  <bean name="scheduledCommit" class="com.attivio.platform.service.ScheduledIndexMessageDescriptor">
    <property name="messageUri" value="indexer" />
    <property name="cronExpression" value="0 0 0 ? * SAT *" />
    <property name="indexMessageType" value="OPTIMIZE" />
  </bean>  
  <!-- back up the index every Sunday at 5:00 AM -->
  <bean name="scheduledCommit" class="com.attivio.platform.service.ScheduledIndexMessageDescriptor">
    <property name="messageUri" value="indexer" />
    <property name="cronExpression" value="0 0 5 ? * SUN *" />
    <property name="indexMessageType" value="BACKUP" />
  </bean>  

</beans>

These scheduled tasks should take effect after you next deploy the modified project configuration.

N.B.: The messageUri value should point to the ingest workflow that contains the content-dispatcher component for your index. By default, this is the indexer subflow, which contains the index-content-dispatcher component for the default index with name index. If you want to commit a different index, you will need to replace indexer with the appropriate workflow name in the above configuration.

Attivio does not recommend committing the index more often than every 15 minutes.

  • No labels