Overview

The Java Client API lets us create independent Java applications that can send documents to an AIE server for ingestion, and can also query AIE and interpret the search results.

View incoming links.

• Architecture ,
• Attivio Modules ,
• Content Ingestion - Concepts and Tools ,
• Create an Attivio Project on Hadoop ,
• Creating Custom Response Transformers ,
• Custom Index Security With ACLs ,
• Dates and Date Formats ,
• Developing with Attivio ,
• Discrete Facets ,
• English (en) ,
• Field Expressions ,
• Filter-Based Facets ,
• In Depth Architecture Diagrams ,
• Java Model API ,
• Load Data and Content ,
• Non-English Queries ,
• Opensearch API ,
• Query and Response - Concepts and Tools ,
• Query Data and Content in Attivio ,
• Query Resubmission ,
• Range Facets ,
• Real-Time Updates ,
• Relational Querying JOIN ,
• Schema Facet ,
• Scope Facets ,
• Search Application Example ,
• Sorting Results ,
• Streaming Query API ,
• Updating Content ,
• Using Java APIs

API implementations are obtained through a Service Discovery framework.  Nodes and other service providers register services with a centralized ZooKeeper (a.k.a. Configuration Server) service.  Clients obtain API implementations from ZooKeeper.  These implementations then work directly with the node or other service provider.  Service Discovery is discussed in more detail below.

Java Client API Overview

The Java Client API contains the interfaces necessary for developing client applications to use with an AIE server.  The Client project depends on the Model project.

The Java Client API consists of the com.attivio.sdk.client  package, which is described below, and the com.attivio.sdk.client.streaming  package, which is described on the Streaming Query API page..

APIs used for ingestion and search:

APIs which interact directly with the AIE store:

APIs for administrative functions

Software Requirements

• Installation of the maven build tool for your platform.
• Installation of Oracle JDK 1.8.
• Installation of Attivio if running integration or client tests.

How to get started

Attivio provides the SDK in a public github repository, branched according to version.  Instructions for building modules and client projects are provided there.

Service Discovery

The ZooKeeper-based Service Discovery framework allows clients to connect to Attivio systems without knowledge of where or how individual services are provided.  Clients are more flexible than in the past (versions previous to 5.x) since the client is no longer required to provide the URI for connection to AIE.  Service discovery also allows an architecture where clients can connect directly to services hosted outside of Attivio nodes (such as Hadoop).  Service Discovery requires only the connection information to ZooKeeper and the name of the Attivio project – all other connection information is obtained from ZooKeeper.

Connection information for ZooKeeper can be provided via the java AIE_ZOOKEEPER system property or the AIE_ZOOKEEPER environment variable.   The project name is set using Platform.instance.setProjectName(name).  The example below illustrates this, comparing the prior and new way of obtaining the ContentStoreClient API:

AieClientFactory clientFactory = new DefaultAieClientFactory();
ContentStoreClient csc = clientFactory.createContentStoreClient(contentStoreHost, contentStorePort);
IngestClient ingestClient = clientFactory.createIngestClient(ingestHost, ingestPort);

System.setProperty("AIE_ZOOKEEPER","hostname:2181"); // Used for Multinode projects
System.setProperty("AIE_ZOOKEEPER","hostname:16980"); // Used for Single node projects
Platform.instance.setProjectName("myproject"); 
Platform.instance.setProjectEnvironment("myenvironment");
ContentStoreClient csc = ServiceFactory.getService(ContentStoreClient.class);
IngestClient ingestClient = ServiceFactory.getService(IngestClient.class);

In this simple example there is not any code savings.  However, with the exception of ZooKeeper whereabouts, the client code is now independent of the topology of the application.  As the application changes (new nodes, moved nodes, etc) the client does not have to be called differently or updated.  Additionally the ServiceFactory class generates dynamic proxies for each API appropriate for their use.  Client connections to services are automatically pooled, load balanced, localized, and recreated as appropriate.

Security

When the application has security enabled for service APIs, the client must supply authorization information prior to obtaining the API.  Since different APIs may require different authorization or other client configuration information, the service discovery framework provides a generic way for registering client API parameters.  The example below shows how security information may be supplied for APIs requiring basic authorization:

ServiceFactory.setFactoryParams(new AuthParams() {
  public String getUsername() { return "admin"; }
  public String getPassword() { return "admin"; }
});
ingestApi = ServiceFactory.getService(IngestApi.class);

Failure to supply required parameters will result in an exception:

Exception: PLATFORM-103 : Client factory class com.attivio.platform.service.RemoteIngestApiFactory for service interface com.attivio.service.IngestApi requires registration of parameters of type interface com.attivio.service.AuthParams
	at com.attivio.service.ServiceFactory.createApiFromInstance(ServiceFactory.java:354)
	at com.attivio.service.ServiceFactory.getServiceInternal(ServiceFactory.java:266)
	at com.attivio.service.ServiceFactory.getService(ServiceFactory.java:210)
	at com.attivio.integration.platform.transport.AuthenticatedTransportTest.authTest(AuthenticatedTransportTest.java:43)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

Bypassing Service Discovery Load Balancing

In some situations it is desirable to bypass the load balancing features of the Service Discovery framework to handle load balancing specifically at the client level.  In particular, when the query endpoints are fairly static and hardware load balancing is desired, the Service Discovery framework load balancing features defeat the purpose.  In this situation, lower-level APIs may be used to access individual service registrations and choose which service is used at runtime.  For example:

/* build a list of registered "nodeNames" for the SearchClient service */
ServiceDiscovery<AieServiceInfo> sd = ServiceDiscoveryHelper.instance();
ArrayList<String> names = new ArrayList<>();
for (ServiceInstance<AieServiceInfo> instance : sd.queryForInstances(SearchClient.class.getName())) {
  names.add(instance.getPayload().getNodeName());
}
/* get service associated with a particular node */
HashMap<String, SearchClient> clients = new HashMap<>();
for (String nm : names) {
  clients.put(nm, ServiceFactory.getService(SearchClient.class, nm);
}
 
////////////////////////////////////////////////////////////////////////////////////////////
/* OR, Get list of http endpoints for search (e.g. use with http load balancer) */
List<String> searchEndpoints = new ArrayList<>();
for (ServiceInstance<AieServiceInfo> instance : sd.queryForInstances(SearchClient.class.getName())) {
  boolean ssl = instance.getSslPort() != null;
  int port = ssl ? instance.getPort() : instance.getSslPort();

  searchEndpoints.add(RemoteSearchClient.getHttpQueryEndpoint(instance.getAddress(), port, ssl));
}

Setting up an HTTP Load Balancer

Determining the endpoints for the load balancer:

/* Get list of http endpoints for search (e.g. use with http load balancer) */
List<String> searchEndpoints = new ArrayList<>();
for (ServiceInstance<AieServiceInfo> instance : sd.queryForInstances(SearchClient.class.getName())) {
  boolean ssl = instance.getSslPort() != null;
  int port = ssl ? instance.getPort() : instance.getSslPort();

  searchEndpoints.add(RemoteSearchClient.getHttpQueryEndpoint(instance.getAddress(), port, ssl));
}

Connecting to the load balancer:

RemoteSearchClient searchClient = new RemoteSearchClient(loadBalancerAddress, loadBalancerPort, useSsl);

Proxying APIs

It is sometimes advantageous to proxy service APIs via an Attivio node.  This case occurs when clients are running in environments where they do not have direct access to the Attivio stores.  For instance, in a secured Hadoop cluster, clients may not be able to directly interact with Attivio's HDFS and HBase stores.  In such a case, required APIs can be proxied by an Attivio node running as a Hadoop edge node.  While proxying is not as scalable and efficient as direct access, it can help with system architecture constraints.  To proxy services, the ServiceApiProxy bean must be added to your project in the conf/beans/root/ directory.   In the example below, the TriggerApi is being proxied:

<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.springframework.org/schema/beans" xmlns:util="http://www.springframework.org/schema/util" xmlns:sec="http://www.springframework.org/schema/security" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-4.1.xsd">
  <bean name="serviceApiProxy" class="com.attivio.service.ServiceApiProxy">
    <property name="proxies">
      <list>
        <value>com.attivio.sdk.client.trigger.TriggerApi</value>
      </list>
    </property>
  </bean>
</beans>

When proxying APIs, the client code does not need to change, the proxied service will be discovered automatically.  Not all APIs can be proxied since certain serialization constraints must be met to proxy an API.  The following APIs have been tested for proxying.  Note, methods which take InputStream and OutputStream parameters must use stream implementations which are Serializable.

• com.attivio.sdk.client.TriggerApi
• com.attivio.sdk.client.EventStoreApi

When using an API that cannot be proxied, a SaxException2 class not found error may occur.  To see the full exception the following dependency may be added to the project:

    <!-- Required for proxied APIs -->
    <dependency>
      <groupId>com.sun.xml.bind</groupId>
      <artifactId>jaxb-core</artifactId>
      <version>2.2.7</version>
    </dependency>

Examples

These are full-length examples with sample code attached:

• Streaming Query API
• Search Application Example
• Ingest Application Example