Overview

Although knowing the overall sentiment of a document is often useful, it is also useful to be able to analyze the sentiment for smaller parts of a document, too. The Attivio Intelligence Engine (AIE) provides several ways to perform Entity Extraction. The Entity Sentiment module assigns sentiment weights to those entities, based on their context within a document.

The sentiment model built by sentiment training is used to assign sentiment to entities as well. Building such a model is the first, and most challenging, step.

View incoming links.

• Arabic (using ALM) ,
• Attivio Glossary ,
• Attivio Modules ,
• Attivio Text Analytics ,
• Chinese (simplified) ,
• Chinese (traditional) ,
• Czech (cs) ,
• Danish (da) ,
• Dutch (nl) ,
• English (en) ,
• English (using ALM) ,
• Entity Extraction ,
• Farsi (using ALM) ,
• Features by Language Table ,
• Finnish (fi) ,
• French (fr) ,
• German (de) ,
• Greek (el) ,
• Hebrew (using ALM) ,
• Hungarian (hu) ,
• Italian (it) ,
• Japanese (ja) ,
• Korean (ko) ,
• Norwegian (nb) ,
• Polish (pl) ,
• Portuguese (pt) ,
• Romanian (ro) ,
• Russian (ru) ,
• Scope Search ,
• Spanish (es) ,
• Swedish (sv) ,
• Thai (th) ,
• Turkish (tr) ,
• Urdu (ur) ,
• Vietnamese (vi)

Prerequisites

To use Entity Sentiment:

• Entity extraction must be turned on. See Dictionary Entity Extraction.
• A sentiment model must exist. See Building Sentiment Models.

• The entity-extraction handlers must include the ScopeTagger. See Scope Search.

  .

Configuration

Entity Sentiment is primarily a matter of configuring the entity sentiment module, and adding it to your ingestion workflow.

When you install AIE with the Entity Sentiment module, your AIE configuration includes the following definition:

<component name="addEntitySentiment"
           class="com.attivio.platform.transformer.ingest.document.AddEntitySentiment">
  <properties>
    <map name="stopWordDictionaries">
      <!-- A minimal stopword dictionary is better for sentiment -->
      <property name="en" value="classifier/dictionaries/small_stopwords_en.csv" />
      <!-- add stopword dictionaries for other languages here -->
    </map>
    <list name="negationPatterns">
      <entry value="not" />
      <entry value=".*n't" />
    </list>
    <property name="modelName" value="${sentiment.defaultModelFileName}" />
    <property name="lowerCase" value="true" />
    <property name="posOutput" value="entity.sentiment.pos"/>
    <property name="posOutputScore" value="entity.sentiment.pos.score"/>
    <property name="negOutput" value="entity.sentiment.neg"/>
    <property name="negOutputScore" value="entity.sentiment.neg.score"/>
    <property name="includingSentences" value="true" />
    <property name="includingNounPhrases" value="false" />
    <property name="useEmptyModelIfNoneFound" value="true" />
    <property name="blurriness" value="10" />
    <property name="minimumPositiveValue" value="0.5"/>
    <property name="minimumNegativeValue" value="0.5"/>
  </properties>
</component>

Each of the properties reflects some aspect of the component's operation:

• the ignoreScopes list (not shown above) specifies scopes which will be ignored by entity sentiment analysis. By default, this list includes the number, date, extracteddate, time, and url scopes.

• the stopWordsDictionary map sets, for each language specified, a file of stopwords for that langauges.

  • The stopWordsDictionary has no impact on the identification of entities. You cannot eliminate an entity by adding it to the stopWordsDictionary.

  • The stopWordsDictionary can be used to eliminate loaded words that skew the sentiment values associated with entities.

• the negationPatterns list specifies the tokens or token patterns that connote negation, or reverse sentiment polarity in a similar way.

• the modelName is the uri for the sentiment model file that this component will use.

  default modelName value

  The default value of modelName should be changed from the default value before use. The default value is empty-sentiment.model, which will both not do anything and trigger empty model log warnings. The model generic-sentiment.model is recommended, unless the user decides to train their own sentiment model. This change may be made directly in the conf file, or by changing the value in the attivio.entitysentiment.properties file.

• the lowerCase property specifies, if true, that the tokens should be lowercased before being looked up the model. (This value should match the training time value.)
• the posOutput property is the name of the field in which to put the entities assigned positive sentiment by the component.
• the posOutputScore property is the name of the field in which to put this scores for the entities assigned positive sentiment by the component.
• the negOutput property is the name of the field in which to put the entities assigned negative sentiment by the component.
• the negOutputScore property is the name of the field in which to put the scores for the entities assigned negative sentiment by the component.
• the includingSentences property is a boolean that specifies whether sentential information should be used for calculating entity sentiment. It is suggested that this property remain set to "true".
• the includingNounPhrases property is a boolean that specifies whether extracted noun phrases should have sentiments extracted for them as for other entity types.
• the useEmptyModelIfNoneFound property is a boolean. Its meaning is the same as in the sentiment module: If "true", load an empty model if no model file can be found.
• the blurriness property is a positive integer. After computing an initial sentiment value for each token, the algorithm will spread this sentiment value to the blurriness tokens before and after it. The default value is 10.
• the minimumPositiveValue and minimumNegativeValue define the minimum sentiment value that a entity must have in a document before being included in the list of entities with positive and negative sentiment, respectively. Their default value is 0.5.

Note that the fields used for this configuration need to be specified in an addition to the schema. When you create a project with createproject and include the entitysentiment module, your project's schema will include the appropriate fields:

<schema name="default" merge="true">
  <fields> <!--  default-search-field="content" -->
    <!-- Just like the "cat" field -->
    <field name="sentiment" type="string" tokenize="false"
           indexed="true" stored="true" facet="true"
           displayName="Sentiment" />
    <field name="sentiment.score" type="float" indexed="true"
           stored="true" facet="false"
           displayName="Score" />
    <field name="entity.sentiment.pos" type="string" tokenize="false"
           indexed="false" stored="true" facet="true"
           displayName="PositiveEntities" />
    <field name="entity.sentiment.pos.score" type="float" tokenize="false"
           indexed="false" stored="true" facet="false"
           displayName="PositiveEntityScores" />
    <field name="entity.sentiment.neg" type="string" tokenize="false"
           indexed="false" stored="true" facet="true"
           displayName="NegativeEntities" />
    <field name="entity.sentiment.neg.score" type="float" tokenize="false"
           indexed="false" stored="true" facet="false"
           displayName="NegativeEntityScores" />
  </fields>
</schema>

If you ensure that your schema includes these fields, and you add the addEntitySentiment component to an ingest workflow, then entity sentiment will be added to the documents processed by that ingest workflow.

Best Practices