Package org.apache.lucene.analysis.opennlp

# Class OpenNLPTokenizer

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.Tokenizer
org.apache.lucene.analysis.util.SegmentingTokenizerBase
org.apache.lucene.analysis.opennlp.OpenNLPTokenizer

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public final class OpenNLPTokenizer extends SegmentingTokenizerBase
Run OpenNLP SentenceDetector and Tokenizer. The index of each sentence is stored in SentenceAttribute.

## Nested Class Summary

## Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

`AttributeSource.State`

## Field Summary

### Fields inherited from class org.apache.lucene.analysis.util.SegmentingTokenizerBase

`buffer, BUFFERMAX, offset`

### Fields inherited from class org.apache.lucene.analysis.Tokenizer

`input`

### Fields inherited from class org.apache.lucene.analysis.TokenStream

`DEFAULT_TOKEN_ATTRIBUTE_FACTORY`

## Constructor Summary

Constructors
Constructor
Description
`OpenNLPTokenizer(AttributeFactory factory, NLPSentenceDetectorOp sentenceOp, NLPTokenizerOp tokenizerOp)`

## Method Summary

Modifier and Type
Method
Description
`void`
`close()`
`protected boolean`
`incrementWord()`
`void`
`reset()`
`protected void`
`setNextSentence(int sentenceStart, int sentenceEnd)`

### Methods inherited from class org.apache.lucene.analysis.util.SegmentingTokenizerBase

`end, incrementToken, isSafeEnd`

### Methods inherited from class org.apache.lucene.analysis.Tokenizer

`correctOffset, setReader, setReaderTestPoint`

### Methods inherited from class org.apache.lucene.util.AttributeSource

`addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString`

### Methods inherited from class java.lang.Object

`clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

## Constructor Details

### OpenNLPTokenizer

public OpenNLPTokenizer(AttributeFactory factory, NLPSentenceDetectorOp sentenceOp, NLPTokenizerOp tokenizerOp) throws IOException

Throws:
  `IOException`

## Method Details

### close

public void close() throws IOException

Specified by:
  `close` in interface `AutoCloseable`
Specified by:
  `close` in interface `Closeable`
Overrides:
  `close` in class `Tokenizer`
Throws:
  `IOException`

### setNextSentence

protected void setNextSentence(int sentenceStart, int sentenceEnd)

Specified by:
  `setNextSentence` in class `SegmentingTokenizerBase`

### incrementWord

protected boolean incrementWord()

Specified by:
  `incrementWord` in class `SegmentingTokenizerBase`

### reset

public void reset() throws IOException

Overrides:
  `reset` in class `SegmentingTokenizerBase`
Throws:
  `IOException`