Package org.apache.lucene.analysis.compound

# Class CompoundWordTokenFilterBase

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.analysis.compound.CompoundWordTokenFilterBase

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `Unwrappable<TokenStream>`

Direct Known Subclasses:
  `DictionaryCompoundWordTokenFilter`, `HyphenationCompoundWordTokenFilter`

public abstract class CompoundWordTokenFilterBase extends TokenFilter
Base class for decomposition token filters.

## Nested Class Summary

Nested Classes
Modifier and Type
Class
Description
`protected class`
`CompoundWordTokenFilterBase.CompoundToken`
Helper class to hold decompounded token information

## Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

`AttributeSource.State`

## Field Summary

Fields
Modifier and Type
Field
Description
`static final int`
`DEFAULT_MAX_SUBWORD_SIZE`
The default for maximal length of subwords that get propagated to the output of this filter
`static final int`
`DEFAULT_MIN_SUBWORD_SIZE`
The default for minimal length of subwords that get propagated to the output of this filter
`static final int`
`DEFAULT_MIN_WORD_SIZE`
The default for minimal word length that gets decomposed
`protected final CharArraySet`
`dictionary`
`protected final int`
`maxSubwordSize`
`protected final int`
`minSubwordSize`
`protected final int`
`minWordSize`
`protected final OffsetAttribute`
`offsetAtt`
`protected final boolean`
`onlyLongestMatch`
`protected final CharTermAttribute`
`termAtt`
`protected final LinkedList<CompoundWordTokenFilterBase.CompoundToken>`
`tokens`

### Fields inherited from class org.apache.lucene.analysis.TokenFilter

`input`

### Fields inherited from class org.apache.lucene.analysis.TokenStream

`DEFAULT_TOKEN_ATTRIBUTE_FACTORY`

## Constructor Summary

Constructors
Modifier
Constructor
Description
`protected`
`CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary)`
`protected`
`CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary, boolean onlyLongestMatch)`
`protected`
`CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch)`

## Method Summary

Modifier and Type
Method
Description
`protected abstract void`
`decompose()`
Decomposes the current `termAtt` and places `CompoundWordTokenFilterBase.CompoundToken` instances in the `tokens` list.
`final boolean`
`incrementToken()`
`void`
`reset()`

### Methods inherited from class org.apache.lucene.analysis.TokenFilter

`close, end, unwrap`

### Methods inherited from class org.apache.lucene.util.AttributeSource

`addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString`

### Methods inherited from class java.lang.Object

`clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

## Field Details

### DEFAULT_MIN_WORD_SIZE

public static final int DEFAULT_MIN_WORD_SIZE
The default for minimal word length that gets decomposed

See Also:
  - Constant Field Values

### DEFAULT_MIN_SUBWORD_SIZE

public static final int DEFAULT_MIN_SUBWORD_SIZE
The default for minimal length of subwords that get propagated to the output of this filter

See Also:
  - Constant Field Values

### DEFAULT_MAX_SUBWORD_SIZE

public static final int DEFAULT_MAX_SUBWORD_SIZE
The default for maximal length of subwords that get propagated to the output of this filter

See Also:
  - Constant Field Values

### dictionary

protected final CharArraySet dictionary

### tokens

protected final LinkedList<CompoundWordTokenFilterBase.CompoundToken> tokens

### minWordSize

protected final int minWordSize

### minSubwordSize

protected final int minSubwordSize

### maxSubwordSize

protected final int maxSubwordSize

### onlyLongestMatch

protected final boolean onlyLongestMatch

### termAtt

protected final CharTermAttribute termAtt

### offsetAtt

protected final OffsetAttribute offsetAtt

## Constructor Details

### CompoundWordTokenFilterBase

protected CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary, boolean onlyLongestMatch)

### CompoundWordTokenFilterBase

protected CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary)

### CompoundWordTokenFilterBase

protected CompoundWordTokenFilterBase(TokenStream input, CharArraySet dictionary, int minWordSize, int minSubwordSize, int maxSubwordSize, boolean onlyLongestMatch)

## Method Details

### incrementToken

public final boolean incrementToken() throws IOException

Specified by:
  `incrementToken` in class `TokenStream`
Throws:
  `IOException`

### decompose

protected abstract void decompose()
Decomposes the current `termAtt` and places `CompoundWordTokenFilterBase.CompoundToken` instances in the `tokens` list. The original token may not be placed in the list, as it is automatically passed through this filter.

### reset

public void reset() throws IOException

Overrides:
  `reset` in class `TokenFilter`
Throws:
  `IOException`