Package org.apache.lucene.analysis.pattern

# Class SimplePatternTokenizerFactory

java.lang.Object
org.apache.lucene.analysis.AbstractAnalysisFactory
org.apache.lucene.analysis.TokenizerFactory
org.apache.lucene.analysis.pattern.SimplePatternTokenizerFactory

public class SimplePatternTokenizerFactory extends TokenizerFactory
Factory for `SimplePatternTokenizer`, for matching tokens based on the provided regexp.

This tokenizer uses Lucene `RegExp` pattern matching to construct distinct tokens for the input stream. The syntax is more limited than `PatternTokenizer`, but the tokenization is quite a bit faster. It takes two arguments:

- "pattern" (required) is the regular expression, according to the syntax described at `RegExp`
- "determinizeWorkLimit" (optional, default 10000) the limit on total effort spent to determinize the automaton computed from the regexp

The pattern matches the characters to include in a token (not the split characters), and the matching is greedy such that the longest token matching at a given point is created. Empty tokens are never created.

For example, to match tokens delimited by simple whitespace characters:

     <fieldType name="text_ptn" class="solr.TextField" positionIncrementGap="100">
       <analyzer>
         <tokenizer class="solr.SimplePatternTokenizerFactory" pattern="[^ \t\r\n]+"/>
       </analyzer>
     </fieldType>

Since:
  6.5.0
See Also:
  - `SimplePatternTokenizer`
WARNING: This API is experimental and might change in incompatible ways in the next release.
SPI Name (case-insensitive: if the name is 'htmlStrip', 'htmlstrip' can be used when looking up the service).
  "simplePattern"

## Field Summary

Fields
Modifier and Type
Field
Description
`static final String`
`NAME`
SPI name
`static final String`
`PATTERN`

### Fields inherited from class org.apache.lucene.analysis.AbstractAnalysisFactory

`LUCENE_MATCH_VERSION_PARAM, luceneMatchVersion`

## Constructor Summary

Constructors
Constructor
Description
`SimplePatternTokenizerFactory()`
Default ctor for compatibility with SPI
`SimplePatternTokenizerFactory(Map<String,String> args)`
Creates a new SimplePatternTokenizerFactory

## Method Summary

Modifier and Type
Method
Description
`SimplePatternTokenizer`
`create(AttributeFactory factory)`

### Methods inherited from class org.apache.lucene.analysis.TokenizerFactory

`availableTokenizers, create, findSPIName, forName, lookupClass, reloadTokenizers`

### Methods inherited from class org.apache.lucene.analysis.AbstractAnalysisFactory

`defaultCtorException, get, get, get, get, get, getBoolean, getChar, getClassArg, getFloat, getInt, getLines, getLuceneMatchVersion, getOriginalArgs, getPattern, getSet, getSnowballWordSet, getWordSet, isExplicitLuceneMatchVersion, require, require, require, requireBoolean, requireChar, requireFloat, requireInt, setExplicitLuceneMatchVersion, splitAt, splitFileNames`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### NAME

public static final String NAME
SPI name

See Also:
  - Constant Field Values

### PATTERN

public static final String PATTERN

See Also:
  - Constant Field Values

## Constructor Details

### SimplePatternTokenizerFactory

public SimplePatternTokenizerFactory(Map<String,String> args)
Creates a new SimplePatternTokenizerFactory

### SimplePatternTokenizerFactory

public SimplePatternTokenizerFactory()
Default ctor for compatibility with SPI

## Method Details

### create

public SimplePatternTokenizer create(AttributeFactory factory)

Specified by:
  `create` in class `TokenizerFactory`