Package org.apache.lucene.analysis.query

# Class QueryAutoStopWordAnalyzer

java.lang.Object
org.apache.lucene.analysis.Analyzer
org.apache.lucene.analysis.AnalyzerWrapper
org.apache.lucene.analysis.query.QueryAutoStopWordAnalyzer

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public final class QueryAutoStopWordAnalyzer extends AnalyzerWrapper
An `Analyzer` used primarily at query time to wrap another analyzer and provide a layer of protection which prevents very common words from being passed into queries.

For very large indexes the cost of reading TermDocs for a very common word can be high. This analyzer was created after experience with a 38 million doc index which had a term in around 50% of docs and was causing TermQueries for this term to take 2 seconds.

Since:
  3.1

## Nested Class Summary

## Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

`Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents`

## Field Summary

Fields
Modifier and Type
Field
Description
`static final float`
`defaultMaxDocFreqPercent`

### Fields inherited from class org.apache.lucene.analysis.Analyzer

`GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY`

## Constructor Summary

Constructors
Constructor
Description
`QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader)`
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for all indexed fields from terms with a document frequency percentage greater than `defaultMaxDocFreqPercent`
`QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader, float maxPercentDocs)`
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for all indexed fields from terms with a document frequency percentage greater than the given maxPercentDocs
`QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader, int maxDocFreq)`
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for all indexed fields from terms with a document frequency greater than the given maxDocFreq
`QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader, Collection<String> fields, float maxPercentDocs)`
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for the given selection of fields from terms with a document frequency percentage greater than the given maxPercentDocs
`QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader, Collection<String> fields, int maxDocFreq)`
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for the given selection of fields from terms with a document frequency greater than the given maxDocFreq

## Method Summary

Modifier and Type
Method
Description
`Term[]`
`getStopWords()`
Provides information on which stop words have been identified for all fields
`String[]`
`getStopWords(String fieldName)`
Provides information on which stop words have been identified for a field
`protected Analyzer`
`getWrappedAnalyzer(String fieldName)`
`protected Analyzer.TokenStreamComponents`
`wrapComponents(String fieldName, Analyzer.TokenStreamComponents components)`

### Methods inherited from class org.apache.lucene.analysis.AnalyzerWrapper

`attributeFactory, createComponents, getOffsetGap, getPositionIncrementGap, initReader, initReaderForNormalization, normalize, wrapReader, wrapReaderForNormalization, wrapTokenStreamForNormalization`

### Methods inherited from class org.apache.lucene.analysis.Analyzer

`close, getReuseStrategy, normalize, tokenStream, tokenStream`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### defaultMaxDocFreqPercent

public static final float defaultMaxDocFreqPercent

See Also:
  - Constant Field Values

## Constructor Details

### QueryAutoStopWordAnalyzer

public QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader) throws IOException
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for all indexed fields from terms with a document frequency percentage greater than `defaultMaxDocFreqPercent`

Parameters:
  `delegate` - Analyzer whose TokenStream will be filtered
  `indexReader` - IndexReader to identify the stopwords from
Throws:
  `IOException` - Can be thrown while reading from the IndexReader

### QueryAutoStopWordAnalyzer

public QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader, int maxDocFreq) throws IOException
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for all indexed fields from terms with a document frequency greater than the given maxDocFreq

Parameters:
  `delegate` - Analyzer whose TokenStream will be filtered
  `indexReader` - IndexReader to identify the stopwords from
  `maxDocFreq` - Document frequency terms should be above in order to be stopwords
Throws:
  `IOException` - Can be thrown while reading from the IndexReader

### QueryAutoStopWordAnalyzer

public QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader, float maxPercentDocs) throws IOException
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for all indexed fields from terms with a document frequency percentage greater than the given maxPercentDocs

Parameters:
  `delegate` - Analyzer whose TokenStream will be filtered
  `indexReader` - IndexReader to identify the stopwords from
  `maxPercentDocs` - The maximum percentage (between 0.0 and 1.0) of index documents which contain a term, after which the word is considered to be a stop word
Throws:
  `IOException` - Can be thrown while reading from the IndexReader

### QueryAutoStopWordAnalyzer

public QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader, Collection<String> fields, float maxPercentDocs) throws IOException
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for the given selection of fields from terms with a document frequency percentage greater than the given maxPercentDocs

Parameters:
  `delegate` - Analyzer whose TokenStream will be filtered
  `indexReader` - IndexReader to identify the stopwords from
  `fields` - Selection of fields to calculate stopwords for
  `maxPercentDocs` - The maximum percentage (between 0.0 and 1.0) of index documents which contain a term, after which the word is considered to be a stop word
Throws:
  `IOException` - Can be thrown while reading from the IndexReader

### QueryAutoStopWordAnalyzer

public QueryAutoStopWordAnalyzer(Analyzer delegate, IndexReader indexReader, Collection<String> fields, int maxDocFreq) throws IOException
Creates a new QueryAutoStopWordAnalyzer with stopwords calculated for the given selection of fields from terms with a document frequency greater than the given maxDocFreq

Parameters:
  `delegate` - Analyzer whose TokenStream will be filtered
  `indexReader` - IndexReader to identify the stopwords from
  `fields` - Selection of fields to calculate stopwords for
  `maxDocFreq` - Document frequency terms should be above in order to be stopwords
Throws:
  `IOException` - Can be thrown while reading from the IndexReader

## Method Details

### getWrappedAnalyzer

protected Analyzer getWrappedAnalyzer(String fieldName)

Specified by:
  `getWrappedAnalyzer` in class `AnalyzerWrapper`

### wrapComponents

protected Analyzer.TokenStreamComponents wrapComponents(String fieldName, Analyzer.TokenStreamComponents components)

Overrides:
  `wrapComponents` in class `AnalyzerWrapper`

### getStopWords

public String[] getStopWords(String fieldName)
Provides information on which stop words have been identified for a field

Parameters:
  `fieldName` - The field for which stop words identified in "addStopWords" method calls will be returned
Returns:
  the stop words identified for a field

### getStopWords

public Term[] getStopWords()
Provides information on which stop words have been identified for all fields

Returns:
  the stop words (as terms)