Package org.apache.lucene.benchmark.byTask.feeds

# Class TrecContentSource

java.lang.Object
org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource
org.apache.lucene.benchmark.byTask.feeds.ContentSource
org.apache.lucene.benchmark.byTask.feeds.TrecContentSource

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public class TrecContentSource extends ContentSource
Implements a `ContentSource` over the TREC collection.

Supports the following configuration parameters (on top of `ContentSource`):

- work.dir - specifies the working directory. Required if "docs.dir" denotes a relative path (default=work).
- docs.dir - specifies the directory where the TREC files reside. Can be set to a relative path if "work.dir" is also specified (default=trec).
- trec.doc.parser - specifies the `TrecDocParser` class to use for parsing the TREC documents content (default=TrecGov2Parser).
- html.parser - specifies the `HTMLParser` class to use for parsing the HTML parts of the TREC documents content (default=DemoHTMLParser).
- content.source.encoding - if not specified, ISO-8859-1 is used.
- content.source.excludeIteration - if true, do not append iteration number to docname

## Field Summary

Fields
Modifier and Type
Field
Description
`static final String`
`DOC`
`static final String`
`DOCNO`
`static final String`
`NEW_LINE`
separator between lines in the byffer
`static final String`
`TERMINATING_DOC`
`static final String`
`TERMINATING_DOCNO`

### Fields inherited from class org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource

`encoding, forever, logStep, verbose`

## Constructor Summary

Constructors
Constructor
Description
`TrecContentSource()`

## Method Summary

Modifier and Type
Method
Description
`void`
`close()`
Called when reading from this content source is no longer required.
`DocData`
`getNextDocData(DocData docData)`
Returns the next `DocData` from the content source.
`Date`
`parseDate(String dateStr)`
`void`
`resetInputs()`
Resets the input for this content source, so that the test would behave as if it was just started, input-wise.
`void`
`setConfig(Config config)`
Sets the `Config` for this content source.

### Methods inherited from class org.apache.lucene.benchmark.byTask.feeds.ContentItemsSource

`addBytes, addItem, collectFiles, getBytesCount, getConfig, getItemsCount, getTotalBytesCount, getTotalItemsCount, printStatistics, shouldLog`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Field Details

### DOCNO

public static final String DOCNO

See Also:
  - Constant Field Values

### TERMINATING_DOCNO

public static final String TERMINATING_DOCNO

See Also:
  - Constant Field Values

### DOC

public static final String DOC

See Also:
  - Constant Field Values

### TERMINATING_DOC

public static final String TERMINATING_DOC

See Also:
  - Constant Field Values

### NEW_LINE

public static final String NEW_LINE
separator between lines in the byffer

## Constructor Details

### TrecContentSource

public TrecContentSource()

## Method Details

### parseDate

public Date parseDate(String dateStr)

### close

public void close() throws IOException
Description copied from class: `ContentItemsSource`
Called when reading from this content source is no longer required.

Specified by:
  `close` in interface `AutoCloseable`
Specified by:
  `close` in interface `Closeable`
Specified by:
  `close` in class `ContentItemsSource`
Throws:
  `IOException`

### getNextDocData

public DocData getNextDocData(DocData docData) throws NoMoreDataException, IOException
Description copied from class: `ContentSource`
Returns the next `DocData` from the content source. Implementations must account for multi-threading, as multiple threads can call this method simultaneously.

Specified by:
  `getNextDocData` in class `ContentSource`
Throws:
  `NoMoreDataException`
  `IOException`

### resetInputs

public void resetInputs() throws IOException
Description copied from class: `ContentItemsSource`
Resets the input for this content source, so that the test would behave as if it was just started, input-wise.

NOTE: the default implementation resets the number of bytes and items generated since the last reset, so it's important to call super.resetInputs in case you override this method.

Overrides:
  `resetInputs` in class `ContentItemsSource`
Throws:
  `IOException`

### setConfig

public void setConfig(Config config)
Description copied from class: `ContentItemsSource`
Sets the `Config` for this content source. If you override this method, you must call super.setConfig.

Overrides:
  `setConfig` in class `ContentItemsSource`