Package org.apache.lucene.index

# Class CheckIndex

java.lang.Object
org.apache.lucene.index.CheckIndex

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

public final class CheckIndex extends Object implements Closeable
Basic tool and API to check the health of an index and write a new segments file that removes reference to problematic segments.

As this tool checks every byte in the index, on a large index it can take quite a long time to run.

WARNING: This API is experimental and might change in incompatible ways in the next release.
  Please make a complete backup of your index before using this to exorcise corrupted documents from your index!

## Nested Class Summary

Nested Classes
Modifier and Type
Class
Description
`static class`
`CheckIndex.CheckIndexException`
The marker RuntimeException used by CheckIndex APIs when index integrity failure is detected.
`static class`
`CheckIndex.Level`
Class with static variables with information about CheckIndex's -level parameter.
`static class`
`CheckIndex.Options`
Run-time configuration options for CheckIndex commands.
`static class`
`CheckIndex.Status`
Returned from `checkIndex()` detailing the health and status of the index.
`static class`
`CheckIndex.VerifyPointsVisitor`
Walks the entire N-dimensional points space, verifying that all points fall within the last cell's boundaries.

## Constructor Summary

Constructors
Constructor
Description
`CheckIndex(Directory dir)`
Create a new CheckIndex on the directory.
`CheckIndex(Directory dir, Lock writeLock)`
Expert: create a directory with the specified lock.

## Method Summary

Modifier and Type
Method
Description
`static boolean`
`assertsOn()`
Check whether asserts are enabled or not.
`CheckIndex.Status`
`checkIndex()`
Returns a `CheckIndex.Status` instance detailing the state of the index.
`CheckIndex.Status`
`checkIndex(List<String> onlySegments)`
Returns a `CheckIndex.Status` instance detailing the state of the index.
`CheckIndex.Status`
`checkIndex(List<String> onlySegments, ExecutorService executorService)`
Returns a `CheckIndex.Status` instance detailing the state of the index.
`void`
`close()`
`int`
`doCheck(CheckIndex.Options opts)`
Actually perform the index check
`void`
`exorciseIndex(CheckIndex.Status result)`
Repairs the index using previously returned result from `checkIndex()`.
`boolean`
`getFailFast()`
See `setFailFast(boolean)`.
`int`
`getLevel()`
See `setLevel(int)`.
`static void`
`main(String[] args)`
Command-line interface to check and exorcise corrupt segments from an index.
`static CheckIndex.Options`
`parseOptions(String[] args)`
Parse command line args into fields
`void`
`setFailFast(boolean v)`
If true, just throw the original exception immediately when corruption is detected, rather than continuing to iterate to other segments looking for more corruption.
`void`
`setInfoStream(PrintStream out)`
Set infoStream where messages should go.
`void`
`setInfoStream(PrintStream out, boolean verbose)`
Set infoStream where messages should go.
`void`
`setLevel(int v)`
Sets Level, the higher the value, the more additional checks are performed.
`void`
`setThreadCount(int tc)`
Set threadCount used for parallelizing index integrity checking.
`static CheckIndex.Status.DocValuesStatus`
`testDocValues(CodecReader reader, PrintStream infoStream, boolean failFast)`
Test docvalues.
`static CheckIndex.Status.FieldInfoStatus`
`testFieldInfos(CodecReader reader, PrintStream infoStream, boolean failFast)`
Test field infos.
`static CheckIndex.Status.FieldNormStatus`
`testFieldNorms(CodecReader reader, PrintStream infoStream, boolean failFast)`
Test field norms.
`static CheckIndex.Status.LiveDocStatus`
`testLiveDocs(CodecReader reader, PrintStream infoStream, boolean failFast)`
Test live docs.
`static CheckIndex.Status.PointsStatus`
`testPoints(CodecReader reader, PrintStream infoStream, boolean failFast)`
Test the points index.
`static CheckIndex.Status.TermIndexStatus`
`testPostings(CodecReader reader, PrintStream infoStream)`
Test the term index.
`static CheckIndex.Status.TermIndexStatus`
`testPostings(CodecReader reader, PrintStream infoStream, boolean verbose, int level, boolean failFast)`
Test the term index.
`static CheckIndex.Status.IndexSortStatus`
`testSort(CodecReader reader, Sort sort, PrintStream infoStream, boolean failFast)`
Tests index sort order.
`static CheckIndex.Status.StoredFieldStatus`
`testStoredFields(CodecReader reader, PrintStream infoStream, boolean failFast)`
Test stored fields.
`static CheckIndex.Status.TermVectorStatus`
`testTermVectors(CodecReader reader, PrintStream infoStream)`
Test term vectors.
`static CheckIndex.Status.TermVectorStatus`
`testTermVectors(CodecReader reader, PrintStream infoStream, boolean verbose, int level, boolean failFast)`
Test term vectors.
`static CheckIndex.Status.VectorValuesStatus`
`testVectors(CodecReader reader, PrintStream infoStream, boolean failFast)`
Test the vectors index.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

## Constructor Details

### CheckIndex

public CheckIndex(Directory dir) throws IOException
Create a new CheckIndex on the directory.

Throws:
  `IOException`

### CheckIndex

public CheckIndex(Directory dir, Lock writeLock)
Expert: create a directory with the specified lock. This should really not be used except for unit tests!!!! It exists only to support special tests (such as TestIndexWriterExceptions*), that would otherwise be more complicated to debug if they had to close the writer for each check.

## Method Details

### close

public void close() throws IOException

Specified by:
  `close` in interface `AutoCloseable`
Specified by:
  `close` in interface `Closeable`
Throws:
  `IOException`

### setLevel

public void setLevel(int v)
Sets Level, the higher the value, the more additional checks are performed. This will likely drastically increase time it takes to run CheckIndex! See `CheckIndex.Level`

### getLevel

public int getLevel()
See `setLevel(int)`.

### setFailFast

public void setFailFast(boolean v)
If true, just throw the original exception immediately when corruption is detected, rather than continuing to iterate to other segments looking for more corruption.

### getFailFast

public boolean getFailFast()
See `setFailFast(boolean)`.

### setThreadCount

public void setThreadCount(int tc)
Set threadCount used for parallelizing index integrity checking.

### setInfoStream

public void setInfoStream(PrintStream out, boolean verbose)
Set infoStream where messages should go. If null, no messages are printed. If verbose is true then more details are printed.

### setInfoStream

public void setInfoStream(PrintStream out)
Set infoStream where messages should go. See `setInfoStream(PrintStream,boolean)`.

### checkIndex

public CheckIndex.Status checkIndex() throws IOException
Returns a `CheckIndex.Status` instance detailing the state of the index.

As this method checks every byte in the index, on a large index it can take quite a long time to run.

WARNING: make sure you only call this when the index is not opened by any writer.

Throws:
  `IOException`

### checkIndex

public CheckIndex.Status checkIndex(List<String> onlySegments) throws IOException
Returns a `CheckIndex.Status` instance detailing the state of the index.

Parameters:
  `onlySegments` - list of specific segment names to check

  As this method checks every byte in the specified segments, on a large index it can take quite a long time to run.

Throws:
  `IOException`

### checkIndex

public CheckIndex.Status checkIndex(List<String> onlySegments, ExecutorService executorService) throws IOException
Returns a `CheckIndex.Status` instance detailing the state of the index.

This method allows caller to pass in customized ExecutorService to speed up the check.

WARNING: make sure you only call this when the index is not opened by any writer.

Throws:
  `IOException`

### testSort

public static CheckIndex.Status.IndexSortStatus testSort(CodecReader reader, Sort sort, PrintStream infoStream, boolean failFast) throws IOException
Tests index sort order.

Throws:
  `IOException`

### testLiveDocs

public static CheckIndex.Status.LiveDocStatus testLiveDocs(CodecReader reader, PrintStream infoStream, boolean failFast) throws IOException
Test live docs.

Throws:
  `IOException`

### testFieldInfos

public static CheckIndex.Status.FieldInfoStatus testFieldInfos(CodecReader reader, PrintStream infoStream, boolean failFast) throws IOException
Test field infos.

Throws:
  `IOException`

### testFieldNorms

public static CheckIndex.Status.FieldNormStatus testFieldNorms(CodecReader reader, PrintStream infoStream, boolean failFast) throws IOException
Test field norms.

Throws:
  `IOException`

### testPostings

public static CheckIndex.Status.TermIndexStatus testPostings(CodecReader reader, PrintStream infoStream) throws IOException
Test the term index.

Throws:
  `IOException`

### testPostings

public static CheckIndex.Status.TermIndexStatus testPostings(CodecReader reader, PrintStream infoStream, boolean verbose, int level, boolean failFast) throws IOException
Test the term index.

Throws:
  `IOException`

### testPoints

public static CheckIndex.Status.PointsStatus testPoints(CodecReader reader, PrintStream infoStream, boolean failFast) throws IOException
Test the points index.

Throws:
  `IOException`

### testVectors

public static CheckIndex.Status.VectorValuesStatus testVectors(CodecReader reader, PrintStream infoStream, boolean failFast) throws IOException
Test the vectors index.

Throws:
  `IOException`

### testStoredFields

public static CheckIndex.Status.StoredFieldStatus testStoredFields(CodecReader reader, PrintStream infoStream, boolean failFast) throws IOException
Test stored fields.

Throws:
  `IOException`

### testDocValues

public static CheckIndex.Status.DocValuesStatus testDocValues(CodecReader reader, PrintStream infoStream, boolean failFast) throws IOException
Test docvalues.

Throws:
  `IOException`

### testTermVectors

public static CheckIndex.Status.TermVectorStatus testTermVectors(CodecReader reader, PrintStream infoStream) throws IOException
Test term vectors.

Throws:
  `IOException`

### testTermVectors

public static CheckIndex.Status.TermVectorStatus testTermVectors(CodecReader reader, PrintStream infoStream, boolean verbose, int level, boolean failFast) throws IOException
Test term vectors.

Throws:
  `IOException`

### exorciseIndex

public void exorciseIndex(CheckIndex.Status result) throws IOException
Repairs the index using previously returned result from `checkIndex()`. Note that this does not remove any of the unreferenced files after it's done; you must separately open an `IndexWriter`, which deletes unreferenced files when it's created.

WARNING: this writes a new segments file into the index, effectively removing all documents in broken segments from the index. BE CAREFUL.

Throws:
  `IOException`

### assertsOn

public static boolean assertsOn()
Check whether asserts are enabled or not.

Returns:
  true iff asserts are enabled

### main

public static void main(String[] args) throws IOException, InterruptedException
Command-line interface to check and exorcise corrupt segments from an index.

Run it like this:

     java -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex pathToIndex [-exorcise] [-verbose] [-segment X] [-segment Y]

- `-exorcise`: actually write a new segments_N file, removing any problematic segments. *LOSES DATA*
- `-segment X`: only check the specified segment(s). This can be specified multiple times, to check more than one segment: `-segment _2 * -segment _a`. You can't use this with the -exorcise option.

WARNING: `-exorcise` should only be used on an emergency basis as it will cause documents (perhaps many) to be permanently removed from the index. Always make a backup copy of your index before running this! Do not run this tool on an index that is actively being written to. You have been warned!

Run without -exorcise, this tool will open the index, report version information and report any exceptions it hits and what action it would take if -exorcise were specified. With -exorcise, this tool will remove any segments that have issues and write a new segments_N file. This means all documents contained in the affected segments will be removed.

This tool exits with exit code 1 if the index cannot be opened or has any corruption, else 0.

Throws:
  `IOException`
  `InterruptedException`

### parseOptions

public static CheckIndex.Options parseOptions(String[] args)
Parse command line args into fields

Parameters:
  `args` - The command line arguments
Returns:
  An Options struct
Throws:
  `IllegalArgumentException` - if any of the CLI args are invalid

### doCheck

public int doCheck(CheckIndex.Options opts) throws IOException, InterruptedException
Actually perform the index check

Parameters:
  `opts` - The options to use for this check
Returns:
  0 iff the index is clean, 1 otherwise
Throws:
  `IOException`
  `InterruptedException`