IDOL 12.13

# Identify and Restore Index Corruption

The most common causes of index corruption are:

- process termination (when a process is forcibly stopped or stops unexpectedly, or when the hardware is switched off)

- storage failures (connection to network storage broken, or disk failure)

- RAM problems (typically rare)

You can normally tell whether the IDOL process was terminated abnormally by inspecting the IDOL Server logs and seeing if the `Version` message is preceded by a clean stop message. If there is no clean stop message, it suggests the previous run was somehow interrupted. System logs also often give an idea of whether a process was terminated abnormally. Storage or RAM failures are typically more difficult to detect and might require additional system tools.

IDOL Server usually detects if it was previously interrupted, and on startup it attempts to validate the internal indexes. During this startup validation the process is unavailable for querying or indexing. You can override this behavior by setting the `ValidateOnInterruption` to `False`. IDOL Server also typically rolls back any index actions that were not committed, and automatically reattempts them.

You can run validation at any time when the server is running by using the `DREVALIDATE` index action, or by using the Validate feature on the Service Control tab of the Console page in the Control section of IDOL Admin.

NOTE: No indexing occurs during the validation process, but the system is still available for queries. Validation is normally an I/O intensive process so it might reduce query performance.

After validation is complete and the server is available for query, IDOL Server displays the result of the validation in the `GetStatus` response. The `GetStatus` response always includes information from the last available validation record. If any index has failed validation then the IDOL Server logs provide more details about the errors detected.

In some cases it is desirable to prevent the system from running with a corrupted index or to stop further validation as soon as a single error has been found. You can control this behavior by using the `ValidateOptions` configuration parameter.

The `diskindex` and `nodetable` indexes are the most fundamental structures in IDOL, and failures in these structures typically means that a server might need to be restored from backup or from source data. In some cases the errors might be isolated to document ranges, in which case it might be possible to just delete those documents and reindex them. You can regenerate the other data structures (`numeric`, `match`, `parametric` and so on) using the `Regenerate*Index` configuration parameters. IDOL Server rebuilds all these structures using data held in the nodetable.

If either of the `diskindex` or `nodetable` indexes are damaged, you can use the following approaches to recover the data (listed in order of speed from fastest to slowest:

- Restore from a `DREBACKUP` (possibly in combination with partial application of some of the following items).

- Replay archived index actions (to do this, you must configure an `ArchivePath` to store index actions).

- Reindex any stored IDX or XML files generated by the connectors.

- `DREEXPORT` from valid `nodetable` data and reindex into a fresh server to rebuild the `diskindex` (and all other indexes).

- Rerun the import process (effectively restarting the entire import and index process).

© Copyright 2022 Micro Focus or one of its affiliates
Version 12.13 | Last updated October 2022

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to swpdl.idoldocsfeedback@microfocus.com.

We appreciate your feedback!

Product: IDOL 12.13

Topic Title: Identify and Restore Index Corruption