Distributed Index Handler 25.1
Distributed Index Handler Help

# Manage the Index Queue

The DIH maintains an internal queue of unfinished index actions that it receives. By default, the index queue size grows if the DIH needs to accept new index actions (that is, it does not discard jobs that have not been completed).

To tune indexing performance, you can control certain characteristics of the queue.

- You can presize the DIH index queue by setting the IndexQueueInitialSize parameter in the `[IndexQueue]` section of the configuration file. When you know that DIH is going to receive a large number of index jobs, you can use this option to reduce the number of times that DIH resizes the index queue.
- You can limit the size of the index queue by setting the IndexQueueMaxPendingItems parameter in the `[IndexQueue]` section of the configuration file. This parameter limits the number of incomplete index actions on the queue. When the queue size reaches this limit, DIH rejects any further index actions. You can override the queue limit for an individual index action by using the IgnoreMaxPendingItems index action parameter.

- You can specify the maximum number of finished (historical) index actions by setting the IndexQueueMaxHistory configuration parameter in the `[IndexQueue]` section of the configuration file. In this case, DIH keeps the most recent finished actions, and removes the oldest.

- By default, the DIH periodically polls each job in the index queue to see whether it is complete. When a job is complete,

  - Specify how often the polling occurs by modifying the PollInterval configuration parameter in the [`Server]` section of the DIH configuration file. The default value is 10 seconds.

    Frequent polling might help to keep the queue size down if you exceed the number of jobs specified in the IndexQueueMaxHistory configuration parameter. However, polling too frequently might affect indexing performance.

  - You can turn off polling completely by setting the Polling configuration parameter to `False`. In this case, DIH marks the job as complete as soon as it sends it to the child servers. DIH keeps the number of finished jobs specified by IndexQueueMaxHistory.

    Turn off polling for all child servers by setting Polling to `False` in the `[Server]` section. Alternatively, turn off polling for individual child servers by setting Polling to `False` in the `[IDOLServerN]` section.

- You can specify a directory for the DIH to use to store index queue status files. By default, it stores the files in the `./main` directory, relative to the DIH executable. Set the Main configuration parameter (in the [`Paths]` section of the DIH configuration file) to the full or relative path to the directory to use.

Copyright 2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery Distributed Index Handler 25.1

Topic Title: Manage the Index Queue