Twitter Connector 24.4
IDOL Twitter Connector Help

# Retrieve Information from a Sampled Stream

To automatically start the synchronize action and retrieve tweets from a sampled stream, create a new fetch task using the following procedure.

NOTE: Twitter allows a user account to connect to only one stream at a time. If you run multiple fetch tasks concurrently (by setting the `TaskThreads` parameter) and these tasks connect to Twitter streams, Twitter might close the connections.

To retrieve information from a sampled stream

1. Stop the connector.
2. Open the connector’s configuration file in a text editor.

3. In the `[Ingestion]` section of the configuration file, set the `IngestBatchSize` parameter:

   `IngestBatchSize` | The number of documents to send to the ingestion server in each batch. If you are processing large numbers of streamed tweets and sending the information to CFS, OpenText recommends setting this parameter to `500` or greater. The Twitter Connector sends metadata-only documents to CFS, and CFS can process large batches more efficiently than small batches.

4. In the `[FetchTasks]` section of the configuration file, specify the number of fetch tasks using the `Number` parameter. If you are configuring the first fetch task, type `Number=1`. If one or more fetch tasks have already been configured, increase the value of the `Number` parameter by one (1). Below the `Number` parameter, specify the names of the fetch tasks, starting from zero (0). For example:

       [FetchTasks]
       Number=1
       0=MyTask

5. Below the `[FetchTasks]` section, create a new TaskName section. The name of the section must match the name of the new fetch task. For example:

       [FetchTasks]
       Number=1
       0=MyTask

       [MyTask]

6. Ensure that the relevant OAuth parameters are included in the configuration file. For example, the following line imports the relevant parameters from a file named `oauth.cfg`. This file is generated by the OAuth configuration tool that is supplied with the connector. For more information about configuring OAuth authentication, see Configure OAuth Authentication.

       [MyTask] < "oauth.cfg" [OAUTH]

   For information about importing parameters into a configuration file, see Include an External Configuration File.

7. In the new section, set the following parameters:

   `ApiVersion` | Set `ApiVersion=V2`.
   `StreamType` | The type of stream to ingest tweets from. To retrieve tweets from a sampled stream, set this parameter to `statuses/sample`.

8. Save and close the configuration file. You can now start the connector.

Copyright 2024 Open Text
Version 24.4 | Last updated September 2024

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: IDOL Twitter Connector 24.4

Topic Title: Retrieve Information from a Sampled Stream