************************ General Queue Parameters ************************ =========================== =========================================================================== **Authors:** `Rainer Gerhards `_ ; Pascal Withopf =========================== =========================================================================== Usage ===== Queue parameters can be used together with the following statements: - :doc:`action() <../configuration/actions>` - ruleset() - main\_queue() Queues need to be configured in the action or ruleset it should affect. If nothing is configured, default values will be used. Thus, the default ruleset has only the default main queue. Specific Action queues are not set up by default. To fully understand queue parameters and how they interact, be sure to read the :doc:`queues <../concepts/queues>` documentation. Configuration Parameters ======================== .. note:: As with other configuration objects, parameters for this object are case-insensitive. queue.filename -------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "word", "none", "no", "``$ActionQueueFileName``" File name to be used for the queue files. If specified, this parameter enables disk-assisted queue functionality. If *not* specified, the queue will operate without saving the queue to disk, either during its operation or when shut down. See the separate ``queue.saveonshutdown`` parameter to configure that option. Please note that this is actually just the file name. A directory can NOT be specified in this parameter. If the files shall be created in a specific directory, specify ``queue.spoolDirectory`` for this. The filename is used to build to complete path for queue files. queue.spoolDirectory -------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "word", "none", "no", "none" This is the directory into which queue files will be stored. Note that the directory must exist, it is NOT automatically created by rsyslog. If no spoolDirectory is specified, the work directory is used. queue.size ---------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "1000/50000", "no", "``$ActionQueueSize``" This is the maximum size of the queue in number of messages. Note that setting the queue size to very small values (roughly below 100 messages) is not supported and can lead to unpredictable results. For more information on the current status of this restriction see the `rsyslog FAQ: "lower bound for queue sizes" `_. The default depends on queue type and rsyslog version, if you need a specific value, please specify it. Otherwise rsyslog selects what it consideres appropriate for the version in question. In rsyslog rsyslog 8.30.0, for example, ruleset queues have a default size of 50000 and action queues which are configured to be non-direct have a size of 1000. queue.dequeueBatchSize ---------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "128/1024", "no", "``$ActionQueueDequeueBatchSize``" Specifies the maximum batch size for dequeue operations. This setting affects performance. As a rule of thumb, larger batch sizes (up to a environment-induced upper limit) provide better performance. For the average system, there usually should be no need to adjust batch sizes as the defaults are sufficient. The default for ruleset queues is 1024 and for action queues 128. Note that this only specifies the **maximum** batch size. Batches will be slower if rsyslog does not have as many messages inside the queue at time of dequeuing it. If you want to set a minimum Batch size as well, you can use `queue.minDequeueBatchSize`. queue.minDequeueBatchSize ------------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "0", "no", "none" Specifies the **minimum** batch size for dequeue operations. This setting is especially useful with outputs like ElasticSearch or ClickHouse, where you want to limit the number of HTTP requests. With this setting, the queue engine waits up to `queue.minDequeueBatchSize.timeout` milliseconds when there are fewer messages currently queued. Note that the minimum batch size cannot be larger than the configured maximum batch size. If so, it is automatically adjusted to match the maximum. So if in doubt, you need to specify both parameters. queue.minDequeueBatchSize.timeout --------------------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "1000", "no", "none" This parameter is only meaningful if use together with `queue.minDequeueBatchSize`, otherwise it is ignored. It specifies the amout of time (in milliseconds) rsyslogs waits for new messages so that the minimum batch size can be reached. After this period, the batch is processed, *even if it is below minimum size*. This capability exists to prevent having messages stalled in an incomplete batch just because no new messages arrive. We would expect that it usually makes little sense to set it to higher than 60.000 (60 seconds), but this is permitted. Just be warned that this potentially delays log processing for that long. queue.maxDiskSpace ------------------ .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "0", "no", "``$ActionQueueMaxDiskSpace``" The maximum size that all queue files together will use on disk. Note that the actual size may be slightly larger than the configured max, as rsyslog never writes partial queue records. queue.highWatermark ------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "90% of queue.size", "no", "``$ActionQueueHighWaterMark``" This applies to disk-assisted queues, only. When the queue fills up to this number of messages, the queue begins to spool messages to disk. Please note that this should not happen as part of usual processing, because disk queue mode is very considerably slower than in-memory queue mode. Going to disk should be reserved for cases where an output action destination is offline for some period. queue.lowWatermark ------------------ .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "70% of queue.size", "no", "``$ActionQueueLowWaterMark``" This applies to disk-assisted queues, only. When the high watermark is reached, the queue will write data to disk. It does so until the low watermark is reached, then the queue reverts back to in-memory mode. queue.fullDelaymark ------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "97% of queue.size", "no", "none" Number of messages when the queue should block delayable messages. Messages are NO LONGER PROCESSED until the queue has sufficient space again. If a message is delayable depends on the input. For example, messages received via imtcp are delayable (because TCP can push back), but those received via imudp are not (as UDP does not permit a push back). The intent behind this setting is to leave some space in an almost-full queue for non-delayable messages, which would be lost if the queue runs out of space. Please note that if you use a DA queue, setting the fulldelaymark BELOW the highwatermark makes the queue never activate disk mode for delayable inputs. So this is probably not what you want. queue.lightDelaymark -------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "70% of queue.size", "no", "none" If this mark is reached the sender will be throttled if possible. queue.discardMark ----------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "80% of queue.size", "no", "``$ActionQueueDiscardMark``" Specifies the threshold at which rsyslog begins to discard less important messages. To define which messages should be discarded use the queue.discardseverity parameter. queue.discardSeverity --------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "8", "no", "``$ActionQueueDiscardSeverity``" As soon as the threshold of the parameter queue.discardMark is reached incoming aswell as queued messages with a priority equal or lower than specified will be erased. With the default no messages will be erased. You have to specify a numeric severity value for this parameter. queue.checkpointInterval ------------------------ .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "0", "no", "``$ActionQueueCheckpointInterval``" Disk queues by default do not update housekeeping structures every time the queue writes to disk. This is for performance reasons. In the event of failure, data will still be lost (except when data is mangled via the file structures). However, disk queues can be set to write bookkeeping information on checkpoints (every n records), so that this can be made ultra-reliable, too. If the checkpoint interval is set to one, no data can be lost, but the queue is exceptionally slow. queue.syncqueuefiles -------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "binary", "off", "no", "``$ActionQueueSyncQueueFiles``" Disk-based queues can be made very reliable by issuing a (f)sync after each write operation. This happens when you set the parameter to "on". Activating this option has a performance penalty, so it should not be turned on without a good reason. Note that the penalty also depends on *queue.checkpointInterval* frequency. queue.samplingInterval ---------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "0", "no", "none" .. versionadded:: 8.23.0 This option allows queues to be populated by events produced at a specific interval. It provides a way to sample data each N events, instead of processing all, in order to reduce resources usage (disk, bandwidth...) queue.type ---------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "word", "Direct", "no", "``$ActionQueueType``" Specifies the type of queue that will be used. Possible options are "FixedArray", "LinkedList", "Direct" or "Disk". For more information read the documentation for :doc:`queues <../concepts/queues>`. queue.workerThreads ------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "1", "no", "``$ActionQueueWorkerThreads``" Specifies the maximum number of worker threads that can be run parallel. queue.workerThreadMinimumMessages --------------------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "queue.size/queue.workerthreads", "no", "``$ActionQueueWorkerThreadMinimumMessages``" Specify the number of messages a worker thread is processing before another worker thread is created. This number is limited by parameter queue.workerThreads. For example if this parameter is set to 200 and in the queue are 201 messages a second worker thread will be created. queue.timeoutWorkerthreadShutdown --------------------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "60000", "no", "``$ActionQueueTimeoutWorkerthreadShutdown``" After starting a worker thread, it will process messages until there are no messages for him to process. This parameter specifies the time the worker thread has to be inactive before it times out. The parameter must be specified in milliseconds. Which means the default of 60000 is 1 minute. queue.timeoutshutdown --------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "10/1500", "no", "``$ActionQueueTimeoutShutdown``" If a queue that still contains messages is terminated it will wait the specified time interval for the worker thread to finish. The time is specfied in milliseconds (1000ms is 1sec). Default for action queues is 10, for ruleset queues it is 1500. queue.timeoutActionCompletion ----------------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "1000", "no", "``$ActionQueueTimeoutActionCompletion``" When a queue is terminated, the timeout shutdown is over and there is still data in the queue, the queue will finish the current data element and then terminate. This parameter specifies the timeout for processing this last element. Parameter is specified in milliseconds (1000ms is 1sec). queue.timeoutEnqueue -------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "2000", "no", "``$ActionQueueTimeoutEnqueue``" This timeout value is used when the queue is full. If rsyslog cannot enqueue a message within the timeout period, the message is discarded. Note that this is setting of last resort (assuming defaults are used for the queue settings or proper parameters are set): all delayable inputs (like imtcp or imfile) have already been pushed back at this stage. Also, discarding of lower priority messages (if configured) has already happened. So we run into one of these situations if we do not timeout quickly enough: * if using imuxsock and no systemd journal is involved, the system would become unresponsive and most probably a hard reset would be required. * if using imuxsock with imjournal forwarding is active, messages are lost because the journal discards them (more agressive than rsyslog does) * if using imjournal, the journal will buffer messages. If journal runs out of configured space, messages will be discarded. So in this mode discarding is moved to a bit later place. * other non-delayable sources like imudp will also loose messages So this setting is provided in order to guard against problematic situations, which always will result either in message loss or system hang. For action queues, one may debate if it would be better to overflow rapidly to the main queue. If so desired, this is easy to acomplish by setting a very large timeout value. The same, of course, is true for the main queue, but you have been warned if you do so! In some other words, you can consider this scenario, using default values. With all progress blocked (unable to deliver a message): * all delayable inputs (tcp, relp, imfile, imjournal, etc) will block indefinantly (assuming queue.lightdelaymark and queue.fulldelaymark are set sensible, which they are by default). * imudp will be loosing messages because the OS will be dropping them * messages arriving via UDP or imuxsock that do make it to rsyslog, and that are a severity high enough to not be filtered by discardseverity, will block for 2 seconds trying to put the message in the queue (in the hope that something happens to make space in the queue) and then be dropped to avoid blocking the machine permanently. Then the next message to be processed will also be tried for 2 seconds, etc. * If this is going into an action queue, the log message will remain in the main queue during these 2 seconds, and additional logs that arrive will accumulate behind this in the main queue. queue.maxFileSize ----------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "1m/16m", "no", "``$ActionQueueMaxFileSize``" Specifies the maximum size for the disk-assisted queue file. Parameter can be spezified in Mebibyte or Gibibyte, default for action queues is 1m and for ruleset queues 16m (1m = 1024*1024). queue.saveOnShutdown -------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "binary", "off", "no", "``$ActionQueueSaveOnShutdown``" This parameter specifies if data should be saved at shutdown. queue.dequeueSlowDown --------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "0", "no", "``$ActionQueueDequeueSlowDown``" Regulates how long dequeueing should be delayed. This value must be specified in microseconds (1000000us is 1sec). It can be used to slow down rsyslog so it won't send things to fast. For example if this parameter is set to 10000 on a UDP send action, the action won't be able to put out more than 100 messages per second. queue.dequeueTimeBegin ---------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "0", "no", "``$ActionQueueDequeueTimeBegin``" With this parameter you can specify rsyslog to process queues during specific time periods. To define a time frame use the 24-hour format without minutes. This parameter specifies the begin and "queue.dequeuetimeend" the end of the time frame. queue.dequeueTimeEnd -------------------- .. csv-table:: :header: "type", "default", "mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "25", "no", "``$ActionQueueDequeueTimeEnd``" With this parameter you can specify rsyslog to process queues during specific time periods. To define a time frame use the 24-hour format without minutes. This parameter specifies the end and "queue.dequeuetimebegin" the begin of the time frame. The default 25 disables the time-window. Examples ======== Example 1 --------- The following is a sample of a TCP forwarding action with its own queue. .. code-block:: none action(type="omfwd" target="192.168.2.11" port="10514" protocol="tcp" queue.filename="forwarding" queue.size="1000000" queue.type="LinkedList" )