omhttpfs: Hadoop HTTPFS Output Module

Module Name: omhttpfs
Available Since: 8.10.0
Author: sskaje <sskaje@gmail.com>

Purpose

This module is an alternative to omhdfs via Hadoop HDFS over HTTP.

Dependencies

  • libcurl

Configuration Parameters

Note

Parameter names are case-insensitive.

Action Parameters

Host

type default mandatory obsolete legacy directive
word 127.0.0.1 no none

HttpFS server host.

Port

type default mandatory obsolete legacy directive
integer 14000 no none

HttpFS server port.

User

type default mandatory obsolete legacy directive
word hdfs no none

HttpFS auth user.

https

type default mandatory obsolete legacy directive
binary off no none

Turn on if your HttpFS runs on HTTPS.

File

type default mandatory obsolete legacy directive
word none yes none

File to write, or a template name.

isDynFile

type default mandatory obsolete legacy directive
binary off no none

Turn this on if your file is a template name. See examples below.

Template

type default mandatory obsolete legacy directive
word RSYSLOG_FileFormat no none

Format your message when writing to file. Default: RSYSLOG_FileFormat

Configure

./configure --enable-omhttpfs

Examples

Example 1

module(load="omhttpfs")
template(name="hdfs_tmp_file" type="string" string="/tmp/%$YEAR%/test.log")
template(name="hdfs_tmp_filecontent" type="string" string="%$YEAR%-%$MONTH%-%$DAY% %MSG% ==\n")
local4.*    action(type="omhttpfs" host="10.1.1.161" port="14000" https="off" file="hdfs_tmp_file" isDynFile="on")
local5.*    action(type="omhttpfs" host="10.1.1.161" port="14000" https="off" file="hdfs_tmp_file" isDynFile="on" template="hdfs_tmp_filecontent")

See also

Help with configuring/using Rsyslog:

  • Mailing list - best route for general questions
  • GitHub: rsyslog source project - detailed questions, reporting issues that are believed to be bugs with Rsyslog
  • Stack Exchange (View, Ask) - experimental support from rsyslog community

See also

Contributing to Rsyslog: