ClusterInput
fluentd.fluent.io / v1alpha1
apiVersion: fluentd.fluent.io/v1alpha1
kind: ClusterInput
metadata:
name: example
apiVersion
string
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
kind
string
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
metadata
object
spec object
ClusterInputSpec defines the desired state of ClusterInput
inputs []object
customPlugin object
Custom plugin type
config
string required
forward object
in_forward plugin
addTagPrefix
string
Adds the prefix to the incoming event's tag.
bind
string
The port to listen to, default is "0.0.0.0"
chunkSizeLimit
string
The size limit of the received chunk. If the chunk size is larger than this value, the received chunk is dropped.
pattern:
^\d+(KB|MB|GB|TB)$
chunkSizeWarnLimit
string
The warning size limit of the received chunk. If the chunk size is larger than this value, a warning message will be sent.
pattern:
^\d+(KB|MB|GB|TB)$client object
The security section of client plugin
host
string
The IP address or hostname of the client. This is exclusive with Network.
network
string
The network address specification. This is exclusive with Host.
sharedKey
string
The shared key per client.
users
string
The array of usernames.
denyKeepalive
boolean
The connections will be disconnected right after receiving a message, if true.
lingerTimeout
integer
The timeout used to set the linger option.
port
integer
The port to listen to, default is 24224.
format:
int32minimum:
1maximum:
65535
resolveHostname
boolean
Tries to resolve hostname from IP addresses or not.
security object
The security section of forward plugin
allowAnonymousSource
string
Allows the anonymous source. <client> sections are required, if disabled.
selfHostname
string
The hostname.
sharedKey
string
The shared key for authentication.
user object
Defines user section directly.
password object
Secret defines the key of a value.
valueFrom object
ValueSource defines how to find a value's key.
secretKeyRef object
Selects a key of a secret in the pod's namespace
key
string required
The key of the secret to select from. Must be a valid secret key.
name
string
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
optional
boolean
Specify whether the Secret or its key must be defined
username object
Secret defines the key of a value.
valueFrom object
ValueSource defines how to find a value's key.
secretKeyRef object
Selects a key of a secret in the pod's namespace
key
string required
The key of the secret to select from. Must be a valid secret key.
name
string
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
optional
boolean
Specify whether the Secret or its key must be defined
userAuth
string
If true, user-based authentication is used.
sendKeepalivePacket
boolean
Enables the TCP keepalive for sockets.
skipInvalidEvent
boolean
Skips the invalid incoming event.
sourceAddressKey
string
The field name of the client's source address. If set, the client's address will be set to its key.
sourceHostnameKey
string
The field name of the client's hostname. If set, the client's hostname will be set to its key.
tag
string
in_forward uses incoming event's tag by default (See Protocol Section).
If the tag parameter is set, its value is used instead.
transport object
The transport section of forward plugin
caCertPath
string
for Cert generated
caPath
string
for Cert signed by public CA
caPrivateKeyPassphrase
string
caPrivateKeyPath
string
certPath
string
certVerifier
string
other parameters
ciphers
string
clientCertAuth
boolean
insecure
boolean
privateKeyPassphrase
string
privateKeyPath
string
protocol
string
The protocol name of this plugin, i.e: tls
version
string
user object
The security section of user plugin
password object
Secret defines the key of a value.
valueFrom object
ValueSource defines how to find a value's key.
secretKeyRef object
Selects a key of a secret in the pod's namespace
key
string required
The key of the secret to select from. Must be a valid secret key.
name
string
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
optional
boolean
Specify whether the Secret or its key must be defined
username object
Secret defines the key of a value.
valueFrom object
ValueSource defines how to find a value's key.
secretKeyRef object
Selects a key of a secret in the pod's namespace
key
string required
The key of the secret to select from. Must be a valid secret key.
name
string
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
optional
boolean
Specify whether the Secret or its key must be defined
http object
in_http plugin
addHttpHeaders
boolean
Adds HTTP_ prefix headers to the record.
addRemoteAddr
string
Adds REMOTE_ADDR field to the record. The value of REMOTE_ADDR is the client's address.
i.e: X-Forwarded-For: host1, host2
bind
string
The port to listen to, default is "0.0.0.0"
bodySizeLimit
string
The size limit of the POSTed element.
pattern:
^\d+(KB|MB|GB|TB)$
corsAllOrigins
string
Whitelist domains for CORS.
corsAllowCredentials
string
Add Access-Control-Allow-Credentials header. It's needed when a request's credentials mode is include
keepaliveTimeout
string
The timeout limit for keeping the connection alive.
pattern:
^\d+(\.[0-9]{0,2})?(s|m|h|d)?$parse object
The parse section of http plugin
customPatternPath
string
Path to the file that includes custom grok patterns.
estimateCurrentEvent
boolean
If true, use Fluent::Eventnow(current time) as a timestamp when time_key is specified.
expression
string
Specifies the regular expression for matching logs. Regular expression also supports i and m suffix.
grok []object
Grok Sections
keepTimeKey
boolean
If true, keep time field in the record.
name
string
The name of this grok section.
pattern
string
The pattern of grok. Required parameter.
timeFormat
string
Process value using specified format. This is available only when time_type is string
timeKey
string
Specify time field for event time. If the event doesn't have this field, current time is used.
timeZone
string
Use specified timezone. one can parse/format the time value in the specified timezone.
grokFailureKey
string
The key has grok failure reason.
grokPattern
string
The pattern of grok.
grokPatternSeries
string
Specify grok pattern series set.
id
string
The @id parameter specifies a unique name for the configuration.
keepTimeKey
boolean
If true, keep time field in th record.
localtime
boolean
If true, uses local time.
logLevel
string
The @log_level parameter specifies the plugin-specific logging level
multiLineStartRegexp
string
The regexp to match beginning of multiline. This is only for "multiline_grok".
timeFormat
string
Process value according to the specified format. This is available only when time_type is string
timeFormatFallbacks
string
Uses the specified time format as a fallback in the specified order. You can parse undetermined time format by using time_format_fallbacks. This options is enabled when time_type is mixed.
timeKey
string
Specify time field for event time. If the event doesn't have this field, current time is used.
timeType
string
parses/formats value according to this type, default is string
enum:
float, unixtime, string, mixed
timeout
string
Specify timeout for parse processing.
pattern:
^\d+(\.[0-9]{0,2})?(s|m|h|d)?$
timezone
string
Uses the specified timezone.
type
string required
The @type parameter specifies the type of the plugin.
enum:
regexp, apache2, apache_error, nginx, syslog, csv, tsv, ltsv, json, multiline, none, grok, multiline_grok
types
string
Specify types for converting field into another, i.e: types user_id:integer,paid:bool,paid_usd_amount:float
utc
boolean
If true, uses UTC.
port
integer
The port to listen to, default is 9880.
format:
int32minimum:
1maximum:
65535
respondsWithEmptyImg
boolean
Responds with an empty GIF image of 1x1 pixel (rather than an empty string).
transport object
The transport section of http plugin
caCertPath
string
for Cert generated
caPath
string
for Cert signed by public CA
caPrivateKeyPassphrase
string
caPrivateKeyPath
string
certPath
string
certVerifier
string
other parameters
ciphers
string
clientCertAuth
boolean
insecure
boolean
privateKeyPassphrase
string
privateKeyPath
string
protocol
string
The protocol name of this plugin, i.e: tls
version
string
id
string
The @id parameter specifies a unique name for the configuration.
label
string
The @label parameter is to route the input events to <label> sections.
logLevel
string
The @log_level parameter specifies the plugin-specific logging level
monitorAgent object
monitor_agent plugin
bind
string
The bind address to listen to.
emitInterval
integer
The interval time between event emits. This will be used when "tag" is configured.
format:
int64
includeConfig
boolean
You can set this option to false to remove the config field from the response.
includeRetry
boolean
You can set this option to false to remove the retry field from the response.
port
integer
The port to listen to.
format:
int64
tag
string
If you set this parameter, this plugin emits metrics as records.
sample object
in_sample plugin
autoIncrementKey
string
If specified, each generated event has an auto-incremented key field.
rate
integer
It configures how many events to generate per second.
format:
int64
sample
string
The sample data to be generated. It should be either an array of JSON hashes or a single JSON hash. If it is an array of JSON hashes, the hashes in the array are cycled through in order.
size
integer
The number of events in the event stream of each emit.
format:
int64
tag
string
The tag of the event. The value is the tag assigned to the generated events.
tail object
in_tail plugin
emitUnmatchedLines
boolean
Emits unmatched lines when <parse> format is not matched for incoming logs.
enableStatWatcher
boolean
Enables the additional inotify-based watcher. Setting this parameter to false will disable the inotify events and use only timer watcher for file tailing.
This option is mainly for avoiding the stuck issue with inotify.
enableWatchTimer
boolean
Enables the additional watch timer. Setting this parameter to false will significantly reduce CPU and I/O consumption when tailing a large number of files on systems with inotify support.
The default is true which results in an additional 1 second timer being used.
encoding
string
Specifies the encoding of reading lines. By default, in_tail emits string value as ASCII-8BIT encoding.
If encoding is specified, in_tail changes string to encoding.
If encoding and fromEncoding both are specified, in_tail tries to encode string from fromEncoding to encoding.
excludePath
[]string
The paths excluded from the watcher list.
followInodes
boolean
Avoid to read rotated files duplicately. You should set true when you use * or strftime format in path.
fromEncoding
string
Specifies the encoding of reading lines. By default, in_tail emits string value as ASCII-8BIT encoding.
If encoding is specified, in_tail changes string to encoding.
If encoding and fromEncoding both are specified, in_tail tries to encode string from fromEncoding to encoding.
group object
The in_tail plugin can assign each log file to a group, based on user defined rules.
The limit parameter controls the total number of lines collected for a group within a rate_period time interval.
pattern
string
Specifies the regular expression for extracting metadata (namespace, podname) from log file path.
Default value of the pattern regexp extracts information about namespace, podname, docker_id, container of the log (K8s specific).
ratePeriod
integer
Time period in which the group line limit is applied. in_tail resets the counter after every rate_period interval.
format:
int32rule object required
Grouping rules for log files.
limit
integer
Maximum number of lines allowed from a group in rate_period time interval. The default value of -1 doesn't throttle log files of that group.
format:
int32
match
object
match parameter is used to check if a file belongs to a particular group based on hash keys (named captures from pattern) and hash values (regexp in string)
ignoreRepeatedPermissionError
boolean
If you have to exclude the non-permission files from the watch list, set this parameter to true. It suppresses the repeated permission error logs.
limitRecentlyModified
integer
Limits the watching files that the modification time is within the specified time range when using * in path.
format:
int32
maxLineSize
integer
The maximum length of a line. Longer lines than it will be just skipped.
format:
int32
multilineFlushInterval
integer
The interval of flushing the buffer for multiline format.
format:
int32
openOnEveryUpdate
boolean
Opens and closes the file on every update instead of leaving it open until it gets rotated.
parse object required
Parse defines various parameters for the parse plugin
customPatternPath
string
Path to the file that includes custom grok patterns.
estimateCurrentEvent
boolean
If true, use Fluent::Eventnow(current time) as a timestamp when time_key is specified.
expression
string
Specifies the regular expression for matching logs. Regular expression also supports i and m suffix.
grok []object
Grok Sections
keepTimeKey
boolean
If true, keep time field in the record.
name
string
The name of this grok section.
pattern
string
The pattern of grok. Required parameter.
timeFormat
string
Process value using specified format. This is available only when time_type is string
timeKey
string
Specify time field for event time. If the event doesn't have this field, current time is used.
timeZone
string
Use specified timezone. one can parse/format the time value in the specified timezone.
grokFailureKey
string
The key has grok failure reason.
grokPattern
string
The pattern of grok.
grokPatternSeries
string
Specify grok pattern series set.
id
string
The @id parameter specifies a unique name for the configuration.
keepTimeKey
boolean
If true, keep time field in th record.
localtime
boolean
If true, uses local time.
logLevel
string
The @log_level parameter specifies the plugin-specific logging level
multiLineStartRegexp
string
The regexp to match beginning of multiline. This is only for "multiline_grok".
timeFormat
string
Process value according to the specified format. This is available only when time_type is string
timeFormatFallbacks
string
Uses the specified time format as a fallback in the specified order. You can parse undetermined time format by using time_format_fallbacks. This options is enabled when time_type is mixed.
timeKey
string
Specify time field for event time. If the event doesn't have this field, current time is used.
timeType
string
parses/formats value according to this type, default is string
enum:
float, unixtime, string, mixed
timeout
string
Specify timeout for parse processing.
pattern:
^\d+(\.[0-9]{0,2})?(s|m|h|d)?$
timezone
string
Uses the specified timezone.
type
string required
The @type parameter specifies the type of the plugin.
enum:
regexp, apache2, apache_error, nginx, syslog, csv, tsv, ltsv, json, multiline, none, grok, multiline_grok
types
string
Specify types for converting field into another, i.e: types user_id:integer,paid:bool,paid_usd_amount:float
utc
boolean
If true, uses UTC.
path
string required
The path(s) to read. Multiple paths can be specified, separated by comma ','.
pathKey
string
Adds the watching file path to the path_key field.
pathTimezone
string
This parameter is for strftime formatted path like /path/to/%Y/%m/%d/.
posFile
string
(recommended) Fluentd will record the position it last read from this file.
pos_file handles multiple positions in one file so no need to have multiple pos_file parameters per source.
Don't share pos_file between in_tail configurations. It causes unexpected behavior e.g. corrupt pos_file content.
posFileCompactionInterval
integer
The interval of doing compaction of pos file.
format:
int32
readBytesLimitPerSecond
integer
The number of reading bytes per second to read with I/O operation. This value should be equal or greater than 8192.
format:
int32
readFromHead
boolean
Starts to read the logs from the head of the file or the last read position recorded in pos_file, not tail.
readLinesLimit
integer
The number of lines to read with each I/O operation.
format:
int32
refreshInterval
integer
The interval to refresh the list of watch files. This is used when the path includes *.
format:
int32
rotateWait
integer
in_tail actually does a bit more than tail -F itself. When rotating a file, some data may still need to be written to the old file as opposed to the new one.
in_tail takes care of this by keeping a reference to the old file (even after it has been rotated) for some time before transitioning completely to the new file.
This helps prevent data designated for the old file from getting lost. By default, this time interval is 5 seconds.
The rotate_wait parameter accepts a single integer representing the number of seconds you want this time interval to be.
format:
int32
skipRefreshOnStartup
boolean
Skips the refresh of the watch list on startup. This reduces the startup time when * is used in path.
tag
string required
The tag of the event.
status
object
ClusterInputStatus defines the observed state of ClusterInput
No matches. Try .spec.inputs for an exact path