org.apache.kafka.connect.transforms.Cast$Key
) or value (org.apache.kafka.connect.transforms.Cast$Value
).
List of fields and the type to cast them to of the form field1:type,field2:type to cast fields of Maps or Structs. A single type to cast the entire value. Valid types are int8, int16, int32, int64, float32, float64, boolean, and string. Note that binary fields can only be cast to string.
Type: | list |
---|---|
Default: | |
Valid Values: | list of colon-delimited pairs, e.g. foo:bar,abc:xyz |
Importance: | high |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |
The name of the headers to be removed.
Type: | list |
---|---|
Default: | |
Valid Values: | non-empty list |
Importance: | high |
org.apache.kafka.connect.transforms.ExtractField$Key
) or value (org.apache.kafka.connect.transforms.ExtractField$Value
).
Field name to extract.
Type: | string |
---|---|
Default: | |
Valid Values: | |
Importance: | medium |
Defines the version of the syntax to access fields. If set to `V1`, then the field paths are limited to access the elements at the root level of the struct or map. If set to `V2`, the syntax will support accessing nested elements. To access nested elements, dotted notation is used. If dots are already included in the field name, then backtick pairs can be used to wrap field names containing dots. E.g. to access the subfield `baz` from a field named "foo.bar" in a struct/map the following format can be used to access its elements: "`foo.bar`.baz".
Type: | string |
---|---|
Default: | V1 |
Valid Values: | (case insensitive) [V1, V2] |
Importance: | high |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |
org.apache.kafka.connect.transforms.Flatten$Key
) or value (org.apache.kafka.connect.transforms.Flatten$Value
).
Delimiter to insert between field names from the input record when generating field names for the output record
Type: | string |
---|---|
Default: | . |
Valid Values: | |
Importance: | medium |
fields
and headers
together identify a field and the header it should be moved or copied to. Use the concrete transformation type designed for the record key (org.apache.kafka.connect.transforms.HeaderFrom$Key
) or value (org.apache.kafka.connect.transforms.HeaderFrom$Value
).
Field names in the record whose values are to be copied or moved to headers.
Type: | list |
---|---|
Default: | |
Valid Values: | non-empty list |
Importance: | high |
Header names, in the same order as the field names listed in the fields configuration property.
Type: | list |
---|---|
Default: | |
Valid Values: | non-empty list |
Importance: | high |
Either move
if the fields are to be moved to the headers (removed from the key/value), or copy
if the fields are to be copied to the headers (retained in the key/value).
Type: | string |
---|---|
Default: | |
Valid Values: | [move, copy] |
Importance: | high |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |
org.apache.kafka.connect.transforms.HoistField$Key
) or value (org.apache.kafka.connect.transforms.HoistField$Value
).
Field name for the single field that will be created in the resulting Struct or Map.
Type: | string |
---|---|
Default: | |
Valid Values: | |
Importance: | medium |
org.apache.kafka.connect.transforms.InsertField$Key
) or value (org.apache.kafka.connect.transforms.InsertField$Value
).
Field name for Kafka offset - only applicable to sink connectors.
Suffix with !
to make this a required field, or ?
to keep it optional (the default).
Type: | string |
---|---|
Default: | null |
Valid Values: | |
Importance: | medium |
Field name for Kafka partition. Suffix with !
to make this a required field, or ?
to keep it optional (the default).
Type: | string |
---|---|
Default: | null |
Valid Values: | |
Importance: | medium |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |
Field name for static data field. Suffix with !
to make this a required field, or ?
to keep it optional (the default).
Type: | string |
---|---|
Default: | null |
Valid Values: | |
Importance: | medium |
Static field value, if field name configured.
Type: | string |
---|---|
Default: | null |
Valid Values: | |
Importance: | medium |
Field name for record timestamp. Suffix with !
to make this a required field, or ?
to keep it optional (the default).
Type: | string |
---|---|
Default: | null |
Valid Values: | |
Importance: | medium |
Field name for Kafka topic. Suffix with !
to make this a required field, or ?
to keep it optional (the default).
Type: | string |
---|---|
Default: | null |
Valid Values: | |
Importance: | medium |
The name of the header.
Type: | string |
---|---|
Default: | |
Valid Values: | non-null string |
Importance: | high |
The literal value that is to be set as the header value on all records.
Type: | string |
---|---|
Default: | |
Valid Values: | non-null string |
Importance: | high |
org.apache.kafka.connect.transforms.MaskField$Key
) or value (org.apache.kafka.connect.transforms.MaskField$Value
).
Names of fields to mask.
Type: | list |
---|---|
Default: | |
Valid Values: | non-empty list |
Importance: | high |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |
Custom value replacement, that will be applied to all 'fields' values (numeric or non-empty string values only).
Type: | string |
---|---|
Default: | null |
Valid Values: | non-empty string |
Importance: | low |
java.util.regex.Pattern
. If the pattern matches the input topic, java.util.regex.Matcher#replaceFirst()
is used with the replacement string to obtain the new topic.
Regular expression to use for matching.
Type: | string |
---|---|
Default: | |
Valid Values: | valid regex |
Importance: | high |
Replacement string.
Type: | string |
---|---|
Default: | |
Valid Values: | |
Importance: | high |
org.apache.kafka.connect.transforms.ReplaceField$Key
) or value (org.apache.kafka.connect.transforms.ReplaceField$Value
).
Fields to exclude. This takes precedence over the fields to include.
Type: | list |
---|---|
Default: | "" |
Valid Values: | |
Importance: | medium |
Fields to include. If specified, only these fields will be used.
Type: | list |
---|---|
Default: | "" |
Valid Values: | |
Importance: | medium |
Field rename mappings.
Type: | list |
---|---|
Default: | "" |
Valid Values: | list of colon-delimited pairs, e.g. foo:bar,abc:xyz |
Importance: | medium |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |
org.apache.kafka.connect.transforms.SetSchemaMetadata$Key
) or value (org.apache.kafka.connect.transforms.SetSchemaMetadata$Value
) schema.
Schema name to set.
Type: | string |
---|---|
Default: | null |
Valid Values: | |
Importance: | high |
Schema version to set.
Type: | int |
---|---|
Default: | null |
Valid Values: | |
Importance: | high |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |
org.apache.kafka.connect.transforms.TimestampConverter$Key
) or value (org.apache.kafka.connect.transforms.TimestampConverter$Value
).
The desired timestamp representation: string, unix, Date, Time, or Timestamp
Type: | string |
---|---|
Default: | |
Valid Values: | [string, unix, Date, Time, Timestamp] |
Importance: | high |
The field containing the timestamp, or empty if the entire value is a timestamp
Type: | string |
---|---|
Default: | "" |
Valid Values: | |
Importance: | high |
A SimpleDateFormat-compatible format for the timestamp. Used to generate the output when type=string or used to parse the input if the input is a string.
Type: | string |
---|---|
Default: | "" |
Valid Values: | |
Importance: | medium |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |
The desired Unix precision for the timestamp: seconds, milliseconds, microseconds, or nanoseconds. Used to generate the output when type=unix or used to parse the input if the input is a Long.Note: This SMT will cause precision loss during conversions from, and to, values with sub-millisecond components.
Type: | string |
---|---|
Default: | milliseconds |
Valid Values: | [nanoseconds, microseconds, milliseconds, seconds] |
Importance: | low |
Format string for the timestamp that is compatible with java.text.SimpleDateFormat
.
Type: | string |
---|---|
Default: | yyyyMMdd |
Valid Values: | |
Importance: | high |
Format string which can contain ${topic}
and ${timestamp}
as placeholders for the topic and timestamp, respectively.
Type: | string |
---|---|
Default: | ${topic}-${timestamp} |
Valid Values: | |
Importance: | high |
Field names on the record value to extract as the record key.
Type: | list |
---|---|
Default: | |
Valid Values: | non-empty list |
Importance: | high |
Whether to replace fields that have a default value and that are null to the default value. When set to true, the default value is used, otherwise null is used.
Type: | boolean |
---|---|
Default: | true |
Valid Values: | |
Importance: | medium |