How can I exclude ipv6 from filebeat field host.ip?

[jmjoy]

I use source type logstach to collect filebeat inputs, the log is json format and has the field host.ip, such as { "host": { "ip": [ "10.4.139.11", "fe80::5054:ff:fe4f:893e" ] } }.

Now I want to exclude the ipv6 address such as fe80::5054:ff:fe4f:893e, how can I reach this? (I found remap don't have the for operation?) Or maybe I have to use lua?

[tobz]

👋🏻

I think you could accomplish this with VRL. VRL actually does now (as of v0.22.0) have iteration support via for_each. Here's my initial thought on how you might achieve this in a remap transform:

# Will hold all of the filtered IPs.
filtered_ips = []

# Iterate over each IP in `.host.ip`. We forcefully make sure that
# `.host.ip` is actually an array, otherwise the VRL program errors.
for_each(array!(.host.ip)) -> |_index, ip| {
  # We're not yet sure if the elements in `.host.ip` are strings,
  # so try and coerce them, since `ipv6_to_ipv4` needs a string.
  ip_str, err = string(ip)
  if err == null {
    # The IP value is a string, so now we run it through `ipv6_to_ipv4`
    # which will try and convert it to IPv4. If it's already IPV4, it's
    # returned unmodified, and if it's an IPv6 address, it may return a
    # IPv4 version if the IPv6 was a IPv4 compatibility address, or an
    # error otherwise.
    #
    # This just means that if we get an error, or if `as_ipv4` is not
    # equal to `ip_str`, that this IP was not already a valid IPv4
    # address, and that we should skip it.
    as_ipv4, err = ipv6_to_ipv4(ip_str)
    if err == null && as_ipv4 == ip_str {
      filtered_ips = push(filtered_ips, as_ipv4)
    }
  }
}

# Replace the original list of IPs with our filtered list.
.host.ip = filtered_ips

[jmjoy]

Thank you very much.

[tobz]

BTW, I just merged in two new VRL functions -- is_ipv4 and is_ipv6 -- to hopefully make this cleaner/easier to do in the future.

[jmjoy]

Good job.