ipv6_lookup plugin

Article
09/11/2024

Applies to: ✅ Azure Data Explorer

The ipv6_lookup plugin looks up an IPv6 value in a lookup table and returns rows with matched values. The plugin is invoked with the evaluate operator.

Syntax

T | evaluate ipv6_lookup( LookupTable , SourceIPv6Key , IPv6LookupKey [, return_unmatched ] )

Parameters

Name	Type	Required	Description
T	`string`	✔️	The tabular input whose column SourceIPv6Key is used for IPv6 matching.
LookupTable	`string`	✔️	Table or tabular expression with IPv6 lookup data, whose column LookupKey is used for IPv6 matching. IPv6 values can be masked using IP-prefix notation.
SourceIPv6Key	`string`	✔️	The column of T with IPv6 string to be looked up in LookupTable. IPv6 values can be masked using IP-prefix notation.
IPv6LookupKey	`string`	✔️	The column of LookupTable with IPv6 string that is matched against each SourceIPv6Key value.
return_unmatched	`bool`		A boolean flag that defines if the result should include all or only matching rows (default: `false` - only matching rows returned).

IP-prefix notation

IP-prefix notation (also known as CIDR notation) is a concise way of representing an IP address and its associated network mask. The format is <base IP>/<prefix length>, where the prefix length is the number of leading 1 bits in the netmask. The prefix length determines the range of IP addresses that belong to the network.

For IPv4, the prefix length is a number between 0 and 32. So the notation 192.168.2.0/24 represents the IP address 192.168.2.0 with a netmask of 255.255.255.0. This netmask has 24 leading 1 bits, or a prefix length of 24.

For IPv6, the prefix length is a number between 0 and 128. So the notation fe80::85d:e82c:9446:7994/120 represents the IP address fe80::85d:e82c:9446:7994 with a netmask of ffff:ffff:ffff:ffff:ffff:ffff:ffff:ff00. This netmask has 120 leading 1 bits, or a prefix length of 120.

Returns

The ipv6_lookup plugin returns a result of join (lookup) based on IPv6 key. The schema of the table is the union of the source table and the lookup table, similar to the result of the lookup operator.

If the return_unmatched argument is set to true, the resulting table includes both matched and unmatched rows (filled with nulls).

If the return_unmatched argument is set to false, or omitted (the default value of false is used), the resulting table has as many records as matching results. This variant of lookup has better performance compared to return_unmatched=true execution.

Note

This plugin covers the scenario of IPv6-based join, assuming a small lookup table size (100K-300K rows), with the input table optionally having a larger size.
The performance of the plugin will depend on the sizes of the lookup and data source tables, the number of columns, and number of matching records.

Examples

IPv6 lookup - matching rows only

// IP lookup table: IP_Data (the data is generated by ChatGPT).
let IP_Data = datatable(network:string, continent_code:string ,continent_name:string, country_iso_code:string, country_name:string)
[
  "2001:0db8:85a3::/48","NA","North America","US","United States",
  "2404:6800:4001::/48","AS","Asia","JP","Japan",
  "2a00:1450:4001::/48","EU","Europe","DE","Germany",
  "2800:3f0:4001::/48","SA","South America","BR","Brazil",
  "2c0f:fb50:4001::/48","AF","Africa","ZA","South Africa",
  "2607:f8b0:4001::/48","NA","North America","CA","Canada",
  "2a02:26f0:4001::/48","EU","Europe","FR","France",
  "2400:cb00:4001::/48","AS","Asia","IN","India",
  "2801:0db8:85a3::/48","SA","South America","AR","Argentina",
  "2a03:2880:4001::/48","EU","Europe","GB","United Kingdom"
];
let IPs = datatable(ip:string)
[
  "2001:0db8:85a3:0000:0000:8a2e:0370:7334", // United States
  "2404:6800:4001:0001:0000:8a2e:0370:7334", // Japan
  "2a02:26f0:4001:0006:0000:8a2e:0370:7334", // France
  "a5e:f127:8a9d:146d:e102:b5d3:c755:abcd",  // N/A
  "a5e:f127:8a9d:146d:e102:b5d3:c755:abce"   // N/A
];
IPs
| evaluate ipv6_lookup(IP_Data, ip, network)

Output

network	continent_code	continent_name	country_iso_code	country_name	ip
2001:0db8:85a3::/48	NA	North America	US	United States	2001:0db8:85a3:0000:0000:8a2e:0370:7334
2404:6800:4001::/48	AS	Asia	JP	Japan	2404:6800:4001:0001:0000:8a2e:0370:7334
2a02:26f0:4001::/48	EU	Europe	FR	France	2a02:26f0:4001:0006:0000:8a2e:0370:7334

IPv6 lookup - return both matching and nonmatching rows

// IP lookup table: IP_Data (the data is generated by ChatGPT).
let IP_Data = datatable(network:string, continent_code:string ,continent_name:string, country_iso_code:string, country_name:string)
[
  "2001:0db8:85a3::/48","NA","North America","US","United States",
  "2404:6800:4001::/48","AS","Asia","JP","Japan",
  "2a00:1450:4001::/48","EU","Europe","DE","Germany",
  "2800:3f0:4001::/48","SA","South America","BR","Brazil",
  "2c0f:fb50:4001::/48","AF","Africa","ZA","South Africa",
  "2607:f8b0:4001::/48","NA","North America","CA","Canada",
  "2a02:26f0:4001::/48","EU","Europe","FR","France",
  "2400:cb00:4001::/48","AS","Asia","IN","India",
  "2801:0db8:85a3::/48","SA","South America","AR","Argentina",
  "2a03:2880:4001::/48","EU","Europe","GB","United Kingdom"
];
let IPs = datatable(ip:string)
[
  "2001:0db8:85a3:0000:0000:8a2e:0370:7334", // United States
  "2404:6800:4001:0001:0000:8a2e:0370:7334", // Japan
  "2a02:26f0:4001:0006:0000:8a2e:0370:7334", // France
  "a5e:f127:8a9d:146d:e102:b5d3:c755:abcd",  // N/A
  "a5e:f127:8a9d:146d:e102:b5d3:c755:abce"   // N/A
];
IPs
| evaluate ipv6_lookup(IP_Data, ip, network, true)