dcount_intersect plugin

  • Article

Applies to: ✅ Azure Data Explorer

Calculates intersection between N sets based on hll values (N in range of [2..16]), and returns N dcount values. The plugin is invoked with the evaluate operator.

Syntax

T | evaluate dcount_intersect(hll_1, hll_2, [, hll_3, ...])

Learn more about syntax conventions.

Parameters

Name Type Required Description
T string ✔️ The input tabular expression.
hll_i The values of set Si calculated with the hll() function.

Returns

Returns a table with N dcount values (per column, representing set intersections). Column names are s0, s1, ... (until n-1).

Given sets S1, S2, .. Sn return values will be representing distinct counts of:
S1,
S1 ∩ S2,
S1 ∩ S2 ∩ S3,
... ,
S1 ∩ S2 ∩ ... ∩ Sn

Examples

// Generate numbers from 1 to 100
range x from 1 to 100 step 1
| extend isEven = (x % 2 == 0), isMod3 = (x % 3 == 0), isMod5 = (x % 5 == 0)
// Calculate conditional HLL values (note that '0' is included in each of them as additional value, so we will subtract it later)
| summarize hll_even = hll(iif(isEven, x, 0), 2),
            hll_mod3 = hll(iif(isMod3, x, 0), 2),
            hll_mod5 = hll(iif(isMod5, x, 0), 2) 
// Invoke the plugin that calculates dcount intersections         
| evaluate dcount_intersect(hll_even, hll_mod3, hll_mod5)
| project evenNumbers = s0 - 1,             //                             100 / 2 = 50
          even_and_mod3 = s1 - 1,           // gcd(2,3) = 6, therefor:     100 / 6 = 16
          even_and_mod3_and_mod5 = s2 - 1   // gcd(2,3,5) is 30, therefore: 100 / 30 = 3

Output

evenNumbers even_and_mod3 even_and_mod3_and_mod5
50 16 3