Optimize array_intersect() using hash-based matching

[mehmetcansahin]

Replaces array_intersect()'s sort-based matching with a hash-based implementation for calls with at least two arrays. Integer and string values use normalized hash keys directly; other values are converted according to the existing string-comparison semantics. Single-array calls retain the generic path.

This removes the type-based fallback and makes matching expected-linear. Empty operands short-circuit after argument validation while preserving the first array's key and bucket metadata.

String conversions now occur while scanning instead of during sort comparisons. This can change warning and __toString() invocation counts and order, which conversion exception is reached, and results for stateful __toString() implementations. This is documented in UPGRADING.

Compared with the previous PR head on an Apple M1, existing integer/string cases are neutral or up to 22% faster, the former fallback cases are about 109–111x faster, and float-only arrays are about 24x faster.

Verification includes the configured full test suite with 0 failures, all relevant array_intersect* tests, the resource-heavy hard-timeout test, and 4,000 differential cases matching the previous implementation for stable conversions and array metadata.

[LamentXU123]

This looks sensible. But could you please provide the "Local benchmarks" you've run for us to verify. These days it's hard to tell a performance improvement without benchmarks.

[mehmetcansahin]

@LamentXU123 Thanks. I reran the benchmarks on an Apple M1, comparing base 3407a6d2a04 with PR head 2e270dcd9be. Each result is the median of 11 runs.

Input	Base	PR	Change
2 x 10 integers	1.113 us	0.135 us	8.24x faster
2 x 100,000 integers	104,655.083 us	1,096.548 us	95.44x faster
2 x 10,000 strings	1,725.658 us	191.142 us	9.03x faster
3 x 10,000 integers	11,936.917 us	108.249 us	110.27x faster
10,000 values, early fallback	7,937.938 us	8,105.664 us	2.11% slower
10,000 values, late fallback	8,033.523 us	8,181.500 us	1.84% slower

Benchmark script:

benchmark.php

<?php

declare(strict_types=1);

const TARGET_SAMPLE_NS = 100_000_000;
const SAMPLE_COUNT = 11;

function makeStrings(int $start, int $size): array
{
    $values = [];
    for ($i = $start, $end = $start + $size; $i < $end; $i++) {
        $values[] = "value_$i";
    }
    return $values;
}

function makeMixed(int $start, int $size): array
{
    $values = [];
    for ($i = $start, $end = $start + $size; $i < $end; $i++) {
        $values[] = ($i & 1) === 0 ? $i : (string) $i;
    }
    return $values;
}

function scenarios(): array
{
    $fallbackFirst = range(0, 9_999);
    array_unshift($fallbackFirst, 0.5);

    $fallbackLast = range(0, 9_999);
    $fallbackLast[] = 0.5;

    return [
        'int-10' => [range(0, 9), range(5, 14)],
        'int-1000' => [range(0, 999), range(500, 1_499)],
        'int-100000' => [range(0, 99_999), range(50_000, 149_999)],
        'string-10000' => [makeStrings(0, 10_000), makeStrings(5_000, 10_000)],
        'mixed-int-string-10000' => [makeMixed(0, 10_000), makeMixed(5_000, 10_000)],
        'int-10000-3-arrays' => [
            range(0, 9_999),
            range(2_500, 12_499),
            range(5_000, 14_999),
        ],
        'fallback-float-first-10000' => [$fallbackFirst, range(5_000, 14_999)],
        'fallback-float-last-10000' => [$fallbackLast, range(5_000, 14_999)],
    ];
}

function measure(array $arrays, int $iterations): array
{
    $checksum = 0;
    $start = hrtime(true);
    for ($i = 0; $i < $iterations; $i++) {
        $checksum += count(array_intersect(...$arrays));
    }
    return [hrtime(true) - $start, $checksum];
}

$allScenarios = scenarios();
$selected = $argv[1] ?? null;
if ($selected === null || !isset($allScenarios[$selected])) {
    fwrite(STDERR, "Usage: php benchmark.php <scenario>\n\nScenarios:\n");
    foreach (array_keys($allScenarios) as $name) {
        fwrite(STDERR, "  $name\n");
    }
    exit(1);
}

$arrays = $allScenarios[$selected];
$iterations = 1;
do {
    [$elapsed] = measure($arrays, $iterations);
    if ($elapsed >= TARGET_SAMPLE_NS || $iterations >= 1_048_576) {
        break;
    }
    $iterations *= 2;
} while (true);

measure($arrays, $iterations);

$samples = [];
$checksum = 0;
for ($sample = 0; $sample < SAMPLE_COUNT; $sample++) {
    [$elapsed, $sampleChecksum] = measure($arrays, $iterations);
    $samples[] = $elapsed / $iterations;
    $checksum ^= $sampleChecksum;
}

sort($samples);
$median = $samples[intdiv(count($samples), 2)];

printf(
    "%s iterations=%d samples=%d median_us=%.3f min_us=%.3f max_us=%.3f checksum=%d\n",
    $selected,
    $iterations,
    SAMPLE_COUNT,
    $median / 1_000,
    $samples[0] / 1_000,
    $samples[array_key_last($samples)] / 1_000,
    $checksum,
);

[LamentXU123]

I don't love the additional code complexity, but the benchmark result seems worth it :/

[arnaud-lb]

Current algo:

• Build a sorted list for each array: O(m (n log n))
• Find intersections: O(mn)

New algo:

• Iterate first array: O(n)
• Flip first array: O(n)
• Find intersections: O(mn)

New algo is clearly superior.

Could the same algorithm be used in all cases, not only string|int arrays? array_intersect() converts values to string before comparison, so all values can be used as hash index. This would eliminate the fallback overhead.

[mehmetcansahin]

I tried the universal hash approach and updated the implementation to use it for all value types. The type-based fallback is now gone.

The former fallback cases are about 109–111x faster, float-only arrays are about 24x faster, and the existing integer/string cases remained neutral or improved by up to 22%.

I also added empty-input short-circuiting, while preserving the first array's key/bucket metadata, and documented the observable conversion-order differences in UPGRADING. The configured full test suite passes with 0 failures, and 4,000 differential cases matched for stable conversions and array metadata.

[mehmetcansahin]

I pushed the latest fixes.

Updated benchmark results:

Input	Base 3407a6d2a04	Current c15fff721e2	Change
2 × 10 integers	1.176 us	0.158 us	7.44× faster
2 × 1,000 integers	725.501 us	8.697 us	83.42× faster
2 × 100,000 integers	109,186.875 us	1,189.661 us	91.78× faster
2 × 10,000 strings	1,818.927 us	203.271 us	8.95× faster
2 × 10,000 mixed integers/strings	5,270.137 us	137.644 us	38.29× faster
3 × 10,000 integers	11,976.055 us	104.426 us	114.68× faster
10,000 values, float first	7,948.281 us	86.429 us	91.96× faster
10,000 values, float last	8,033.435 us	86.289 us	93.10× faster

The safety fix adds approximately 2–20% overhead compared with the previous hash implementation, while remaining 7.44–114.68× faster than the base implementation.

[arnaud-lb]

This looks good to me apart from a few nits. I will merge once those are resolved.

Feel free to add a NEWS entry.

[arnaud-lb]

Did you consider using ZEND_BITSET_FOREACH() here?

[mehmetcansahin]

The bitset tracks live-entry positions, not bucket indexes. Since zend_array_dup() can compact holes after conversion, using ZEND_BITSET_FOREACH() would still require mapping each position back to the corresponding bucket.