Conversation
Merging this PR will improve performance by 92.57%
Performance Changes
Comparing Footnotes
|
Polar Signals Profiling ResultsLatest Run
Previous Runs (4)
Powered by Polar Signals Cloud |
Benchmarks: PolarSignals ProfilingVortex (geomean): 0.973x ➖ datafusion / vortex-file-compressed (0.973x ➖, 0↑ 0↓)
|
Benchmarks: TPC-H SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.967x ➖, 2↑ 1↓)
datafusion / vortex-compact (0.968x ➖, 0↑ 1↓)
datafusion / parquet (0.978x ➖, 2↑ 1↓)
datafusion / arrow (0.941x ➖, 5↑ 0↓)
duckdb / vortex-file-compressed (0.952x ➖, 3↑ 0↓)
duckdb / vortex-compact (0.960x ➖, 0↑ 0↓)
duckdb / parquet (1.002x ➖, 1↑ 3↓)
duckdb / duckdb (0.956x ➖, 4↑ 0↓)
Full attributed analysis
|
Benchmarks: FineWeb NVMeVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.967x ➖, 1↑ 0↓)
datafusion / vortex-compact (1.003x ➖, 0↑ 1↓)
datafusion / parquet (0.988x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.972x ➖, 1↑ 1↓)
duckdb / vortex-compact (0.998x ➖, 0↑ 0↓)
duckdb / parquet (1.000x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.901x ➖, 51↑ 0↓)
datafusion / vortex-compact (0.913x ➖, 35↑ 0↓)
datafusion / parquet (0.931x ➖, 26↑ 0↓)
duckdb / vortex-file-compressed (0.899x ✅, 48↑ 0↓)
duckdb / vortex-compact (0.909x ➖, 30↑ 0↓)
duckdb / parquet (0.943x ➖, 12↑ 1↓)
duckdb / duckdb (0.904x ➖, 41↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=10 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.979x ➖, 2↑ 1↓)
datafusion / vortex-compact (0.971x ➖, 3↑ 2↓)
datafusion / parquet (0.973x ➖, 1↑ 0↓)
datafusion / arrow (0.958x ➖, 3↑ 0↓)
duckdb / vortex-file-compressed (0.998x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.995x ➖, 0↑ 0↓)
duckdb / parquet (1.002x ➖, 0↑ 0↓)
duckdb / duckdb (0.999x ➖, 0↑ 1↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.924x ➖, 3↑ 2↓)
datafusion / vortex-compact (1.081x ➖, 1↑ 5↓)
datafusion / parquet (1.056x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed (0.980x ➖, 1↑ 3↓)
duckdb / vortex-compact (0.965x ➖, 1↑ 2↓)
duckdb / parquet (1.134x ➖, 0↑ 6↓)
Full attributed analysis
|
Benchmarks: FineWeb S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.174x ➖, 0↑ 2↓)
datafusion / vortex-compact (1.311x ❌, 0↑ 5↓)
datafusion / parquet (1.077x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (0.933x ➖, 1↑ 1↓)
duckdb / vortex-compact (1.018x ➖, 0↑ 0↓)
duckdb / parquet (1.012x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Random AccessVortex (geomean): 0.862x ✅ unknown / unknown (0.953x ➖, 9↑ 0↓)
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) duckdb / vortex-file-compressed (0.964x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.980x ➖, 0↑ 0↓)
duckdb / parquet (0.975x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Clickbench on NVMEVerdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.064x ➖, 0↑ 12↓)
datafusion / parquet (1.065x ➖, 0↑ 5↓)
duckdb / vortex-file-compressed (1.054x ➖, 4↑ 14↓)
duckdb / parquet (1.021x ➖, 0↑ 1↓)
duckdb / duckdb (0.933x ➖, 5↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=10 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.371x ❌, 0↑ 12↓)
datafusion / vortex-compact (0.964x ➖, 4↑ 2↓)
datafusion / parquet (0.986x ➖, 4↑ 4↓)
duckdb / vortex-file-compressed (1.101x ➖, 0↑ 7↓)
duckdb / vortex-compact (1.096x ➖, 0↑ 4↓)
duckdb / parquet (1.073x ➖, 1↑ 4↓)
Full attributed analysis
|
Benchmarks: CompressionVortex (geomean): 1.043x ➖ unknown / unknown (1.042x ➖, 0↑ 17↓)
|
Resolve conflicts in alp_rd array/mod keeping BufferMut optimization from this branch while adopting develop's structural changes (Arc<Array>, require_child!, Validity::from_mask). Signed-off-by: Joe Isaacs <joe@spiraldb.com>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
|
this looks fine to me, but I think it would be better if someone more familiar with this encoding reviewed this? |
Use in-place alp decompression