A universal adaptive lossless compression system engineered for backend microservices, real-time data ingestion, and multi-language runtimes. Designed around a locked CDP1 container format, multi-profile entropy codecs, and data-aware prefilters.
"Stable format, evolving encoder, portable decoder." Decoders remain lightweight and permanent, while encoders can introduce aggressive heuristics without breaking existing software.
From uncompressed data ingestion to bit-level entropy emission and CRC32c integrity verification.
Choose between microsecond latency, balanced HTTP payloads, or maximum archival ratio.
Byte-aligned LZ engine engineered for maximum throughput. Features SIMD vectorized match extension, 4-bit nibble token encoding, and zero-allocation token emission.
LZ matching coupled with Canonical Huffman entropy coding. Maximum code length bounded to 15 bits via Kraft inequality rebalancing with O(1) table decoding.
Compact LZ combined with table-based Asymmetric Numeral Systems (tANS / Finite State Entropy). Deterministic state machine (L=1024) with branchless lookup decoding.
Delta prefilters (stride 1..8) flatten numeric and time-series telemetry. Static domain dictionaries (JSON & Web) and dynamic phrase tokenizers shrink micro-payloads before entropy stages.
High-entropy inputs (pre-encrypted or already compressed files) automatically drop back to Block Type 0 (RAW verbatim passthrough), guaranteeing compressed data never exceeds original size.
Codrop delivers first-class SDKs and zero native compilation headaches across the developer ecosystem.
Core Rust reference engine with #![deny(unsafe_code)], safe high-level APIs, and standard C ABI FFI.
Official Node.js & WebAssembly SDK. Zero native build dependencies; runs everywhere from Edge to Server.
Universal high-performance adaptive compression Python bindings with native C FFI integration.
High-performance universal adaptive compression system for Dart and Flutter with multi-profile codecs.
Source repository containing the Rust core, WebAssembly build, CLI utility, test vectors, and specs.
Idiomatic, simple APIs across every major language runtime.
.png){kind=icon}
use libcodrop::{compress, decompress, CompressionLevel};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let payload = b"Codrop delivers universal adaptive compression across runtimes.";
// Compress with balanced profile (LZ + Canonical Huffman)
let compressed = compress(payload, CompressionLevel::Balanced)?;
println!("Compressed {} bytes to {} bytes", payload.len(), compressed.len());
// Decompress and verify roundtrip
let restored = decompress(&compressed)?;
assert_eq!(restored.as_slice(), payload);
println!("Lossless verification successful!");
Ok(())
}const codrop = require('codrop');
// Uncompressed data buffer
const data = Buffer.from('High performance universal adaptive compression for Node.js.');
// Compress (.cdp format with balanced profile)
const compressed = codrop.compress(data, codrop.LEVEL_BALANCED);
console.log(`Original: ${data.length}B, Compressed: ${compressed.length}B`);
// Decompress
const restored = codrop.decompress(compressed);
console.log(restored.toString('utf-8'));import codrop
# Input bytes
data = b"Hello Codrop! High performance adaptive compression for modern Python backends."
# Compress (Profiles: LEVEL_AUTO, LEVEL_FAST, LEVEL_BALANCED, LEVEL_COMPACT)
compressed = codrop.compress(data, level=codrop.LEVEL_BALANCED)
print(f"Original: {len(data)} bytes, Compressed: {len(compressed)} bytes")
# Decompress
decompressed = codrop.decompress(compressed)
assert decompressed == data
print("Roundtrip lossless verification verified!")import 'dart:typed_data';
import 'package:codrop/codrop.dart';
void main() {
final data = Uint8List.fromList('Dart and Flutter backend compression'.codeUnits);
// Compress
final compressed = Codrop.compress(data, profile: CodropProfile.balanced);
print('Compressed: ${compressed.length} bytes');
// Decompress
final restored = Codrop.decompress(compressed);
print('Restored: ${String.fromCharCodes(restored)}');
}Deterministic exit codes, streaming stdin/stdout pipelines, and deep metadata inspection.
# Compress a file using Balanced profile (default)
codrop compress file.txt -o file.cdp
# Compress with Fast profile (LZF)
codrop compress telemetry.log -l fast -o telemetry.cdp
# Compress with Compact profile (LZA / tANS)
codrop compress backup.tar -l compact -o backup.cdp
# Unix streaming pipeline from stdin to stdout
cat data.json | codrop compress - -o - > data.cdp
# Safe decompression with 64MB memory limit
cat data.cdp | codrop decompress - -o - --max-size 67108864 > out.json
# Inspect .cdp container metadata and blocks
codrop inspect file.cdp
The codrop command-line executable outputs standard exit codes for automated CI/CD pipelines and container orchestration:
| Code | Status | Description |
|---|---|---|
| 0 | Success | Operation completed successfully |
| 1 | General Failure | Argument syntax error or file I/O error |
| 2 | Malformed Stream | Invalid magic bytes, corrupted header, or CRC32c mismatch |
| 3 | Unsupported | Unsupported block type or incompatible format version |
| 4 | Limit Exceeded | Decompressed stream exceeded --max-size protection limit |
Built for multi-decade reliability in mission-critical infrastructure.
Core codec enforces #![deny(unsafe_code)] in Rust. Unsafe operations are strictly isolated and documented in C ABI FFI wrappers with automatic panic containment.
Permanent cross-version regression corpus verifying every codec block type, static dictionary, streaming chunk size, and CRC32c tamper detection.
Native protection against compression bombs with strict configurable byte limits (--max-size) and single-pass stream boundaries.