ItdaStreamKafka-Compatible Interface · Tiered Storage · Real-time Iceberg Streaming
ItdaStream Key Features
Overcoming Kafka limitations by separating compute and storage for unlimited retention and lower cost — and landing topic messages into an Iceberg lakehouse in real time, exactly-once.
Full Kafka Protocol Compatibility
Seamless migration using existing Kafka clients and ecosystem without any code changes.
No-Code Kafka → Iceberg Streaming
Point a topic at a target Iceberg table in the Admin UI and messages land in Iceberg exactly-once — no code. When records carry no event-time column, an ingestion-time column is synthesised and the table is hidden-partitioned by hour, so time-pruned scans work out of the box. A built-in Flink-style engine (controller broker as master, the rest as workers) needs no separate streaming cluster. Set a WAP branch in the sink config to stage streaming writes on a non-main branch, audit them in isolation, then publish to main with a fast-forward via REST (/admin/iceberg/publish) — Write-Audit-Publish.
SDK Multi-Sink Pipelines
Submit filter/select/map/flatMap transforms and multi-sink pipelines — Iceberg, Elasticsearch, JDBC, Kafka — with the Java SDK or submit.sh. Custom map/flatMap transform JARs are uploaded with the job and loaded by a per-job URLClassLoader, so no broker restart is needed. Credentials are registered once in the Connection Registry and referenced by connectionId.
Separation of Compute & Storage
Cloud-native architecture with high-performance Stateless brokers and independent S3 storage scaling.
S3-based Tiered Storage
Solves disk capacity limits by keeping fresh data in Memory/SSD (Hot tier) and aged data in S3 (Cloud tier).
Superior Performance & Latency
Ensures minimal latency for real-time data processing through asynchronous S3 Flush and Tail Read optimization, even in S3-based environments.
Unlimited Data Retention
Secure long-term data preservation using S3 Object Storage, independent of local disk constraints.
Enterprise Security & KMS
Enhanced security with built-in KMS applying AES-GCM 256-bit Envelope Encryption.
Zero-Rebalancing Scaling
Instant broker scaling without the heavy data re-replication load during node addition or removal.
Integrated Admin UI
Real-time TPS monitoring, topic management, IAM policies, and integrated log browsing.
Drastic Infrastructure Cost Savings
Reduces storage costs by up to 80% compared to traditional Kafka by utilizing S3 over expensive SSDs.
Key Use Cases
Large-scale Log Aggregation
Cost-effectively store and query terabytes of logs for the long term.
Event-driven Microservices
Handle MSA events based on standard Kafka APIs with minimal operational overhead.
Real-time Lakehouse Ingestion
Land topic JSON/Avro messages into Iceberg tables exactly-once with no-code setup — auto-partitioned by time and ready for immediate analytics.
Data Lake Integration
Directly accumulate and analyze real-time events into S3 data lakes.
Cloud Native Migration
Seamlessly transition legacy Kafka infrastructure to elastic cloud environments.
Considering ItdaStream for your infrastructure?
Kafka API Compatible. S3 Tiered Storage. Unlimited Retention.
Innovate with the next-generation distributed streaming platform that enables cost savings and unlimited scalability.