> ## Documentation Index > Fetch the complete documentation index at: https://docs.condense.io/llms.txt > Use this file to discover all available pages before exploring further. # DIY OSS Kafka ##### **Condense and Apache Kafka (OSS) represent the choice between deploying a finished application and managing raw infrastructure. Apache Kafka OSS is a distributed engine that moves bytes reliably, but it requires you to build, scale, and maintain the entire ecosystem including separate clusters for any data processing. Condense is a unified streaming platform that wraps the Kafka engine with an autonomous execution layer, allowing you to run business logic directly on the stream without managing the underlying "plumbing."** ## **How does Condense compare against Apache Kafka?** | **Feature** | **Condense** | **Apache kafka** | | :--------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | System Design | A unified environment that bundles Kafka, Kubernetes, and the App Runtime into one integrated stack. | Distributed design based on a Java/Scala engine that utilizes the JVM, OS page cache, and internal KRaft for cluster metadata. | | Infrastructure | Managed BYOC: Deploys a fully managed stack (Kafka + K8s + Logic) into your VPC with one click from cloud marketplace of AWS, Azure and GCP. | DIY Infrastructure: You must manually provision VMs, networking, storage, and KRaft. | | Unified Platform | Unified Application Fabric: Merges the Kafka engine with a native event-driven microservice runtime. It is an "Execution Fabric." | Fragmented Infrastructure: Operates as a standalone distributed log. Requires external, manual integration with separate processing layers (Flink, Spark, or custom JVM apps). | | Logic Execution | Managed App Runtime: Custom transforms run in a managed, serverless-style environment. Condense handles containerization, state management, and orchestration. | External Services: Requires separate Flink, Spark, or Microservices to process data. | | Scaling Scope | Full-Stack: Automatically scales brokers and custom code in lockstep based on real-time lag. | Infrastructure-Only: Manual broker provisioning and time-consuming partition rebalancing. | | Storage | Native Object Storage: Direct offloading to S3/GCS/Azure Blob; data stays in your buckets for infinite retention. | Local Disk (Default): Requires manual configuration of Tiered Storage and complex cleanup policies. | | Performance | Yes (Serverless Scale and Zero-Hop): Running GBps+ enterprise data workload and autonomous scale for spikes and variations. Application logic runs "local" to the broker, eliminating the network latency of external processors. | Network-Heavy: Every transformation usually requires a network hop to an external processor. | | **Feature** | **Condense** | **Apache kafka** | | :------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------- | | Operational Effort | Zero-Ops, platform handles upgrades, patching, and self-healing autonomously. | High-Ops: Your team is responsible for cluster health, JVM tuning, and server maintenance. | | Industry Readiness | Dual-Layer Ecosystem: Access a vast library of generic connectors alongside industry specific domain-aware transforms for Mobility, IoT, and FinTech (e.g., VIN parsing, Geofencing) out of the box. | Standard Connect: Generic connectors that require manual configuration and worker management. | | Deployment Model | BYOC-First: Runs 100% in your private VPC to keep data sovereign and eliminate egress fees. | Self-Managed: Runs anywhere, but you own the complexity of the security and network stack. | | Developer Experience | Integrated AI IDE: Build, test, and deploy Java or Python logic natively with built-in AI for debugging and code generation. | Command Line / External: No native UI; requires external tools like Kafka-UI or AKHQ. | | Observability | Full-Stack: Single dashboard tracing an event from raw ingestion through logic to the sink. | Fragmented: Requires stitching together Prometheus, Grafana, and external APM tools. | | **Feature** | **Condense** | **Apache kafka** | | :------------- | :--------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------- | | Access Experts | Direct Engineering: Built in support widget to raise ticket. Condense offers support priority based TAT for resolution | Community / Consultants: Reliance on StackOverflow or expensive third-party support contracts. | | Security | Own Cloud -Sovereign: Inherits your IAM and security groups; 100% data localization. | You must build and maintain security policies, certificates, and auditing. | | SLAs | 99.95% Full-Stack: Covers the brokers, the processing logic, and the connectors under one SLA. | None: Uptime is entirely dependent on your internal DevOps team's expertise. | | TCO | Predictable BYOC: Pay for compute (vCPU/hr) inside your VPC. No throughput or egress fees; uses your existing cloud credits (EDP). | Kafka is free, but engineering hours for operations are not. | ## **Why Switch to Condense?** OSS Kafka is a "broker-only" technology. To build a real-world application, you must manually integrate and manage separate clusters for Kafka Connect, Schema Registry, Flink/Spark (for logic), and Prometheus/Grafana (for monitoring). Condense collapses these silos into a single Unified Application Fabric. It merges the Kafka engine with a native, event-driven microservice runtime. You don't "glue" tools together; the tools are built into the fabric. In OSS Kafka processing typically happens in external microservices or Flink clusters. Data must travel from the broker over the network to the processor and often back again, adding significant network latency and egress costs. Condense allows you to deploy your Java, Python, or Go logic local to the data. By running code on the same substrate as the broker, Condense eliminates the network "hop," delivering sub-millisecond end-to-end latency for mission-critical alerts. In OSS Kafka you own the code, but you also own the 3 AM "broker down" phone calls. Scaling, patching, and rebalancing are manual, high-risk operations. Condense offers a Bring Your Own Cloud (BYOC) model. It deploys directly into your AWS/GCP/Azure account via the marketplace leveraging your cloud credits and IAM, but with fully managed platform without headache of operations and maintenance OSS Kafka is a generic "dumb pipe." It has no idea what a VIN (Vehicle Identification Number), a geofence, or a telematics packet is. You have to write every line of that domain logic from scratch. Condense ships with Vertical Blueprints. It includes pre-built, domain-aware modules for Mobility (Trip Formation, Driver Scoring) and IoT (VSS Mapping). This allows teams to go live in days rather than the months required to build these patterns on OSS. In OSS Kafka the license is "free," the Total Cost of Ownership is massive. You need a dedicated team of 3–5 specialized SREs to manage a production-grade Kafka ecosystem. Condense reduces TCO by up to 60%. It eliminates the need for a specialized Kafka Ops team and uses a flat, predictable vCPU-based pricing model. There are no "hidden taxes" for throughput, partitions, or data egress within your VPC. ## **Why is Condense the Best Way to Enable Agentic AI and Real-Time Data** | **WITHOUT CONDENSE** | **THE CONDENSE WAY** | | :----------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Coding Connectors Requires specialized Java/Scala skills to write and maintain industry specific connectors | Universal & Industry-Ready Connectors Deploy universal or specialized connectors
(e.g., Telematics for Mobility) that come with built-in parsing for complex schemas | | Complex Management Development and maintenance of ever changing industry connectors becomes difficult | Configurable Output Sinks Configure and deploy pre-built sink/source connectors and through UI into the data pipeline
| | Maintenance & Scalability Managing scale and failover of connectors become a challenge as the load increases | | | **WITHOUT CONDENSE** | **THE CONDENSE WAY** | | :---------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Research & Glue Code Weeks spent finding libraries and writing boilerplate just to connect components | In-Built AI IDE & Git Sync Use purpose built AI agents to create, test, and build your custom transforms with GIT support. Publish them to be used in data pipeline directly | | Maintenance & Scalability Multiple workflows and automations are required to manage availability and failover of stream processing applications | Native Stream Processing No external engine needed. Deploy your custom logic as reusable connectors or transforms that runs as containerized services | | Disjointed Lifecycle Constant context switching between IDE, Git, Cloud Console, and CI/CD tools | Management & Scalability Complete lifecycle, versioning and scalability of services managed by Custom Transform Framework | | **WITHOUT CONDENSE** | **THE CONDENSE WAY** | | :--------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------- | | Absence of Insights Creation of observability layer based on disjointed CLI tools, multiple available monitoring stacks and manual log aggregation | Native Dashboard Built-in visual pipeline view to see data moving in real-time, Check and act based on states of services, logs and configurations | | Manual Tracking Manually monitoring usage, over-

provisioning, and under-utilization risks that impacts availability and increases operational cost | Comprehensive Observability Seamless integration with external tools industry accepted observability stacks for platform and cloud Ops | | |
Purpose Built AI Agents

Autonomously checks system to generate actionable insights | | **WITHOUT CONDENSE** | **THE CONDENSE WAY** | | :-------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Complex Setup Manual provisioning of Cloud compute resources and networking. Deployment of platform architecture for streaming usecases | Automated Provisioning Automated deployment of cloud resources and platform tailor-made for data streaming on your cloud subscription | | Maintenance Nightmare Difficult to manage uptime between Infra and other system upgrades and cross dependencies | Fully Managed Maintenance All upgrades, patches, and downtime recovery are handled by the Condense team. User stays on a stable interface with 99.95% availability | | Security Considerations Maintaining custom build governance workflows, cloud security and compliances becomes difficult over time | Security & Compliance Enterprise grade governance, audits, Information security and compliance certifications out-of-the-box |