ksqlDB

​

Overview

ksqlDB is a powerful, SQL-like streaming engine for Apache Kafka that enables real-time data processing and transformation. Integrating ksqlDB with Condense will enable users to perform complex data transformations and aggregations on Kafka topics directly from the platform.

​

Key Features

​

Real-Time Data Processing

1. Execute SQL-like queries on streaming data for immediate insights.
2. Supports filtering, joins, and aggregations in real time.

​

Stream and Table Management

• Create, update, and delete streams and tables from Kafka topics.
• Streams for unbounded event sequences
• Tables for stateful data

​

Transient Queries

• Run queries that process data on the fly without storing results.
• Ideal for real-time monitoring and alerting

​

User-Friendly Interface

• Integrated directly into the Condense UI for seamless query execution.
• Visualise query results in real time.

​

Role-Based Access Control

• Admin/Developer: Full access to create and manage queries.
• Viewer: Read-only access (no KSQL permissions)

​

How to Use KSQL

​

Prerequisites

• Admin or Developer role in Condense
• Access to Kafka topics with read/write permissions
• Basic knowledge of SQL syntax

​

Step 1: Access KSQL Interface

1. Log in to Condense with your credentials
2. Navigate to KSQL from the main menu
3. Select New Query to open the editor

​

Step 2: Create a Stream

1. For processing real-time event data, in the query editor, enter your query
2. Click Execute to create the stream

​

Step 3: Create a Table

1. For stateful data, in the query editor, enter your query
2. Click Execute to create the table

​

Step 4: Run queries

1. Execute transient queries for real-time analysis
2. Click Execute to execute and view real-time results

​

Step 5: Monitor & Troubleshoot

Review Error Logs:

1. Access logs below the query editor
2. Common issues: topic permissions, schema mismatches

​

Frequently Asked Questions (FAQs)

​

Q1: What is RBAC in KSQL?

RBAC (Role-Based Access Control) restricts access based on predefined user roles:

• Admin: Full create/edit/delete permissions
• Developer: Create/edit permissions
• Viewer: Read-only access (no KSQL access in initial release)

​

Q2: Can we allow KSQL to read from some topics but not others?

1. Grant the user “Developer” or “Admin” role in RBAC
2. Ensure ACLs allow topic creation permissions

​

Q3: How to allow KSQL to create new streams and tables?

Yes, ACLs can be configured at the topic level to grant or restrict access to specific topics.