Overview
The HTTP Output Connector enables Condense to publish data from Apache Kafka topics to external HTTP or HTTPS endpoints. It acts as a Sink Connector, consuming Kafka records and delivering them to REST APIs, web services, webhooks, and enterprise applications in real time. The connector supports multiple authentication mechanisms, SSL/TLS security, configurable batch processing, and flexible error-handling strategies, making it suitable for integrating Kafka-based streaming applications with external systems.Key Features
The HTTP Output Connector provides the following capabilities:- Publishes Kafka messages to HTTP and HTTPS endpoints
- Supports REST API integration
- Supports multiple authentication methods
- Supports One-Way and Two-Way SSL (Mutual TLS)
- Configurable batch processing
- Configurable retry policies
- Multiple payload formats
- Flexible error handling
- No-code deployment through the Condense UI
Supported HTTP Methods
The connector supports the following HTTP methods for sending data to external services.Supported Authentication
The connector supports multiple authentication mechanisms.SSL/TLS Support
Secure communication with HTTPS endpoints is supported through the following SSL modes.Supported Payload Formats
The connector supports the following message formats:- JSON
- String
- Avro
- Binary
Batch Processing
The connector supports configurable batch processing to improve throughput when sending records to HTTP endpoints. Batch processing options include:- Batch Size
- Batch Timeout
- Request Timeout
Error Handling
The connector supports multiple strategies for handling failed requests.Typical Use Cases
The HTTP Output Connector can be used for:- Sending Kafka events to REST APIs
- Webhook notifications
- CRM integration
- ERP integration
- Third-party SaaS integration
- Event-driven microservices
- Data synchronization
- Business workflow automation
Prerequisites
Before configuring the connector, ensure:- The target HTTP or HTTPS endpoint is accessible.
- Required authentication credentials are available.
- SSL certificates are available if HTTPS is used.
- The Kafka topic containing source data exists.
- Appropriate permissions are available to deploy connectors.
How to Configure the HTTP Output Connector
Step 1. Navigate to Connectors
- Log in to Condense.
- Navigate to Connectors.
- Click Create Connector.
- Select HTTP Output Connector.
Add Screenshot: Connector Selection
Step 2. Configure Basic Information
Provide:- Connector Name
- Description (optional)
- Workspace
Step 3. Configure Kafka Source
Specify:- Input Kafka Topic
- Message Format
- Record Key (optional)
Add Screenshot: Kafka Configuration
Step 4. Configure HTTP Endpoint
Provide:- Endpoint URL
- HTTP Method
- Content Type
- Request Headers
- Query Parameters (optional)
Add Screenshot: Endpoint Configuration
Step 5. Configure Authentication
Select the required authentication method. Supported options include:- None
- Basic Authentication
- JWT
- API Key
- OAuth2
- Digest Authentication
Add Screenshot: Authentication Settings
Step 6. Configure SSL
If the endpoint uses HTTPS, configure:- One-Way SSL
- Mutual TLS (Two-Way SSL)
Add Screenshot: SSL Configuration
Step 7. Configure Batch Processing
Configure:- Batch Size
- Batch Timeout
- Request Timeout
Step 8. Configure Error Handling
Choose the preferred strategy.
Configure retry interval and maximum retry attempts where applicable.
Step 9. Deploy the Connector
- Review the configuration.
- Click Deploy Connector.
- Monitor deployment from the Pipelines page.
Editing the HTTP Output Connector
- Navigate to Pipelines.
- Select the deployed HTTP Output Connector.
- Click Edit Configurations.
- Update the required settings.
- Click Deploy Connector to apply the changes.
Add Screenshot: Edit Configuration
Deleting the HTTP Output Connector
- Navigate to Pipelines.
- Select the deployed connector.
- Click Delete Connector.
- Confirm the deletion.
Best Practices
- Always use HTTPS in production environments.
- Use Mutual TLS when connecting to secure enterprise APIs.
- Configure appropriate retry intervals for transient network failures.
- Validate endpoint accessibility before deployment.
- Avoid excessively large batch sizes for latency-sensitive applications.
- Monitor connector health after deployment.
- Verify authentication credentials before enabling production workloads.
- Use appropriate error-handling strategies based on business requirements.
Limitations
- Each connector publishes to a single HTTP endpoint.
- Throughput depends on the performance of the target service.
- Authentication mechanisms depend on the capabilities of the target API.
- SSL certificates must remain valid for secure communication.
Related Documentation
- HTTP Input Connector
- Apache Kafka Output Connector
- Security
- SSL Configuration
- Connectors
Frequently Asked Questions (FAQs)
What is the HTTP Output Connector used for?
The HTTP Output Connector consumes records from Kafka topics and delivers them to external HTTP or HTTPS endpoints.Which HTTP methods are supported?
The connector currently supports:- POST
- PUT
- PATCH
Which authentication methods are supported?
The connector supports:- None
- Basic Authentication
- JWT
- API Key
- OAuth2
- Digest Authentication
Does the connector support HTTPS?
Yes. Both HTTP and HTTPS endpoints are supported.Is Mutual TLS supported?
Yes. The connector supports both One-Way SSL and Two-Way SSL (Mutual TLS).Which payload formats are supported?
Supported payload formats include:- JSON
- String
- Avro
- Binary