> ## Documentation Index
> Fetch the complete documentation index at: https://docs.condense.io/llms.txt
> Use this file to discover all available pages before exploring further.

# HTTP (Input, Stream)

# Overview

The HTTP Input Connector enables Condense to ingest data from external HTTP or HTTPS endpoints and publish it to Apache Kafka topics for real-time stream processing.

The connector supports both **Polling** and **Webhook** modes, allowing applications to either periodically fetch data from REST APIs or receive events instantly through HTTP requests. It also supports multiple authentication methods, SSL/TLS security, configurable polling intervals, and flexible data formats, making it suitable for integrating external systems with streaming pipelines.

***

# Key Features

The HTTP Input Connector provides the following capabilities:

* Supports Polling and Webhook modes
* Ingests data from HTTP and HTTPS endpoints
* Publishes incoming events to Kafka topics
* Supports multiple authentication mechanisms
* Supports One-Way and Two-Way SSL (Mutual TLS)
* Supports JSON, String, Avro, and Binary payloads
* Configurable polling intervals
* Configurable batch processing
* Retry and error-handling strategies
* No-code configuration through the Condense UI

***

# Supported Input Modes

| Mode | Description |
| - | - |
| **Polling** | Periodically calls an HTTP endpoint and publishes the response to Kafka. |
| **Webhook** | Receives HTTP POST requests and immediately streams the received payload into Kafka. |

***

# Supported Authentication

The connector supports multiple authentication methods depending on the target HTTP service.

| Authentication | Supported |
| - | :-: |
| None | ✅ |
| Basic Authentication | ✅ |
| JWT Bearer Token | ✅ |
| API Key | ✅ |
| OAuth 2.0 | ✅ |
| Digest Authentication | ✅ |

***

# SSL/TLS Support

The connector supports secure communication with HTTPS endpoints.

| SSL Mode | Description |
| - | - |
| HTTP | Unencrypted communication |
| One-Way SSL | Validates the server certificate |
| Two-Way SSL (Mutual TLS) | Validates both client and server certificates |

***

# Supported Payload Formats

The connector supports the following data formats:

* JSON
* String
* Avro
* Binary

***

# Typical Use Cases

The HTTP Input Connector is commonly used for:

* Polling REST APIs
* Receiving webhook notifications
* Collecting IoT events
* Integrating third-party SaaS applications
* Importing business events into Kafka
* Event-driven microservices
* Real-time data ingestion

***

# Prerequisites

Before configuring the connector, ensure:

* The HTTP or HTTPS endpoint is accessible.
* Required authentication credentials are available.
* SSL certificates are available if HTTPS is used.
* A Kafka topic exists to receive incoming events.
* Appropriate permissions are available to deploy connectors.

***

# How to Configure the HTTP Input Connector

## Step 1. Navigate to Connectors

1. Log in to Condense.
2. Navigate to **Connectors**.
3. Click **Create Connector**.
4. Select **HTTP Input Connector**.

> **Add Screenshot:** Connector Selection

***

## Step 2. Configure Basic Information

Provide the following information:

* Connector Name
* Description (optional)
* Target Workspace

***

## Step 3. Configure the HTTP Endpoint

Enter:

* Endpoint URL
* HTTP Method (if applicable)
* Request Headers
* Query Parameters (optional)

> **Add Screenshot:** Endpoint Configuration

***

## Step 4. Select Input Mode

Choose one of the following:

### Polling Mode

Configure:

* Polling Interval
* Request Timeout
* Batch Size

### Webhook Mode

Configure:

* Webhook Endpoint
* Listener Configuration

> **Add Screenshot:** Mode Configuration

***

## Step 5. Configure Authentication

Select the required authentication type.

Supported options include:

* None
* Basic Authentication
* JWT
* API Key
* OAuth2
* Digest Authentication

Provide the required credentials based on the selected authentication mechanism.

> **Add Screenshot:** Authentication Settings

***

## Step 6. Configure SSL

If HTTPS is used, configure:

* One-Way SSL
* Mutual TLS (Two-Way SSL)

Upload certificates if required.

> **Add Screenshot:** SSL Configuration

***

## Step 7. Configure Kafka Settings

Specify:

* Output Kafka Topic
* Message Format
* Key Configuration (optional)

***

## Step 8. Configure Error Handling

Select one of the available strategies:

| Strategy | Description |
| - | - |
| Retry | Retries failed requests automatically |
| Skip | Ignores failed records |
| Fail | Stops connector execution |

Configure retry interval and maximum retries if Retry is selected.

***

## Step 9. Deploy the Connector

1. Review all configurations.
2. Click **Deploy Connector**.
3. Monitor deployment from the **Pipelines** page.

***

# Editing the HTTP Input Connector

1. Navigate to **Pipelines**.
2. Select the deployed HTTP Input Connector.
3. Click **Edit Configurations**.
4. Update the required settings.
5. Click **Deploy Connector** to apply the changes.

> **Add Screenshot:** Edit Configuration

***

# Deleting the HTTP Input Connector

1. Navigate to **Pipelines**.
2. Select the deployed connector.
3. Click **Delete Connector**.
4. Confirm the deletion.

***

# Best Practices

* Prefer HTTPS for production deployments.
* Use Mutual TLS when connecting to secure enterprise services.
* Configure polling intervals based on API rate limits.
* Avoid overly aggressive polling frequencies.
* Validate authentication credentials before deployment.
* Use Retry for transient network failures.
* Monitor connector health after deployment.
* Validate payload structure before publishing to Kafka.

***

# Limitations

* Each connector monitors a single endpoint.
* Polling frequency depends on endpoint availability and response time.
* Authentication requirements depend on the external service.
* SSL certificates must remain valid for secure communication.

***

# Related Documentation

* HTTP Output Connector
* Apache Kafka Input Connector
* MQTT Input Connector
* Security
* SSL Configuration

***

# Frequently Asked Questions (FAQs)

### What is the HTTP Input Connector used for?

It ingests data from HTTP or HTTPS endpoints and publishes it to Kafka topics for downstream processing.

***

### Which input modes are supported?

The connector supports:

* Polling
* Webhook

***

### Which authentication methods are supported?

The connector supports:

* None
* Basic Authentication
* JWT
* API Key
* OAuth2
* Digest Authentication

***

### Can I connect to HTTPS endpoints?

Yes. The connector supports both HTTP and HTTPS endpoints.

***

### Does the connector support Mutual TLS?

Yes. Both One-Way SSL and Two-Way SSL (Mutual TLS) are supported.

***

### Which payload formats are supported?

Supported formats include:

* JSON
* String
* Avro
* Binary

***

### Can I periodically poll REST APIs?

Yes. Polling mode allows the connector to fetch data at configurable intervals.

***

### Can the connector receive webhook events?

Yes. Webhook mode allows external systems to push events directly into Kafka.

***

### Can I configure retry behaviour?

Yes. Retry, Skip, and Fail strategies are available for handling request failures.

***

### Can I edit a deployed connector?

Yes. Existing connector configurations can be modified and redeployed from the **Pipelines** page.

***

### Can multiple HTTP Input Connectors publish to the same Kafka topic?

Yes. Multiple connectors can publish to the same topic if required by your streaming architecture.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.