SFTP (Output, Store)

• https://datatracker.ietf.org/doc/html/rfc4253
• https://man.openbsd.org/sftp

• Enterprise file exchanges
• Banking and financial reporting
• Regulatory submissions
• Partner data sharing
• Legacy application integration
• Batch ingestion workflows

• Streams Kafka events into files.
• Securely transfers files using SFTP.
• Supports configurable remote directories.
• Supports username/password authentication.
• Buffers events before upload for efficient transfers.
• Compatible with cloud-hosted and on-premises SFTP servers.

1. The connector subscribes to the configured Kafka topic.
2. Incoming events are buffered.
3. Buffered events are written into a file using the configured format.
4. The file is uploaded securely to the remote SFTP server.
5. Upload status and connector metrics are available through Condense observability.

​

Title

• Description: Friendly name for the connector.
• Example: SFTP_Financial_Exports

​

Connection Host

• Description: Hostname or IP address of the destination SFTP server.
• Examples:
  • sftp.partner.com
  • 192.168.10.20
• Requirements:
  • The server must be reachable from the Condense deployment.

​

Connection Port

• Description: TCP port used by the SFTP server.
• Default: 22
• Examples:
  • 22
  • 2022

​

Username

• Description: Username used for authenticating with the SFTP server.
• Requirements:
  • The account must have write access to the destination directory.

​

Password

• Description: Password associated with the configured username.
• Security Recommendation: Store credentials securely and rotate them regularly.

​

Directory

• Description: Remote directory where uploaded files will be stored.
• Examples:
  • /incoming/reports/
  • /exports/events/
• Requirements:
  • Directory must already exist.
  • User must have write permissions.

​

Input Topic

• Description: Kafka topic inside Condense from which events are consumed.
• Example: payment.events.processed

{
  "title": "SFTP_Financial_Exports",
  "connectionHost": "sftp.partner.com",
  "connectionPort": 22,
  "username": "condense_writer",
  "password": "********",
  "directory": "/incoming/reports/",
  "inputTopic": "payment.events.processed"
}

​

Connection Refused

• Incorrect hostname.
• Incorrect port.
• Firewall restrictions.

• Verify network connectivity.
• Confirm the configured host and port.
• Ensure outbound access from Condense.

​

Authentication Failed

• Invalid username or password.
• Account disabled or locked.

• Verify credentials.
• Confirm the account has SFTP access.

​

Permission Denied

• User lacks write permission.
• Destination directory is read-only.

• Verify directory permissions.
• Ensure the configured account has write access.

​

Files Not Uploaded

• Kafka topic contains no events.
• Upload interrupted.
• Directory path is incorrect.

• Verify data is available in the input topic.
• Check connector logs.
• Confirm the remote directory exists.

​

Upload Timeout

• Slow network.
• Idle connection timeout.

• Verify network stability.
• Review server timeout settings.

​

File Naming

condense-20260713-154500.json

• Custom filename templates
• File rotation by size
• File rotation by time
• Dynamic directory structures

​

Output Formats

• JSON
• NDJSON
• CSV

​

Security

• Restrict users to dedicated upload directories.
• Rotate credentials regularly.
• Limit server access using firewalls or VPNs.
• Use dedicated service accounts.

1. Create dedicated SFTP accounts for Condense integrations.
2. Ensure destination directories exist before deployment.
3. Organize files using logical directory structures.
4. Monitor upload status through Condense observability and server logs.
5. Validate generated files before integrating with downstream applications.