tinsaetadesse/t-connector

By tinsaetadesse

•Updated 3 days ago

A production-ready container image for running Kafka Connect with Apache Kafka and Schema Registry.

Image

Message queues

185

Overview Tags

tinsaetadesse/t-connector repository overview

⁠T-Connector Image

A production-ready container image for running Kafka Connect⁠ as a distributed worker with Kafka⁠ and Schema Registry⁠ connectivity over SSL/TLS.

The image is designed to run securely as a non-root user and includes the Java runtime, configuration scripts, logging configuration, and plugin support required by T-Connector.

⁠Features

The T-Connector image provides the following key features:

Java 21 runtime — Temurin JRE 21 is included in the image.
Apache Kafka 4.2.1 — Kafka runtime components are packaged into the image.
SSL/TLS enabled — Kafka, REST HTTPS, and Schema Registry connections can use SSL/TLS.
Mutual TLS support — Client authentication is enabled for the HTTPS REST listener and Kafka SSL configuration.
Schema Registry integration — Supports Confluent Avro⁠ serialization through Schema Registry.
Kafka Connect distributed mode — Multiple connector instances can operate as a single Kafka Connect cluster.
Plugin support — Custom Kafka Connect plugins are packaged under a dedicated plugin directory.
Non-root execution — The application runs as the t-soft user.
Configurable through environment variables — Kafka Connect settings are supplied through CONNECTOR_* environment variables.
Multiple REST endpoints — HTTP and HTTPS listeners are configured.
Multi-architecture build support — The image supports AMD64 and ARM64 architectures.

⁠Container Image

The image contains the T-Connector runtime and its supporting components, so end users do not need to install Java or Kafka inside the container.

⁠Main image components

Component	Version / Location
Base OS	Alpine Linux 3.21
Java	Temurin JRE 21.0.10+7
Apache Kafka	4.2.1
Scala binary version	2.13
Confluent base Java	8.2.1
Application user	`t-soft`
Application home	`/usr/local/t-soft/t-connector`
Configuration	`/etc/t-soft/t-connector`
Secrets	`/etc/t-soft/t-connector/secrets`
Plugins	`/usr/local/t-soft/t-connector/plugins`
Logs	`/var/log/t-soft/t-connector`

⁠Configuration

T-Connector is configured primarily through environment variables.

The Compose file uses the CONNECTOR_* naming convention.

For example:

environment:
  CONNECTOR_BOOTSTRAP_SERVERS: 'broker-1.tsoft.internal:9192,broker-2.tsoft.internal:9192,broker-3.tsoft.internal:9192'
  CONNECTOR_SECURITY_PROTOCOL: 'SSL'
  CONNECTOR_GROUP_ID: 't-connector'

This makes the deployment easy to configure without rebuilding the image.

⁠SSL/TLS

SSL/TLS is a central part of the T-Connector image.

To enable SSL for Kafka, include the below line in your Compose script:

CONNECTOR_SECURITY_PROTOCOL: 'SSL'

To require client authentication for Kafka clients, add the below line:

CONNECTOR_SSL_CLIENT_AUTH_REQUIRED: 'true'

Note: SSL credential files should be referenced from the secrets directory.

⁠REST Listeners

Containers can be configured to expose both HTTP and HTTPS REST listeners.

For example, connector-1 below advertises HTTP and HTTPS on two different ports.

http://connector-1.tsoft.internal:8383
https://connector-1.tsoft.internal:8483

If client authentication is required by HTTPS:

CONNECTOR_LISTENERS_HTTPS_SSL_CLIENT_AUTH: 'required'

⁠Connector Plugins

Custom plugins are stored in:

/usr/local/t-soft/t-connector/plugins

The Compose script references this directory as:

CONNECTOR_PLUGIN_PATH: '/usr/local/t-soft/t-connector/plugins'

⁠Deployment Example

services:
  connector-1:
    image: docker.io/tinsaetadesse/t-connector:latest
    container_name: connector-1
    hostname: connector-1.tsoft.internal
    ports:
      - '18383:8383'
      - '28483:8483'
    environment:
      CONNECTOR_BOOTSTRAP_SERVERS: 'broker-1.tsoft.internal:9192,broker-2.tsoft.internal:9192,broker-3.tsoft.internal:9192'
      CONNECTOR_SSL_KEYSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/databus-connector.keystore.jks'
      CONNECTOR_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/star.tsoft.internal.truststore.jks'
      CONNECTOR_SSL_KEY_CREDENTIAL_FILE: 'databus-client.ssl.key.credential'
      CONNECTOR_SSL_KEYSTORE_CREDENTIAL_FILE: 'databus-client.ssl.keystore.credential'
      CONNECTOR_SSL_TRUSTSTORE_CREDENTIAL_FILE: 'star.tsoft.internal.ssl.truststore.credential'
      CONNECTOR_SSL_CLIENT_AUTH_REQUIRED: 'true'
      CONNECTOR_SECURITY_PROTOCOL: 'SSL'
      CONNECTOR_REST_ADVERTISED_LISTENER: 'http'
      CONNECTOR_REST_ADVERTISED_HOST_NAME: 'connector-1.tsoft.internal'
      CONNECTOR_REST_ADVERTISED_PORT: 8383
      CONNECTOR_REST_PORT: 8383
      CONNECTOR_LISTENERS: 'http://connector-1.tsoft.internal:8383,https://connector-1.tsoft.internal:8483'
      CONNECTOR_LISTENERS_HTTPS_SSL_KEYSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/connector.client.keystore.jks'
      CONNECTOR_LISTENERS_HTTPS_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/star.tsoft.internal.truststore.jks'
      CONNECTOR_LISTENERS_HTTPS_SSL_KEYSTORE_CREDENTIAL_FILE: 'connector.client.ssl.keystore.credential'
      CONNECTOR_LISTENERS_HTTPS_SSL_KEY_CREDENTIAL_FILE: 'connector.client.ssl.key.credential'
      CONNECTOR_LISTENERS_HTTPS_SSL_TRUSTSTORE_CREDENTIAL_FILE: 'star.tsoft.internal.ssl.truststore.credential'
      CONNECTOR_LISTENERS_HTTPS_SSL_CLIENT_AUTH: 'required'
      CONNECTOR_GROUP_ID: 't-connector'
      CONNECTOR_CONFIG_STORAGE_TOPIC: '_connector-configs'
      CONNECTOR_OFFSET_STORAGE_TOPIC: '_connector-offsets'
      CONNECTOR_STATUS_STORAGE_TOPIC: '_connector-status'
      CONNECTOR_KEY_CONVERTER: 'org.apache.kafka.connect.storage.StringConverter'
      CONNECTOR_VALUE_CONVERTER: 'io.confluent.connect.avro.AvroConverter'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: 'https://schema-registry-1.tsoft.internal:8181,https://schema-registry-2.tsoft.internal:8181'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_KEYSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/metahub.client.keystore.jks'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_KEYSTORE_CREDENTIAL_FILE: 'metahub.client.ssl.keystore.credential'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/star.tsoft.internal.truststore.jks'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_TRUSTSTORE_CREDENTIAL_FILE: 'star.tsoft.internal.ssl.truststore.credential'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_KEY_CREDENTIAL_FILE: 'metahub.client.ssl.key.credential'
      CONNECTOR_PLUGIN_PATH: '/usr/local/t-soft/t-connector/plugins'
    volumes:
      - ./include/secrets:/etc/t-soft/t-connector/secrets:ro,z
    networks:
      my-network:
        aliases:
          - connector-1.tsoft.internal

  connector-2:
    image: docker.io/tinsaetadesse/t-connector:latest
    container_name: connector-2
    hostname: connector-2.tsoft.internal
    ports:
      - '28383:8383'
      - '38483:8483'
    environment:
      CONNECTOR_BOOTSTRAP_SERVERS: 'broker-1.tsoft.internal:9192,broker-2.tsoft.internal:9192,broker-3.tsoft.internal:9192'
      CONNECTOR_SSL_KEYSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/databus-connector.keystore.jks'
      CONNECTOR_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/star.tsoft.internal.truststore.jks'
      CONNECTOR_SSL_KEY_CREDENTIAL_FILE: 'databus-client.ssl.key.credential'
      CONNECTOR_SSL_KEYSTORE_CREDENTIAL_FILE: 'databus-client.ssl.keystore.credential'
      CONNECTOR_SSL_TRUSTSTORE_CREDENTIAL_FILE: 'star.tsoft.internal.ssl.truststore.credential'
      CONNECTOR_SSL_CLIENT_AUTH_REQUIRED: 'true'
      CONNECTOR_SECURITY_PROTOCOL: 'SSL'
      CONNECTOR_REST_ADVERTISED_LISTENER: 'http'
      CONNECTOR_REST_ADVERTISED_HOST_NAME: 'connector-2.tsoft.internal'
      CONNECTOR_REST_ADVERTISED_PORT: 8383
      CONNECTOR_REST_PORT: 8383
      CONNECTOR_LISTENERS: 'http://connector-2.tsoft.internal:8383,https://connector-2.tsoft.internal:8483'
      CONNECTOR_LISTENERS_HTTPS_SSL_KEYSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/connector.client.keystore.jks'
      CONNECTOR_LISTENERS_HTTPS_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/star.tsoft.internal.truststore.jks'
      CONNECTOR_LISTENERS_HTTPS_SSL_KEYSTORE_CREDENTIAL_FILE: 'connector.client.ssl.keystore.credential'
      CONNECTOR_LISTENERS_HTTPS_SSL_KEY_CREDENTIAL_FILE: 'connector.client.ssl.key.credential'
      CONNECTOR_LISTENERS_HTTPS_SSL_TRUSTSTORE_CREDENTIAL_FILE: 'star.tsoft.internal.ssl.truststore.credential'
      CONNECTOR_LISTENERS_HTTPS_SSL_CLIENT_AUTH: 'required'
      CONNECTOR_GROUP_ID: 't-connector'
      CONNECTOR_CONFIG_STORAGE_TOPIC: '_connector-configs'
      CONNECTOR_OFFSET_STORAGE_TOPIC: '_connector-offsets'
      CONNECTOR_STATUS_STORAGE_TOPIC: '_connector-status'
      CONNECTOR_KEY_CONVERTER: 'org.apache.kafka.connect.storage.StringConverter'
      CONNECTOR_VALUE_CONVERTER: 'io.confluent.connect.avro.AvroConverter'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: 'https://schema-registry-1.tsoft.internal:8181,https://schema-registry-2.tsoft.internal:8181'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_KEYSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/metahub.client.keystore.jks'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_KEYSTORE_CREDENTIAL_FILE: 'metahub.client.ssl.keystore.credential'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION: '/etc/t-soft/t-connector/secrets/star.tsoft.internal.truststore.jks'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_TRUSTSTORE_CREDENTIAL_FILE: 'star.tsoft.internal.ssl.truststore.credential'
      CONNECTOR_VALUE_CONVERTER_SCHEMA_REGISTRY_SSL_KEY_CREDENTIAL_FILE: 'metahub.client.ssl.key.credential'
      CONNECTOR_PLUGIN_PATH: '/usr/local/t-soft/t-connector/plugins'
    volumes:
      - ./include/secrets:/etc/t-soft/t-connector/secrets:ro,z
    networks:
      my-network:
        aliases:
          - connector-2.tsoft.internal

networks:
  my-network:
    external: true

⁠Security Recommendations

For production deployments:

Do not commit SSL keys or credential files to Git.
Keep include/secrets protected on the host.
Use a read-only secret mount.
Run the image as the non-root t-soft user.
Use HTTPS when exposing the REST API outside the trusted container network.
Keep client authentication enabled where required.
Use certificates whose SANs match the DataBus, MetaHub, and T-Connector hostnames.
Verify that all referenced keystore and truststore files exist before deployment.

Tag summary

Recent tags

Content type

Image

Digest

sha256:0f7834710…

Size

488.6 MB

Last updated

3 days ago

docker pull tinsaetadesse/t-connector

Help

System theme

Docker Suite