Kafka SSL

For safe usage of Kafka, it is recommended to use mutual TSL for security. This setup means,
that both brokers and clients will have their own certificate. Also, because SSL isn't trusting
by default we need to make sure, that the other side's certificates are trusted.

Kafka Configuration

Kafka is by default plaintext only. To enable SSL you need to configure following:

Service configuration

• advertised.listeners: configuration where Kafka broker listens for client conenctions, recommended value: PLAINTEXT://kafka3:9092,SSL://kafka3:9093
• ssl.keystore.filename: filename/location of the key store, example value: kafka_keystore.jks
• ssl.keystore.password: password of the key store, example value: changeit
• ssl.truststore.filename: filename/location of the trust store, example value: kafka_truststore.jks
• ssl.truststore.password: password of the trust store, example value: changeit
• security.inter.broker.protocol: protocol, that will be used for communication between Kafka brokers, for local and dev
  deployment PLAINTEXT is recommended. For production deployment: SSL
• ssl.client.auth: this field enables/disables client authorization on broker's side. To enable, set value to: required
• security.protocol: protocol, that will be used for verification. Use SSL or leave blank

Example configuration

advertised.listeners: PLAINTEXT://kafka:9092,SSL://kafka:9093
ssl.keystore.filename: kafka-keystore.jks
ssl.keystore.password: kafka-keystore-creds
ssl.key.password: changeit
ssl.truststore.location: kafka-truststore.jks
ssl.truststore.password changeit
security.inter.broker.protocol: PLAINTEXT
ssl.client.auth: 'required'
security.protocol: SSL

Docker configuration

We need to configure same things as in service configuration, but for Docker we use env variables. These variables correspond
to fields in service, but they are uppercase, use instead of . and have prefix KAFKA.

• KAFKA_ADVERTISED_LISTENERS: configuration where Kafka broker listens for client connections, recommended value: PLAINTEXT://kafka3:9092,SSL://kafka3:9093
• KAFKA_SSL_KEYSTORE_FILENAME: filename/location of the key store, example value: kafka_keystore.jks
• KAFKA_SSL_KEYSTORE_CREDENTIALS: filename/location of keystore file, example value: kafka-keystore-creds
• KAFKA_SSL_KEY_CREDENTIALS: filename/location of key credential file, example value: kafka-key-creds
• KAFKA_SSL_TRUSTSTORE_FILENAME: filename/location of the trust store, example value: kafka_truststore.jks
• KAFKA_SSL_TRUSTSTORE_CREDENTIALS: filename/location of truststore file, example value: kafka-truststore-creds
• KAFKA_SECURITY_INTER_BROKER_PROTOCOL: protocol, that will be used for communication between Kafka brokers, for local and dev
  deployment PLAINTEXT is recommended. For production deployment: SSL
• KAFKA_SSL_CLIENT_AUTH: this field enables/disables client authorization on broker's side. To enable, set value to: required
• KAFKA_SECURITY_PROTOCOL: protocol, that will be used for verification. Use SSL or leave blank

Example configuration

KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,SSL://kafka:9093
KAFKA_SSL_KEYSTORE_FILENAME: kafka-keystore.jks
KAFKA_SSL_KEYSTORE_CREDENTIALS: kafka-keystore-creds
KAFKA_SSL_KEY_CREDENTIALS: kafka-key-creds
KAFKA_SSL_TRUSTSTORE_FILENAME: kafka-truststore.jks
KAFKA_SSL_TRUSTSTORE_CREDENTIALS: kafka-truststore-creds
KAFKA_SECURITY_INTER_BROKER_PROTOCOL: PLAINTEXT
KAFKA_SSL_CLIENT_AUTH: 'required'
KAFKA_SECURITY_PROTOCOL: SSL

Local

We will use Kafka with SSL in Docker, because we need to do some changes in configuration and configuring through
Docker compose is the easiest option. In sample-hub-instance directory is located sample
docker-compose. But to use it, we need to generate keystores and truststores
for both brokers and client (our application).

Generating keystores and truststores

To make this process less painful, we have script that helps this
process. Script is used like this:

./generate-stores.sh KEY_ALIAS TARGET_KEYSTORE.jks TARGET_TRUSTSTORE.jks
Where:

• KEY_ALIAS is alias, that the key will have in keystore and truststore with suffix public
• TARGET_KEYSTORE is location of the keystore, you want to add key to. If keystore doesn't exist it will be created.
• TARGET_TRUSTSTORE is location of the truststore, you want to add public part of key to. If truststore doesn't exist it will be created.

Script uses Java's keytool, so all interaction in script is handled by keytool.

Script's workflow is as follows:

1. Key generation - you will be prompted for keystore's password (twice if keystore doesn't exist yet)
2. Public key extraction - you will be promoted for keystore's password
3. Public key import to trust store - you will be promoted for trust store's password (twice if trust store doesn't exist yet)