Sending Notifications to a Kafka server

Owned by Eli Pielet (Writec) (Unlicensed)
Last updated: 09 Oct 2023 by Alon Men
2 min read

ZEN Master can export event notifications and SCTE 35 messages to an Apache Kafka server. Kafka is a data streaming platform that allows you to publish and subscribe to streams of data, store them, and process them. ZEN Master is a “publisher” that uses the Kafka Producer API to send event data to a Kafka server. Once the event data from ZEN Master is streaming to a Kafka server, connectors are available to pull data from the Kafka server into third party applications and services including Elasticsearch, Splunk, Azure Event Hubs, AWS CloudWatch, AWS S3, and more.

Kafka integration operates with admin privileges, so that all events that are logged in the system will be sent to Kafka regardless of Tags. You can stop events from being sent to Kafka by disabling the Events checkbox in the configuration. Also, if you disable logging for certain types of events on certain objects (such as a Source), those events will no longer be written to the ZEN Master Events database or sent to Kafka servers.


To add a new Kafka configuration:

  1. Go to Configuration > Notifications.
  2. Click on the Kafka Configuration tab.
  3. Click + Add.
    The Create New Kafka Configuration window is displayed.
  4. In the Name field, enter a name for the Kafka Configuration.
  5. In the Servers field, enter one or more of your Kafka servers, using the following syntax: Kafka broker/host combination delimited by comma, e.g. kafka-1.us-east-1.myapp.com:9093,kafka-2.us-east-1.myapp.com:9093.

  6. In the Topic field, enter the Topic created on the Kafka server where messages will go.

    The Topic needs to be pre-configured in the Kafka server. If the topic hasn't been created in Kafka, the messages won't be delivered and you will receive an error message.

  7. In the Username field, enter your username for the Kafka account.
  8. In the Password field, enter your password for the Kafka account.
  9. If you would like to send Event notifications to the Kafka server, select the Events checkbox.
  10. If you would like to send SCTE35 messages to the Kafka server, select the SCTE35 checkbox.
  11. Click Save.
  12. If you would like to test the server connection, in the Kafka Configuration tab, under Actions in the row of the new server, click on the Test icon. A pop-up window will appear, stating if the server has connected successfully or not. You can also test the server connection by checking if the notification was received on the Kafka server.