Traffic Gateway - Next Generation

The old Traffic Gateway generation had a number of limitations:

Despite the fact that Cinegy tried to support the most common industry BXF format, we had to polish and adapt parser for each vendor due to small changes in implementation and workflow. Other option was a simple, universal Cinegy CSV input format that is widely used though requiring translation from the customer format.

In most cases customers control several channels at once, which requires different settings, content and graphics.

The generated playlists are put to the same folder which requires the post-processing actions.

Often customers need some business rules and extra logic that must be incorporated into the playlist generation that was not easy to do in old fixed workflow. At the same time, old Cinegy Traffic Gateway was simple enough to set up and maintain.

Therefore, we have built a more flexible, extendable but still simple brand-new Cinegy Traffic Gateway.

It runs natively on Windows, Linux and also fully supports containerization, e.g. with Docker.

The configuration file is stored in a JSON format for ease of use and automatic modifications. The same configuration file can be accessed via REST API:

Cinegy Traffic Gateway has built-in machine-readable logging and telemetry enabled on all stages of processing, leaving no "dark places to hide". Categories and message priorities allow easy filtering and automatic processing of the generated logs.

Cinegy Traffic Gateway can run in fully automated mode, being triggered by watch folders, and store generated playlists in the specified folders. It also can trigger playlist processing in response to the REST API call. The information about the corresponding task status can also be requested via REST API.

Cinegy Traffic Gateway was re-architectured to expose its functionality as re-usable modules, which simplifies custom business logic implementation, where it is required.

For example, SMPTE BXF, Cinegy Media Asset Management and Cinegy MCR modules provide a list of helpful commands to manipulate corresponding business entities. Commands for creating a new playlist and filling it with Programs, Blocks and Items make PowerShell business logic scripts creation an easy and less error prone task.

The last but not the least, Cinegy Traffic Gateway natively supports multiple channels with different settings and workflows.

The number of stages of the playlist processing may be customized. Moreover, each stage may be enclosed by pre- and post-processing events. As a common format, the Cinegy Air playlist is used on all stages which unifies the data transferring from start to finish.

The general workflow can be triggered by sending a new task for processing via REST API or via configured Watch folders. Cinegy Traffic Gateway supports multiple simultaneous tasks execution and provides reports over the queue status and currently active tasks via REST API. The final result of the task processing can also be obtained via REST API or stored as a playlist in the specified folder.

When the task is picked up from the queue, its processing starts from the input handler. It is responsible for the initial transformation of the input playlist format into Cinegy Air Playlist one, that is used as a native internal pipeline format. The input handler takes SMPTE BXF, Cinegy CSV or other custom format and performs the initial conversion. In case the playlist contains all the required information, the processing can end immediately.

However, additional business logic is involved usually in order to fill in the commercial blocks with the paid Ads provided by the external system, for example, or to resolve the required media locations from different sources (Cinegy Archive, Amazon S3, local library, etc.).

The final step is storying the complete playlist on the network share or to load it directly into Cinegy Air via API.

Each processing step is independent from the others. The data is passed from one stage to the next one preserving the changes made. This allows to repeat the corresponding step in case of the processing errors in future versions of Cinegy Traffic Gateway.

The processing pipeline is flexible and can include any number of processing steps. Every step also has an option to execute the PowerShell script before and after processing, allowing further extension and custom logic to be executed.

The final processing step usually stores the generated playlist into specified network shares. Alternatively, the processing result for the corresponding task can be retrieved via Cinegy Traffic Gateway REST API.

Here is the list of workflow steps to follow during our demo:

We start our demo with creating a playlist draft in Cinegy Air Playlist Editor. This is actually a playlist template consisting of programs which are divided into blocks, containing dummy items. Let’s assume we have created a basic playlist structure based on a planned events list for the next day. We know what programs are scheduled for the broadcast at specific times and general program structure (logical blocks of the broadcast). So we compose our playlist from the dummy items with the default duration, specify the exact start times for the specific items and add empty ads blocks to the schedule – they are expected to be filled automatically at later stages.

Then the corresponding playlist will be stored in the folder monitored by Cinegy Traffic Gateway to start the processing. The Cinegy Traffic Gateway watch folder picks up a new file, checks the channel that this playlist belongs to and puts a new task into the queue. Another option here could be adding a new task via Traffic Gateway REST API by some 3rd party system.

According to the channel configuration, the first step is filling in the empty commercial blocks with actual advertisement clips. Usually, this requires the external business logic, for example, REST Service that receives commercial block ID, planned duration and date/time of the planned playback. The service provides a list of actual advertisement clips that are inserted in corresponding blocks.

In the result of this step execution the empty ads blocks are filled in with the IDs of the required items to be broadcast.

Now the playlist is prepared to be filled in by real media based on the specified material traffic IDs. The first configured provider is Cinegy Archive. For each item in the playlist the Cinegy Archive database is scanned for the Traffic ID. When a corresponding clip or a Sequence is found, its metadata is inserted into the playlist replacing the dummy item with real media items, including updated item duration.

The process is repeated for each dummy item in the playlist.

Now the playlist is prepared and filled in with references to the real media items in Cinegy Archive. Dummy items have been replaced with the actual metadata.

For the purpose of the demo, we have intentionally left several items in the playlist unresolved by Cinegy Archive. These items will be resolved by media from dedicated Amazon S3 buckets.

We have prepared three Amazon S3 buckets with additional media that will be used in the playlist. The missing media items are replaced with links to the object storage and the playlist is ready to be generated.

Each of the three pre-configured Amazon S3 buckets are scanned for the media. The process continues until the media is resolved or no more media sources are left.

When the playlist is ready and filled in with the links to the real media, it is stored in the other watch folder monitored by the Cinegy Air control application.

Once the Cinegy Air control application detects the new playlist availability and verifies its validity, it is automatically appended to the active playlist.

While Traffic Gateway can handle most of the common requirements, in some cases integration with the 3rd party systems is required - when the workflow becomes complex and involves other services, the probability of unexpected execution results and delays increases greatly.

In such case, for example, for enterprise level orchestration and workflow control, Apache Airflow can be used. It comes with a lot of handy features like graphical UI that helps to understand the workflow execution history, advanced logging to trace every single detail of the process, automatic step repetition and failure control and many more.

Apache Airflow also gives you more control on the workflow graph where you can build a simple, advanced and even sophisticated graphs (hopefully you will never need something like this in your regular broadcast production).

When systems like Apache Airflow are in use, existing Cinegy Traffic Gateway modules can be integrated into the corresponding 3rd party system with minimal adaptations required.

Our wonderful Cinegy Professional Services team is always ready to help with installing and running it.

So, what may help you deciding which approach to take – standalone Cinegy Traffic Gateway or enterprise orchestration system with Cinegy Traffic Gateway modules?

As an approximation you can consider assessing the following requirements:

Installation size – Cinegy Traffic Gateway is the most suitable for the small and medium size installations. When the number of broadcast channels grow there is a limit when operator can conveniently control and maintain them within the simple system. Apache Airflow can help you keeping a large number of channels under your control without enormous efforts.

Processing scale – consider the number of playlist processing transactions during working hours. More tasks to be completed – more chances something goes wrong and will require operator assistance.

Business logic – in case your workflow involves several 3rd party systems and complex decision making during the playlist creation it worth considering Apache Airflow path to keep an eye on all processing steps on the go.

Complexity – define your goals. Either you are under time pressure and should be running ASAP or you are ready to spend some of your precious time planning and implementing large system with custom integrations that should work flawlessly under heavy load.

Redundancy – Cinegy Traffic Gateway will happily get you through most of the workflows and some basic failures. Although in case you expect the system to self-diagnosis and self-healing when possible – consider designing your system with the help of our Professional Services team and Apache Airflow as a background.

Now you have enough information to plan your next steps with Cinegy products and specifically Cinegy Traffic Gateway for your next small and large scale projects. In case of any doubt, please feel free to contact our Professional Services team to help you reaching your valuable goals in the most efficient way.