VidiFlow Release Notes [VF RN]

The following items on the list encompass breaking changes, features, and fixes that are relevant for the major release.

Release 23.1

Setup and Operations

Dependencies and Component Updates

The following central components were updated and will be shipped in the noted version, along with relevant product and service dependencies:

Features

New Services, Agents and Tasks

Copy Chunked Files Via Agent

A new task was introduced to allow copying of growing files to and from cloud storages like S3 or GCP. The agent uses a chunking mechanism that works transparently in all Vidispine products. After the complete file has been copied into chunks, the file will automatically be consolidated into an MXF file. All temporary files will get a deletion lock that expires after a timespan to be configured in the TranscoderS4M configuration use case page.

Support for partial restore on cloud storages (like S3)

Several VidiFlow tasks are updated to work on cloud storages like S3 in the same way as on local storages. This includes, among others, the task for MXF index creation and the Trim MXF task.

Support of Google Cloud Storage

The Vidiflow tasks processing media now support Google Cloud Storage.

Workflow API: New call: POST message

In the workflow API, a new call is available. This allows sending messages to a Camunda workflow. This can be used to pause a workflow and wait for a message to be sent from an external system.

List of Updated Tasks

Housekeeping service and workflow

The housekeeping service and workflows were adapted to react to new notifications from VidiCore passing the child IDs if deletion locks for expired collections. This allows triggering the deletion of the contained items or sub-collections in a more efficient way.

Workflow Monitor 

Manual resync of workflow index

It may happen that the workflow index for a workflow is not up to date. To repair this, it is possible now to select a single workflow in the Workflow Monitor and trigger a manual resync for it.

Show instance ID and relation ID in the details view

The instance id and relation id of each workflow will now be shown in the details view.

State FAILED removed from the workflow status filter

The deprecated workflow status “FAILED” was removed from the filter for workflow states.

Adapted default column size

The default column size of different columns was changed to allow a better user experience for new installations or with cleared browser cache.

VidiFlow Portal

Support of deep links for items, collections, and libraries

It is now possible to call the VidiFlow Portal with a URI deep linking to an item, collection, or library. This allows integration in 3rd party monitoring applications.

Performance, stabilization, and other improvements

Workflow indexing performance improvements

The workflow index represents the state of all Camunda workflows plus additional information to be shown in the Workflow Monitor. The index is updated using a message queue.

In high-load scenarios, it could happen that all current workflows in the Workflows Monitor were not shown or up to date because processing these messages took longer than required.

To avoid or at least reduce this effect, the workflows index update messages are now split according to priorities

• Priority 1: workflows status updates

• Priority 2: task status updates

• Priority 3: all other updates

This should lead to the situation that priority 1 messages are processed very close to real-time, whereas other messages may be delayed. In the Workflow Monitor all workflows should be shown with the correct workflow status, but other information like “steps done” or “steps in progress” may be delayed.

To allow monitoring of the queue behavior, two new metrics were introduced:

• Outdated Counter: Counter of messages (with lower priority) which were ignored as the final workflows state was already achieved.

• Ignored Counter: Count of messages where no workflow was available in the index (should always be 0).

Camunda workflow metrics deletion

The time to live for Camunda metrics can now be configured separately by the configuration parameter “CamundaMetricsHistoryCleanupInDays” (Default 1 day).

Script to change Camunda workflow time-to-live for dedicated workflows

A new script is provided to allow setting the time-to-live for dedicated workflows filtered by name von version to new values. This can be used e.g. in migration scenarios, where a very high amount of migration workflows may slow down the system.

Batch thumbnail processing in Transcoder S4M

The Transcoder S4M posts thumbnails now in batches to VidiCore to reduce the API load. The tasks use batches of 50 thumbnails or posts all collected thumbnails every 30 seconds, depending on which limit has been reached first.

Password encryption

Different passwords to 3rd party systems are now encrypted in the VidiFlow deployment using the ConfigPortal decrypt API.

In other scenarios, passwords were moved from a config map into Kubernetes secrets.

Improved logging for media agents

The readability and quality of log files for media agents have been improved in different situations.

Standard settings for transcoder modules

For new installations, VidiFlow deploys default values to the use case page “TranscoderS4M\Config” in ConfigPortal.

Fixes