Third Party Retry

Communication with third-party services is encapsulated in an internal controller, which consists of two modules: ThirdPartyControl and ThirdPartyRetryControl.

ThirdPartyControl

This module is designed to communicate with external service. It uses adapters to interact with different third-party services, executing tasks based on the provided data. The module is controlled by a gRPC service and includes methods for pinging services, notifying them, and creating objects.

Failing Tasks are inserted in an internal database table, see Third Party Retry | Database-table-for-retries.

Adapters

Adapters act as intermediaries that translate and execute tasks between VidiControl and external services. They ensure that the data and commands sent to external services are in the correct format and understood by those services.

Two adapters are integrated in VidiControl:

1. VidiflowWorkflowAdapter:

   • Starts a workflow in VidiFlow without waiting for the result.

   • Input is a JSON with the workflow name, version, and parameters.

2. VidiCoreAdapter:

   • Manages objects in VidiCore.

   • Can create, modify, and prepare deletion of booking-related objects.

ThirdPartyRetryControl

This module manages retries for failed tasks. It periodically checks the database table for tasks that need to be retried and triggers the appropriate methods in ThirdPartyControl to attempt the tasks again. It ensures that tasks are retried a specified number of times before marking them as failed.

Database table for retries

The ThirdPartyRetry table is used to store information about tasks that need to be retried due to failures in communication with third-party services. It has these columns:

• ThirdPartyRetryId: The unique identifier for each retry entry.

• ServiceName: The name of the external service that was requested.

• Data: The payload of the request that needs to be retried.

• Reason: The reason why the retry is needed.

• TimeInserted: The timestamp when the retry entry was created.

• LastRetry: The timestamp of the last retry attempt.

• RetryCount: The number of retry attempts made so far.

• ThirdPartyMethod: The method in ThirdPartyControl that was called.

  • Notify (0)

  • CreateObjects (1)

  • ModifyObjects (2)

  • DeleteObjects (3)

  • UpdateParent (4)

  • Unknown (99)

• ExecutionState: The current state of the retry entry

  • Pending (0) - Retry can be processed by ThirdPartyRetryControl

  • InProgress (1) - ThirdPartyRetryControl has triggered ThirdPartyControl again and ThirdPartyControl is processing the task

  • Error (3) - task finally failed, no more retries (e.g. max number of retries reached)

• CreationTime: The timestamp when the entry was created.

• ModificationTime: The timestamp when the entry was last modified.

Benefits

1. Enhanced Reliability:
   ThirdPartyControl ensures smooth communication with external services, while ThirdPartyRetryControl manages and retries failed tasks. This combination significantly reduces the chances of permanent failures, ensuring tasks are completed successfully.

2. Improved Error Handling:
   With ThirdPartyRetryControl in place, any errors encountered by ThirdPartyControl are automatically managed and retried. This means that transient issues or temporary outages in external services are handled gracefully without manual intervention.

3. Increased Efficiency:
   ThirdPartyControl efficiently executes tasks with external services, and ThirdPartyRetryControl ensures that any failed tasks are retried promptly. This leads to a more efficient system where tasks are completed faster and more reliably.

4. Automated Recovery:
   ThirdPartyRetryControl provides an automated recovery mechanism for failed tasks. This reduces the need for manual monitoring and intervention, saving time and resources.

5. Consistency in Operations:
   The combination ensures that tasks are consistently retried until they succeed or reach the retry limit. This consistency helps maintain the integrity of operations and data within your system.

6. Operational Continuity:
   Even if a third-party system is temporarily unavailable, the combination of these modules ensures that VidiControl can continue operating normally. Failed tasks are queued for retry, allowing the system to maintain its workflow without significant disruption.

Configuration

The following Config Values are used to configure Third Party Retry:

• ThirdPartyCreateVidiCoreCollectionsHidden
  True if collections shall be created hidden for the media portal.

• ThirdPartyCreateVidiCoreItemsHidden
  True if items shall be created hidden for the media portal.

• ThirdPartyGrpcDeadlineInSeconds
  Timeout for GRPC call to ThirdParty Grpc service. No global setting, must be set on each request. Default is 0 (no timeout).

• ThirdPartyMaxNumberOfRetries
  Retry count for the ThirdParty Connector (VidiCore).

• ThirdPartyRetryControlDbCheckIntervalInSeconds
  ThirdPartyRetryControl will check the DB every x seconds.

• ThirdPartyRetryIntervalInSeconds
  ThirdPartyRetryControl processes retry entries with last try at least x seconds ago.

• ThirdPartyVidiCoreChunkItemLockIntervalInHours
  Length of the item lock put on created items in VidiCore.

Please refer to Config Values for more details.