Timeline Sequences Data [VE OG]
Data Handling explained how a project is persisted in VidiEditor. It leaves out a detailed explanation of how a timeline with all its different options such as effects, transitions, tracks and other options is modelled and persisted. This article will be focus of this section explaining the different options in detail.
Please note that an object model is implemented that deals with the timeline objects in a flexible and interactive way. This section itself concentrates on how the data looks like that is persisted and can be used to exchange timelines.
Overview of timeline data in VidiEditor
As explained in Data Handling the timeline description (EDL) is stored as metadata on the a projects library. Anyway there are 2 different kind of timeline descriptions used.
VidiEditor Timeline Data Model
This is a an EDL stored as JSON describing all necessary parameter to visualize the timeline on the users screen graphically and to render the timelines media in the timeline when using web render engine. It is also used in MediaPortal when transforming a VidiEditor timeline to an Adobe Premiere Sequence.
The Model itself will be discussed in deeper detail below.
VidiCore Sequence Document
This is an EDL usually stored as XML that contains needed information for conforming a timeline using the VidiCoder and controlling the conform output with all the different possibilities VidiVore offers. It is described in detail under: https://apidoc.vidispine.com/5.7/ref/item/sequence.html?highlight=item%20sequence and will not explained here again.
Following figure gives a rough overview what is used where:
Please note that above explanation leaves out VidiFlow. It is not of interest as in case of using VidiFlow workflows for publish also, the flow is the same with the only difference that the Sequence Document is handed over in the workflow first controlling then, the same conforming by a workflow task.
VidiEditor Timeline Data Model (Version 1.0)
Even a small timeline easily results in a JSON that has more then 500 lines. So a complete example will not be presented here. It can be easily generated in VidiEditor while using its log report or EDL export option as described in: Log Report Download and in EDL Import / Export [VE UG].
This section goes through the individual option step by step starting from the highest level and iterating into the document deeper step by step.
Timeline Settings
The top layer describes general settings about the timeline and project used. It has the following elements assuming all sub elements are empty:
{
"version": "1.0",
"duration": 15000,
"id": "ITEM-VX-232937",
"lastOutTime": 1647,
"startTimeCode": "00:00:00:00@PAL",
"title": "timeline",
"videoTracks": [],
"audioTracks": [],
"animationTracks": [],
"frameRate": {}
}
The elements have following meaning:
version
The version of the JSON structure used. This might change if this description of the VidiEditor JSON changes to allow proper versioning of the data model for migrations and compatibility questions. New version might be introduced with new releases allowing new options and might require migrations and adoptions.
Name | Value | Used in VidiEditor | Used in WRE |
---|---|---|---|
version | 1.0 | Yes | Yes |
duration
The total duration of a timeline in absolute frames. Not meaning its the last frame of media in the timeline but the complete duration of the timeline possible.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
duration | 15000 | Yes | No |
id
The Id of the timeline stored in VidiCore so VidiEditor can reference to the correct item object and persist it.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
id | Item-VX-1234 | Yes | No |
lastOutTime
The last visible frame existing on a timeline. This can be different to the total timeline duration presented on the screen.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
lastOutTime | 1647 | Yes | No |
startTimeCode
The first frame of the timeline allowing to add a constant offset to the timeline, where the frame count starts at the given value and will count from this values on. It requires the timebase besides the timecode to calculate the absolute frames accordingly. See VidiCore Timebase notation for information about the given notation.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
startTimeCode | 10:00:00:00@PAL | Yes | No |
title
A title for the timeline free to choose by the user in the GUI.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
title | MyNiceTimeline | Yes | No |
videoTracks
The video tracks where the video and image segments can be arranged on to create the desired media composition. It holds information about the displayed media and various attributes discussed in th following.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
videoTracks | JSON Array | Yes | Yes |
audioTracks
The audio tracks where the audio segments can be arranged on to create the desired media composition. It holds information about the displayed media and various attributes discussed in th following.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
audioTracks | JSON Array | Yes | Yes |
animationTracks
The 1 animation track where that can be used to add the animated graphics and arrange animated graphics on top of the rest of the timeline.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
animationTracks | JSON Array | Yes | No |
frameRate
Stores the frame rate set for the timeline to define the playback timebase as well as the media that can be added to the timeline.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
frameRate | JSON Array | Yes | Yes |
Tracks and Clips
The tracks are an essential element to be able arranging the individual media elements over time and over multiple tracks. There are different track types available called:
videoTracks
audioTracks
animationTracks
But all 3 kinds share the same interface in the object model. So the following parameters are available doesn’t matter for what kind a track is used. Anyway the usage differs depending on the kind of track that will be generated which will be explained for the individual parameters.
A basic example looks as follows:
"videoTracks": [
{
"level": 1,
"mute": false,
"clips": [
{
"clipDuration": 3001,
"clipSourceIn": 0,
"id": "ITEM-VX-232143",
"in": 1153,
"itemDuration": 3001,
"itemUniqueId": "59576323-7f5c-d1d4-1d11-027b57c0cc52",
"linkId": "92b43475-ec56-4509-9e48-a4d4eb021ed2",
"local": false,
"mediaType": "Video",
"name": "",
"out": 1647,
"path": "s3://team-vidieditorbucket-s3/voiceover/FILE-VX-14889.mxf?endpoint=s3.eu-central-1.amazonaws.com®ion=eu-central-1",
"sourceIn": 1153,
"audioMapping": [],
"effects": [],
"sourceOut": 1647,
"isSelected": true,
"state": "IDLE",
"trackLevel": 1,
"trackType": 0,
"isAudio": false,
"voiceoverStatus": 1
}
],
"channelMute": [],
"effects": [],
"transitions": [],
"type": 0,
"pseudoClips": [],
"isActive": false,
"isSelected": true,
"isLoading": false
}
]
level
Defines the ordering of the tracks for display in GUI and Preview using WRE.
Value: 1 based
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
level | 1 | Yes | Yes |
mute
Allows to deactivate the track without deleting / adding it. When set to true the playback in WRE will be ignored.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
mute | true / false | Yes | Yes |
clips
The individual media segments used on the video track with various parameters possible .
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
clips | JSON Array | Yes | Yes |
…Tracks.clips.clipDuration
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
clipDuration | 3001 | Yes | Yes |
…Tracks.clips.clipSourceIn
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
clipSourceIn | 0 | Yes | Yes |
…Tracks.clips.id
The ID of the related source media item in VidiCore that is used.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
id | ITEM-VX-1234 | Yes | No |
…Tracks.clips.in
It describes the timecode of the segments in on a timeline, so the position of the first visible frame (video) or first hearable sample (audio) of the medias segment. The value is an absolute frame count.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
in | 0 | Yes | Yes |
…Tracks.clips.itemDuration
The total duration of the source media used on timeline. This is used for example to calculate how much the media can be extended on the timeline (E.g.: on Trimming Segments). The value is an absolute frame count.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
itemDuration | 1127 | Yes | No |
…Tracks.clips.itemUniqueID
A unique ID used to identify each timeline segment individually and explicitly. The value must be a GUID.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
itemUniqueID |
| Yes | No |
…Tracks.clips.linkID
A unique ID used to identify the possible linking between audio and video segments. The value must be a GUID.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
linkID |
| Yes | No |
…Tracks.clips.local
A boolean flag describing if the media is available on a central storage or only on a local PC. This is used for Voice Over recordings but might be used additionally in future for local media files.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
local | true / false | Yes | No |
…Tracks.clips.mediaType
The type defines what type of media is used, this can be either video, audio or image depending on the used physical media. Please note that voice Over recordings will be defined as audio while animated graphis are separated via there own track.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
mediaType | Video / Audio / Image | Yes | Yes |
…Tracks.clips.name
A String attribute that can be used for naming the segment, for example VidiEditor uses this to blend in a configurable metadata field to the GUI.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
name | MyTitle | Yes | No |
…Tracks.clips.out
It describes the timecode of the segments out on a timeline, so the position of the last visible frame (video) or last hearable sample (audio) of the medias segment. The value is an absolute frame count.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
out | 557 | Yes | Yes |
…Tracks.clips.path
This holds the path information to a physical file to be referenced for the clip/segment. Depending on the use case this can hold different file path informations. Usually it is used in the context of VidiEditor to reference the file path of the proxy file to be streamed to the client by VidiStream and to be rendered using WRE.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
path | s3://{Bucket}/{File}?{params} | Yes | Yes |
…Tracks.clips.sourceIn
The sourceIn is used to reference the in point of the source media that is used on the timelines clip in position. So the sourceIn is the absolute frame of the media to be presented at the in position of the segment / clip on timeline.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
sourceIn | 570 | Yes | Yes |
…Tracks.clips.audioMapping
A JSON element used for mapping audio channels of clips / segments of mediaType audio. The options will be explained below in an own section.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
audioMapping | JSON Array | Yes | Yes |
…Tracks.clips.effects
A JSON element used for storing informations of effect parameters per clip / segment that will be presented in the playback and GUI. The individual options will be explained in an own section.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
effects | 570 | Yes | Yes |
…Tracks.clips.sourceOut
The sourceOut is used to reference the out point of the source media that is used on the timelines clip out position. So the sourceOut is the absolute frame of the media to be presented at the out position of the segment / clip on timeline.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
sourceOut | 1127 | Yes | Yes |
…Tracks.clips.isSelected
A boolean flag describing if the clip / segment is currently selected or not.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
isSelected | true / false | Yes | No |
…Tracks.clips.state
The state usable to define the current state of a clip / segment on timeline. At this time 2 states are defined as clip is either static positioned on timeline (IDLE) or in ove currently by the user (MOVE).
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
state | IDLE / MOVE | Yes | No |
…Tracks.clips.groupID
A unique identifier as GUID that can bind different clips together. It is used for multiselecting different clips / segments in one go.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
groupID |
| Yes | No |
…Tracks.clips.trackLevel
Define which track of current clip locate to. Same as Level
Value: 1 based
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
trackLevel | 1 | Yes | No |
…Tracks.clips.trackType
This defines the the kind of track to be of value 0 - video, 1 - audio or 2 - animated graphics.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
trackType | 0 | Yes | No |
…Tracks.clips.isAudio
This boolean flag defines if a track is usable for audio or not. It can be helpful having this explicitly set beside trackType when using the interface.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
isAudio | true / false | Yes | No |
…Tracks.clips.voiceoverStatus
A specific state for voice over status determining the status of a voicer over clip / segment. This is used to identify if the clip is local, centrally available or pending. The values are 0 - pending/ingesting, 1 - centrally ingested, 2 - local.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
voiceoverStatus | 0 | Yes | No |
…Tracks.channelMute
This defines if a dedicated audio channel is muted which differs from the “mute” option used for complete tracks and which is explained above. It can hold to boolean value defining the state per channel.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
channelMute | [true, false] | Yes | Yes |
…Tracks.effects
This defines effects being applied to a track. In current implementation this is only used for audio leveling on a track while all other effects are handled on clip level. It might be usable when needing more effects are on track level.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
effects | [] | Yes | Yes |
…Tracks.transitions
A JSON element holding the needed informations about transitions used on or in between clips / segments. It will be explained in detail below
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
transitions | [] | Yes | Yes |
…Tracks.type
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
type | [] | Yes | Yes |
…Tracks.pseudoClips
A specific kind of clips / segments not working as media clips / segments. It is used for gaps between timeline segments being stored under this JSON element.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
pseudoClips | [] | Yes | Yes |
…Tracks.isActive
A boolean flag only used for audio tracks holding voice over data. The state describes if the track is currently in recording state or not. If not it acts like a normal audio playback track.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
isActive | true / false | Yes | No |
…Tracks.isSelected
A boolean flag that defines if the track itself is selected or not.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
isSelected | true / false | Yes | No |
…Tracks.isLoading
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
is Loading | true / false | Yes | No |
…Tracks.channelCount
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
channelCount | 2 | Yes | Yes |
…Tracks.channelMute
This defines if channels of a track are muted with 2 boolean flags inside an array where the 1st parameter defines for the first channel of the track if it is muted or not and the 2nd one for channel 2.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
level | [true false] | Yes | Yes |
Tracks.Effects
This can hold effects parameters on a track level. This is similar to effects on a clip / segment level with the difference that only audio leveling is used at this point of time for track effects and keyframes ar not supported on a track effect.
type = gain
This effect allows to change the level of an audio signal while controlling its gain. It allows to set a gain factor that defines how much the level will be adjusted. A gain of 1 means no adjustment while values between 0 and 1 lower the signal and values above 1 bring up the signal.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
gain |
CODE
| Yes | Yes |
Tracks.Clips.Effects
As discussed above video and audio effects can be applied on clip / segment level. This section explains the individual settings possible. In general it can be said that following effects are possible on a clip / segment level:
Video
Opacity
Scale
Rotation
Position
Blur
Mosaic
Audio
Leveling
An example looks as follows:
"effects": [
{
"type": "opacity",
"opacity": {
"default": 0.97
}
},
{
"type": "scale",
"link": true,
"horiz": {
"default": 1.04
},
"vert": {
"default": 1.04
}
},
{
"type": "rotation",
"angle": {
"default": 6
}
},
{
"type": "position",
"y": {
"default": 0.07
},
"x": {
"default": -0.08
}
},
{
"type": "blur",
"id": "252e7408-3b33-4be6-8b30-3d8b624bda34",
"blurriness": {
"default": 0
},
"x": {
"keyFrames": [
{
"position": 478,
"type": "linear",
"value": 0.28
},
{
"position": 580,
"type": "linear",
"value": 0.71
}
]
},
"y": {
"default": 0.01
},
"h": {
"default": 0.55
},
"w": {
"default": 0.37
},
"maskType": {
"default": 1
},
"border": {
"default": 2
}
}
]
type = opacity
Defines the opacity of a clip in percentage from 0 to 100.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
opacity |
CODE
| Yes | Yes |
type = scale
Defines the scaling of of a clip between where 0 is the minimum limit value resulting in a non visible frame while the maximum value can be in theory of any number. Anyway the usage is limited to a maximum border of 200 in VidiEditor for better usage. 100 means a scaling being exactly in the dimensions of the target (E.g. the player dimensions). Note also that horizontal and vertical scaling can be adjusted independently to stretch or squeeze the video.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
scale |
CODE
| Yes | Yes |
type = rotation
Defines the rotation of the video in degrees. This basically is possible in infinite range of values resulting in repeating rotation. Anyway in VidiEditor the usage is limited in the range from 0 to 360 degrees.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
rotation |
CODE
| Yes | Yes |
type = position
Defines the positioning of of a clip between -1 and +1 where 0 for vertical and horizontal scaling means the video is in center of the player while -1 / +1 means the video is positioned exactly outside of the visible area.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
position |
CODE
| Yes | Yes |
type = blur
Adding this effect allows to blur the video picture where 0 means not blurred at all and the max value is set to 100 resulting in a strong blur. Additionally it is possible to set a mask for the blur area to only blur a part of the video. The mask can be either an rectangle or an ellipse. Following parameters exist:
id - GUI to identify the effect that might be added multiple times
blurriness 0 -100
x - the x coordinate / position
y- the y coordinate / position
h - the horizontal dimension
w - the vertical dimension or width
maskType - Either 0 for rectangle or 1 for ellipse
border - Integer value defining strength in pixels
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
blur |
CODE
| Yes | Yes |
type = mosaic
Adding this effect allows to pixelate the video picture in a mosaic kind of style where the horizBlocks and vertBlocks define the strength of the visual effect in number of blocks. Additionally it is possible to set a mask for the mosaic area to only pixelate a part of the video. The mask can be either an rectangle or an ellipse. Following parameters exist:
id - GUI to identify the effect that might be added multiple times
horizBlocks 0 -200 (number of blocks to be displayed horizontal)
vertBlocks 0 -200 (number of blocks to be displayed vertical)
x - the x coordinate / position
y- the y coordinate / position
h - the horizontal dimension
w - the vertical dimension or width
maskType - Either 0 for rectangle or 1 for ellipse
border - Integer value defining strength in pixels
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
mosaic |
CODE
| Yes | Yes |
type = gain
This effect allows to change the level of an audio signal while controlling its gain. It allows to set a gain factor that defines how much the level will be adjusted. A gain of 1 means no adjustment while values between 0 and 1 lower the signal and values above 1 bring up the signal.
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
gain |
CODE
| Yes | Yes |
Tracks.Clips.Effects.{Parameter}.keyFrames
Keyframes can be used on various effect parameter to adjust the value in a linear way over time where an array of keyframes can be set to vary the effect value. So when 2 keyframes are defined the effect value will change from the earlier to the later keyframes position in a liner way.
position defines the timecode of the keyframe as absolute frame
type defines ho the value will be changed over time, only linear is supported at this point of time#
value defines the acutal value of the keyframe adjusted parameter for this frame position
Name | Value Example | Used in VidiEditor GUI | Used in WRE |
---|---|---|---|
keyframes |
CODE
| Yes | Yes |