What we are working on: Generative AI Meets Video Editing - 30/11/2023
By Ralf Jansen and Ulrich Ening
Published on
Clever Cutting Makes for Smart Editing
With generative AI on everyone's lips, we would like to draw attention to some of the topics we are working on regarding the matter.
We would like to introduce you to a work-in-progress feature we are dubbing “The AutoCut”. It uses a large language model to find relevant moments along with statements within a video.
Although this in and of itself may not necessarily cause a wow effect, the magic happens when integrating this algorithm into a tool chain consisting of a search portal for media professionals such as (MediaPortal) coupled together with a browser-based video editor (VidiEditor). Employed in this context, it would allow a video’s most relevant sequences to be edited in no time.
Walk me through it
A typical workflow would involve a media professional browsing and picking their media through MediaPortal’s refined search functions.
Picking a video in MediaPortal
On every media item, in this case a video, the user can start configured workflows, which in this case is a workflow called “OpenAI AutoCut”. The user can then specify how many relevant moments they would like to have proposed.
Preparing the AutoCut and adding intuition
Additionally, the user can define related term(s) to add intuition to the algorithm to focus on. Furthermore, the language of the reasoning is configurable. This makes it particularly interesting if the video’s contained language is not spoken.
As shown below, all relevant and time accurate moments of the video – a press conference in this case – can be viewed and navigated in MediaPortal as well.
Generated segments in MediaPortal resulting from the AutoCut
And here comes the benefit of automation. Handing over these snippets to an integrated VideoEditor works like a charm. When carried over these are linked after another, segment after segment to create a pre-cut timeline along with all of the metadata generated by the AutoCut.
Generated segments in MediaPortal are carried over into VidiEditor to generate a timeline
Want to know more?
Want to know more about AutoCut or VidiNet? Drop us a mail at hello@vidispine.com.
About the Authors
Ralf Jansen
Media technology savant meets code cruncher meets passionate ping pong player. Ralf Jansen is a Senior Software Architect & Innovation Lead at Arvato Systems and based on the right side of the Rhine, Cologne.
Ulrich Ening
Product Manager at Vidispine. He has a background as a Broadcast and IT specialist, with years of experience in projects across Europe, North America, and the Middle East, working in roles as an Engineer and Solution Architect. Since 2019, he has been leading Vidispine's MediaPortal team from Cologne.