Import A File Into A New Item [GL IG]

Motivation

File import describes scenarios where a file is copied to a VidiCore-managed storage by an external component and a new VidiCore item shall be created. This applies (at least) to these scenarios:

• feed ingest ingest;

• file upload via a fat client (e.g. MediaIngest’s ControlCenter);

• backend hires processing that create a new item (e.g. fusing, trimming, HDF normalisation, rewrapping).

Uploads via a web frontend or via a VidiFlow check-in API may have to be handled differently. Backend operations that create an additional shape on the existing VidiCore item also require a different handling.

Workflow

As VidiCore items, shapes, components, and files cannot be created independently via the VidiCore API, the VidiCore placeholder import is the way to go (see Placeholder Imports in APIdoc).

The necessary steps for single-file assets (Op1a MXF, self-contained MOV, etc.) are:

1. Create the placeholder item with a shape with one container component, but no further components like audio, video, etc. via POST /API/import/placeholder?container=1  (documentation):

   • this will return the ID of the newly created item;

   • the actual component types and count need not be known in advance, but you need to specify at least one container component.

   • Note: if you want to retrieve the shapes of a placeholder item please specify the placeholder  parameter: GET /API/item/VX-100/shape?placeholder=true.

2. Specify a MetadataDocument as body to provide an initial set of metadata for the placeholder item. Set at least the title metadata field directly with the initial API call for placeholder creation.

3. If available, attach further metadata to the item.

4. The field V3_ExpectedMediaType (string) on item-level should be set to video so that the MediaPortal connector can identify these items as videos even if they don’t have a continuous hires shape, yet.

5. Create the file object via POST /API/storage/<storageId>/file?path=<relativePathOfFileOnStorage>&state=TO_APPEAR&createOnly=true:

   • this is to create the file entity manually and will return the file ID of the newly created file;

   • createOnly=true acts as a safeguard not to create the same physical file twice; if the API call returns an error the import process should stop. Of course, such error scenarios should be avoided at an earlier stage of the ingest workflow for a better user experience.

   • Important: other applications may rely on the existence of a physical file if the file state is OPEN. Keep it on TO_APPEAR if you cannot tell for sure if the file exists.

6. Start writing the file to the storage.

   • It is important to create the file object before starting to write; otherwise the above API would fail as the file object may have already been created by VidiCore.

   • If it is not possible to create the file object properly creating the file (e.g. for some video servers) a possible solution is to define a specific storage with method != NONE in VidiCore. In this case no file has to be written. In order to access the file later an agent has to be developed to retrieve the file from the video server. The relativePathOfFileOnStorage Parameter will point to the item on the video server.

7. When you are 100% sure that the physical file exists on the storage change the state of file object to OPEN via PUT /API/storage/file/<fileId>/state/OPEN. Important: other applications may rely on the existence of a physical file if the file state is OPEN. Keep it on TO_APPEAR if you cannot determine for sure if the file exists.

   • If the file is suitable for processing while still being written, fill the placeholder via POST /API/import/placeholder/<itemId>/container?fileId=<fileId>.

   • VidiCore will start an import job on the item and will return its job ID.

   • The import job will extract technical metadata from the file, create the appropriate components on the placeholder shape, and attach the technical metadata to the shape.

   • To automatically create a proxy video for the item specify the desired shaped tag in the tag parameter of the above call and add the destination storage as described in Transcoding With VidiCoder [GL IG].
     For MXF files you can assume that they can be processed while being written, esp. if the MXF files have a ClosedComplete header. Files in an mp4 container usually are not suitable for processing while being written.

8. When file is copied completely, change the state of file object to CLOSED via PUT /API/storage/file/<fileId>/state/CLOSED.

   • This is not mandatory as VidiCore will detect the closed file itself if the browse flag is set to true on the storage, but it will speed up the ingest process and thus should always be done.

9. If your file cannot be processed while being written then fill the placeholder via POST /API/import/placeholder/<itemId>/container?fileId=<fileId> after the file was set to CLOSED.

   • VidiCore will start an import job on the item and will return its job ID.

   • The import job will extract technical metadata from the file, create the appropriate components on the placeholder shape, and will attach the technical metadata to the shape.

   • To automatically create a proxy video for the item specify the desired shaped tag in the tag parameter of the above call and add the destination storage as described in Transcoding With VidiCoder [GL IG].

10. If you’ve started this import job while the file still was being written, we recommend to run a shape deduction job on the newly created shape: POST /API/item/<itemId>/shape/<shapeId>/update.

    • VidiCore will start a shape deduction job on the shape and will return its job ID.

    • The shape deduction job will extract technical metadata from the file, update the appropriate components on the shape, and update the technical metadata to the shape.