When you use your own cloud storage (S3 or Azure), every file in MediaLab can be stored in different storage types, from fast and always available to low-cost long-term archive. What you can do with a stored item depends on whether it is the source of a file or one of its formats.

Sources and formats

• Source: the original file as it was uploaded. It is the master copy that everything else is made from.
• Formats: versions made from the source, such as an MP4 for playback in the browser, a streaming version, a thumbnail or a lower resolution download. A format can always be made again from the source.
• Same-as-source formats: sometimes the source can be used as a format as is, for example when an uploaded MP4 is already suitable for playback. That format uses the source and has no separate copy of its own, so it always follows the source.

You can see the source and formats of a file, and where and how each is stored, on the file's Storage tab.

Storage types

Each item in your cloud storage (a "blob") has a storage type. MediaLab uses the same four storage types for S3 and Azure, and maps them to the storage classes and access tiers of your provider:

Availability per storage type

On S3, content in cold storage needs a restore before it can be read, just like archive. MediaLab therefore treats S3 cold storage as archived: a file whose source is in S3 cold storage is marked as archived, in the same way as a file whose source is in archive storage (see below). On Azure, cold storage stays immediately available.

With S3 Intelligent-Tiering (INTELLIGENT_TIERING), the storage provider moves content between access tiers (instant, archive and deep archive) by itself, based on how often it is used. Content in the archive or deep archive access tier is treated as archived.

The storage types you can choose from depend on the storage container. Some containers, such as a local archive or a single-tier archive container, only offer archive.

The archive tiers (archive, and on S3 also cold) require the Archiving add-on to be enabled for your account.

What you can do with sources and formats

Why formats can't be moved to archive

Content in archive storage can't be read until it has been brought back from the archive. That is done per file and brings back the file's source; there is no way to bring back a single format. A format in archive storage would therefore stay unusable, and playback, sharing and everything else that uses that format would stop working.

Because formats can always be made again from the source, there is no need to archive them. Archive the file (its source) instead. A stored item that is used by a format can't be moved to a storage type that archives it, also not directly from the blob details: that is archive, and on S3 also cold. Same-as-source formats don't block this, because they follow the source.

What happens when a source moves to archive

When a source is moved to archive (or, on S3, to cold storage), the file is marked as archived as well. Only the source is archived; the formats stay where they are, so the file keeps working:

• Still works: viewing and playback, sharing, downloading formats, and transcription (it uses the MP4 or MP3 format when one is available).
• Not available: downloading the source, pushing the source to a cloud connection, and anything else that needs the original, such as creating new formats.

When the source is back out of archive, the file is fully available again.

Getting a source back from archive

There are two ways to get an archived source back, both started from the file (or the folder):

• Copy from archive makes a new copy of the file from the archived source, in a folder you choose, in a regular storage type. The original stays in the archive, untouched. When you only need the copy for a while, set Delete copy after (from 1 day up to 30 days) to make it a temporary copy: it is moved to the trash automatically at the end of that period, so you don't keep paying for a second copy you no longer need.
• Restore from archive takes the original source out of the archive and moves it back to a regular storage type. The file is no longer archived, so if you want it in the archive again later, it has to be archived again.

Your account can limit Restore from archive to users of a certain level (Restore from archive min. user level); other users with restore rights can still use Copy from archive.

Early deletion fees

Cloud providers charge a minimum storage duration for their archive tiers. Taking content out of such a tier before that period has passed (by restoring it, deleting it or moving it to another tier) is billed as if it had been stored for the full period. These early deletion fees are charged by your storage provider, for example:

Check your provider's current pricing for the exact terms.

Restore from archive removes the content from the archive tier, so it can trigger early deletion fees when the content was archived recently. Copy from archive leaves the archived original in place and avoids that, at the cost of storing a second copy outside the archive. When you only need the source for a while, a temporary copy (with Delete copy after set) is usually the better choice; restore is meant for content that should leave the archive for good.

How long it takes

The storage provider first has to bring the content back, which takes time:

• S3 cold (Glacier Flexible Retrieval): from minutes to about 12 hours.
• S3 archive (Glacier Deep Archive): up to 48 hours.
• Azure archive: several hours.

You are notified about the progress, and the source is available as soon as the copy or restore has finished.

Viewing and managing stored items

Managers can browse everything stored in your cloud containers with the blob viewer:

1. Open the Storage tab on your dashboard.
2. Click View all blobs to see the items of all containers, or View blobs on a container to see only that container. You can also click one of a container's status or storage type rows to open the viewer with that filter already set. In Settings → Cloud containers, the viewer can be opened for all containers from the table header, or for one container through its actions menu.
3. Filter on container, status, storage type, access tier and archived state, or search on a name, ID or UUID.
4. Click an item to see its details. From there you can:
   • see which files and formats use it, and open them in a new window;
   • Verify storage tier: check with the storage provider which storage type the item is actually in. This also works for an item that seems stuck, for example one that is still uploading.
   • Change storage type: move the item to another storage type. Storage types that archive the item are only offered for items that are a file's source. Moving an item out of cool, cold or archive storage before the minimum storage duration can also lead to early deletion fees.

Checks and changes run in the background. Reopen the item after a while to see the result.

Requirements

• Changing storage types requires the cloud storage add-on, and the Allow setting the storage type setting must be enabled for your account.
• The archive tiers (archive, and on S3 also cold), and copying or restoring from archive, require the Archiving add-on.
• Browsing stored items, verifying storage types and changing storage types requires manager rights.
• MediaLab's internal storage has no storage types: verifying or changing the storage type is only available for content in your own S3 or Azure storage.
• An archived item can't be changed directly. Restore the file first.

For developers

Everything above is also available through the API: list stored items with filters on storage type, access tier and archived state, verify an item's storage type, and get or change an item's storage type. See the DFS section of the API documentation.