Cloudinary Blog

Migrating your Media Assets to the Cloud Using Cloudinary

Easily Migrate your Media Assets to the Cloud with Cloudinary

When analyzing use of website or mobile application storage, there’s no doubt that media files, such as video clips and images, use the most space. Migrating these files to the cloud and storing them in a location where they are available online makes perfect sense, but images and videos often require additional transformations before they are delivered to end users.

Images need to be transformed to optimal formats, resized and cropped for various aspect ratios, especially if you have a responsive site. In addition, you may want to apply special effects and overlays. Videos, too, need to be optimized for web delivery, transcoded to support different resolution options and streamed using adaptive streaming, as well as making other modifications. Then, when your media files are stored on the cloud, you’ll want fast delivery via content deliver networks (CDNs) to ensure a great user experience.

Cloudinary, a cloud-based service that supports all image and video management needs for websites and mobile apps, delivers all of these capabilities. In order to start using Cloudinary, you first need to set up a free plan to start. Then you will have to migrate your images and videos to your Cloudinary account. Here’s how you get started:

Requirements

The code samples provided here are in Node.js. However Cloudinary publishes SDKs in several languages. You can find the various options listed here.

In case your development language isn’t there, Cloudinary supports the option of uploading images using a direct HTTP call to the API, as explained here.

Image manipulation and delivery also can be implemented by providing the transformation parameters as part of the URL directly.

Installation

  1. Open a Cloudinary free account.
    Go to Cloudinary and click “sign up for free.” Fill out the registration form. Here you can change your default cloud name to your preferred name. In these examples my cloud name is “cld-name.”

  2. Follow the instructions according to the required SDK documentation as listed here.

Migration Planning

Much like when you move to a new apartment, you need to ask yourself, do I need to move all my files or is it a good opportunity to leave some of it behind? To answer those questions, let’s consider several migration options, each fitting with a different scenario.

Migrating all existing content in one phase – This requires uploading all your media files. This option fits one of the following use cases:

  • All of your images and videos are actively used.
  • You intend to shut down your existing media storage.

Implement this migration by creating a script that runs on your media assets and for each file it calls the upload API call. Cloudinary’s upload API call supports uploading files from various sources, including a local path, a remote HTTP or HTTPS URL, a remote S3 URL, a Data URI or an FTP URL.

When uploading a file, you can define your own unique Id (as shown below). If it is not defined, the call will create one randomly. Here is an example of uploading a file from your local computer:

Copy to clipboard
cloudinary.v2.uploader.upload("local_folder/image.jpg", {public_id: "my_image"}, 
function(error, result) { console.log(error,result) });

Lazy migration – Uploading a media file only when it is requested by your website or app user for the first time. This option is effective when you have a long tail of media assets, not all of them are active, and you are not sure which ones are still in use.

Using the Cloudinary management console or the API, you can define a mapping between a folder name in your Cloudinary media library and a base remote URL, which links to your images online folder. For example, if your images are available at: https://example.fileserver.com/media/, the API call shown below will map https://example.fileserver.com/media/ to a Cloudinary folder called: media

Copy to clipboard
cloudinary.v2.api.create_upload_mapping('media',
    { template: "https://example.fileserver.com/media/" },
    function(error, result) { console.log(error,result) });

Now, in order to automatically upload koala.jpg, you need to replace, on your front end, the image URL:

Copy to clipboard
https://example.fileserver.com/media/koala.jpg

With the following URL:

Copy to clipboard
https://res.cloudinary.com/cld-name/image/upload/media/koala.jpg

The first user who calls the Cloudinary URL will trigger an automatic upload of the image to Cloudinary. Any subsequent request for the same image will be delivered via the CDN.

The Cloudinary URL is constructed as follows:

Copy to clipboard
res.cloudinary.com/<cloud-name>/image/upload/<mapped-folder>/<partial-path-of-remote-image>*

The following API call also will return the required URL:

Copy to clipboard
cloudinary.url("media/koala.jpg");

Hybrid approach – Run a script to upload the “hot” group of your most commonly used media assets and use the “lazy migration” option to upload the rest. This option works best when you have a defined subset of your media assets that drives most of your traffic.

Fetch assets – Fetch media assets from remote locations and store them for a predefined period. Use this option when your images and videos originate from various online sources and they are used for a short timespan, as in news items.

For example, the following code is used to deliver a remote image of Jennifer Lawrence fetched by Cloudinary from WikiMedia.

Copy to clipboard
cloudinary.url("http://upload.wikimedia.org/wikipedia/commons/4/46/Jennifer_Lawrence_at_the_83rd_Academy_Awards.jpg", {type: "fetch"});

The equivalent URL is:

Copy to clipboard
https://res.cloudinary.com/cld-name/image/fetch/http://upload.wikimedia.org/wikipedia/commons/4/46/Jennifer_Lawrence_at_the_83rd_Academy_Awards.jpg

Uploading Large Files – If you are uploading files larger than 100MB, there is an option to do a chunked upload:

Copy to clipboard
cloudinary.v2.uploader.upload_large("my_large_image.tiff", 
    { resource_type: "image", chunk_size: 6000000 }, 
    function(error, result) { console.log(error,result) });

Which files to upload?

Cloudinary is able to manipulate images and videos on-the-fly upon request or upon upload, so you only need to upload the highest resolution of one image or video. There is no need to upload large/medium/small variants.

Access Control

If all of your media assets are not public, you can upload them and restrict their availability:

  • Private files – The original files are not available unless accessed via a signed URL. Another option to provide access is creating a time-expired download URL. The following code example uploads the file as private:
Copy to clipboard
cloudinary.v2.uploader.upload("local_folder/private_image.jpg", {type: "private"}, 
function(error, result) { console.log(error,result) });
  • Authenticated files - The original files, as well as their derived ones, are not available unless accessed via a signed URL. For increased security, cookie-based authentication can be setup as well, in order to access them.

  • Whitelisting referral domains – An additional optional security layer that restricts the access to your media assets is to setup a whitelist of referral domains. Then only URL requests arriving from those domains are allowed to access the file.

Setting an Upload Policy Using an Upload Preset

A convenient way to create a centralized upload policy is defining an upload preset. This enables you to define the transformations you would like to do once, then use the preset name to activate it upon upload. You can define several upload presets and use them according to different policies you have, for example watermark all images or transcode a video rendition of 640p wide resolution.

When you define an upload preset, you can set a transformation that will change the original file and then only the transformed file will be stored. This option is called an incoming transformation. You can also define transformations that will be created as derived files, which will be stored in addition to the original file. This process is called an eager transformation. Using Cloudinary, you can transform the images and video on-the-fly, therefore these options are required for cases where you would like to process the transformation immediately upon upload.

As an example, the following code creates an upload preset that adds the tag remote. The unsigned parameter determines if the preset can be used for unsigned uploads, which can be done from the client side without having the API secret. The allowed_formats parameter defines the file formats allowed to be used with this preset.

Copy to clipboard
cloudinary.v2.api.create_upload_preset({ name: "my_preset", 
    unsigned: true, tags: "remote", allowed_formats: "jpg,png" },
    function(error, result) { console.log(error,result) });

The following code uploads an image using this upload preset:

Copy to clipboard
cloudinary.v2.uploader.upload("smiling_man.jpg", 
    { public_id: “smile”, upload_preset: "my_preset" }, 
    function(error, result) { console.log(error,result) });

Upload Response

As a response to the upload call, you get some important information back. The response looks like this:

Copy to clipboard
{ public_id: 'smile',
  version: 1482935950,
  signature: 'bfd5019ee4f513f30226cc06c750b2ad6eccceef',
  width: 1743,
  height: 1307,
  format: 'jpg',
  resource_type: 'image',
  created_at: '2016-12-28T14:39:10Z',
  tags: [],
  bytes: 642781,
  type: 'upload',
  etag: '7c26a0d7b72b6621bba1110da25d099e',
  url: 'https://res.cloudinary.com/hadar-staging/image/upload/v1482935950/smile.jpg',
  secure_url: 'https://res.cloudinary.com/hadar-staging/image/upload/v1482935950/smile.jpg',
  original_filename: 'smiling_man' }

Notification URL

You can tell Cloudinary to notify your application as soon as the upload completes by adding the notification_url parameter to the upload method and setting it to any valid HTTP or HTTPS URL. You also can set the notification_url parameter globally for all uploads on the Upload Settings page in the Cloudinary Management Console, instead of individually for each upload call.

Uploading New Files

Following a successful migration, you need to start uploading all new media assets to Cloudinary. There are several ways to do it: manually via the Cloudinary account console, calling the upload API, or using automatic upload. Another easy way to do this is using the ready-made upload widget.

Upload Widget

Cloudinary's upload widget includes a complete graphical interface. The widget supports a drag and drop functionality, interactive cropping, upload progress indication and thumbnail previews. The widget also monitors and handles uploading errors. The following code example shows how to open the widget:

Copy to clipboard
<script src="//widget.cloudinary.com/global/all.js" type="text/javascript">  
</script>
<script>
  cloudinary.setCloudName('cld-name');
  cloudinary.openUploadWidget({upload_preset: 'my_preset'}, 
  function(error, result) {console.log(error, result)});
</script>

More information regarding the widget functionality is available here.

Summary

The steps detailed above are just the beginning of the journey of moving your media assets to the cloud. Once uploaded, Cloudinary supports a long list of image manipulation options, and the same goes for video. In addition, images can be optimized for faster delivery and support responsive design.

Now is the time to let go, send your media assets to the cloud and set them free.

Recent Blog Posts

Generate Waveform Images from Audio with Cloudinary

This is a reposting of an article written by David Walsh. Check out his blog HERE!
I've been working a lot with visualizations lately, which is a far cry from your normal webpage element interaction coding; you need advanced geometry knowledge, render and performance knowledge, and much more. It's been a great learning experience but it can be challenging and isn't always an interest of all web developers. That's why we use apps and services specializing in complex tasks like Cloudinary: we need it done quickly and by a tool written by an expert.

Read more
Make All Images on Your Website Responsive in 3 Easy Steps

Images are crucial to website performance, but most still don't implement responsive images. It’s not just about fitting an image on the screen, but also making the the image size relatively rational to the device. The srcset and sizes options, which are your best hit are hard to implement. Cloudinary provides an easier way, which we will discuss in this article.

Read more

The Future of Audio and Video on the Web

By Prosper Otemuyiwa
The Future of Audio and Video on the Web

Web sites and platforms are becoming increasingly media-rich. Today, approximately 62 percent of internet traffic is made up of images, with audio and video constituting a growing percentage of the bytes.

Read more

Embed Images in Email Campaigns at Scale

By Sourav Kundu
Embed Images in Email Campaigns at Scale

tl;dr

Cloudinary is a powerful image hosting solution for email marketing campaigns of any size. With features such as advanced image optimization and on-the-fly image transformation, backed by a global CDN, Cloudinary provides the base for a seamless user experience in your email campaigns leading to increased conversion and performance.

Read more
Build the Back-End For Your Own Instagram-style App with Cloudinary

Github Repo

Managing media files (processing, storage and manipulation) is one of the biggest challenges we encounter as practical developers. These challenges include:

A great service called Cloudinary can help us overcome many of these challenges. Together with Cloudinary, let's work on solutions to these challenges and hopefully have a simpler mental model towards media management.

Read more

Build A Miniflix in 10 Minutes

By Prosper Otemuyiwa
Build A Miniflix in 10 Minutes

Developers are constantly faced with challenges of building complex products every single day. And there are constraints on the time needed to build out the features of these products.

Engineering and Product managers want to beat deadlines for projects daily. CEOs want to roll out new products as fast as possible. Entrepreneurs need their MVPs like yesterday. With this in mind, what should developers do?

Read more