ikorason

Image Resize in Azure Functions using Rust

May 6, 2024 / 13 min read

rustwarpapiazureserverless functionsimage processing

I've known about serverless functions for a while now, but I've never actually used them. I've always been curious about how they work and how they can be used in real-world scenarios.

I was browsing through this article on creating Azure Functions using custom handlers with languages like Go or Rust. After reading the article, I was intrigued by the idea of using Rust with Azure Functions. So I try to find something like a tutorial about Rust and Azure Functions if someone built something but I can't find anything, then I found this tutorial from Mohamad Lawand Youtube video where he created a simple image processing with Azure Functions & Service Bus using .Net Core. So I was very excited to try to create the same thing but using Rust.

At first, I thought it would be difficult to use Rust with Azure services like Service Bus and Blob Storage, but after doing some research, I've found that there is already an unofficial Rust Azure SDK which can be used to interact with Azure services. So I decided to create an image resizing function using Azure Functions and Rust.

The basic idea is simple, first create an API endpoint which makes a POST request to upload the image to the Azure Blob Storage and send a message to the Azure Service Bus. Then Azure Functions will be utilized to listen to the Service Bus using Azure Service Bus Queue trigger and resize the image and save it back to the Blob Storage.

image resize plan

What are Azure Functions?

Azure Functions is a serverless compute service that lets you run event-triggered code without having to explicitly provision or manage infrastructure. You can use Azure Functions to build web APIs, respond to database changes, process IoT data, and more.

Pre-requisites

This tutorial assumes that you have a basic understanding of Rust programming language and Azure Functions. If you are new to Rust, I recommend you to read the official Rust Book.

Also make sure you have Azure account and Azure CLI installed on your machine, if not, you can create a free account on Azure and install Azure CLI on your machine from the above links.

Create API endpoint to upload image

First thing first, we need to create a simple API endpoint to upload the image to the Azure Blob Storage and send a message to the Azure Service Bus. We will use warp for creating the API endpoint. Let's create a directory called image-resize and create a new directory called api and create a new Rust project using cargo:

bash
mkdir image-resize && cd image-resize && mkdir api && cd api
cargo init

This will create a new Rust project with the name image-resize. Now, let's add the required dependencies to the Cargo.toml file:

toml
[package]
name = "image-resize"
version = "0.1.0"
edition = "2021"


[dependencies]
warp = "0.3"
tokio = { version = "1.12", features = ["macros", "fs", "rt-multi-thread"] }
futures = { version = "0.3", default-features = false }
bytes = "1.0"
azure_core = "0.20.0"
azure_storage = "0.20.0"
azure_storage_blobs = "0.20.0"
azure_messaging_servicebus = "0.20.0"
serde = "1.0.200"
serde_json = "1.0"

Now let's add the code to the main.rs file:

rust
// api/src/main.rs
use warp::{
    http::StatusCode,
    multipart::FormData,
    Filter, Rejection, Reply,
};
use std::{convert::Infallible};


#[tokio::main]
async fn main() {
    let upload_route = warp::path("upload")
        .and(warp::post())
        .and(warp::multipart::form().max_length(5 * 1024 * 1024)) // Max image size: 5MB
        .and_then(upload_file);


    let routes = upload_route
        .recover(handle_rejection);


    println!("Server started at http://localhost:3030");
    warp::serve(routes).run(([127, 0, 0, 1], 3030)).await;
}


async fn upload_file(form: FormData) -> Result<impl Reply, Rejection> {
    Ok(format!("Hello, world!"))
}


async fn handle_rejection(err: Rejection) -> std::result::Result<impl Reply, Infallible> {
    let (code, message) = if err.is_not_found() {
        (StatusCode::NOT_FOUND, "Not Found".to_string())
    } else if err.find::<warp::reject::PayloadTooLarge>().is_some() {
        (StatusCode::BAD_REQUEST, "Payload too large".to_string())
    } else {
        eprintln!("unhandled error: {:?}", err);
        (
            StatusCode::INTERNAL_SERVER_ERROR,
            "Internal Server Error".to_string(),
        )
    };


    Ok(warp::reply::with_status(message, code))
}

This will set up a basic API endpoint at http://localhost:3030/upload which will accept a POST request with a multipart/form-data containing the image file.

You can now try to run the API using the following command:

bash
cargo run -r

You then should see a message Server started at http://localhost:3030 in the console. Now you can test the API by sending a POST request to http://localhost:3030/upload with an image file.

bash
CURL -X POST -F 'file=@/path/to/image.jpeg' http://localhost:3030/upload

It should return Hello, world! for now.

Now let's add the code to read the image file from the FormData and store it in a Vec<u8> which we will use to upload the image to the Azure Blob Storage.

rust
// api/src/main.rs
use bytes::BufMut;
use futures::TryStreamExt;
use warp::{
    http::StatusCode,
    multipart::{FormData, Part},
    Filter, Rejection, Reply,
};
use std::{convert::Infallible};


async fn upload_file(form: FormData) -> Result<impl Reply, Rejection> {
    let uploaded_files: Vec<_> = form
        .and_then(|mut part: Part| async move {
            let mut bytes: Vec<u8> = Vec::new();


            // read the part stream
            while let Some(content) = part.data().await {
                let content = content.unwrap();
                bytes.put(content);
            }


            // return the part name, filename and bytes as a tuple
            Ok((
                part.name().to_string(),
                part.filename().unwrap().to_string(),
                String::from_utf8_lossy(&*bytes).to_string(),
            ))
        })
        .try_collect()
        .await
        .map_err(|_| warp::reject::reject())?;


    Ok(format!("Uploaded files: {:?}", uploaded_files))
}

Next let's implement the code to upload the image to the Azure Blob Storage, we will use the azure-storage and azure-storage-blobs crate for this purpose. Add the following code to the main.rs file:

rust
// api/src/main.rs
use azure_storage::StorageCredentials;
use azure_storage_blobs::prelude::ClientBuilder;
use bytes::BufMut;
use futures::TryStreamExt;
use warp::{
    http::StatusCode,
    multipart::{FormData, Part},
    Filter, Rejection, Reply,
};
use std::{convert::Infallible, env};


async fn upload_file(form: FormData) -> Result<impl Reply, Rejection> {
    let uploaded_files: Vec<_> = form
        .and_then(|mut part: Part| async move {
            let mut bytes: Vec<u8> = Vec::new();


            // read the part stream
            while let Some(content) = part.data().await {
                let content = content.unwrap();
                bytes.put(content);
            }


            if !bytes.is_empty() {
                // Azure Blob Storage credentials
                let storage_account = env::var("AZURE_STORAGE_ACCOUNT").expect("Missing AZURE_STORAGE_ACCOUNT env var");
                let storage_access_key = env::var("AZURE_STORAGE_ACCESS_KEY").expect("Missing AZURE_STORAGE_ACCESS_KEY env var");
                let container_name = env::var("AZURE_STORAGE_CONTAINER").expect("Missing AZURE_STORAGE_CONTAINER env var");
                let blob_name = part.filename().unwrap().to_string();


                // create Azure Blob Storage client
                let storage_credentials = StorageCredentials::access_key(storage_account.clone(), storage_access_key);
                let blob_client = ClientBuilder::new(storage_account, storage_credentials).blob_client(&container_name, blob_name);


                // upload file to Azure Blob Storage
                match blob_client
                    .put_block_blob(bytes.clone())
                    .content_type("image/jpeg")
                    .await {
                        Ok(_) => println!("Blob uploaded successfully"),
                        Err(e) => println!("Error uploading blob: {:?}", e),
                    }


                println!("Uploaded file url: {}", blob_client.url().expect("Failed to get blob url"));
            }


            // return the part name, filename and bytes as a tuple
            Ok((
                part.name().to_string(),
                part.filename().unwrap().to_string(),
                String::from_utf8_lossy(&*bytes).to_string(),
            ))
        })
        .try_collect()
        .await
        .map_err(|_| warp::reject::reject())?;


    Ok(format!("Uploaded files: {:?}", uploaded_files))
}

Make sure to have the following environment variables set in your system:

bash
export AZURE_STORAGE_ACCOUNT="your-storage-account-name"
export AZURE_STORAGE_ACCESS_KEY="your-storage
export AZURE_STORAGE_CONTAINER="your-container-name"

Also make sure to create a blob container in the Azure Portal and set the container name in the environment variable AZURE_STORAGE_CONTAINER. You can also find the access key and storage account name in the Azure Portal.

Now let's create a function to send a message to the Azure Service Bus. Add the following code to the main.rs file:

rust
// api/src/main.rs
use azure_messaging_servicebus::service_bus::QueueClient;
... existing imports ...


#[derive(Serialize, Deserialize, Debug)]
struct Image {
    filename: String,
    image_container: String,
}


async fn upload_file(form: FormData) -> Result<impl Reply, Rejection> {
    let uploaded_files: Vec<_> = form
        .and_then(|mut part: Part| async move {
            ... existing code ...


            if !bytes.is_empty() {
              ... existing code ...


              let image = Image {
                  filename: part.filename().unwrap().to_string(),
                  image_container: container_name,
              };


              send_message_to_queue(image).await;
            }


            // return the part name, filename and bytes as a tuple
            Ok((
                part.name().to_string(),
                part.filename().unwrap().to_string(),
                String::from_utf8_lossy(&*bytes).to_string(),
            ))
        })
        .try_collect()
        .await
        .map_err(|_| warp::reject::reject())?;


    Ok(format!("Uploaded files: {:?}", uploaded_files))
}


async fn send_message_to_queue(image: Image) {
    let service_bus_namespace = env::var("AZURE_SERVICE_BUS_NAMESPACE").expect("Please set AZURE_SERVICE_BUS_NAMESPACE env variable first!");
    let queue_name = env::var("AZURE_QUEUE_NAME").expect("Please set AZURE_QUEUE_NAME env variable first!");
    let policy_name = env::var("AZURE_POLICY_NAME").expect("Please set AZURE_POLICY_NAME env variable first!");
    let policy_key = env::var("AZURE_POLICY_KEY").expect("Please set AZURE_POLICY_KEY env variable first!");


    let http_client = azure_core::new_http_client();


    let client = QueueClient::new(
        http_client,
        service_bus_namespace,
        queue_name,
        policy_name,
        policy_key
    ).expect("Failed to create client");


    let message_to_send = serde_json::to_string(&image).expect("Failed to serialize image");


    client
        .send_message(message_to_send.as_str())
        .await
        .expect("Failed to send message");


    println!("Message sent to Azure Service Bus queue successfully!");
    println!("Message: {}", message_to_send);
}

Here we are sending a message to the Azure Service Bus queue with the image filename and the container name where the image is stored, so we can later use this information to get the correct image from the blob container and resize the image and save it back to the Blob Storage.

Make sure to create a Azure Service Bus and Queue item in your Azure Portal, and also make sure to have all the required environment variables set in your system:

bash
export AZURE_SERVICE_BUS_NAMESPACE="your-service-bus-namespace"
export AZURE_QUEUE_NAME="your-queue-name"
export AZURE_POLICY_NAME="your-policy-name"
export AZURE_POLICY_KEY="your-policy-key"

Now if you run the API and upload an image, if everything is ok you should see the following output:

bash
Blob uploaded successfully
Uploaded file url: https://storage_account_name.blob.core.windows.net/container_name/filename.jpeg
Message sent to Azure Service Bus queue successfully!
Message: {"filename":"filename.jpeg","image_container":"container_name"}

Go ahead and check the Azure Blob Storage and Azure Service Bus to see if the image is uploaded and the message is sent to the queue.

If everything is successful, we can now move on to the next step which is to create an Azure Function to listen to the Service Bus queue and resize the image.

Create Azure Function to resize image

Creating a Azure function can be done using both Azure Portal or Azure CLI, also there is easier option which uses VSCode. In this tutorial, I will use Azure CLI to create the Azure Function. Make sure you have Azure CLI installed on your machine.

Navigate to the image-resize directory and run the following command to create a new Azure Function:

bash
mkdir functions && cd functions
cargo init

This will create a new Rust project with the name functions. Now let's add the required dependencies to the Cargo.toml file:

toml
[package]
name = "handler" // Azure Function name
version = "0.1.0"
edition = "2021"


[dependencies]
warp = "0.3"
tokio = { version = "1.12", features = ["macros", "fs", "rt-multi-thread"] }
futures = { version = "0.3", default-features = false }
serde = "1.0.200"
serde_json = "1.0"
azure_core = "0.20.0"
azure_storage = "0.20.0"
azure_storage_blobs = "0.20.0"
azure_messaging_servicebus = "0.20.0"
tracing = "0.1.40"
image = "0.25.1"

Now let's create host.json and local.settings.json file, this is needed for Azure Functions.

json
// host.json
{
  "version": "2.0",
  "logging": {
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      }
    }
  },
  "extensionBundle": {
    "id": "Microsoft.Azure.Functions.ExtensionBundle",
    "version": "[4.*, 5.0.0)"
  },
  "customHandler": {
    "description": {
      "defaultExecutablePath": "handler", // make sure this is the same as the binary name
      "workingDirectory": "",
      "arguments": []
    }
  },
  "concurrency": {
    "dynamicConcurrencyEnabled": true,
    "snapshotPersistenceEnabled": true
  }
}


// make sure this file is in .gitignore
// local.settings.json
{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "",
    "FUNCTIONS_WORKER_RUNTIME": "custom",
    // make sure to replace with your service bus namespace, and access key
    "servicebusnamespace_SERVICEBUS": "Endpoint=sb://servicebusnamespace.servicebus.windows.net/;SharedAccessKeyName=SharedAccessKeyName;SharedAccessKey=SharedAccessKey"
  }
}

Now let's create a function using func new command:

bash
func new --template "Azure Service Bus Queue trigger" --name "process_image_resize"

This will create a new folder called process_image_resize with the functions.json file, let's modify the functions.json file:

json
{
  "bindings": [
    {
      "name": "mySbMsg",
      "type": "serviceBusTrigger",
      "direction": "in",
      "queueName": "queue_name", // replace queue_name with your queue name
      "connection": "servicebusnamespace_SERVICEBUS" // replace servicebusnamespace with your service bus namespace
    }
  ]
}

Now let's add the following code to the src/main.rs file:

rust
use azure_messaging_servicebus::service_bus::QueueClient;
use azure_storage::StorageCredentials;
use azure_storage_blobs::prelude::BlobServiceClient;
use serde::{Deserialize, Serialize};
use futures::StreamExt;
use tracing::trace;
use std::{env, io::Cursor};


#[derive(Serialize, Deserialize, Debug)]
struct ImageNode {
    filename: String,
    image_container: String,
}


#[tokio::main]
async fn main() -> azure_core::Result<()> {
    let service_bus_namespace = env::var("AZURE_SERVICE_BUS_NAMESPACE").expect("Please set AZURE_SERVICE_BUS_NAMESPACE env variable first!");
    let queue_name = env::var("AZURE_QUEUE_NAME").expect("Please set AZURE_QUEUE_NAME env variable first!");
    let policy_name = env::var("AZURE_POLICY_NAME").expect("Please set AZURE_POLICY_NAME env variable first!");
    let policy_key = env::var("AZURE_POLICY_KEY").expect("Please set AZURE_POLICY_KEY env variable first!");


    let http_client = azure_core::new_http_client();


    let client = QueueClient::new(
        http_client,
        service_bus_namespace,
        queue_name,
        policy_name,
        policy_key
    ).expect("Failed to create client");


    let received_message = client
        .receive_and_delete_message()
        .await
        .expect("Failed to receive message");


    if received_message.is_empty() {
        println!("No message received");
        return Ok(())
    }


    println!("Received message: {:?}", received_message);


    // grab the image from the message
    match serde_json::from_str::<ImageNode>(&received_message) {
        Ok(image) => {
            println!("Deserialized image: {:?}", image);


            // Azure Blob Storage credentials
            let storage_account = env::var("AZURE_STORAGE_ACCOUNT").expect("Missing AZURE_STORAGE_ACCOUNT env var");
            let storage_access_key = env::var("AZURE_STORAGE_ACCESS_KEY").expect("Missing AZURE_STORAGE_ACCESS_KEY env var");
            let container_name = image.image_container;


            let blob_name = &*image.filename;


            // create Azure Blob Storage client
            let storage_credentials = StorageCredentials::access_key(storage_account.clone(), storage_access_key);
            let service_client = BlobServiceClient::new(storage_account, storage_credentials);
            let blob_client = service_client
                .container_client(&container_name)
                .blob_client(blob_name);


            trace!("Requesting blob");


            let mut bytes: Vec<u8> = Vec::new();
            // stream a blob, 8KB at a time
            let mut stream = blob_client.get().chunk_size(0x2000u64).into_stream();
            while let Some(value) = stream.next().await {
                let data = value?.data.collect().await?;
                println!("received {:?} bytes", data.len());
                bytes.extend(&data);
            }


            // load the image from the bytes
            let img = image::load_from_memory(&bytes).expect("Failed to load image");
            // resize the image
            let resized_img = img.resize(100, 100, image::imageops::FilterType::Triangle);
            // write the resized image to the buffer
            let mut resized_bytes: Vec<u8> = Vec::new();
            resized_img.write_to(&mut Cursor::new(&mut resized_bytes), image::ImageFormat::Jpeg).expect("Failed to write image");


            // change the filename to include the word "resized"
            let new_blob_name = format!("resized_{}", blob_name);


            let blob_client = service_client
                .container_client(&container_name)
                .blob_client(&new_blob_name);


            blob_client.put_block_blob(resized_bytes)
                .content_type("image/jpeg")
                .await
                .expect("Failed to upload blob");


            println!("Resized image uploaded successfully");


        },
        Err(e) => {
            println!("Failed to deserialize image: {:?}", e);
            return Ok(())
        }
    };


    Ok(())
}

The above code will listen to the Azure Service Bus queue and receive the message, then it will get the image from the Azure Blob Storage and resize the image and save it back to the Blob Storage with the filename prefixed with resized_.

Now with everything in place, let's start our server and upload an image:

bash
// image-resize/api
cargo run -r


// upload an image
CURL -X POST -F 'file=@/path/to/image.jpeg' http://localhost:3030/upload

Now let's start the Azure Function by running the following command:

bash
// image-resize/functions
cargo build -r && cp ./target/debug/handler . && func start

If everything is successful, you should see the following output:

bash
Received message: "{\"filename\":\"example.jpeg\",\"image_container\":\"image_container_name\"}"
Deserialized image: ImageNode { filename: "avatar-1.jpeg", image_container: "image_container_name" }
received 8192 bytes
received 8192 bytes
...
...
Resized image uploaded successfully

If you go back to the Azure Blob Storage, you should see the resized image with the filename prefixed with resized_.

Conclusion

This is just a simple example of how you can use Azure Functions with Rust to create a simple image processing function, there are many other possibilities and use cases where you can use Azure Functions with Rust. Overall it was a fun experience to use Rust with Azure Functions, and I learned a lot about Azure Functions and Rust in the process.

Even though Azure SDK for Rust is still in unofficial state, it is still very good and easy to use, and I hope it will be officially supported by Microsoft in the future.