In this article, we will be taking a look at how we can create a image processor using WebAssembly and react. This will be running completely on client side. The goal of this article is to understand how we can utilize the power of assembly to run heavy computational tasks in browser. I will not be foucssing too much on image processing in this article.
WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications.
We will be writing some image processing functions in rust and we will be compiling that to wasm, and load the wasm module in react. So let’s start with coding image processing functions, for simplicity I will only be writing 3 functions: 1) resize, 2) gray scale and 3) blur.
So first initialize a rust project by running command:
cargo init
This will initialize a project, now open main.rs inside src/main.rs and add image module by running command:
cargo add image
rustup target add wasm32-unknown-unknown
cargo install wasm-bindgen-cli
And modify you cargo.toml to look like:
[package]
name = "img"
version = "0.1.0"
edition = "2021"
[dependencies]
image = "0.25.5"
wasm-bindgen = "0.2"
[lib]
crate-type = ["cdylib"]
[dev-dependencies]
wasm-bindgen-test = "0.3"
Now you can write all the functoins as:
1) resize: take image, height and width as input and returns new image with new size
2) gray_scale: takes image as input and return the image after making it gray
2) blur: takes image and sigma as input and returns image after making it blur
First, we will be taking a look at resize function:
#[wasm_bindgen]
pub fn resize(image_data: &[u8], width: u32, height: u32) -> Vec<u8> {
let image = image::load_from_memory(image_data).expect("Failed to open the file");
let resized_image = image.resize(width, height, image::imageops::FilterType::Lanczos3);
let mut buf = Cursor::new(Vec::new());
resized_image.write_to(&mut buf, image::ImageFormat::Png).expect("Failed to write the image");
buf.into_inner()
}
1) #[wasm_bindgen]: this attribute is provided by wasm-bindgen crate, it makes the function callable from javascript, it also helps in making the interaction between rust and js seamless. For Example: in this case, this attribute will convert &[u8] from javascript’s Uint8Array, and also convert Vec<u8> to a javascript consumable format.
2) Now this functions takes image_data, width and height as input. image_data is a slice of bytes representing the raw image data.
3) load_from_memory is a function provided by image crate, which from &[u8] array, converts it to an image.
4) Then we call image.resize to resize the image to given formats, and also use a filter provided by a filter provided by image crate.
5) Next we create a mutable vector named buf and next we write to it the resized image in png format.
6) Then we return the vector, buf.into_inner() extracts the underlying vector from cursor.
Similarly you can take a look at other functions:
#[wasm_bindgen]
pub fn grayscale(image_data: &[u8]) -> Vec<u8> {
let image = image::load_from_memory(image_data).expect("Failed to open the file");
let gray_image = image.grayscale();
let mut buf = Cursor::new(Vec::new());
gray_image.write_to(&mut buf, image::ImageFormat::Png).expect("Failed to write the image");
buf.into_inner()
}
#[wasm_bindgen]
pub fn blur(image_data: &[u8], sigma: f32) -> Vec<u8> {
let image = image::load_from_memory(image_data).expect("Failed to open the file");
let blurred_image = image.blur(sigma);
let mut buf = Cursor::new(Vec::new());
blurred_image.write_to(&mut buf, image::ImageFormat::Png).expect("Failed to write the image");
buf.into_inner()
}
And here’s the complete code for your reference:
use wasm_bindgen::prelude::*;
use std::io::Cursor;
#[wasm_bindgen]
pub fn resize(image_data: &[u8], width: u32, height: u32) -> Vec<u8> {
let image = image::load_from_memory(image_data).expect("Failed to open the file");
let resized_image = image.resize(width, height, image::imageops::FilterType::Lanczos3);
let mut buf = Cursor::new(Vec::new());
resized_image.write_to(&mut buf, image::ImageFormat::Png).expect("Failed to write the image");
buf.into_inner()
}
#[wasm_bindgen]
pub fn grayscale(image_data: &[u8]) -> Vec<u8> {
let image = image::load_from_memory(image_data).expect("Failed to open the file");
let gray_image = image.grayscale();
let mut buf = Cursor::new(Vec::new());
gray_image.write_to(&mut buf, image::ImageFormat::Png).expect("Failed to write the image");
buf.into_inner()
}
#[wasm_bindgen]
pub fn blur(image_data: &[u8], sigma: f32) -> Vec<u8> {
let image = image::load_from_memory(image_data).expect("Failed to open the file");
let blurred_image = image.blur(sigma);
let mut buf = Cursor::new(Vec::new());
blurred_image.write_to(&mut buf, image::ImageFormat::Png).expect("Failed to write the image");
buf.into_inner()
}
Now we need to create javascript bindings and compile this to web assmebly by running the following command:
wasm-pack build --target web
This will create a pkg folder with .wasm and .js files in it.
Next we create a simple frontend in html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Image Processing</title>
</head>
<body>
<h1>Image Processing</h1>
<input type="file" id="imageInput">
<button id="resizeButton">Resize</button>
<button id="grayscaleButton">Grayscale</button>
<button id="blurButton">Blur</button>
<script src="./app.js" type="module"></script>
</body>
</html>
and the javascript code is as belows:
import init, { resize, grayscale, blur } from './pkg/img.js';
async function main() {
await init();
const fileInput = document.getElementById('imageInput');
const resizeButton = document.getElementById('resizeButton');
const grayscaleButton = document.getElementById('grayscaleButton');
const blurButton = document.getElementById('blurButton');
resizeButton.addEventListener('click', async () => {
const file = fileInput.files[0];
if (file) {
const arrayBuffer = await file.arrayBuffer();
const resizedImage = resize(new Uint8Array(arrayBuffer), 200, 200);
downloadImage(resizedImage, 'resized_image.png');
}
});
grayscaleButton.addEventListener('click', async () => {
const file = fileInput.files[0];
if (file) {
const arrayBuffer = await file.arrayBuffer();
const grayImage = grayscale(new Uint8Array(arrayBuffer));
downloadImage(grayImage, 'gray_image.png');
}
});
blurButton.addEventListener('click', async () => {
const file = fileInput.files[0];
if (file) {
const arrayBuffer = await file.arrayBuffer();
const blurredImage = blur(new Uint8Array(arrayBuffer), 5.0);
downloadImage(blurredImage, 'blurred_image.png');
}
});
function downloadImage(imageData, fileName) {
const blob = new Blob([imageData], { type: 'image/png' });
const url = URL.createObjectURL(blob);
const link = document.createElement('a');
link.href = url;
link.download = fileName;
link.click();
}
}
main();
Now as you can see we can import resize, gray_scale and blur functions from wasm file directly here.
There is also an init function, which initializes and loads the wasm module.
And then we basically get all the buttons and add events on them to call respective functions.
So try this out and create cool projects using web assmebly. Thanks for reading!!