Polotno SDK includes four AI image tools: Expand, Generative fill, Upscale and Background removal. The SDK does the editor work: selections, frames, pixel preparation, progress, undo and cancellation. You connect the AI model you want, on your own server, with your own keys.
The hard part is not the model call
Calling an image model is a short piece of code. Adding that call to a design editor so that it feels finished takes much more work:
- The user needs a way to mark what to change: a frame to drag past the image edges, or a selection to draw on the photo.
- The model needs the pixels the user sees, not the original file. That means the current crop, flips and stretch, at a size the model accepts, with a mask that has the same pixel size.
- The result must go back into the design in the correct position, with the element's filters and effects still editable, in one undo step.
- Users cancel, undo, or move the image while the request runs. A late result must not overwrite their newer work.
- Failures need a clear message, and a canceled request needs no message at all.
Polotno SDK does all of this. Your code does one thing: it sends an image to a model and returns the result.
Four tools
Expand. The user drags the frame past the edges of a photo, with an optional prompt. The model fills the new area, and the image grows to the new frame. Use it to turn a portrait photo into a banner, or to make room for a headline.
Generative fill. The user selects an area on the image. Replace fills it from a prompt, such as "open sky" or "a plain wall". Remove erases an object and fills the area from its surroundings.
Upscale. One click increases the resolution of an image.
Background removal. One click removes the background. It can use Polotno Cloud, or your own provider.
The tools show in the AI edit menu of the image toolbar. Each tool appears only when you connect it, so users never see a button that does nothing.
Why you connect the model
For Expand, Generative fill and Upscale, the SDK includes no AI provider and uploads nothing. That is deliberate:
- Choose the model. Image models change fast. You can change the provider without changing the editor.
- Control the cost. You decide which plans get which tools, and how many edits a user can make.
- Keep the keys and data on your side. Requests go to your server, so provider keys can stay there, and you decide where images are processed and stored.
- Show your own UI where it matters. Credits, plans and model choice belong to your product. You can keep the stock menu and panel, or build your own buttons, and the SDK still draws the frames, selections and progress on the canvas.
What connecting a tool looks like
Each tool is one function that receives the prepared image and returns the new one. This connects Upscale to an endpoint on your own server:
import { setUpscaleFunc } from 'polotno/config';
setUpscaleFunc(async ({ image, signal }) => {
const body = new FormData();
body.append('image', image, 'image.png');
// `signal` stops the upload when the user cancels.
const res = await fetch('/api/upscale', { method: 'POST', body, signal });
if (!res.ok) throw new Error(`Upscale failed (HTTP ${res.status})`);
const { url } = await res.json();
return { src: url };
});Expand and Generative fill work the same way, and also receive a mask of the area to change. Helpers resize the image and mask for your provider, pad the image for Expand, and convert the mask for providers that use another mask format. A tool can also apply its result in its own way, for example as a new layer above the photo.
Try it
The AI Image Tools guide has a live demo of every tool. It runs on Polotno's AI endpoints and allows a small number of edits per visitor each day. The guide also covers the request and result format, cancellation, error messages and custom UI.
