Editing pictures

Obraz edits pictures with the tools of a photo editor: crop, turn, resize and frame a picture; change its tone, give it a look or a colour grade; blur, mosaic, sharpen, vignette and grain it; write text, lay stickers and shapes on it; put it on a canvas or several pictures in a collage; and, through an image-model service, let a model generate a picture, remove or replace its background, erase an object, extend the frame, restyle, colourise, restore, retouch or relight it.

Three ways reach it: the editor in a browser at the address Obraz serves, POST /v1/edits for programs, and obraz::client::ObrazClient::edit for Rust callers. New to Obraz? Start with the quickstart.

An edit is one plan: a list of steps run in order on a working picture. The working picture starts as input 0, or is made by a first blank, collage or generate step. Every step names its values; where a field may be left out, leaving it out leaves the picture as it is (no turn, full opacity, normal blending, an overlay's own size) or uses what the font says (its line height). Obraz chooses no strength, size or colour for anybody.

The whole plan is checked before any pixel changes: a plan that would be refused at step nine does not spend a model call at step two. What only the running picture can answer, such as a crop outside a picture whose size earlier steps decided, is refused when that step runs.

The endpoint

POST /v1/edits             run one plan, answer the edited picture
GET  /v1/edits/operations  the steps a plan can hold, with their fields

POST /v1/edits
x-obraz-secret: <the deployment's secret>

{"plan": {"steps": [
   {"op": "crop_ratio", "ratio": "4:5", "anchor": "center"},
   {"op": "resize", "width": 1080, "height": 1350, "fit": "cover", "filter": "lanczos3"},
   {"op": "adjust", "contrast": 0.15, "saturation": 0.25},
   {"op": "text", "text": "Summer in the city", "font": "brand", "size": 96,
    "color": "#ffffff", "x": 540, "y": 1280, "anchor": "bottom",
    "stroke": {"width": 4, "color": "#000000"}}]},
 "inputs": ["<base64 of a picture>"],
 "assets": {"brand": "<base64 of a TTF or OTF font>"},
 "output": {"format": "png"}}

• inputs: pictures as base64, in any format the Rust image crate reads (PNG, JPEG, WebP, GIF, BMP, TIFF and others); steps name them by position, input 0 first.
• assets: named files as base64 — fonts for text, .cube tables for lut.
• output: format is png, jpeg, webp, tiff, bmp or gif; JPEG needs quality on the encoder's scale, 1 to 100, and no other format takes one. JPEG holds no transparency, so a picture with a transparent pixel is refused rather than flattened onto a colour nobody chose.

The answer is success, job_id, mime_type, width, height, image_base64, and steps: for every step its number (from 1), its op, the size of the picture it left, and for a model step the route: the model the image-model service says answered. A refusal is success: false with error: {"step", "op", "reason"}, or only reason when the refusal is about the whole request. A Rust caller uses obraz::client::ObrazClient::edit.

Any field a step does not take, and an op Obraz does not have, is refused by the JSON reader with 422 and its own sentence, which names the field or the variant, for example unknown variant `sparkle`.

Values every step uses

• Colours are #RRGGBB or #RRGGBBAA; without the alpha pair a colour is opaque.
• Anchors say which point of a box a position holds: top_left, top, top_right, left, center, right, bottom_left, bottom, bottom_right.
• Fits: stretch distorts a picture to a frame, contain shows all of it and fills the rest, cover fills the frame and cuts what sticks out.
• Filters for resampling, as the image crate names them: nearest, triangle, catmull_rom, gaussian, lanczos3.
• Fills paint what a picture does not cover: {"color": "#RRGGBB"}; {"blur": SIGMA}, the working picture covering the frame and blurred, opaque even where the picture is transparent; {"gradient": {"from", "to", "angle"}}, 0 degrees running left to right and 90 top to bottom; {"input": N}, an input picture covering the frame. Pictures are resampled for a fill, a collage and an overlay with Lanczos3.
• Blends for an overlay: normal, multiply, screen, overlay, darken, lighten, color_dodge, color_burn, hard_light, soft_light, difference, exclusion, hue, saturation, color, luminosity, add.
• Regions are {"x", "y", "width", "height"} in pixels from the top-left corner, and must lie inside the picture.
• Opacity is a share from 0 to 1.

Canvas

Geometry

Tone and colour

Effects

Text, stickers and shapes

Model steps

These steps are done by an image model, through the image-model service the deployment names (configuration): any service that speaks the OpenAI Images API. In a Wisent deployment that is Brama, whose image-model alias the deployment names and which owns which provider answers and who pays; outside one it can be the OpenAI API itself. A generation is POST /v1/images/generations; an edit is POST /v1/images/edits as the OpenAI edit form, the working picture as the image file and the mask as the mask file. A step asks the model it names in model, or the deployment's OBRAZ_IMAGE_MODEL. Every model step is a billed provider call.

A model answers at a size of its own, so an edit is resized back to the size of the picture it was given (Lanczos3). erase and expand keep the original pixels everywhere outside the area the model was asked to paint.

The service's refusal comes back in its own words as the step's reason, with 502: the image model service at <address> refused (<status>): <code or type>: <message> — for example an unrouted image-model on Brama, a key the provider rejects, or an account with no credit. Other answers Obraz cannot use are named: the image model service at <address> could not be reached: <cause>, the image model service at <address> answered <status> with a body that is not JSON: <body>, the image model service at <address> answered <status> with no image in `data`: <body>, the image model service's image carries neither b64_json nor url: <item>, the image model service's b64_json is not base64: <cause>, the picture at <url> answered <status>, OBRAZ_IMAGE_MODEL_TOKEN_FILE <path> cannot be read: <cause>.

The editor

obraz serve answers GET / with the editor: a page where a person adds pictures and assets, starts from an example (a 4:5 post, a 9:16 story on a blurred backdrop, a round avatar, a mosaic over a face, a logo watermark, a collage, a product photo on white), adds steps from the list GET /v1/edits/operations gives, and presses Edit. The page sends the same POST /v1/edits any caller sends, with the caller secret the person types, and shows the picture, its size after every step, a download link, or the refusal with its status, step and reason.

Refusals

400, before anything is edited:

the plan has no steps

no input picture was given and the first step edits one: give an input, or begin with blank, collage or generate

input <n> is not base64: <cause>

input <n> is not a picture Obraz can read: <cause>

asset <name> is not base64: <cause>

a JPEG output needs a quality from 1 to 100

JPEG quality is 1 to 100 on the encoder's scale, not <quality>

quality applies to JPEG only; <mime type> is written without one

and, prefixed step <n> (<op>): :

this step is done by an image model, and this Obraz has no image-model service: its deployment sets OBRAZ_IMAGE_MODEL_URL, OBRAZ_IMAGE_MODEL_TOKEN_FILE, OBRAZ_IMAGE_MODEL

input <n> was named and <count> input picture(s) were given

<width>x<height> has no pixels

a blur fill blurs the working picture, and a blank canvas starts without one

a collage needs at least one input and one column

ratio "<ratio>" is not two whole numbers above zero as W:H

a turn of <degrees> degrees uncovers corners: name the fill colour for them

a contain resize leaves bars: name the fill for them

only a contain resize leaves room for a fill; stretch and cover fill the frame

a rounded_rectangle mask needs a radius

an ellipse mask takes no radius

a border needs a width of at least 1 pixel

adjust names no control: give at least one of brightness, contrast, saturation, vibrance, exposure, temperature, tint, highlights, shadows, hue, fade, gamma

posterize needs at least 2 levels, not <levels>

the <font|LUT> asset "<name>" was not given; the assets given are [<names>]

LUT asset "<name>": line <n>: LUT_3D_SIZE must be a whole number of at least 2

LUT asset "<name>": line <n>: LUT_3D_INPUT_RANGE needs two numbers

LUT asset "<name>" is not text

LUT asset "<name>": declares no LUT_3D_SIZE

LUT asset "<name>": is a 1D table; Obraz applies 3D tables (LUT_3D_SIZE)

LUT asset "<name>": LUT_3D_SIZE <size> needs <count> entries and the file has <count>

LUT asset "<name>": line <n> is not three numbers

LUT asset "<name>": DOMAIN_MAX must lie above DOMAIN_MIN on every channel

a pixelate block must be at least 1 pixel

font asset "<name>" is not a font Obraz can read: <cause>

text has nothing to write

text that runs over several lines needs align: left, center or right

an overlay's width and height must be at least 1 pixel

a line has no inside to fill; give it a stroke

a line needs a stroke to be seen

only a rectangle takes a corner_radius

a shape needs a fill, a stroke or both

expand needs at least one side to grow

<field> must be a number, not <value>

<field> must be above 0, not <value>

<field> must be 0 or more, not <value>

opacity is a share from 0 to 1, not <value>

<prompt|style|lighting> must say something

422, while editing — a step the picture as it then is cannot take, or a result the format cannot hold:

step <n> (<op>): the region <w>x<h> at (<x>, <y>) reaches (<right>, <bottom>), outside the <width>x<height> picture

step <n> (scale): scaling the <width>x<height> picture by <factor> leaves no pixel

the picture has transparent pixels and JPEG cannot hold them: fill the background first (canvas or blank with a fill) or write PNG or WebP

502: a model step that the image-model service or its provider refused, as above. 401 unauthorized: a missing or wrong x-obraz-secret.