Images from Code

A picture you load in a pictures.js scene can be changed in code: cut out its subject, resize it, crop, rotate, trace it into vector shapes, read what is in it. Each change gives you a new picture in memory, ready to draw, to give to a Sprite, or to save.

const img = loadImage("shoe.jpg")                 // a picture beside this scene
const cut = await img.removeBackground()          // the shoe, background transparent (Apple devices only)
const small = await cut.resize({ width: 800 })    // one step after another
await small.save("shoe cutout.png")               // optional: write it as a file

function draw() {
  background("white")
  ctx.drawImage(small, 40, 40)
}

These are the methods a Python scene has in Images from Code for Pyctures, with JavaScript's spelling: remove_background is removeBackground, and the options go in one pair of braces, { width: 800 }.

Where each method runs

The work is done by the Circuitry app, so these methods run in scenes inside Circuitry. They are not part of the pictures.js library on its own, and not of a scene exported as a web page: there a picture still loads and draws, and calling one of these stops the scene with a message that says they work inside Circuitry (Use Your Scenes Outside Circuitry).

Inside Circuitry, what a method needs depends on the method:

• Any device Web, Mac, Windows, Linux, iPhone, iPad and Android: resize, crop, rotate, flip, convert, trace, threshold, edgeDetect, cannyEdges, findContours, detectShapes, watermark, stripMetadata, faviconSet, appIconSet, save.
• Apple devices The Circuitry app on a Mac, iPhone or iPad, or a browser connected to Circuit running on a Mac: removeBackground, enhance, sepia, mono, noir, vintage, blur, sharpen, vibrance, and the photo tools classify, findFaces, readCodes, photoDetails, hideFaces, scanDocument, smartCrop, straighten.

The photo tools on a Mac need the current version of the Circuitry desktop app, or of Circuit when a browser, phone or tablet is connected to one. With an older one they are not offered: available() leaves them out, and calling one stops with a message that says what it needs.

The Apple-only ones use the picture tools built into Apple's systems (Vision and Core Image), which work on the device itself. They are not Apple Intelligence, and nothing is sent to a server.

System versions. iOS 17 / macOS 14 or later removeBackground needs iOS 17, iPadOS 17 or macOS 14 or later. Every other method here runs on any system version the Circuitry app itself runs on.

A method called where it cannot run stops with a message that says where it does. Every method lists each one with where it runs, and await img.available() gives the names this device can run.

How it works

• loadImage("name.png") loads a picture from the folder your scene is saved in. You can draw it straight away, as before.
• A picture from a web address can be drawn, but most cannot be changed: a browser lets a page read a picture's pixels only when the picture's own site allows it. Save the picture beside your scene and load it from there.
• Every method is awaited: await img.resize({ width: 800 }). Put the line at the top of the scene, or inside an async function such as async function setup(). Do not call one in draw(): change the picture once, and draw the result every frame.
• A method never changes the picture you called it on. It gives a new one, so you can keep both, or chain: await (await img.removeBackground()).resize({ width: 800 }).
• The new picture has finished loading when you get it, so small.width and small.height are right at once. It is drawn like any other: ctx.drawImage(small, x, y), picture(small, x, y, size), new Sprite(small, x, y, size).
• The work happens in the app, not in your scene, and a scene cannot reach the app: it is handed the new picture and nothing else (what a scene can reach).
• Results are remembered. Running the scene again, or recording a video of it, reuses the results instead of redoing the work, as long as the picture and the options are the same.
• Nothing is written anywhere unless you call save().

Remove a background

const product = loadImage("bottle.jpg")
const cutout = await product.removeBackground()

function draw() {
  background("#f4f1ea")
  picture(cutout, width / 2, height / 2, height * 0.8)
}

removeBackground() finds the main subject (a product, a person, a pet) and makes everything else transparent. It uses Apple's own on-device subject lifting, so it needs the Circuitry app on a Mac (macOS 14 or later), iPhone or iPad (iOS / iPadOS 17 or later), or a browser connected to Circuit running on a Mac. Anywhere else the line stops with: removeBackground() needs an Apple device. On iPhone and iPad it needs a real device; it does not run in the iOS Simulator.

Resize a picture

const hero = await img.resize({ width: 1280 })              // height follows the shape
const thumb = await img.resize({ percent: 25 })
const square = await img.resize({ width: 1080, height: 1080 })                 // whole picture, centred
const cover = await img.resize({ width: 1080, height: 1080, fit: "cover" })    // fill the square, trim the edges

Give width, height or percent. With both width and height, fit says how the picture fills the box: "contain" (the default: the whole picture, with clear margins), "cover" (fills the box and trims the edges) or "stretch". Big reductions are made in halves before the final size, so detail and text stay sharp. Transparency is kept.

Trace a picture into shapes

const logo = loadImage("logo.png")
const vec = await logo.trace({ colors: 6, detail: "medium" })
await vec.save("logo.svg")

function draw() {
  background("white")
  picture(vec, width / 2, height / 2, 300)    // sharp at any size
}

trace() turns a picture into flat-colour vector shapes, the same tracing 2Vec uses. colors is how many colours to keep; detail is "low" (smoother, fewer points), "medium" or "high". A logo or an illustration with clear areas of colour traces best; a photo becomes a poster-like picture. What you get is a picture you can draw at any size, with vec.svg (its SVG text) and vec.shapes (how many shapes it has). It saves as .svg. It has no other methods: trace last.

Data from a picture

Some methods read a picture instead of changing it, and give plain JavaScript data. classify, findFaces, readCodes and photoDetails need an Apple device (the Circuitry app on a Mac, iPhone or iPad, or Circuit running on a Mac); findContours and detectShapes run everywhere.

const info = await img.classify({ maxLabels: 5 })
for (const item of info.labels) {             // e.g. { label: "shoe", confidence: 0.93 }
  console.log(item.label, item.confidence)
}

const codes = await img.readCodes({ overlay: false })
for (const code of codes.codes) {             // each has symbology and payload
  console.log(code.payload)
}

Methods that can draw what they found (findFaces, readCodes, findContours, detectShapes) give { found, overlay } when overlay is true (their default): found is the data and overlay a picture with the finds drawn on. Pass { overlay: false } for the data alone.

Saving

await pic.save("name.png") writes the picture as a file (.png, .jpg or .webp; the format follows the name) and gives its path. It replaces a file of that name. A traced picture saves as .svg. faviconSet() and appIconSet() make a set of files: icons.names lists them, and await icons.save("icons") writes them into a folder.

A saved picture goes where the scene keeps its other files: beside the scene in the Circuitry app on a phone, a tablet or a computer, and in the browser when Circuitry runs in a web browser. Where files and databases are kept has the whole table, and the same rules about names hold: a scene writes inside its own folder, and never a file that is run.

What this device can run

console.log(await img.available())    // the method names this device can run

Most methods run everywhere. Apple's effects and the photo tools need an Apple device: the Mac app, iPhone or iPad, or Circuit on a Mac. On a Mac the photo tools need the updated desktop app or Circuit, so the list is the way to know what this one has.

Mistakes say what is wrong

• An option the method does not take: resize() has no option "wdith". It takes width, height, percent, fit.
• A method no picture has: A picture has no removeBackgroud().
• A device that cannot run it: removeBackground() needs an Apple device: the Circuitry app on a Mac, iPhone or iPad, or Circuit running on a Mac.

Each stops the scene on its own line, like any other mistake (Finding Mistakes). To carry on without the picture, catch it: const cut = await img.removeBackground().catch(() => img).

Every method

Every picture from loadImage has these methods. Each is awaited (await img.resize({ width: 800 })) and takes its options in one pair of braces. An option you leave out takes the value shown; one shown with no value is simply not used. await img.run("name", options) runs any of them by name. The Python spellings (remove_background, max_labels) work too.

Related

• Images from Code for Pyctures: the same methods in Python, and in Pymations.
• Files and Databases: the other files a scene writes, and where they are kept.
• Drawing in 2D: loading and drawing pictures.
• From Python: bringing a Pyctures scene across.