Branded Illustration Generation

Brand-safe illustrations by everyone

01 · The Challenge

A global financial service's branded illustration library was a cost problem disguised as a design problem.

Commissioning bespoke assets across a global network — many offices lacking in-house design — was slow and expensive. A centralised generic library couldn't serve local needs: landmarks, cultural context, market-specific relevance. The opportunity was a generation tool for marketers that could produce brand-consistent output directed to any market's needs, without a design intermediary.

The question was whether generative tools had reached the necessary bar. They hadn't — not out of the box. We proved it, and then built the alternative.

02 · The System

A pipeline from simple text input to curated, brand-accurate illustrations.

A user prompt, combined with an injected quality prompt, passes to a fine-tuned SDXL model generating eight candidates. PickScore aesthetic scoring surfaces the top four.

Frontend: React app serialising user input to JSON, calling a local ComfyUI API on a dedicated 4070 Ti.

QA: Output quality was evaluated manually via structured internal scoring — semantic relevance, human realism, brand coherence — with findings presented to the client showing both the model's current state and the iteration trajectory.

Infrastructure was scoped as a proof of concept. Scaling was discussed after delivery but wasn't part of the original brief.

03 · The Decisions

The central problem was fidelity — making generated outputs indistinguishable from the client's existing illustration library. Two entangled variables: training parameters and inference parameters.

Primary evaluation method was XY plots — systematically comparing outputs across parameter values. It's a manual process by necessity, because illustration fidelity is a judgment call, not a metric.

Example of an XY plot with the desired images in box

ELLA with SD 1.5 produced richer outputs but injected uncontrolled details. Style fidelity held; user control didn't. Rejected: a tool that produces beautiful outputs the user can't direct is not a tool for marketers.

Adobe Firefly — including its fine-tuning tools — was tested and invalidated. The client had reached the same conclusion independently, meaning the case for a custom pipeline was a shared finding, not just my recommendation.

Fine-tuned SDXL with locked inference values gave controllable results — the right trade-off for a tool meant to be operated by marketers, not ML engineers.

04 · The Complexity

The main client was an innovation lead in Singapore with APAC remit. The real decision-maker was her boss in Hong Kong. Approval required passing the client's internal AI council, then a presentation to the regional CEO. The pipeline wasn't just a technical deliverable — it was a governance case.

Work spanned Singapore, Japan, and Korea, each with different local use cases. The Hong Kong brand custodian held the final call on visual output, so evaluation criteria were designed as proxies for how a non-technical senior stakeholder would read the results.

Upon approval, the client rolled the tool out at a regional internal event, where I ran a prompt workshop — the moment the system moved from demonstration to actual marketer use.

05 · The Evidence

Three phases of structured documentation: model training, model optimisation, and prompt optimisation — each run as iterative loops between ComfyUI, internal QA scoring, and client review.

Experiments log tracks LoRA versions across training set size, repeat count, epoch count, and total steps, with hypotheses and results per iteration.

Structured batch scoring: each prompt tested across candidates, scored against semantic relevance (prompt adherence), human realism (no deformities), and brand coherence (colours, shapes). Pass/fail tracked per image, per market, per scenario.

XY plots documented parameter optimisation: LoRA Master and CLIP strength tested across a grid, optimal range marked. A data flow diagram shows the full ComfyUI pipeline — base SDXL model, LoRA, IP Adapter, KSampler, CLIP encode, latent space, PNG decode, SVG vectorisation — through to the frontend.

Examples of images generated in the three sizes

Examples of images in training set

Infrastructure: Local 4090 / 4070 Ti instances accessed remotely via Tailscale and RustDesk, with Runpod for Kohya SS training runs.

06 · My Contribution

Technical lead and client-facing lead for all technical matters. Led a rotating team across three phases. Designed and iterated the pipeline architecture, developed the LoRA evaluation process, built the ComfyUI inference pipeline, ran the XY plot methodology, and presented at each client review. Delivered a prompt workshop to regional marketing teams at the internal rollout event.

On the client side: regular conversations with the innovation lead and her team across Singapore, Hong Kong, Japan, and Korea — showing possibilities, explaining constraints, running feasibility tests. The client team valued the transparency; being equipped to explain the work to their own stakeholders was as important to them as the outputs. That relationship was the direct reason a phase two was requested.

Phase two was ultimately rejected — not by the client, but by the client company's AI council. Productionising would have required handing over the pipeline architecture and ways of working, which the agency couldn't agree to. The project ended at the boundary between a successful proof of concept and a transfer of capability the client couldn't have on those terms.

Collaborators

R/GA x MetLife