Skip to content
Creator Link Studio

LoRA training dataset tips that fix character drift

A LoRA is only as good as the images you feed it. Here is how to build, clean, caption and test a dataset so your character looks like the same person every time.

Creator Link Studio8-minute read

Most LoRA problems get blamed on settings. Learning rate, steps, network rank, the base model. Those matter at the margins, but if your character's face shifts between generations, the dataset is almost always the cause. A LoRA learns whatever is consistent across your images. If the only consistent thing is a studio background and a three-quarter angle, that is what it learns.

These tips assume you want one character who stays recognisable across many images, outfits and scenes, not a style LoRA. The work is unglamorous: collecting, cropping, deleting, captioning. Budget most of your time there and the training run becomes a short, boring step at the end.

Decide what the LoRA has to memorise

Before you collect a single image, write down what must stay fixed and what is allowed to change. Fixed usually means face structure, eye colour, hair colour and texture, body proportions, and any permanent marks like a mole or tattoo. Variable means pose, outfit, lighting, background, expression and camera distance.

This list is not admin. It decides every later choice. Anything on the fixed list needs to appear in nearly every image, identically. Anything on the variable list needs to change often, because repetition teaches the model that it is part of the character.

A common failure: every reference image has the same haircut and the same jacket, so the LoRA treats the jacket as part of the person and refuses to put the character in anything else. Another: half the images have heavy warm lighting, so the character comes out orange no matter what you prompt. If you are still deciding how your character should look at all, settle that first in how to train a LoRA character that stays consistent, then come back and build the set.

How many images, and what mix

There is no magic number, and more is not automatically better. A tight set of 20 to 40 clean, varied images of one person usually beats 200 images where half are near-duplicates or low quality. Extra images only help when each one adds information the others do not have.

Build a variety budget

Think of your set as a budget you spend across categories rather than a pile of pictures. A workable split for a character LoRA:

  • Close-up face shots, several angles: roughly a third of the set. These carry identity.
  • Upper body and waist-up: another third. These connect the face to the body.
  • Full body, standing and seated: the remainder, enough that the model learns proportions.
  • A handful of unusual angles: slight profile, looking down, three-quarter from above.

Within that, vary expression (neutral, smiling, talking), lighting (soft daylight, indoor warm, flat overcast) and background (plain, interior, outdoor). Keep clothing varied too, unless a specific outfit is genuinely part of the character.

What to leave out

Cut anything where the face is blurred, obscured by hair or hands, or smaller than a few hundred pixels across. Cut heavy filters, strong colour grades, motion blur and harsh compression artefacts. Cut images with other people in frame unless you crop them out entirely, because the model will happily learn a second face. Cut anything where the character's identity would be ambiguous to a stranger looking at the image cold.

Deleting feels wasteful. It is the highest-leverage thing you will do all day.

Prepare the files before you think about settings

Once you have your selects, spend an hour on file hygiene. This is where most of the quality comes from.

Resolution and cropping

Work at or above the resolution your training setup expects, never below. Upscaling a small image to meet a requirement adds nothing and often adds mush. If you only have a small source image, either cut it or use it as a close-up crop where it is actually sharp.

Crop with intent. For face shots, leave some headroom and shoulders rather than cropping tight to the jaw, so the model learns head shape rather than just features. For full body, keep feet in frame. Avoid awkward crops that slice through a knee or elbow, because the model learns those amputations and reproduces them.

Hunt down near-duplicates

Burst shots are the enemy. Ten frames from the same two seconds look varied to you and identical to the model, and they will dominate training. Pick the single best frame from each burst and delete the rest. The same applies to images generated from one seed with small prompt tweaks.

Keep consistent quality, not consistent content

Mix resolutions and aspect ratios if you must, but keep sharpness and exposure broadly even. One very dark image in a set of bright ones will not break anything, but five will push everything dim. The same logic that governs reference sets for prompting applies here, and it is worth reading how to keep a consistent AI character across images alongside this, because the reference discipline is the same discipline.

Caption for what changes, not what stays

Captioning is where people overwork and get worse results. The principle is simple: caption the variable things, leave the fixed things to the trigger word.

Use one short, unusual trigger token for the character, the same token in every caption. Then describe what is different in that specific image: the pose, the clothing, the setting, the lighting, the camera distance. Do not describe the eye colour, face shape or hair colour in every caption if those are meant to be baked into the character, because describing them teaches the model they are optional attributes it can be asked to change.

A caption pattern that works

Keep captions consistent in structure so the model sees a clean pattern:

[trigger], [shot type], [pose], [clothing], [setting], [lighting]

For example: tokenname, close-up portrait, head turned slightly left, black knit sweater, plain grey background, soft window light.

Short and literal beats poetic. Avoid words like "beautiful", "stunning" or "cinematic" in training captions; they describe your opinion, not the image, and they drag in whatever the base model associates with those words.

Check captions against the image, one by one

Auto-captioning tools save time and get things wrong. They hallucinate objects, miss clothing details and sometimes invent a second person. Read every caption next to its image once. On a 30-image set that takes fifteen minutes and it catches the errors that would otherwise show up as mystery artefacts later. For the full pipeline around this step, including the training run itself, see Flux LoRA training: a practical guide for creators.

Test the result like a sceptic

After training, do not judge by one lucky generation. Run a fixed test battery and run the same battery after every retrain so you can compare like for like.

A reasonable battery: a neutral close-up, a full body in plain clothing, the character in three outfits you never trained on, the character in two unfamiliar settings, and one shot at an unusual angle. Generate several images per prompt at different seeds.

Read the failures as dataset notes

Each failure points back at the set:

  • Face drifts between seeds: not enough close-ups, or close-ups too similar to each other.
  • Character always wears the same thing: that garment was over-represented, or you captioned it inconsistently.
  • Body proportions wrong at full length: not enough full-body images.
  • Strange colour cast: lighting in your set was too uniform.
  • Character ignores the prompt's setting: backgrounds in your set were too repetitive.
  • Extra limbs or warped hands at distance: too many tight crops, too few clean full-body frames.

Fix the dataset, retrain, run the same battery. Two or three rounds of that usually gets you somewhere stable. If you would rather not run the pipeline yourself, character generation in Creator Link Studio trains a character once and then generates photos and video of the same person on demand, which is the same loop with the infrastructure handled.

Keep the dataset as an asset

Treat your final set as a durable thing, not scratch files. Keep the selects, the captions and a short text note recording the trigger word, the base model and the settings you used. When a better base model arrives, you retrain in an afternoon instead of rebuilding from nothing.

Version it. Folder names like character-v1, character-v2-more-fullbody are enough. Note what changed between versions and what the test battery showed. Six months in, you will not remember why v3 was better than v2 unless you wrote it down.

Then put the character to work

A consistent character is only worth the effort if it feeds something. The usual path is a public page where the images live as posts, pay-per-view unlocks, bundles and membership tiers, with payouts to your bank. On Creator Link Studio that page sits at one address with everything on it, and the platform take rate runs from 3% to 20% depending on plan, falling as you grow. If you are still choosing where to host all of this, how to compare and choose an online creator platform walks through the criteria that actually change your income.

And if the character exists but the offer does not, sequence the business side the same way you sequenced the dataset. The 30-day plan to monetize online content gives you a week-by-week order so the work does not stall at "I have a great character and nothing to sell".

Say you want $1,000 a month from 50 members. That is a $20 tier, and a trained character means you can actually deliver fresh images to those 50 people every week without a shoot. The dataset work is what makes that sustainable.

Frequently asked questions

How many images do I need for a LoRA character dataset?
A tight set of roughly 20 to 40 clean, varied images of one person is usually enough for a character LoRA. More images only help when each one adds something the others do not: a new angle, a new lighting condition, a different outfit. Two hundred near-duplicate frames will train worse than thirty deliberate ones.
Should I caption the character's face and hair colour?
Generally no. Caption what changes between images, such as pose, clothing, setting and lighting, and leave the fixed identity to a single trigger word used in every caption. Describing eye colour or face shape in every caption teaches the model that those traits are adjustable attributes rather than part of the character.
Why does my LoRA character keep wearing the same outfit?
Because that outfit was over-represented in the dataset, or it was not captioned consistently, so the model folded it into the character's identity. Fix it by adding images in several different outfits and making sure every caption names the clothing in that specific image.
Can I use AI-generated images in my training dataset?
You can, and many people bootstrap a character that way, but be strict about near-duplicates. Images generated from one seed with small prompt tweaks look varied to you and almost identical to the trainer. Pick one frame per concept, keep sharpness even, and check that faces are genuinely consistent before you include them.
How do I know if the problem is the dataset or the training settings?
Run a fixed test battery after every training run: a neutral close-up, a full body, three unfamiliar outfits, two unfamiliar settings, one unusual angle. Settings problems tend to show as global blur, burnt-in artefacts or a LoRA that does nothing. Dataset problems show as specific drift, a stuck outfit, wrong proportions or a colour cast.

Keep reading

  1. Guide

    How to train a LoRA character that stays consistent

    Training a LoRA character is mostly dataset work, not settings work. Here is how to build the images, caption them, run the training, test the result, and put a consistent character to work.

    , 8-minute read

  2. Guide

    Online creator platform: how to compare and choose one

    An online creator platform is where your audience, your offers and your money meet. Here is how to compare the options on the four things that actually change your income, and how to set one up without stalling.

    , 7-minute read

  3. Guide

    Flux LoRA training: a practical guide for creators

    Flux LoRA training is mostly dataset work with a short compute step at the end. Here is how to build the images, caption them, run the training, test the result, and put a consistent character to work on a page that sells.

    , 8-minute read