Tsubaki.3

Tsubaki.3 composes text-to-image and in-context tasks into one unified pretraining pipeline. It starts from one shared backbone and forks into two branches. Text-to-image pretraining gives the model world general concept knowledge, and text-to-image mid-training provides anime concept knowledge. The shared base model that follows takes both images and text as input. From there, training splits into two tracks that specialize the same backbone toward two different jobs. One track generates from text, and the other generates from a reference. Each track then carries its own distillation. The text-to-image track finally runs a post-distillation RL task for last-mile alignment. The in-context track continues pretraining on image-conditioned data, and it distills from there. The complete training sequence is shown in Figure 1.

Tsubaki.3 supports two broad categories of generation: text-to-image and in-context generation. Text-to-image prompts are descriptive: they specify what belongs in the image. In-context prompts are instructive: they specify what to do with the input images. We further divide in-context generation into three task families: local editing, reference generation, and reference editing. The table below summarizes the inputs and preservation requirements of text-to-image generation and these three in-context families.

At multi-node scale, each synchronized step waits for the slowest training process, or rank. We record data transfer, forward, backward, optimizer, and checkpoint timings on every rank and compare them with GPU telemetry from NVIDIA Data Center GPU Manager (DCGM) and NVIDIA Management Library (NVML). Phase timings show where a delay appears; device and interconnect counters help distinguish a compute imbalance from a hardware or communication problem.

Resolution and native aspect ratio change the shape of the workload. A long 1536 bucket raises image-token count, activation residency, and collective volume together, so each resolution and token geometry receives its own performance baseline. Within each baseline, we compare typical rank times with the slowest ranks. A widening gap shows that faster ranks spend more time waiting, even when average GPU utilization remains high.

The 1536 phase also exposed a late activation peak during the first full transformer pass. Selective checkpointing provided the best path at shorter sequences, while full-block checkpointing gave the high-resolution run a stable memory envelope. FlashAttention^[8] forward outputs remain saved inside that policy, since recomputing attention can cost more than retaining its compact result. This policy saves memory by recomputing surrounding operations while retaining the attention results that are expensive to reproduce.

The distributed layout follows the hardware topology. Tensor and sequence parallel groups stay within each eight-GPU NVLink domain; FSDP2 carries model-state sharding across nodes. An initialization audit verifies that every trainable parameter is represented as a DTensor and that each rank owns the expected local shard. This small preflight check catches mesh and wrapping errors before the first sustained allocation. Parameters remain in bf16, while reductions accumulate in FP32.

DCGM profiling values are interval averages, so we use them as a fleet-level locator and open a focused profiler trace on the affected rank for kernel attribution. Counter deltas are ranked by rate, concentration on specific ranks or links, and alignment with the step-time change. The DCGM field catalog provides the device and fabric counters; health watches keep passive coverage during training, and active diagnostics run before a large allocation and after an isolated hardware event.

Checkpoint cadence is set against measured restart cost. Model, optimizer, random-state, and data-position shards become visible through one completed manifest, which gives replacement ranks one consistent recovery point. A watchdog combines collective heartbeats with hardware health events, fences the affected worker, and resumes from the latest complete checkpoint. This turns intermittent GPU and node events into bounded interruptions while preserving the exact training stream.

We construct the SFT dataset in two filtering stages. First, automated selection reduces the pre-training pool to 30k candidates using our in-house aesthetic scorer alongside complexity, style, and semantic-balance metrics. Next, in-house annotators trained in style alignment select 7k images based on consistent style and high visual aesthetics. We fine-tune the model using a uniform timestep schedule with a 0% caption-dropout rate to preserve the pre-trained unconditional prior. As shown in Figure 11 of Text-to-image evaluation, SFT improves aesthetic quality over the base model while prompt following degrades. That figure carries the quantitative comparison. We also find that after SFT, anatomical and hand artifacts become more likely to appear. These issues are addressed in the subsequent RL stage.

RL runs as a two-stage cascade: correctness alignment first, aesthetic enhancement second. Early experiments showed that correctness RL alone hurts style diversity: the reward model is biased toward plain, rigid images, which sacrifices the aesthetic qualities an anime illustration model should preserve. To counteract this, we introduce Stage 2, aesthetic RL, which reverses the drift. Stage 1, the correctness stage, uses a reward system built from four task-specific rubrics. Stage 2 uses two aesthetic rewards: our in-house aesthetic scorer and a VLM-based aesthetic rubric evaluator distilled from illustration-industry standards. Figure 6 summarizes the cascade.

Both stages share one optimization objective: an online group-relative policy optimization (GRPO)-style^[9] RL algorithm for diffusion-based image generation^[10][11]. For each prompt the policy generates a group of k candidate images. The group is scored and normalized inside itself, so each prompt is its own baseline. Above-average trajectories pull the policy, and below-average ones push it away. The policy is scored on its latest generations, which lifts the optimization ceiling above what offline data allows. Rollouts use a deterministic ODE sampler with classifier-free guidance (CFG)^[12] at CFG = 1.0. Training touches the conditional branch only, with no explicit KL penalty against a reference policy. Ablations show that a larger group size k and more prompts per batch both raise the final reward ceiling, so we maximize both. The per-iteration cost rises, and the run converges quickly in return: it plateaus at about 100 optimization steps.

The correctness alignment stage is divided across four specialized tasks: body anatomy, multi-subject, interaction, and text rendering. Each task brings its own prompt pool and its own reward system. We use mixed-task training with a cyclic batch mechanism: at every step of a cycle, one task is drawn at random and optimized against its own reward. Two tasks never share a batch, so each reward keeps its own scale, and no dominant gradient washes out a weaker one. Both the data and the reward systems are built agentically.

Figure 7 shows the effect directly in the pixels, with every error circled in red on the SFT side. The malformed hand and the shapeless companion cat are corrected (1). The extra arm in the clash disappears, and the punch lands exactly where the prompt says it should, on the opponent's stomach (2). The jumbled road recovers its spatial logic (3), and the twisted table legs come out right (4). As also shown in Figure 11, after the correctness RL stage, prompt following climbs well above the pretrained baseline, while the aesthetic score gives back part of the SFT gain but still stays above where it started.

Stage 2 starts from the final Stage 1 checkpoint and uses two aesthetic rewards: an in-house aesthetic scorer and a rubric-based evaluator.

In-house aesthetic scorer. A ViT-based scorer that maps an image to a continuous aesthetic reward. Our annotation team first built a large set of fine-grained aesthetic quality ratings, and the scorer was trained on these human preferences.

Aesthetic rubric evaluator. What makes an illustration good depends on the kind of illustration it means to be. A flat-color piece lives on clean silhouettes and a controlled palette, a painterly one on brushwork and edge discipline, and a cinematic scene on light and depth of field. Drawing on the principles and techniques of illustration, we wrote concrete rubrics for different artistic styles and scenes, and a VLM judge scores each image against the rules that apply to it.

In this stage, we use mixed training with the GDPO^[13] strategy. Each reward is first normalized into its own group-relative advantage, and the advantages are combined only after that. A scale difference between a rubric score and a model score therefore cannot let one signal dominate the other. The training prompts for this stage are drawn from high-quality user data, including both raw user prompts and prompts processed by our automatic prompt-enhancement pipeline, described in the Prompt Helper section. During this stage, aesthetics rises while prompt following drops, so we pick the point on the Pareto frontier that balances the two as the final Tsubaki.3 model.

Many projects use a VLM judge as the reward, but one unified reward model cannot serve every image. What is critical for one prompt could be trivial for another, and in our early experiments, we found that one shared standard accelerated reward hacking. We therefore built an agentic rubric reward system that writes rubrics per prompt.

The agent carries six tools. Every arrow in Figure 8 is a call to one of them.

The agent runs an autonomous four-stage loop. Each step issues one tool call and receives the tool's output as the next observation, which keeps every decision attributable and makes the trace easy to audit.

• Draft rubrics. For one training prompt, the agent routes it to a category, inherits the shared checks, writes the case-specific checks, and weights each item by importance.
• Recalibrate the verifier when a rubric needs it. The verifier is a VLM with fixed instructions, frozen by default. Occasionally a new prompt needs a check it cannot yet judge, so the agent adds an instruction for it. Any such edit counts as a model change and must pass the golden anchors, a human-labeled set, on two conditions: agreement with the human labels stays above threshold, and scoring the same case twice gives the same result. Only then is the verifier frozen again.
• Probe on rollouts. The rollout sampler scores a batch of images from the current policy under the approved rubric. The agent reads the spread to tell whether the prompt is saturated, too hard, or in the range that still carries signal.
• Prune and backfill. Prompts where all rollouts pass or all fail are pruned. The data retriever fetches replacements and the cycle restarts.

The hardest problem in RL is reward hacking. We defend against it in two ways.

The first is detection and early stopping. Under reward hacking, the training reward and validation scores can keep climbing while visual quality degrades. We therefore keep a fixed, hand-picked evaluation set, generated with fixed seeds at every checkpoint for a quick visual review. The set is chosen for its sensitivity to known failure modes: washed-out overexposure, synthetic grain, and distorted glyphs. When the scores rise and visual quality on the evaluation set degrades, training stops.

The second is recovery. After detecting reward hacking and stopping the run, we apply a short SFT pass to the stopped checkpoint to pull the model out of the local optimum. Early experiments show that this brings the model back toward the real-data distribution within a small number of steps. RL then resumes from the corrected state. This recovery cycle is illustrated in Figure 9.

Evaluation is organized around the ways users address the model: named long-tail concepts, combinations of several concepts, character identity, style treatment, readable text, and high-resolution detail. Each capability keeps its own slice, while paired comparisons and confidence intervals resolve close checkpoints. This preserves the distinction between identity, style, composition, and surface quality when selecting the next training stage.

The fixed-bank slices are paired with a shared rubric for reference generation and editing. Consistency, instruction following, and output quality apply to both settings; their weights shift with the task. Full-frame checks remain beside crop-level detail measurements because high-resolution texture can mature earlier than character identity, relation accuracy, or source-layout preservation. This combination gives each training phase a compact, capability-oriented reading.

Every candidate checkpoint lands on the same two axes. The horizontal axis is weighted quality, which combines HPSv3^[16] and QScore in one z-scored number. HPSv3 measures visual preference, while QScore, our in-house image quality model, measures how well an image is made. The vertical axis is the inhouse anime SigLIP2 target hit rate. The legend reports full-cast preservation alongside the two plotted measurements for each checkpoint on the post-training path.

The four colored points in Figure 11 follow one path through the pipeline. Pretraining sets the starting point. SFT raises quality and full-cast preservation, and it costs a little target fidelity. Correctness RL then lifts the target hit rate to the highest value on the chart, at some cost in quality. Aesthetic RL recovers the quality and holds the target hit rate near the top. A longer aesthetic RL run buys more quality again while giving back a small amount of target fidelity and some full-cast preservation; its target hit rate remains slightly above pretraining.

The point marked Tsubaki.3 represents the full, undistilled model. It carries most of the quality gain while the target hit rate stays near its best value. The adjacent checkpoints show the trade-offs on either side of this operating point.

A typical request on our platform is a handful of words: a few tags, a sentence in Japanese or Korean, often both. The captions the generator learned from are long, specific, and written in a house dialect of the training pipeline. That is the first gap, and closing it is worth real quality.

The second gap sits on top of the first. A perfectly clear request can still be a dull one. A girl sitting on a chair fixes the subject and leaves the camera, the light, the mood and the story open, and something has to decide those before the image becomes interesting. Prompt Helper does both jobs: it translates the request into the generator's dialect, and it reads the context to make the choices the user left open. Anything stated explicitly carries through untouched; the model writes into the space left blank.

Teaching that mapping requires pairs, and each half comes from a different world. Genuine user prompts arrive without an answer; training captions are ideal answers nobody would type. We assemble the dataset from both directions, so each end of the mapping is anchored in something real.

We tuned the forward rewriter by measurement. We rendered images from candidate rewrites, put them through large-scale human arena comparison, and let the winning version generate at scale. Backward synthesis is conditioned on a target length drawn from the empirical user distribution, so the short, casual prompts that make up roughly a third of real traffic keep their place in the data. Matching how users write turned out to matter as much as matching what they mean.

On top of the pairs we generate reasoning traces: letting the model first reconstruct what was being asked for measurably improves what it writes afterwards. A modest amount of deliberate reweighting finishes the mix. Artistically dense and visually inventive material gets extra representation, scene and object material keeps the model fluent beyond portraits, and prompts headed toward photorealism carry a light medium cue. The aim is coverage wide enough that both art-directed and plainly photographic requests come out well.

The model also knows when to hold back. When a user has loaded a character or style low-rank adaptation (LoRA)^[6], those visual choices are already made. We precompute variants of every target caption with traits stripped, style stripped, or both, and train on the variant matching the switch state, so restraint is learned behavior rather than an inference-time instruction.

Hitting a synthetic target and making a better picture are two different achievements. The next stage optimizes the rewriter against the images it actually causes to exist, under two families of signal. One is measured on the generated image: how good it looks and how much it is preferred. The other is checked against the text. For each request we precompute a checklist of the concrete things the user specified, every attribute, object, pose and style cue, and score how many survive the rewrite. Safety and constraint checks act as a veto over the combined score.

The faithfulness score is computed deterministically from a structured judgment, which narrows the surface available for reward hacking. Every reward component must clear held-out human-labeled cases before it is permitted to move a gradient.

The pool is built around the cases that matter most in production. Three patterns recur, and each gets a dedicated slice of data. The bad outputs are fluent, well-structured, and wrong, so a larger pool of ordinary traffic does not remove them.

Ordinary traffic forms the base, and one slice keeps its retrieval clean and untouched, so the model learns correction and trust in references side by side. Throughout, we favor prompts that are difficult without being impossible: a batch where everything succeeds and a batch where everything fails teach equally little.

Landing inside the training distribution has a ceiling: optimize for that alone and the best output converges on the distribution's average. We generated thousands of images from the same source prompts through many rewriting strategies and ranked them pairwise with two independent evaluators. Several strategies beat the shipping baseline on both.

What separates the winners is judgment rather than vocabulary: deciding where the eye should land, choosing light with a visible source, letting composition follow from the kind of image being made. The strategy that stamped one fixed stylistic treatment onto every prompt finished last on both evaluators, which is our clearest evidence against a house look. The two evaluators also rank the strategies differently, so a strategy that wins on one of them has yet to win.

These findings became the stronger expansion settings. Expansion is a spectrum: off, translation and retrieval only, the standard rewrite, and a fully art-directed mode that classifies the image type first and fills in only the dimensions the user left blank. The user's explicit choices override the model's additions, and the model's additions override defaults.

We have open-sourced Tagger 1.0. Check it out at https://huggingface.co/pixai-labs/pixai-tagger-v1.0.

The architecture starts with the ViTDet-style^[18] image backbone from SAM3 and processes images at 1008² resolution. A classification head reads the backbone features and predicts the 30,877 tags. During training, a small reconstruction decoder reads the same features and encourages them to retain fine visual details. Tag prediction and evaluation use the classification head alone.

The evaluated instance is the 80,531-image test split, held out from training and never used for checkpoint selection. Each image is scored against the model's own 30,877-tag vocabulary: for every tag, confusion counts are accumulated at 100 threshold points from 0.00 to 0.99 in steps of 0.01. Macro averaging computes precision and recall per tag, then means across tags. Every tag counts equally, so tags near the 300-image admission floor pull the score down. Micro averaging pools true positives, false positives, and false negatives across all tags before it takes one ratio, so frequent tags dominate. For each displayed category and aggregation we report the threshold that maximizes F1 on this split (ties resolved to the plateau midpoint).

The difference between micro-F1 and macro-F1 shows where the model performs well. Micro-F1 is influenced most by common tags, while macro-F1 gives common and rare tags equal weight. A high micro-F1 with a much lower macro-F1 therefore means that common tags work well but rare tags remain difficult. This pattern is strongest for Meta and is also visible for General. Character and Copyright show smaller gaps, suggesting more consistent performance across frequent and infrequent tags. Rating contains only four tags and is less affected by this imbalance.

General tags have an additional evaluation challenge: the ground-truth annotations are often incomplete. An image may contain a valid attribute that its annotator did not record, so some predictions counted as errors may actually be correct. In practice, General and Meta predictions are best used as a strong first pass followed by human review—General because valid attributes may be missing from the source labels, and Meta because performance is less consistent on rare tags.

We compare Tagger 1.0 with our previous release, pixai-tagger v0.9, under one controlled protocol.

The comparison includes only ground-truth tags that appear at least five times and belong to both models' vocabularies. This produces a shared set of 8,407 General tags and 2,099 Character tags. Tags outside these intersections do not enter the scores. F1 denotes micro-F1; macro-F1 gives every tag equal weight, and mAP is computed from predictions retained above the 0.05 sparse-storage floor.

On the shared vocabulary, Tagger 1.0 improves General micro-F1 by 6.8 points and Character micro-F1 by 10.4 points over v0.9. Macro-F1 and mAP improve in both categories as well. Because both rows use the same controlled protocol and shared vocabulary, these differences measure the models under the same conditions rather than comparing separate release benchmarks.

1. M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdulmohsin, N. Parthasarathy, T. Evans, L. Beyer, Y. Xia, B. Mustafa, O. Hénaff, J. Harmsen, A. Steiner, and X. Zhai. SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features. arXiv:2502.14786, 2025.
2. E. Pizzi, S. Dutta Roy, S. Nagavara Ravindra, P. Goyal, and M. Douze. A Self-Supervised Descriptor for Image Copy Detection. arXiv:2202.10261, 2022.
3. M. Douze and others. The Faiss library. arXiv:2401.08281, 2024.
4. H. V. Vo, V. Khalidov, T. Darcet, T. Moutakanni, N. Smetanin, M. Szafraniec, H. Touvron, C. Couprie, M. Oquab, A. Joulin, H. Jégou, P. Labatut, and P. Bojanowski. Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach. arXiv:2405.15613, 2024.
5. L. Zhang, A. Rao, and M. Agrawala. Adding Conditional Control to Text-to-Image Diffusion Models. 2023.
6. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-Rank Adaptation of Large Language Models. 2021.
7. C. Meng, Y. He, Y. Song, J. Song, J. Wu, J.-Y. Zhu, and S. Ermon. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations. 2021.
8. J. Shah, G. Bikshandi, Y. Zhang, V. Thakkar, P. Ramani, and T. Dao. FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision. arXiv:2407.08608, 2024.
9. Z. Shao and others. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. 2024.
10. J. Liu, G. Liu, J. Liang, Z. Yuan, X. Liu, M. Zheng, X. Wu, Q. Wang, M. Xia, and others. Improving Video Generation with Human Feedback. arXiv:2501.13918, 2025.
11. K. Zheng, H. Chen, H. Ye, H. Wang, Q. Zhang, K. Jiang, H. Su, S. Ermon, J. Zhu, and M.-Y. Liu. DiffusionNFT: Online Diffusion Reinforcement with Forward Process. arXiv:2509.16117, 2025.
12. J. Ho and T. Salimans. Classifier-Free Diffusion Guidance. 2022.
13. S.-Y. Liu, X. Dong, X. Lu, S. Diao, P. Belcak, M. Liu, M.-H. Chen, H. Yin, Y.-C. F. Wang, K.-T. Cheng, Y. Choi, J. Kautz, and P. Molchanov. GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization. arXiv:2601.05242, 2026.
14. T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, and T. Park. One-step Diffusion with Distribution Matching Distillation. CVPR, 2024.
15. D. Jiang and others. Distribution Matching Distillation Meets Reinforcement Learning. arXiv:2511.13649, 2025.
16. Y. Ma, Y. Shui, X. Wu, K. Sun, and H. Li. HPSv3: Towards Wide-Spectrum Human Preference Score. arXiv:2508.03789, 2025.
17. N. Carion and others. SAM 3: Segment Anything with Concepts. arXiv:2511.16719, 2025.
18. Y. Li, H. Mao, R. Girshick, and K. He. Exploring Plain Vision Transformer Backbones for Object Detection. arXiv:2203.16527, 2022.