Abstract
Hellow fellow AI-generated goblins of r/StableDiffusion For a few weeks I’ve been stress-testing my tool, Arthemy Krea-2 Tuner: https://github.com/aledelpho/comfyui-arthemy-krea2-tuner I wanted to investigate if nudging the weights of single blocks of Krea-2 can teach us something about its internal structure. https://preview.redd.it/ouhfy67e1gvh1.png?width=1600&format=png&auto=webp&s=d96358fb07c9b2d6e81145d79cf93d5aa8c1bb61 I worked on two parts: the model and the text encoder (I only started on the last one recently but it seems much more interesting than the previous one) . The underlying question was whether the models already holds “ knobs ” that change style, look, colours or literally anything else in the image, without training anything. After weeks of work and thousands upon thousands of renders I found a grand total of… one ! (underwhelming, I know, but better than nothing) The upside, if something cool comes out of it: It’s still sci-fi at this point, but if we manage to map the inside of a model this way, we might be able to make presets that do the job of a small fine-tune with nothing trained: the presets I use now are 34 numbers, a few hundred bytes, against the tens or hundreds of MB of a typical LoRA. I know the approach is crude. It’s like shooting a car with a shotgun to understand how it works. But it can still tell where the windows are (they make a specific noise) and which parts are vital. Breaking a piece and watching what stops working is an old trick in interpretability and it tells where something happens, not how . Said so: when we know so little about these models, I don’t think that’s useless. Of course, I fixed the test rules before looking at the data, and I started summoning my girlfriend (who doesn't know anything about AI or what I'm trying to do) to do the most important blind tests. Limits: one model, almost always one observer (yours truly) , small samples. When a conclusion goes beyond the data, I mark it with "🔎" that’s a lead, not proof. TEXT ENCODER 1. Text-encoder edits change what is drawn (genre, style, who the character is) and hurt image quality much less than edits to the drawing model Each row is one character with the same seed and prompt. Left: the original. Right: the same seed after DOUBLING one layer of the text encoder. Nothing gets noisy or broken, but the style and even the person change, and the same layer pushes the two characters in different directions. This part I measured : with standard quality metrics, text-encoder edits don’t make the image worse, while a preset on the drawing model does. Even at full strength (x2.0), no picture fell apart. In exchange, they are less “consistent”: the same edit pushes different characters in different directions. Only 4 edits out of 20 moved them all the same way. 🔎 My impression is that instead of damaging the picture, these edits break the links between the words of the prompt. I tried to test it, but on the images I had I couldn’t define it consistently, so I need renders made for the purpose. 2. Anime isn’t the only fallback for illustration. https://preview.redd.it/rr88gxsmvfvh1.png?width=1600&format=png&auto=webp&s=2cb02f9546af6ed1ca194d5d779ac5b5b82d50d3 People often say that "anime" is the default of almost every model when it comes to illustration. In Krea-2 it isn’t. ( 🔎 I was surprised because I was coming from Anima and Illustrious where Anime seems to be a stronger "default" fallback) When I removed the style words from the prompt, one character became anime every time, while seven others turned into paintings or photographs: each one has its own fallback. Weight edits, on the other hand, don’t lead to the character’s fallback . Each edit has its own destination (one towards realism, one towards cartoon), whoever is being drawn. 🔎 My hypothesis: every concept falls back on the style, and the traits, that the model has seen with it most often. A text-encoder edit doesn’t seem to push toward one default, but toward some of the styles that concept already lives near. That would explain why a character can change style so heavily, but only within a small set: my "elf" prompt turned into comics, anime or cartoons, never into a photo, while the same edit turned most other characters realistic. What I found points that way, though it doesn’t prove it. I have no access to Krea-2’s training data, and the same effect could come from an aesthetic fine-tuning stage or from the text encoder’s own associations. Some clues: Single words, generated on their own, carry preconceptions: “female dwarf” gets a beard 4 times out of 4 (Gimli approved); “uniform” becomes an anime schoolgirl; “western” draws cowboys. What a surprise, right? When I alter the text encoder, characters lose their specific traits first and drift towards a more “average” version later. My half-orc becomes an ordinary human woman, still drawn as a comic and THEN it loses its style. A random push of the same strength doesn’t do this: it depends on where the edit points, not on how strong it is. https://preview.redd.it/qcsmi2l42gvh1.png?width=1600&format=png&auto=webp&s=debb17fb96f0d7f172800b123468b5224d99776a 3. Commas seem to act as borders https://preview.redd.it/yhfsnolzvfvh1.png?width=1600&format=png&auto=webp&s=9936729238e8c722726b2e39cc74e94cc121c374 When I edit the text encoder, the meaning the model gives each word shifts a little . The shift has two parts. One is the same for the whole prompt, like a filter laid over everything. The other is local: neighbouring words move together in packets of 2–4, and the packets tend to break exactly where there’s a comma. For example, in “dark green hooded cowl, brass buckles”, “dark green hooded cowl” moves as one block and “brass buckles” as another. 🔎 This might suggests the text encoder uses commas to decide which words belong together. For prompt writers that could be an important lead: maybe putting an object and its colour inside the same comma-phrase helps keep them bound. But I measured this on the numbers the text encoder hands to the image model, not on the pictures. I haven’t yet checked whether moving commas actually changes the result, or by how much. 4. Shuffled prompts (still under study). 🔎 Edits to the drawing model seem to give more or less the same look even when the phrases of the prompt are shuffled, but that test came out inconclusive. With the text encoder it depends on the edit: some send every phrase order to the same result, others scatter them across different genres. I don’t know why yet, I’m working on it. Each column shows the same prompt with its phrases rearranged. the first row shows the baseline and it gives you an idea about what I was supposed to get - now take a look at the last row ... what the hell is going on? it's the same edit! 5. Switch off the first layer of the text encoder, and the model writes your prompt instead of drawing it. https://preview.redd.it/dint6566kgvh1.png?width=1600&format=png&auto=webp&s=ee191728324fa575da5402e445b19aea89e3e072 With the attention of layer 0 switched off, Krea-2 drew pages of text in 4 prompts out of 4, with readable fragments of the prompt (“close-up”, “frontal view”; in the cleanest case realistic western comics style and the prompt’s own Lighting: section). Switching off the MLP did it in 2 of 4. Switching off the whole layer gave no text at all, only repeating noise. So it isn’t “the model sees raw words and writes them”: it takes a half-broken layer, and I don’t know why . https://preview.redd.it/qd0dy7r7kgvh1.png?width=1600&format=png&auto=webp&s=b29e3c557573e86b60826a02afb43d91d0c9428e THE MODEL 1. Different blocks do different things. https://preview.redd.it/yp5xuef73gvh1.png?width=1600&format=png&auto=webp&s=c3c3834a958a27226acca9e9e75efe088da9bd26 The last blocks touch colour and grain. The middle ones change the content, and you can push them harder with fewer visual defects: surface damage in 19% of cases, against 79% for the last ones. They’re not free, though. 2. Block 23 alone works as a saturation knob. Writing “colorful” or “high saturation” in the prompt changes the content much more. That said, it works less well than a correction in post-production (afterall these aren't true knobs, just approximations of knobs) . it can still be useful to tweak it, if your merges tend, on average, towards colours that are too vivid or too dull. https://preview.redd.it/aqdfpmh5wfvh1.png?width=1600&format=png&auto=webp&s=332042b7652ea03c3c3e1e3d84f70d2f4ad2e059 3. A specific calibration of the weights keeps its look even on characters it has never seen. I calibrated a set of weight shifts by eye (a preset) on four portraits and applied it to three new characters. Someone who didn’t know which image carried the preset picked it as closer to the look I had described 56 times out of 56. Faces do change a bit more than a change of seed would. The Preset: It was meant to be a very light touch towards animation: flatter colours, simpler lines and shapes. CONFIRMED WITH PRE-REGISTERED TESTS Renders reproduce identically, pixel for pixel , even days apart, and the tuner at zero adds nothing. It sounds trivial, but everything rests on it: the differences come from the edit, not from the pipeline. Without this, the whole study would have been pointless. You can predict how much an image will change without rendering it , by measuring how far the edit moves the text encoder’s output (correlation 0.71 and 0.80). It works mostly for telling different kinds of edit apart. Quality metrics don’t see everything. DINOv2, BRISQUE and the like missed exactly the content changes made by the middle blocks, which were obvious by eye. On top of that, I haven’t yet found a reliable automatic “impartial judge”: in my attempts with LLMs, the judgements weren’t trustworthy. So for now I look at many images by hand, like a primitive. WHAT’S NEXT? I don’t know whether any of this holds for other models, but it’s not yet time to experiment on other architectures. First I want to collect many properties of these shifts, to have solid ground (and to understand what they’re really doing) . Then I’ll check whether they hold on models with different architectures. If they do, they would be properties of diffusion models in general. One last tip for those interested in asking Claude or Gemini for help. I'm not afraid to say that I use LLMs to help me produce the data, and I’ve already seen some problems: numbers quoted from memory (with all the errors that brings) and hasty conclusions. If you want to run experiments with Claude Code or Antigravity, have the AI write only scripts. That way the numbers never pass through the “generative” side of the AI, only through linear processes that you run yourself, outside its environment. The full report, with every image and the things that turned out to be wrong: https://aledelpho.github.io/diffusion-models-weight-steering-report/results/index.html I'll let you know if something interesting comes out from it, Cheers! submitted by /u/ItalianArtProfessor to r/StableDiffusion [link] [comments]