AbstractPhil commited on
Commit
53a1e73
·
verified ·
1 Parent(s): 9f34b45

e005_lora_neutral_control: evaluation + result

Browse files
.gitattributes CHANGED
@@ -441,3 +441,5 @@ experiments/e005_lora_neutral_control/samples/step0/00_a-photo-of-a-bowl-of-frui
441
  experiments/e005_lora_neutral_control/samples/step0/01_a-photo-of-a-sailboat-on-a-lake.png filter=lfs diff=lfs merge=lfs -text
442
  experiments/e005_lora_neutral_control/samples/step0/02_a-photo-of-a-playground-in-a-park.png filter=lfs diff=lfs merge=lfs -text
443
  experiments/e005_lora_neutral_control/samples/step0/03_a-photo-of-a-beach-with-an-umbrella.png filter=lfs diff=lfs merge=lfs -text
 
 
 
441
  experiments/e005_lora_neutral_control/samples/step0/01_a-photo-of-a-sailboat-on-a-lake.png filter=lfs diff=lfs merge=lfs -text
442
  experiments/e005_lora_neutral_control/samples/step0/02_a-photo-of-a-playground-in-a-park.png filter=lfs diff=lfs merge=lfs -text
443
  experiments/e005_lora_neutral_control/samples/step0/03_a-photo-of-a-beach-with-an-umbrella.png filter=lfs diff=lfs merge=lfs -text
444
+ experiments/e005_lora_neutral_control/eval/sheet_epochs.jpg filter=lfs diff=lfs merge=lfs -text
445
+ experiments/e005_lora_neutral_control/eval/sheet_final.jpg filter=lfs diff=lfs merge=lfs -text
experiments/e005_lora_neutral_control/README.md CHANGED
@@ -30,7 +30,24 @@ A control: the numbers are reported, and read against e004: the control is QUIET
30
  **The mood judge.** Each image is scored on its pixels only, with CLIP ViT-L/14: 100 x (the mean cosine similarity to three upbeat phrases, "a cheerful, upbeat image", "a happy, joyful scene", "a bright, uplifting photo", minus the same for three downbeat phrases, "a gloomy, downbeat image", "a sad, melancholy scene", "a dark, depressing photo"). On the stock model, writing upbeat words into a prompt raises this score by 1.81 over the neutral prompt (experiment e001). *Content kept* is the CLIP image cosine between an image and the no-LoRA image of the same prompt and seed.
31
 
32
  ## Result
33
- Running.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
  ## Files
36
  - `lora/epochN/adapter_model.safetensors`: every saved epoch (diffusers format).
 
30
  **The mood judge.** Each image is scored on its pixels only, with CLIP ViT-L/14: 100 x (the mean cosine similarity to three upbeat phrases, "a cheerful, upbeat image", "a happy, joyful scene", "a bright, uplifting photo", minus the same for three downbeat phrases, "a gloomy, downbeat image", "a sad, melancholy scene", "a dark, depressing photo"). On the stock model, writing upbeat words into a prompt raises this score by 1.81 over the neutral prompt (experiment e001). *Content kept* is the CLIP image cosine between an image and the no-LoRA image of the same prompt and seed.
31
 
32
  ## Result
33
+ control: mood effect +0.30 +- 0.09 at scale 1; CONTROL QUIET against e004
34
+
35
+ | epoch | scale | effect (mean +- SE) | cells moving the expected way | content kept |
36
+ |---|---|---|---|---|
37
+ | 2 | 1.0 | +0.331 +- 0.129 | 81% | 0.972 |
38
+ | 4 | 1.0 | +0.398 +- 0.126 | 81% | 0.971 |
39
+ | 6 | 1.0 | +0.431 +- 0.102 | 81% | 0.970 |
40
+ | 8 | 1.0 | +0.353 +- 0.097 | 66% | 0.968 |
41
+ | 10 | 0.5 | +0.228 +- 0.098 | 62% | 0.979 |
42
+ | 10 | 1.0 | +0.304 +- 0.089 | 72% | 0.969 |
43
+
44
+ ![no LoRA, scale 0.5, scale 1](eval/sheet_final.jpg)
45
+
46
+ Rows: the held-out scenes at seed 101. Columns: no LoRA, the final LoRA at 0.5, at 1.
47
+
48
+ ![the effect across training](eval/sheet_epochs.jpg)
49
+
50
+ Columns: no LoRA, then each saved epoch at scale 1.
51
 
52
  ## Files
53
  - `lora/epochN/adapter_model.safetensors`: every saved epoch (diffusers format).
experiments/e005_lora_neutral_control/eval/epochs.json ADDED
@@ -0,0 +1,104 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "epoch": 2,
4
+ "scale": 1.0,
5
+ "mean": 0.3306457232683897,
6
+ "se": 0.12893882014295094,
7
+ "frac_pos": 0.8125,
8
+ "n": 32,
9
+ "OUTCOME": "CONTROL",
10
+ "content_kept": 0.9721308220177889,
11
+ "mood_score": 1.0970723628997803,
12
+ "pixels": {
13
+ "luma": 0.47827833238989115,
14
+ "sat": 0.4129191283136606,
15
+ "warmth": 0.07002029614523053,
16
+ "contrast": 0.30231243651360273
17
+ }
18
+ },
19
+ {
20
+ "epoch": 4,
21
+ "scale": 1.0,
22
+ "mean": 0.39772655814886093,
23
+ "se": 0.12554822719425363,
24
+ "frac_pos": 0.8125,
25
+ "n": 32,
26
+ "OUTCOME": "CONTROL",
27
+ "content_kept": 0.9713291935622692,
28
+ "mood_score": 1.1641532182693481,
29
+ "pixels": {
30
+ "luma": 0.5139990951865911,
31
+ "sat": 0.39739791490137577,
32
+ "warmth": 0.07716651912778616,
33
+ "contrast": 0.30508847814053297
34
+ }
35
+ },
36
+ {
37
+ "epoch": 6,
38
+ "scale": 1.0,
39
+ "mean": 0.43084683269262314,
40
+ "se": 0.10241579250589981,
41
+ "frac_pos": 0.8125,
42
+ "n": 32,
43
+ "OUTCOME": "CONTROL",
44
+ "content_kept": 0.9698804393410683,
45
+ "mood_score": 1.1972734928131104,
46
+ "pixels": {
47
+ "luma": 0.5110723646357656,
48
+ "sat": 0.40154851227998734,
49
+ "warmth": 0.07681594509631395,
50
+ "contrast": 0.306139855645597
51
+ }
52
+ },
53
+ {
54
+ "epoch": 8,
55
+ "scale": 1.0,
56
+ "mean": 0.3534310292452574,
57
+ "se": 0.09737796709325917,
58
+ "frac_pos": 0.65625,
59
+ "n": 32,
60
+ "OUTCOME": "CONTROL",
61
+ "content_kept": 0.968297814950347,
62
+ "mood_score": 1.119857668876648,
63
+ "pixels": {
64
+ "luma": 0.508654854260385,
65
+ "sat": 0.39800949580967426,
66
+ "warmth": 0.07533816434442997,
67
+ "contrast": 0.3027775869704783
68
+ }
69
+ },
70
+ {
71
+ "epoch": 10,
72
+ "scale": 0.5,
73
+ "mean": 0.22760832123458385,
74
+ "se": 0.09838567401052334,
75
+ "frac_pos": 0.625,
76
+ "n": 32,
77
+ "OUTCOME": "CONTROL",
78
+ "content_kept": 0.9789615999907255,
79
+ "mood_score": 0.9940349459648132,
80
+ "pixels": {
81
+ "luma": 0.4826270332559943,
82
+ "sat": 0.3920339746400714,
83
+ "warmth": 0.0700287283398211,
84
+ "contrast": 0.2903351611457765
85
+ }
86
+ },
87
+ {
88
+ "epoch": 10,
89
+ "scale": 1.0,
90
+ "mean": 0.30393643490970135,
91
+ "se": 0.0890526511161117,
92
+ "frac_pos": 0.71875,
93
+ "n": 32,
94
+ "OUTCOME": "CONTROL",
95
+ "content_kept": 0.9691903609782457,
96
+ "mood_score": 1.0703630447387695,
97
+ "pixels": {
98
+ "luma": 0.4851257065311074,
99
+ "sat": 0.41412709187716246,
100
+ "warmth": 0.07607955252751708,
101
+ "contrast": 0.30458328127861023
102
+ }
103
+ }
104
+ ]
experiments/e005_lora_neutral_control/eval/final.json ADDED
@@ -0,0 +1,507 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "baseline_scores": [
3
+ 2.886000156402588,
4
+ 2.5024890899658203,
5
+ 2.7197673320770264,
6
+ 3.107909917831421,
7
+ 1.438605785369873,
8
+ 1.9624769687652588,
9
+ 1.5230387449264526,
10
+ 1.421992540359497,
11
+ 0.808367133140564,
12
+ 0.6213948130607605,
13
+ 0.9683996438980103,
14
+ 0.034049153327941895,
15
+ 2.0528719425201416,
16
+ 2.170698404312134,
17
+ 1.63912034034729,
18
+ 1.8219664096832275,
19
+ 2.213168144226074,
20
+ 1.3614521026611328,
21
+ 2.2692770957946777,
22
+ 1.4685227870941162,
23
+ -2.909550189971924,
24
+ -2.0798280239105225,
25
+ -0.8588194847106934,
26
+ -3.6645233631134033,
27
+ 1.403592586517334,
28
+ 1.5216946601867676,
29
+ 0.9342372417449951,
30
+ 0.3674924373626709,
31
+ -1.7905964851379395,
32
+ -1.3536632061004639,
33
+ -1.2247874736785889,
34
+ -0.8111640810966492
35
+ ],
36
+ "cells": {
37
+ "0.5": [
38
+ {
39
+ "subject": "a bowl of fruit on a counter",
40
+ "seed": 101,
41
+ "score": 2.879464626312256,
42
+ "diff": -0.006535530090332031,
43
+ "content_kept": 0.9907717108726501
44
+ },
45
+ {
46
+ "subject": "a bowl of fruit on a counter",
47
+ "seed": 202,
48
+ "score": 2.352367401123047,
49
+ "diff": -0.15012168884277344,
50
+ "content_kept": 0.9858805537223816
51
+ },
52
+ {
53
+ "subject": "a bowl of fruit on a counter",
54
+ "seed": 303,
55
+ "score": 2.506575107574463,
56
+ "diff": -0.21319222450256348,
57
+ "content_kept": 0.9674177765846252
58
+ },
59
+ {
60
+ "subject": "a bowl of fruit on a counter",
61
+ "seed": 404,
62
+ "score": 3.0892207622528076,
63
+ "diff": -0.01868915557861328,
64
+ "content_kept": 0.9952624440193176
65
+ },
66
+ {
67
+ "subject": "a sailboat on a lake",
68
+ "seed": 101,
69
+ "score": 1.4362826347351074,
70
+ "diff": -0.002323150634765625,
71
+ "content_kept": 0.9873384237289429
72
+ },
73
+ {
74
+ "subject": "a sailboat on a lake",
75
+ "seed": 202,
76
+ "score": 2.0700976848602295,
77
+ "diff": 0.1076207160949707,
78
+ "content_kept": 0.9944259524345398
79
+ },
80
+ {
81
+ "subject": "a sailboat on a lake",
82
+ "seed": 303,
83
+ "score": 1.3253360986709595,
84
+ "diff": -0.19770264625549316,
85
+ "content_kept": 0.9942442178726196
86
+ },
87
+ {
88
+ "subject": "a sailboat on a lake",
89
+ "seed": 404,
90
+ "score": 1.7584428787231445,
91
+ "diff": 0.33645033836364746,
92
+ "content_kept": 0.9834393262863159
93
+ },
94
+ {
95
+ "subject": "a playground in a park",
96
+ "seed": 101,
97
+ "score": 0.8007705211639404,
98
+ "diff": -0.007596611976623535,
99
+ "content_kept": 0.987339198589325
100
+ },
101
+ {
102
+ "subject": "a playground in a park",
103
+ "seed": 202,
104
+ "score": 0.8148401975631714,
105
+ "diff": 0.1934453845024109,
106
+ "content_kept": 0.990993320941925
107
+ },
108
+ {
109
+ "subject": "a playground in a park",
110
+ "seed": 303,
111
+ "score": 0.6784245371818542,
112
+ "diff": -0.289975106716156,
113
+ "content_kept": 0.9953629970550537
114
+ },
115
+ {
116
+ "subject": "a playground in a park",
117
+ "seed": 404,
118
+ "score": 0.10215640068054199,
119
+ "diff": 0.0681072473526001,
120
+ "content_kept": 0.9895880818367004
121
+ },
122
+ {
123
+ "subject": "a beach with an umbrella",
124
+ "seed": 101,
125
+ "score": 2.7740061283111572,
126
+ "diff": 0.7211341857910156,
127
+ "content_kept": 0.9813739061355591
128
+ },
129
+ {
130
+ "subject": "a beach with an umbrella",
131
+ "seed": 202,
132
+ "score": 3.057976007461548,
133
+ "diff": 0.8872776031494141,
134
+ "content_kept": 0.9334126710891724
135
+ },
136
+ {
137
+ "subject": "a beach with an umbrella",
138
+ "seed": 303,
139
+ "score": 2.372386932373047,
140
+ "diff": 0.7332665920257568,
141
+ "content_kept": 0.9687939882278442
142
+ },
143
+ {
144
+ "subject": "a beach with an umbrella",
145
+ "seed": 404,
146
+ "score": 2.8800458908081055,
147
+ "diff": 1.058079481124878,
148
+ "content_kept": 0.9460639953613281
149
+ },
150
+ {
151
+ "subject": "a coffee cup on a desk",
152
+ "seed": 101,
153
+ "score": 2.3412511348724365,
154
+ "diff": 0.1280829906463623,
155
+ "content_kept": 0.9848528504371643
156
+ },
157
+ {
158
+ "subject": "a coffee cup on a desk",
159
+ "seed": 202,
160
+ "score": 1.732630968093872,
161
+ "diff": 0.37117886543273926,
162
+ "content_kept": 0.9928879737854004
163
+ },
164
+ {
165
+ "subject": "a coffee cup on a desk",
166
+ "seed": 303,
167
+ "score": 2.1630496978759766,
168
+ "diff": -0.10622739791870117,
169
+ "content_kept": 0.995288610458374
170
+ },
171
+ {
172
+ "subject": "a coffee cup on a desk",
173
+ "seed": 404,
174
+ "score": 1.45057213306427,
175
+ "diff": -0.01795065402984619,
176
+ "content_kept": 0.9863153696060181
177
+ },
178
+ {
179
+ "subject": "an old man sitting on a porch",
180
+ "seed": 101,
181
+ "score": -1.3022392988204956,
182
+ "diff": 1.6073108911514282,
183
+ "content_kept": 0.9629941582679749
184
+ },
185
+ {
186
+ "subject": "an old man sitting on a porch",
187
+ "seed": 202,
188
+ "score": -1.7370045185089111,
189
+ "diff": 0.34282350540161133,
190
+ "content_kept": 0.9440629482269287
191
+ },
192
+ {
193
+ "subject": "an old man sitting on a porch",
194
+ "seed": 303,
195
+ "score": -2.4625329971313477,
196
+ "diff": -1.6037135124206543,
197
+ "content_kept": 0.9099425077438354
198
+ },
199
+ {
200
+ "subject": "an old man sitting on a porch",
201
+ "seed": 404,
202
+ "score": -3.2880232334136963,
203
+ "diff": 0.37650012969970703,
204
+ "content_kept": 0.9821407794952393
205
+ },
206
+ {
207
+ "subject": "a lighthouse on a coast",
208
+ "seed": 101,
209
+ "score": 1.4871597290039062,
210
+ "diff": 0.08356714248657227,
211
+ "content_kept": 0.9683668613433838
212
+ },
213
+ {
214
+ "subject": "a lighthouse on a coast",
215
+ "seed": 202,
216
+ "score": 1.6578078269958496,
217
+ "diff": 0.13611316680908203,
218
+ "content_kept": 0.9958996772766113
219
+ },
220
+ {
221
+ "subject": "a lighthouse on a coast",
222
+ "seed": 303,
223
+ "score": 1.2624218463897705,
224
+ "diff": 0.3281846046447754,
225
+ "content_kept": 0.9622445106506348
226
+ },
227
+ {
228
+ "subject": "a lighthouse on a coast",
229
+ "seed": 404,
230
+ "score": 0.9394571185112,
231
+ "diff": 0.571964681148529,
232
+ "content_kept": 0.9911675453186035
233
+ },
234
+ {
235
+ "subject": "an office with computers",
236
+ "seed": 101,
237
+ "score": -0.519290566444397,
238
+ "diff": 1.2713059186935425,
239
+ "content_kept": 0.987614095211029
240
+ },
241
+ {
242
+ "subject": "an office with computers",
243
+ "seed": 202,
244
+ "score": -1.026512622833252,
245
+ "diff": 0.3271505832672119,
246
+ "content_kept": 0.985089123249054
247
+ },
248
+ {
249
+ "subject": "an office with computers",
250
+ "seed": 303,
251
+ "score": -0.8340209722518921,
252
+ "diff": 0.3907665014266968,
253
+ "content_kept": 0.9929459691047668
254
+ },
255
+ {
256
+ "subject": "an office with computers",
257
+ "seed": 404,
258
+ "score": -0.9540006518363953,
259
+ "diff": -0.1428365707397461,
260
+ "content_kept": 0.9932496547698975
261
+ }
262
+ ],
263
+ "1.0": [
264
+ {
265
+ "subject": "a bowl of fruit on a counter",
266
+ "seed": 101,
267
+ "score": 2.7869715690612793,
268
+ "diff": -0.0990285873413086,
269
+ "content_kept": 0.9791156053543091
270
+ },
271
+ {
272
+ "subject": "a bowl of fruit on a counter",
273
+ "seed": 202,
274
+ "score": 2.3026599884033203,
275
+ "diff": -0.1998291015625,
276
+ "content_kept": 0.9844703078269958
277
+ },
278
+ {
279
+ "subject": "a bowl of fruit on a counter",
280
+ "seed": 303,
281
+ "score": 2.52089786529541,
282
+ "diff": -0.1988694667816162,
283
+ "content_kept": 0.9657233953475952
284
+ },
285
+ {
286
+ "subject": "a bowl of fruit on a counter",
287
+ "seed": 404,
288
+ "score": 2.577543258666992,
289
+ "diff": -0.5303666591644287,
290
+ "content_kept": 0.9732822179794312
291
+ },
292
+ {
293
+ "subject": "a sailboat on a lake",
294
+ "seed": 101,
295
+ "score": 1.2233734130859375,
296
+ "diff": -0.21523237228393555,
297
+ "content_kept": 0.9759174585342407
298
+ },
299
+ {
300
+ "subject": "a sailboat on a lake",
301
+ "seed": 202,
302
+ "score": 1.9424229860305786,
303
+ "diff": -0.020053982734680176,
304
+ "content_kept": 0.9888590574264526
305
+ },
306
+ {
307
+ "subject": "a sailboat on a lake",
308
+ "seed": 303,
309
+ "score": 1.4817655086517334,
310
+ "diff": -0.04127323627471924,
311
+ "content_kept": 0.9877967834472656
312
+ },
313
+ {
314
+ "subject": "a sailboat on a lake",
315
+ "seed": 404,
316
+ "score": 1.8321216106414795,
317
+ "diff": 0.4101290702819824,
318
+ "content_kept": 0.9658820629119873
319
+ },
320
+ {
321
+ "subject": "a playground in a park",
322
+ "seed": 101,
323
+ "score": 0.8859455585479736,
324
+ "diff": 0.07757842540740967,
325
+ "content_kept": 0.9713751673698425
326
+ },
327
+ {
328
+ "subject": "a playground in a park",
329
+ "seed": 202,
330
+ "score": 1.1901170015335083,
331
+ "diff": 0.5687221884727478,
332
+ "content_kept": 0.9817947745323181
333
+ },
334
+ {
335
+ "subject": "a playground in a park",
336
+ "seed": 303,
337
+ "score": 1.0032594203948975,
338
+ "diff": 0.03485977649688721,
339
+ "content_kept": 0.9861829280853271
340
+ },
341
+ {
342
+ "subject": "a playground in a park",
343
+ "seed": 404,
344
+ "score": 0.2668246626853943,
345
+ "diff": 0.2327755093574524,
346
+ "content_kept": 0.9747189879417419
347
+ },
348
+ {
349
+ "subject": "a beach with an umbrella",
350
+ "seed": 101,
351
+ "score": 2.863939046859741,
352
+ "diff": 0.8110671043395996,
353
+ "content_kept": 0.9800201058387756
354
+ },
355
+ {
356
+ "subject": "a beach with an umbrella",
357
+ "seed": 202,
358
+ "score": 2.570101737976074,
359
+ "diff": 0.39940333366394043,
360
+ "content_kept": 0.9458404779434204
361
+ },
362
+ {
363
+ "subject": "a beach with an umbrella",
364
+ "seed": 303,
365
+ "score": 2.921839714050293,
366
+ "diff": 1.282719373703003,
367
+ "content_kept": 0.9668902158737183
368
+ },
369
+ {
370
+ "subject": "a beach with an umbrella",
371
+ "seed": 404,
372
+ "score": 3.034708023071289,
373
+ "diff": 1.2127416133880615,
374
+ "content_kept": 0.9310640692710876
375
+ },
376
+ {
377
+ "subject": "a coffee cup on a desk",
378
+ "seed": 101,
379
+ "score": 2.6942670345306396,
380
+ "diff": 0.48109889030456543,
381
+ "content_kept": 0.9805902242660522
382
+ },
383
+ {
384
+ "subject": "a coffee cup on a desk",
385
+ "seed": 202,
386
+ "score": 2.0595908164978027,
387
+ "diff": 0.6981387138366699,
388
+ "content_kept": 0.979941725730896
389
+ },
390
+ {
391
+ "subject": "a coffee cup on a desk",
392
+ "seed": 303,
393
+ "score": 1.9175543785095215,
394
+ "diff": -0.35172271728515625,
395
+ "content_kept": 0.9845702648162842
396
+ },
397
+ {
398
+ "subject": "a coffee cup on a desk",
399
+ "seed": 404,
400
+ "score": 1.5551865100860596,
401
+ "diff": 0.08666372299194336,
402
+ "content_kept": 0.9663214087486267
403
+ },
404
+ {
405
+ "subject": "an old man sitting on a porch",
406
+ "seed": 101,
407
+ "score": -1.4199450016021729,
408
+ "diff": 1.489605188369751,
409
+ "content_kept": 0.9654241800308228
410
+ },
411
+ {
412
+ "subject": "an old man sitting on a porch",
413
+ "seed": 202,
414
+ "score": -1.3965070247650146,
415
+ "diff": 0.6833209991455078,
416
+ "content_kept": 0.939691960811615
417
+ },
418
+ {
419
+ "subject": "an old man sitting on a porch",
420
+ "seed": 303,
421
+ "score": -1.472003698348999,
422
+ "diff": -0.6131842136383057,
423
+ "content_kept": 0.8956541419029236
424
+ },
425
+ {
426
+ "subject": "an old man sitting on a porch",
427
+ "seed": 404,
428
+ "score": -3.49432373046875,
429
+ "diff": 0.17019963264465332,
430
+ "content_kept": 0.9300631284713745
431
+ },
432
+ {
433
+ "subject": "a lighthouse on a coast",
434
+ "seed": 101,
435
+ "score": 1.4700889587402344,
436
+ "diff": 0.06649637222290039,
437
+ "content_kept": 0.9730719327926636
438
+ },
439
+ {
440
+ "subject": "a lighthouse on a coast",
441
+ "seed": 202,
442
+ "score": 1.803627610206604,
443
+ "diff": 0.2819329500198364,
444
+ "content_kept": 0.9914504289627075
445
+ },
446
+ {
447
+ "subject": "a lighthouse on a coast",
448
+ "seed": 303,
449
+ "score": 1.3072997331619263,
450
+ "diff": 0.37306249141693115,
451
+ "content_kept": 0.9564805626869202
452
+ },
453
+ {
454
+ "subject": "a lighthouse on a coast",
455
+ "seed": 404,
456
+ "score": 0.8462980389595032,
457
+ "diff": 0.4788056015968323,
458
+ "content_kept": 0.9691883325576782
459
+ },
460
+ {
461
+ "subject": "an office with computers",
462
+ "seed": 101,
463
+ "score": -1.02289617061615,
464
+ "diff": 0.7677003145217896,
465
+ "content_kept": 0.9834281206130981
466
+ },
467
+ {
468
+ "subject": "an office with computers",
469
+ "seed": 202,
470
+ "score": -1.017390251159668,
471
+ "diff": 0.3362729549407959,
472
+ "content_kept": 0.9655550122261047
473
+ },
474
+ {
475
+ "subject": "an office with computers",
476
+ "seed": 303,
477
+ "score": -0.3324732184410095,
478
+ "diff": 0.8923142552375793,
479
+ "content_kept": 0.9849458932876587
480
+ },
481
+ {
482
+ "subject": "an office with computers",
483
+ "seed": 404,
484
+ "score": -0.6512463092803955,
485
+ "diff": 0.15991777181625366,
486
+ "content_kept": 0.9887806177139282
487
+ }
488
+ ]
489
+ },
490
+ "read": {
491
+ "epoch": 10,
492
+ "scale": 1.0,
493
+ "mean": 0.30393643490970135,
494
+ "se": 0.0890526511161117,
495
+ "frac_pos": 0.71875,
496
+ "n": 32,
497
+ "OUTCOME": "CONTROL",
498
+ "content_kept": 0.9691903609782457,
499
+ "mood_score": 1.0703630447387695,
500
+ "pixels": {
501
+ "luma": 0.4851257065311074,
502
+ "sat": 0.41412709187716246,
503
+ "warmth": 0.07607955252751708,
504
+ "contrast": 0.30458328127861023
505
+ }
506
+ }
507
+ }
experiments/e005_lora_neutral_control/eval/sheet_epochs.jpg ADDED

Git LFS Details

  • SHA256: 518ec5fcbfe3b0f7bc5b434375e509359f09aa8181fb83dadb1d638253d74e72
  • Pointer size: 131 Bytes
  • Size of remote file: 508 kB
experiments/e005_lora_neutral_control/eval/sheet_final.jpg ADDED

Git LFS Details

  • SHA256: 07632ab1c8bd7487bbc1066b78d231edf3fba12bbe7f1a51a35206db5001ac0c
  • Pointer size: 131 Bytes
  • Size of remote file: 468 kB
experiments/e005_lora_neutral_control/meta.json CHANGED
@@ -3,7 +3,7 @@
3
  "title": "Control LoRA: the model's own neutral images, neutral captions",
4
  "date": "2026-10-03",
5
  "kind": "lora",
6
- "status": "running",
7
  "spec": {
8
  "id": "e005_lora_neutral_control",
9
  "title": "Control LoRA: the model's own neutral images, neutral captions",
@@ -26,5 +26,164 @@
26
  "timestep shift": "3.0 (the checkpoint's own)",
27
  "precision": "bf16 (the LoRA is saved in bf16)",
28
  "evaluation": "8 held-out scenes x seeds 101, 202, 303, 404, 20 steps, guidance 4.5"
29
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
30
  }
 
3
  "title": "Control LoRA: the model's own neutral images, neutral captions",
4
  "date": "2026-10-03",
5
  "kind": "lora",
6
+ "status": "done",
7
  "spec": {
8
  "id": "e005_lora_neutral_control",
9
  "title": "Control LoRA: the model's own neutral images, neutral captions",
 
26
  "timestep shift": "3.0 (the checkpoint's own)",
27
  "precision": "bf16 (the LoRA is saved in bf16)",
28
  "evaluation": "8 held-out scenes x seeds 101, 202, 303, 404, 20 steps, guidance 4.5"
29
+ },
30
+ "epochs": [
31
+ {
32
+ "epoch": 2,
33
+ "scale": 1.0,
34
+ "mean": 0.3306457232683897,
35
+ "se": 0.12893882014295094,
36
+ "frac_pos": 0.8125,
37
+ "n": 32,
38
+ "OUTCOME": "CONTROL",
39
+ "content_kept": 0.9721308220177889,
40
+ "mood_score": 1.0970723628997803,
41
+ "pixels": {
42
+ "luma": 0.47827833238989115,
43
+ "sat": 0.4129191283136606,
44
+ "warmth": 0.07002029614523053,
45
+ "contrast": 0.30231243651360273
46
+ }
47
+ },
48
+ {
49
+ "epoch": 4,
50
+ "scale": 1.0,
51
+ "mean": 0.39772655814886093,
52
+ "se": 0.12554822719425363,
53
+ "frac_pos": 0.8125,
54
+ "n": 32,
55
+ "OUTCOME": "CONTROL",
56
+ "content_kept": 0.9713291935622692,
57
+ "mood_score": 1.1641532182693481,
58
+ "pixels": {
59
+ "luma": 0.5139990951865911,
60
+ "sat": 0.39739791490137577,
61
+ "warmth": 0.07716651912778616,
62
+ "contrast": 0.30508847814053297
63
+ }
64
+ },
65
+ {
66
+ "epoch": 6,
67
+ "scale": 1.0,
68
+ "mean": 0.43084683269262314,
69
+ "se": 0.10241579250589981,
70
+ "frac_pos": 0.8125,
71
+ "n": 32,
72
+ "OUTCOME": "CONTROL",
73
+ "content_kept": 0.9698804393410683,
74
+ "mood_score": 1.1972734928131104,
75
+ "pixels": {
76
+ "luma": 0.5110723646357656,
77
+ "sat": 0.40154851227998734,
78
+ "warmth": 0.07681594509631395,
79
+ "contrast": 0.306139855645597
80
+ }
81
+ },
82
+ {
83
+ "epoch": 8,
84
+ "scale": 1.0,
85
+ "mean": 0.3534310292452574,
86
+ "se": 0.09737796709325917,
87
+ "frac_pos": 0.65625,
88
+ "n": 32,
89
+ "OUTCOME": "CONTROL",
90
+ "content_kept": 0.968297814950347,
91
+ "mood_score": 1.119857668876648,
92
+ "pixels": {
93
+ "luma": 0.508654854260385,
94
+ "sat": 0.39800949580967426,
95
+ "warmth": 0.07533816434442997,
96
+ "contrast": 0.3027775869704783
97
+ }
98
+ },
99
+ {
100
+ "epoch": 10,
101
+ "scale": 0.5,
102
+ "mean": 0.22760832123458385,
103
+ "se": 0.09838567401052334,
104
+ "frac_pos": 0.625,
105
+ "n": 32,
106
+ "OUTCOME": "CONTROL",
107
+ "content_kept": 0.9789615999907255,
108
+ "mood_score": 0.9940349459648132,
109
+ "pixels": {
110
+ "luma": 0.4826270332559943,
111
+ "sat": 0.3920339746400714,
112
+ "warmth": 0.0700287283398211,
113
+ "contrast": 0.2903351611457765
114
+ }
115
+ },
116
+ {
117
+ "epoch": 10,
118
+ "scale": 1.0,
119
+ "mean": 0.30393643490970135,
120
+ "se": 0.0890526511161117,
121
+ "frac_pos": 0.71875,
122
+ "n": 32,
123
+ "OUTCOME": "CONTROL",
124
+ "content_kept": 0.9691903609782457,
125
+ "mood_score": 1.0703630447387695,
126
+ "pixels": {
127
+ "luma": 0.4851257065311074,
128
+ "sat": 0.41412709187716246,
129
+ "warmth": 0.07607955252751708,
130
+ "contrast": 0.30458328127861023
131
+ }
132
+ }
133
+ ],
134
+ "final": {
135
+ "epoch": 10,
136
+ "scale": 1.0,
137
+ "mean": 0.30393643490970135,
138
+ "se": 0.0890526511161117,
139
+ "frac_pos": 0.71875,
140
+ "n": 32,
141
+ "OUTCOME": "CONTROL",
142
+ "content_kept": 0.9691903609782457,
143
+ "mood_score": 1.0703630447387695,
144
+ "pixels": {
145
+ "luma": 0.4851257065311074,
146
+ "sat": 0.41412709187716246,
147
+ "warmth": 0.07607955252751708,
148
+ "contrast": 0.30458328127861023
149
+ }
150
+ },
151
+ "final_diffs": [
152
+ -0.0990285873413086,
153
+ -0.1998291015625,
154
+ -0.1988694667816162,
155
+ -0.5303666591644287,
156
+ -0.21523237228393555,
157
+ -0.020053982734680176,
158
+ -0.04127323627471924,
159
+ 0.4101290702819824,
160
+ 0.07757842540740967,
161
+ 0.5687221884727478,
162
+ 0.03485977649688721,
163
+ 0.2327755093574524,
164
+ 0.8110671043395996,
165
+ 0.39940333366394043,
166
+ 1.282719373703003,
167
+ 1.2127416133880615,
168
+ 0.48109889030456543,
169
+ 0.6981387138366699,
170
+ -0.35172271728515625,
171
+ 0.08666372299194336,
172
+ 1.489605188369751,
173
+ 0.6833209991455078,
174
+ -0.6131842136383057,
175
+ 0.17019963264465332,
176
+ 0.06649637222290039,
177
+ 0.2819329500198364,
178
+ 0.37306249141693115,
179
+ 0.4788056015968323,
180
+ 0.7677003145217896,
181
+ 0.3362729549407959,
182
+ 0.8923142552375793,
183
+ 0.15991777181625366
184
+ ],
185
+ "first_epoch_beyond_3se": 4,
186
+ "train_seconds": 120,
187
+ "summary": "control: mood effect +0.30 +- 0.09 at scale 1; CONTROL QUIET against e004",
188
+ "finished_utc": "2026-10-03 23:38"
189
  }