deepfacelab框架迁移为pytorch的问题

day270010678

星级打分

• 1
• 2
• 3
• 4
• 5

平均分:NAN  参与人数:0  我的评分:未评

我现在已经完全迁移完的项目是，提取脸部，遮罩训练，但是到了遮罩预训练阶段，我发现一个问题，就是deepfacelab的MPSharedList机制，按照我的初期设想是不重构多进程框架Subprocessor，而是在这个框架下重构，但是现在我发现一个问题，就是在数据集PackedFaceset.py中 我给返回的是PyTorch数据集对象，但是在SampleLoader.py中

1.                 samples[sample_type] = MPSharedList(result)
   
2.         elif    sample_type == SampleType.FACE_TEMPORAL_SORTED:
   
3.                 result = SampleLoader.load (SampleType.FACE, samples_path)
   
4.                 result = SampleLoader.upgradeToFaceTemporalSortedSamples(result)
   
5.                 samples[sample_type] = MPSharedList(result)

复制代码

他按照原始的框架是调用列表，但是我这里返回的PyTorch数据集对象，这就导致一个问题，我在pytorch一致性的准则下，有两种解决办法，这里直接把pytorch转换为列表，另外一种是修改MPSharedList，让他支持pytorch对象，这是最中性的修改，但是我突然发现 似乎最完美的方法是PyTorch的DataLoader机制代替MPSharedList机制，这就涉及一个项目的重构了，这简直是重写deepfacelab啊，需要把DeepFaceLab的Subprocessor架构转为PyTorch DataLoader架构， 形成真正的pytorch一致性。但是这是个大项目，按照我拖拖拉拉的偶尔写写停停的时间，现在是假如不重构，我觉得没意思，但是重构吧，恐怕要很久，强迫症犯了。 不知道怎么好，现在都丫的因为这个问题，想不修改了，直接到这里就行了或者简单的重构后面的训练，合成，就不弄预训练了，这该死的完美主义。弄得都不想写了。


在这里我必须负责任的说一句，deepfacelab由于TensorFlow框架的缘故，导致我们很多性能不能发挥，所以迁移到pytorch是必须的，就简单的提取脸部来说，我测试，用原始的TensorFlow2.7提取162张图片，质量100的情况下耗费时间是二分钟多一点， 假如把ensorFlow升级到2.10+python3.10的话，耗时是 1分20秒， 假如框架迁移到pytorch后，耗时是40秒，这还是没有完全重构的情况下的数据对比， 我遮罩训练是260ms一次迭代。

写到这里 我来说说 显卡4000和5000，这个不要过多听别人吹牛，现有的不管那个框架都支持，只是性能发挥的问题，那问题来了，假如我迁移到pytorch后能不能完全发挥性能？答案是肯定的，因为现在最高的pytorch2.91也不能发挥完全性能，至于理由，懂的都懂，真要完全发挥必须用他的测试版加cu30或许好点。

leeso

main9dog 发表于 2026-1-31 22:29
我覺得最需要的是手動上遮照這塊，全世界還沒有高手研究出來怎麼寫嗎？

每次搞這個搞超久....

lispmox

我之前花了很多时间用pytorch重构了整个DFL的工作流，后来发现对于个人玩家意义不大。现在的高分辨模型还是靠算力和数据堆上去的，在架构没有进步的前提下重写DFL也只是雕花了，投入的时间和精力成本不如多训练几天，冬天跑还能取暖。楼主既然如果有精力有算力也可以试试改进模型架构

辣椒炒肉

巧了，我也在做，但是仅实现LIAE架构与训练步骤，还有面部相关工具

day270010678

lispmox 发表于 2026-1-27 00:40
我之前花了很多时间用pytorch重构了整个DFL的工作流，后来发现对于个人玩家意义不大。现在的高分辨模型还是 ...

别提了，昨天晚上心血来潮，重构了他整体框架， 今天调试，  调试了半天还是有错误，都懒得弄了。

1. Loading XSeg model...
   
2. === ON_INITIALIZE_OPTIONS CALLED ===
   
3. self id: 2126717295248
   
4. Press enter in 2 seconds to override model settings.=== ON_INITIALIZE CALLED ===
   
5. self id: 2126717295248
   
6. Using 3 segmented samples.
   
7. === ONGETPREVIEW CALLED ===
   
8. onGetPreview: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
   
9. onGetPreview: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
   
10. onGetPreview: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
    
11. === VIEW FUNCTION CALLED ===
    
12. input_tensor id: 2127144431216, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
    
13. input_tensor min/max: 0.000000/1.000000
    
14. input_tensor data sample: [1. 1. 1. 1. 1.]
    
15. onGetPreview: I shape: (3, 256, 256, 3), M shape: (3, 256, 256, 3), IM shape: (3, 256, 256, 3)
    
16. onGetPreview: I min/max: 0.000000/1.000000, M min/max: 0.000000/1.000000, IM min/max: 0.000000/0.999058
    
17. process_samples called for src
    
18. process_samples: sample_tensor shape: torch.Size([4, 3, 256, 256]), device: cuda:0, min/max: 0.000000/0.976471
    
19. === VIEW FUNCTION CALLED ===
    
20. input_tensor id: 2127144431936, shape: torch.Size([4, 3, 256, 256]), dtype: torch.float32, device: cuda:0
    
21. input_tensor min/max: 0.000000/0.976471
    
22. input_tensor data sample: [0.22156863 0.21862745 0.23333333 0.24313726 0.22352941]
    
23. process_samples called for dst
    
24. process_samples: sample_tensor shape: torch.Size([4, 3, 256, 256]), device: cuda:0, min/max: 0.000000/0.866667
    
25. === VIEW FUNCTION CALLED ===
    
26. input_tensor id: 2127144431936, shape: torch.Size([4, 3, 256, 256]), dtype: torch.float32, device: cuda:0
    
27. input_tensor min/max: 0.000000/0.866667
    
28. input_tensor data sample: [0.15294118 0.15294118 0.15294118 0.15294118 0.15294118]
    
29. =================== Model Summary ===================
    
30. ==                                                 ==
    
31. ==             Model name: XSeg                    ==
    
32. ==                                                 ==
    
33. ==      Current iteration: 123                     ==
    
34. ==                                                 ==
    
35. ==----------------- Model Options -----------------==
    
36. ==                                                 ==
    
37. ==              face_type: wf                      ==
    
38. ==               pretrain: False                   ==
    
39. ==             batch_size: 4                       ==
    
40. == pytorch_cuda_available: True                    ==
    
41. ==        pytorch_version: 2.9.1+cu130             ==
    
42. ==                                                 ==
    
43. ==------------------ Running On -------------------==
    
44. ==                                                 ==
    
45. ==           Device index: 0                       ==
    
46. ==                   Name: NVIDIA GeForce RTX 3050 ==
    
47. ==                   VRAM: 6.00GB                  ==
    
48. ==                                                 ==
    
49. =====================================================
    
50. Starting. Press "Enter" to stop training and save model.
    
51. Time      Iter      Time/Iter   Loss Values...
    
52. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
    
53. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
    
54. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
    
55. onTrainOneIter: total_pixels=196608, positive_pixels=63210, negative_pixels=133398, positive_ratio=0.321503
    
56. === TRAIN FUNCTION CALLED ===
    
57. input_tensor id: 2127144432944, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
    
58. target_tensor id: 2127144431792, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
    
59. input_tensor min/max: 0.000000/1.000000
    
60. target_tensor min/max: 0.000000/1.000000
    
61. input_tensor data sample: [0.12258932 0.12264245 0.12273288 0.12257486 0.12219308]
    
62. target_tensor data sample: [0. 0. 0. 0. 0.]
    
63.   pred_logits min/max: -26.082487/7.937037
    
64.   pred_logits sample: [-1.2224133 -1.0619926 -1.3118495 -1.6369269 -2.51067  ]
    
65.   target_tensor stats: total_pixels=196608, positive_pixels=63210, negative_pixels=133398, positive_ratio=0.321503
    
66.   Total grad_norm: 9.716019e+00, avg: 1.104093e-01
    
67.   Zero gradients: 15/88 (17.0%)
    
68.   Loss: 0.263407
    
69. === ONGETPREVIEW CALLED ===[0.2634]
    
70. onGetPreview: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
    
71. onGetPreview: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
    
72. onGetPreview: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
    
73. === VIEW FUNCTION CALLED ===
    
74. input_tensor id: 2127144431504, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
    
75. input_tensor min/max: 0.000000/1.000000
    
76. input_tensor data sample: [1. 1. 1. 1. 1.]
    
77. onGetPreview: I shape: (3, 256, 256, 3), M shape: (3, 256, 256, 3), IM shape: (3, 256, 256, 3)
    
78. onGetPreview: I min/max: 0.000000/1.000000, M min/max: 0.000000/1.000000, IM min/max: 0.000000/0.998625
    
79. process_samples called for src
    
80. process_samples: sample_tensor shape: torch.Size([4, 3, 256, 256]), device: cuda:0, min/max: 0.000000/0.976471
    
81. === VIEW FUNCTION CALLED ===
    
82. input_tensor id: 2127144431792, shape: torch.Size([4, 3, 256, 256]), dtype: torch.float32, device: cuda:0
    
83. input_tensor min/max: 0.000000/0.976471
    
84. input_tensor data sample: [0.22156863 0.21862745 0.23333333 0.24313726 0.22352941]
    
85. process_samples called for dst
    
86. process_samples: sample_tensor shape: torch.Size([4, 3, 256, 256]), device: cuda:0, min/max: 0.000000/0.866667
    
87. === VIEW FUNCTION CALLED ===
    
88. input_tensor id: 2127144431792, shape: torch.Size([4, 3, 256, 256]), dtype: torch.float32, device: cuda:0
    
89. input_tensor min/max: 0.000000/0.866667
    
90. input_tensor data sample: [0.15294118 0.15294118 0.15294118 0.15294118 0.15294118]
    
91. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.011410/1.000000
    
92. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
    
93. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
    
94. onTrainOneIter: total_pixels=196608, positive_pixels=68769, negative_pixels=127839, positive_ratio=0.349777
    
95. === TRAIN FUNCTION CALLED ===
    
96. input_tensor id: 2127144535392, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
    
97. target_tensor id: 2127144534384, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
    
98. input_tensor min/max: 0.011410/1.000000
    
99. target_tensor min/max: 0.000000/1.000000
    
100. input_tensor data sample: [1. 1. 1. 1. 1.]
     
101. target_tensor data sample: [0. 0. 0. 0. 0.]
     
102.   pred_logits min/max: -21.311359/5.924034
     
103.   pred_logits sample: [-7.4202623  1.7539611 -4.432145  -5.049247  -5.240492 ]
     
104.   target_tensor stats: total_pixels=196608, positive_pixels=68769, negative_pixels=127839, positive_ratio=0.349777
     
105.   Total grad_norm: 6.423314e+00, avg: 7.299221e-02
     
106.   Zero gradients: 16/88 (18.2%)
     
107.   Loss: 0.227651
     
108. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
109. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
110. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
111. onTrainOneIter: total_pixels=196608, positive_pixels=64904, negative_pixels=131704, positive_ratio=0.330119
     
112. === TRAIN FUNCTION CALLED ===
     
113. input_tensor id: 2127117534992, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
114. target_tensor id: 2127144534240, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
115. input_tensor min/max: 0.000000/1.000000
     
116. target_tensor min/max: 0.000000/1.000000
     
117. input_tensor data sample: [0.3203014  0.32011262 0.31642017 0.29747993 0.28607503]
     
118. target_tensor data sample: [0. 0. 0. 0. 0.]
     
119.   pred_logits min/max: -21.281088/8.497002
     
120.   pred_logits sample: [-3.0622835 -1.7201625 -1.9710674 -3.2355402 -3.0791936]
     
121.   target_tensor stats: total_pixels=196608, positive_pixels=64904, negative_pixels=131704, positive_ratio=0.330119
     
122.   Total grad_norm: 3.317015e+01, avg: 3.769335e-01
     
123.   Zero gradients: 14/88 (15.9%)
     
124.   Loss: 0.469815
     
125. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
126. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
127. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
128. onTrainOneIter: total_pixels=196608, positive_pixels=62948, negative_pixels=133660, positive_ratio=0.320170
     
129. === TRAIN FUNCTION CALLED ===
     
130. input_tensor id: 2127117534992, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
131. target_tensor id: 2127117536864, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
132. input_tensor min/max: 0.000000/1.000000
     
133. target_tensor min/max: 0.000000/1.000000
     
134. input_tensor data sample: [0.29013884 0.28985968 0.2895732  0.28930765 0.28898674]
     
135. target_tensor data sample: [0. 0. 0. 0. 0.]
     
136.   pred_logits min/max: -20.726892/7.293363
     
137.   pred_logits sample: [-2.4902973 -1.8111492 -1.4735903 -2.8097928 -3.127867 ]
     
138.   target_tensor stats: total_pixels=196608, positive_pixels=62948, negative_pixels=133660, positive_ratio=0.320170
     
139.   Total grad_norm: 1.098472e+01, avg: 1.248264e-01
     
140.   Zero gradients: 16/88 (18.2%)
     
141.   Loss: 0.263270
     
142. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/0.990774
     
143. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
144. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
145. onTrainOneIter: total_pixels=196608, positive_pixels=65615, negative_pixels=130993, positive_ratio=0.333735
     
146. === TRAIN FUNCTION CALLED ===
     
147. input_tensor id: 2127117534992, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
148. target_tensor id: 2127117539888, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
149. input_tensor min/max: 0.000000/0.990774
     
150. target_tensor min/max: 0.000000/1.000000
     
151. input_tensor data sample: [0.06192069 0.06084388 0.05979334 0.05928424 0.05830612]
     
152. target_tensor data sample: [0. 0. 0. 0. 0.]
     
153.   pred_logits min/max: -23.783371/7.119821
     
154.   pred_logits sample: [-2.8762927  -1.6264055  -0.98988074 -2.3853555  -2.479313  ]
     
155.   target_tensor stats: total_pixels=196608, positive_pixels=65615, negative_pixels=130993, positive_ratio=0.333735
     
156.   Total grad_norm: 1.191719e+01, avg: 1.354226e-01
     
157.   Zero gradients: 16/88 (18.2%)
     
158.   Loss: 0.296378
     
159. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
160. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
161. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
162. onTrainOneIter: total_pixels=196608, positive_pixels=59914, negative_pixels=136694, positive_ratio=0.304738
     
163. === TRAIN FUNCTION CALLED ===
     
164. input_tensor id: 2127117534992, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
165. target_tensor id: 2127117537008, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
166. input_tensor min/max: 0.000000/1.000000
     
167. target_tensor min/max: 0.000000/1.000000
     
168. input_tensor data sample: [0.42745098 0.42745098 0.43529412 0.38039216 0.46666667]
     
169. target_tensor data sample: [0. 0. 0. 0. 0.]
     
170.   pred_logits min/max: -22.802631/9.289175
     
171.   pred_logits sample: [-3.4657896 -1.158008  -1.7567021 -2.7105498 -4.217543 ]
     
172.   target_tensor stats: total_pixels=196608, positive_pixels=59914, negative_pixels=136694, positive_ratio=0.304738
     
173.   Total grad_norm: 1.422873e+01, avg: 1.616901e-01
     
174.   Zero gradients: 15/88 (17.0%)
     
175.   Loss: 0.382004
     
176. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
177. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
178. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
179. onTrainOneIter: total_pixels=196608, positive_pixels=60038, negative_pixels=136570, positive_ratio=0.305369
     
180. === TRAIN FUNCTION CALLED ===
     
181. input_tensor id: 2127117537008, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
182. target_tensor id: 2127117540608, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
183. input_tensor min/max: 0.000000/1.000000
     
184. target_tensor min/max: 0.000000/1.000000
     
185. input_tensor data sample: [0.50262064 0.5055404  0.5037708  0.51130486 0.53210354]
     
186. target_tensor data sample: [0. 0. 0. 0. 0.]
     
187.   pred_logits min/max: -18.638235/10.057793
     
188.   pred_logits sample: [-4.906348   0.6163085 -1.3778951 -2.8572128 -2.77842  ]
     
189.   target_tensor stats: total_pixels=196608, positive_pixels=60038, negative_pixels=136570, positive_ratio=0.305369
     
190.   Total grad_norm: 5.369219e+00, avg: 6.101385e-02
     
191.   Zero gradients: 17/88 (19.3%)
     
192.   Loss: 0.195395
     
193. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
194. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
195. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
196. onTrainOneIter: total_pixels=196608, positive_pixels=66596, negative_pixels=130012, positive_ratio=0.338725
     
197. === TRAIN FUNCTION CALLED ===
     
198. input_tensor id: 2127117540608, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
199. target_tensor id: 2127117535712, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
200. input_tensor min/max: 0.000000/1.000000
     
201. target_tensor min/max: 0.000000/1.000000
     
202. input_tensor data sample: [0.52460366 0.5244897  0.524363   0.52421945 0.52408665]
     
203. target_tensor data sample: [0. 0. 0. 0. 0.]
     
204.   pred_logits min/max: -21.231295/9.863961
     
205.   pred_logits sample: [-4.5722823 -0.8571365 -1.4108061 -4.0296717 -4.0217266]
     
206.   target_tensor stats: total_pixels=196608, positive_pixels=66596, negative_pixels=130012, positive_ratio=0.338725
     
207.   Total grad_norm: 8.823583e+00, avg: 1.002680e-01
     
208.   Zero gradients: 17/88 (19.3%)
     
209.   Loss: 0.280314
     
210. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
211. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
212. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
213. onTrainOneIter: total_pixels=196608, positive_pixels=65776, negative_pixels=130832, positive_ratio=0.334554
     
214. === TRAIN FUNCTION CALLED ===
     
215. input_tensor id: 2127117535712, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
216. target_tensor id: 2127117541184, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
217. input_tensor min/max: 0.000000/1.000000
     
218. target_tensor min/max: 0.000000/1.000000
     
219. input_tensor data sample: [0.73016024 0.7404815  0.75060666 0.7508073  0.7453999 ]
     
220. target_tensor data sample: [0. 0. 0. 0. 0.]
     
221.   pred_logits min/max: -18.398451/10.207720
     
222.   pred_logits sample: [-4.2961097   0.24168685 -0.7255419  -2.3670042  -2.714671  ]
     
223.   target_tensor stats: total_pixels=196608, positive_pixels=65776, negative_pixels=130832, positive_ratio=0.334554
     
224.   Total grad_norm: 8.700741e+00, avg: 9.887206e-02
     
225.   Zero gradients: 16/88 (18.2%)
     
226.   Loss: 0.313794
     
227. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.009528/1.000000
     
228. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
229. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
230. onTrainOneIter: total_pixels=196608, positive_pixels=62885, negative_pixels=133723, positive_ratio=0.319850
     
231. === TRAIN FUNCTION CALLED ===
     
232. input_tensor id: 2127117535712, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
233. target_tensor id: 2127117535280, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
234. input_tensor min/max: 0.009528/1.000000
     
235. target_tensor min/max: 0.000000/1.000000
     
236. input_tensor data sample: [0.9843137 0.9843137 0.9843137 0.9843137 0.9843137]
     
237. target_tensor data sample: [0. 0. 0. 0. 0.]
     
238.   pred_logits min/max: -15.099107/9.275981
     
239.   pred_logits sample: [-5.9354715  2.48184   -3.4873571 -5.2332025 -5.772358 ]
     
240.   target_tensor stats: total_pixels=196608, positive_pixels=62885, negative_pixels=133723, positive_ratio=0.319850
     
241.   Total grad_norm: 6.640438e+00, avg: 7.545952e-02
     
242.   Zero gradients: 17/88 (19.3%)
     
243.   Loss: 0.221051
     
244. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
245. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
246. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
247. onTrainOneIter: total_pixels=196608, positive_pixels=59960, negative_pixels=136648, positive_ratio=0.304972
     
248. === TRAIN FUNCTION CALLED ===
     
249. input_tensor id: 2127117535712, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
250. target_tensor id: 2127117541760, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
251. input_tensor min/max: 0.000000/1.000000
     
252. target_tensor min/max: 0.000000/1.000000
     
253. input_tensor data sample: [0.33630127 0.32305497 0.307116   0.3292566  0.3842591 ]
     
254. target_tensor data sample: [0. 0. 0. 0. 0.]
     
255.   pred_logits min/max: -16.089781/9.791711
     
256.   pred_logits sample: [-4.495612  -1.9841919 -2.935329  -3.786987  -3.5490012]
     
257.   target_tensor stats: total_pixels=196608, positive_pixels=59960, negative_pixels=136648, positive_ratio=0.304972
     
258.   Total grad_norm: 2.765088e+01, avg: 3.142145e-01
     
259.   Zero gradients: 13/88 (14.8%)
     
260.   Loss: 0.603987
     
261. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.005607/1.000000
     
262. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
263. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
264. onTrainOneIter: total_pixels=196608, positive_pixels=60896, negative_pixels=135712, positive_ratio=0.309733
     
265. === TRAIN FUNCTION CALLED ===
     
266. input_tensor id: 2127117535712, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
267. target_tensor id: 2127117533840, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
268. input_tensor min/max: 0.005607/1.000000
     
269. target_tensor min/max: 0.000000/1.000000
     
270. input_tensor data sample: [0.22499394 0.21625811 0.31263733 0.3163157  0.317699  ]
     
271. target_tensor data sample: [0. 0. 0. 0. 0.]
     
272.   pred_logits min/max: -15.114578/8.362134
     
273.   pred_logits sample: [-3.8244817 -2.4959428 -2.8394878 -3.6102636 -4.0377254]
     
274.   target_tensor stats: total_pixels=196608, positive_pixels=60896, negative_pixels=135712, positive_ratio=0.309733
     
275.   Total grad_norm: 8.240419e+00, avg: 9.364112e-02
     
276.   Zero gradients: 16/88 (18.2%)
     
277.   Loss: 0.230042
     
278. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
279. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
280. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
281. onTrainOneIter: total_pixels=196608, positive_pixels=62055, negative_pixels=134553, positive_ratio=0.315628
     
282. === TRAIN FUNCTION CALLED ===
     
283. input_tensor id: 2127117533840, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
284. target_tensor id: 2127117542336, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
285. input_tensor min/max: 0.000000/1.000000
     
286. target_tensor min/max: 0.000000/1.000000
     
287. input_tensor data sample: [0.7985813  0.795848   0.79368013 0.79222304 0.7900003 ]
     
288. target_tensor data sample: [0. 0. 0. 0. 0.]
     
289.   pred_logits min/max: -15.357988/7.228970
     
290.   pred_logits sample: [-4.1696863  -1.3473691  -0.58398455 -3.9897606  -3.883157  ]
     
291.   target_tensor stats: total_pixels=196608, positive_pixels=62055, negative_pixels=134553, positive_ratio=0.315628
     
292.   Total grad_norm: 1.233695e+01, avg: 1.401926e-01
     
293.   Zero gradients: 16/88 (18.2%)
     
294.   Loss: 0.294917
     
295. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
296. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
297. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
298. onTrainOneIter: total_pixels=196608, positive_pixels=62153, negative_pixels=134455, positive_ratio=0.316127
     
299. === TRAIN FUNCTION CALLED ===
     
300. input_tensor id: 2127117533840, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
301. target_tensor id: 2127117533984, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
302. input_tensor min/max: 0.000000/1.000000
     
303. target_tensor min/max: 0.000000/1.000000
     
304. input_tensor data sample: [0.06908149 0.06904165 0.06914473 0.06926265 0.06937093]
     
305. target_tensor data sample: [0. 0. 0. 0. 0.]
     
306.   pred_logits min/max: -17.118107/7.950355
     
307.   pred_logits sample: [-3.8110635 -4.100872  -4.1111913 -3.4802969 -4.0880437]
     
308.   target_tensor stats: total_pixels=196608, positive_pixels=62153, negative_pixels=134455, positive_ratio=0.316127
     
309.   Total grad_norm: 7.727636e+00, avg: 8.781404e-02
     
310.   Zero gradients: 16/88 (18.2%)
     
311.   Loss: 0.206417
     
312. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
313. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
314. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
315. onTrainOneIter: total_pixels=196608, positive_pixels=65836, negative_pixels=130772, positive_ratio=0.334859
     
316. === TRAIN FUNCTION CALLED ===
     
317. input_tensor id: 2127117533840, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
318. target_tensor id: 2127117536288, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
319. input_tensor min/max: 0.000000/1.000000
     
320. target_tensor min/max: 0.000000/1.000000
     
321. input_tensor data sample: [0.3332027  0.32920265 0.3213459  0.31719378 0.30544254]
     
322. target_tensor data sample: [0. 0. 0. 0. 0.]
     
323.   pred_logits min/max: -14.921435/7.857790
     
324.   pred_logits sample: [-4.045929  -2.456232  -1.8165385 -3.446359  -3.5477982]
     
325.   target_tensor stats: total_pixels=196608, positive_pixels=65836, negative_pixels=130772, positive_ratio=0.334859
     
326.   Total grad_norm: 6.278506e+00, avg: 7.134666e-02
     
327.   Zero gradients: 16/88 (18.2%)
     
328.   Loss: 0.215464
     
329. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
330. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
331. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
332. onTrainOneIter: total_pixels=196608, positive_pixels=63483, negative_pixels=133125, positive_ratio=0.322891
     
333. === TRAIN FUNCTION CALLED ===
     
334. input_tensor id: 2127117533840, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
335. target_tensor id: 2127117536576, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
336. input_tensor min/max: 0.000000/1.000000
     
337. target_tensor min/max: 0.000000/1.000000
     
338. input_tensor data sample: [0.90201896 0.90201896 0.90201896 0.90201896 0.90201896]
     
339. target_tensor data sample: [0. 0. 0. 0. 0.]
     
340.   pred_logits min/max: -15.289414/7.923834
     
341.   pred_logits sample: [-5.1892467  0.3257783 -3.137312  -3.9267967 -2.9589558]
     
342.   target_tensor stats: total_pixels=196608, positive_pixels=63483, negative_pixels=133125, positive_ratio=0.322891
     
343.   Total grad_norm: 2.001761e+01, avg: 2.274728e-01
     
344.   Zero gradients: 15/88 (17.0%)
     
345.   Loss: 0.327865
     
346. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
347. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
348. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
349. onTrainOneIter: total_pixels=196608, positive_pixels=66364, negative_pixels=130244, positive_ratio=0.337545
     
350. === TRAIN FUNCTION CALLED ===
     
351. input_tensor id: 2127117536288, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
352. target_tensor id: 2127117537440, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
353. input_tensor min/max: 0.000000/1.000000
     
354. target_tensor min/max: 0.000000/1.000000
     
355. input_tensor data sample: [0.93682104 0.9366163  0.9301388  0.9306011  0.92951435]
     
356. target_tensor data sample: [0. 0. 0. 0. 0.]
     
357.   pred_logits min/max: -17.635593/9.228088
     
358.   pred_logits sample: [-5.3889318  -0.08361167 -2.1326015  -2.2848048  -2.4422824 ]
     
359.   target_tensor stats: total_pixels=196608, positive_pixels=66364, negative_pixels=130244, positive_ratio=0.337545
     
360.   Total grad_norm: 2.013925e+01, avg: 2.288551e-01
     
361.   Zero gradients: 15/88 (17.0%)
     
362.   Loss: 0.356524
     
363. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
364. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
365. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
366. onTrainOneIter: total_pixels=196608, positive_pixels=65618, negative_pixels=130990, positive_ratio=0.333750
     
367. === TRAIN FUNCTION CALLED ===
     
368. input_tensor id: 2127144432944, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
369. target_tensor id: 2127117536288, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
370. input_tensor min/max: 0.000000/1.000000
     
371. target_tensor min/max: 0.000000/1.000000
     
372. input_tensor data sample: [0.6234512  0.62185526 0.6202409  0.6186082  0.6169569 ]
     
373. target_tensor data sample: [0. 0. 0. 0. 0.]
     
374.   pred_logits min/max: -18.993696/5.892333
     
375.   pred_logits sample: [-2.4105806   0.32982633 -2.3814206  -1.399349   -2.8512943 ]
     
376.   target_tensor stats: total_pixels=196608, positive_pixels=65618, negative_pixels=130990, positive_ratio=0.333750
     
377.   Total grad_norm: 1.207343e+01, avg: 1.371981e-01
     
378.   Zero gradients: 14/88 (15.9%)
     
379.   Loss: 0.295563
     
380. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.033946/1.000000
     
381. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
382. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
383. onTrainOneIter: total_pixels=196608, positive_pixels=60177, negative_pixels=136431, positive_ratio=0.306076
     
384. === TRAIN FUNCTION CALLED ===
     
385. input_tensor id: 2127117535856, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
386. target_tensor id: 2127117540320, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
387. input_tensor min/max: 0.033946/1.000000
     
388. target_tensor min/max: 0.000000/1.000000
     
389. input_tensor data sample: [0.5714462  0.5824624  0.58264655 0.57580924 0.57451284]
     
390. target_tensor data sample: [0. 0. 0. 0. 0.]
     
391.   pred_logits min/max: -20.553726/8.566295
     
392.   pred_logits sample: [-3.9655604   0.96089125 -3.0078368  -2.1678395  -2.479184  ]
     
393.   target_tensor stats: total_pixels=196608, positive_pixels=60177, negative_pixels=136431, positive_ratio=0.306076
     
394.   Total grad_norm: 7.800439e+00, avg: 8.864135e-02
     
395.   Zero gradients: 16/88 (18.2%)
     
396.   Loss: 0.214262
     
397. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/0.949539
     
398. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
399. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
400. onTrainOneIter: total_pixels=196608, positive_pixels=66052, negative_pixels=130556, positive_ratio=0.335958
     
401. === TRAIN FUNCTION CALLED ===
     
402. input_tensor id: 2127117535856, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
403. target_tensor id: 2127117537008, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
404. input_tensor min/max: 0.000000/0.949539
     
405. target_tensor min/max: 0.000000/1.000000
     
406. input_tensor data sample: [0.13139182 0.13866419 0.13650595 0.13696605 0.13724683]
     
407. target_tensor data sample: [0. 0. 0. 0. 0.]
     
408.   pred_logits min/max: -18.889837/8.157580
     
409.   pred_logits sample: [-2.952438  -2.9505057 -3.1360536 -3.050571  -3.6535916]
     
410.   target_tensor stats: total_pixels=196608, positive_pixels=66052, negative_pixels=130556, positive_ratio=0.335958
     
411.   Total grad_norm: 5.657471e+00, avg: 6.428944e-02
     
412.   Zero gradients: 17/88 (19.3%)
     
413.   Loss: 0.188542
     
414. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
415. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
416. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
417. onTrainOneIter: total_pixels=196608, positive_pixels=59883, negative_pixels=136725, positive_ratio=0.304581
     
418. === TRAIN FUNCTION CALLED ===
     
419. input_tensor id: 2127117535856, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
420. target_tensor id: 2127117540752, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
421. input_tensor min/max: 0.000000/1.000000
     
422. target_tensor min/max: 0.000000/1.000000
     
423. input_tensor data sample: [0.9339851  0.93063235 0.920338   0.9100436  0.91274273]
     
424. target_tensor data sample: [0. 0. 0. 0. 0.]
     
425.   pred_logits min/max: -17.082052/7.269665
     
426.   pred_logits sample: [-5.706052   -0.43251267 -5.8534803  -7.1242924  -5.9445167 ]
     
427.   target_tensor stats: total_pixels=196608, positive_pixels=59883, negative_pixels=136725, positive_ratio=0.304581
     
428.   Total grad_norm: 9.922237e+00, avg: 1.127527e-01
     
429.   Zero gradients: 16/88 (18.2%)
     
430.   Loss: 0.271777
     
431. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
432. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
433. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
434. onTrainOneIter: total_pixels=196608, positive_pixels=66281, negative_pixels=130327, positive_ratio=0.337123
     
435. === TRAIN FUNCTION CALLED ===
     
436. input_tensor id: 2127117535856, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
437. target_tensor id: 2127117535424, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
438. input_tensor min/max: 0.000000/1.000000
     
439. target_tensor min/max: 0.000000/1.000000
     
440. input_tensor data sample: [0.910467   0.9105796  0.90350425 0.8964073  0.8965049 ]
     
441. target_tensor data sample: [0. 0. 0. 0. 0.]
     
442.   pred_logits min/max: -19.048122/6.673307
     
443.   pred_logits sample: [-6.365427  -2.1181576 -3.0997455 -5.8022437 -3.8753278]
     
444.   target_tensor stats: total_pixels=196608, positive_pixels=66281, negative_pixels=130327, positive_ratio=0.337123
     
445.   Total grad_norm: 6.779262e+00, avg: 7.703707e-02
     
446.   Zero gradients: 16/88 (18.2%)
     
447.   Loss: 0.228261
     
448. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
449. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
450. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
451. onTrainOneIter: total_pixels=196608, positive_pixels=62250, negative_pixels=134358, positive_ratio=0.316620
     
452. === TRAIN FUNCTION CALLED ===
     
453. input_tensor id: 2127117535424, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
454. target_tensor id: 2127117539168, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
455. input_tensor min/max: 0.000000/1.000000
     
456. target_tensor min/max: 0.000000/1.000000
     
457. input_tensor data sample: [0.40016812 0.4001449  0.4019328  0.4006391  0.39729145]
     
458. target_tensor data sample: [0. 0. 0. 0. 0.]
     
459.   pred_logits min/max: -19.240149/6.941235
     
460.   pred_logits sample: [-3.5946527 -2.404446  -2.034595  -2.8078148 -3.102897 ]
     
461.   target_tensor stats: total_pixels=196608, positive_pixels=62250, negative_pixels=134358, positive_ratio=0.316620
     
462.   Total grad_norm: 1.018770e+01, avg: 1.157693e-01
     
463.   Zero gradients: 16/88 (18.2%)
     
464.   Loss: 0.264548
     
465. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.056059/1.000000
     
466. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
467. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
468. onTrainOneIter: total_pixels=196608, positive_pixels=63742, negative_pixels=132866, positive_ratio=0.324209
     
469. === TRAIN FUNCTION CALLED ===
     
470. input_tensor id: 2127117535424, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
471. target_tensor id: 2127117534416, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
472. input_tensor min/max: 0.056059/1.000000
     
473. target_tensor min/max: 0.000000/1.000000
     
474. input_tensor data sample: [0.51105714 0.51164067 0.5121992  0.51267946 0.45662424]
     
475. target_tensor data sample: [0. 0. 0. 0. 0.]
     
476.   pred_logits min/max: -19.424805/7.869545
     
477.   pred_logits sample: [-3.8387825 -1.8366293 -2.0311072 -5.2345695 -2.5528834]
     
478.   target_tensor stats: total_pixels=196608, positive_pixels=63742, negative_pixels=132866, positive_ratio=0.324209
     
479.   Total grad_norm: 5.092747e+00, avg: 5.787213e-02
     
480.   Zero gradients: 17/88 (19.3%)
     
481.   Loss: 0.137636
     
482. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.070583/1.000000
     
483. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
484. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
485. onTrainOneIter: total_pixels=196608, positive_pixels=60911, negative_pixels=135697, positive_ratio=0.309809
     
486. === TRAIN FUNCTION CALLED ===
     
487. input_tensor id: 2127117535424, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
488. target_tensor id: 2127117542192, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
489. input_tensor min/max: 0.070583/1.000000
     
490. target_tensor min/max: 0.000000/1.000000
     
491. input_tensor data sample: [0.7975298  0.7978341  0.79855996 0.7994592  0.80059665]
     
492. target_tensor data sample: [0. 0. 0. 0. 0.]
     
493.   pred_logits min/max: -18.442959/8.092695
     
494.   pred_logits sample: [-6.1459017 -2.4756172 -4.3615456 -3.950786  -4.515348 ]
     
495.   target_tensor stats: total_pixels=196608, positive_pixels=60911, negative_pixels=135697, positive_ratio=0.309809
     
496.   Total grad_norm: 7.955581e+00, avg: 9.040433e-02
     
497.   Zero gradients: 17/88 (19.3%)
     
498.   Loss: 0.208783
     
499. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
500. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
501. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
502. onTrainOneIter: total_pixels=196608, positive_pixels=60226, negative_pixels=136382, positive_ratio=0.306325
     
503. === TRAIN FUNCTION CALLED ===
     
504. input_tensor id: 2127117535424, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
505. target_tensor id: 2127117538016, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
506. input_tensor min/max: 0.000000/1.000000
     
507. target_tensor min/max: 0.000000/1.000000
     
508. input_tensor data sample: [0.6816234  0.6811441  0.6863018  0.70137674 0.79480106]
     
509. target_tensor data sample: [0. 0. 0. 0. 0.]
     
510.   pred_logits min/max: -20.050505/7.682349
     
511.   pred_logits sample: [-3.5655038 -1.8527833 -4.715788  -1.7698956 -4.470682 ]
     
512.   target_tensor stats: total_pixels=196608, positive_pixels=60226, negative_pixels=136382, positive_ratio=0.306325
     
513.   Total grad_norm: 1.282556e+01, avg: 1.457450e-01
     
514.   Zero gradients: 15/88 (17.0%)
     
515.   Loss: 0.308620
     
516. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000443/1.000000
     
517. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
518. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
519. onTrainOneIter: total_pixels=196608, positive_pixels=67241, negative_pixels=129367, positive_ratio=0.342005
     
520. === TRAIN FUNCTION CALLED ===
     
521. input_tensor id: 2127117535424, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
522. target_tensor id: 2127117539600, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
523. input_tensor min/max: 0.000443/1.000000
     
524. target_tensor min/max: 0.000000/1.000000
     
525. input_tensor data sample: [0.2906826  0.29090518 0.2910909  0.29129034 0.29142997]
     
526. target_tensor data sample: [0. 0. 0. 0. 0.]
     
527.   pred_logits min/max: -17.146389/7.148095
     
528.   pred_logits sample: [-3.1176658 -2.602241  -2.7719784 -3.6554322 -3.2226985]
     
529.   target_tensor stats: total_pixels=196608, positive_pixels=67241, negative_pixels=129367, positive_ratio=0.342005
     
530.   Total grad_norm: 5.208597e+00, avg: 5.918860e-02
     
531.   Zero gradients: 16/88 (18.2%)
     
532.   Loss: 0.197295
     
533. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
534. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
535. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
536. onTrainOneIter: total_pixels=196608, positive_pixels=63587, negative_pixels=133021, positive_ratio=0.323420
     
537. === TRAIN FUNCTION CALLED ===
     
538. input_tensor id: 2127117539600, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
539. target_tensor id: 2127117533408, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
540. input_tensor min/max: 0.000000/1.000000
     
541. target_tensor min/max: 0.000000/1.000000
     
542. input_tensor data sample: [0.32651213 0.32723528 0.32737562 0.32758117 0.3278699 ]
     
543. target_tensor data sample: [0. 0. 0. 0. 0.]
     
544.   pred_logits min/max: -24.800102/9.366374
     
545.   pred_logits sample: [-2.9568644 -2.9865596 -2.1183672 -3.5754726 -4.0290723]
     
546.   target_tensor stats: total_pixels=196608, positive_pixels=63587, negative_pixels=133021, positive_ratio=0.323420
     
547.   Total grad_norm: 5.289403e+00, avg: 6.010685e-02
     
548.   Zero gradients: 17/88 (19.3%)
     
549.   Loss: 0.155098
     
550. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
551. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
552. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
553. onTrainOneIter: total_pixels=196608, positive_pixels=62260, negative_pixels=134348, positive_ratio=0.316671
     
554. === TRAIN FUNCTION CALLED ===
     
555. input_tensor id: 2127117539600, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
556. target_tensor id: 2127117536864, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
557. input_tensor min/max: 0.000000/1.000000
     
558. target_tensor min/max: 0.000000/1.000000
     
559. input_tensor data sample: [0.77622557 0.77949905 0.7803165  0.77980065 0.78029484]
     
560. target_tensor data sample: [0. 0. 0. 0. 0.]
     
561.   pred_logits min/max: -15.465410/8.292643
     
562.   pred_logits sample: [-5.1960363 -2.7262805 -4.2124124 -3.4014785 -3.5092165]
     
563.   target_tensor stats: total_pixels=196608, positive_pixels=62260, negative_pixels=134348, positive_ratio=0.316671
     
564.   Total grad_norm: 1.733326e+01, avg: 1.969689e-01
     
565.   Zero gradients: 15/88 (17.0%)
     
566.   Loss: 0.270896
     
567. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.007880/1.000000
     
568. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
569. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
570. onTrainOneIter: total_pixels=196608, positive_pixels=63661, negative_pixels=132947, positive_ratio=0.323797
     
571. === TRAIN FUNCTION CALLED ===
     
572. input_tensor id: 2127117536864, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
573. target_tensor id: 2127117540896, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
574. input_tensor min/max: 0.007880/1.000000
     
575. target_tensor min/max: 0.000000/1.000000
     
576. input_tensor data sample: [0.9487566  0.9545759  0.96072006 0.9586823  0.9516822 ]
     
577. target_tensor data sample: [0. 0. 0. 0. 0.]
     
578.   pred_logits min/max: -15.762245/8.872600
     
579.   pred_logits sample: [-7.0780697 -1.9001844 -7.3066106 -7.5993633 -7.7355614]
     
580.   target_tensor stats: total_pixels=196608, positive_pixels=63661, negative_pixels=132947, positive_ratio=0.323797
     
581.   Total grad_norm: 9.250301e+00, avg: 1.051171e-01
     
582.   Zero gradients: 16/88 (18.2%)
     
583.   Loss: 0.200962
     
584. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
585. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
586. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
587. onTrainOneIter: total_pixels=196608, positive_pixels=59623, negative_pixels=136985, positive_ratio=0.303258
     
588. === TRAIN FUNCTION CALLED ===
     
589. input_tensor id: 2127117536864, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
590. target_tensor id: 2127117537440, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
591. input_tensor min/max: 0.000000/1.000000
     
592. target_tensor min/max: 0.000000/1.000000
     
593. input_tensor data sample: [0.14610593 0.1946109  0.24477606 0.26804468 0.26333246]
     
594. target_tensor data sample: [0. 0. 0. 0. 0.]
     
595.   pred_logits min/max: -15.081598/7.973622
     
596.   pred_logits sample: [-3.2265728 -3.394641  -2.7356813 -2.5273912 -2.807359 ]
     
597.   target_tensor stats: total_pixels=196608, positive_pixels=59623, negative_pixels=136985, positive_ratio=0.303258
     
598.   Total grad_norm: 2.116366e+01, avg: 2.404961e-01
     
599.   Zero gradients: 14/88 (15.9%)
     
600.   Loss: 0.355708
     
601. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
602. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
603. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
604. onTrainOneIter: total_pixels=196608, positive_pixels=62786, negative_pixels=133822, positive_ratio=0.319346
     
605. === TRAIN FUNCTION CALLED ===
     
606. input_tensor id: 2127117537440, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
607. target_tensor id: 2127117536288, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
608. input_tensor min/max: 0.000000/1.000000
     
609. target_tensor min/max: 0.000000/1.000000
     
610. input_tensor data sample: [0.5472278  0.5470247  0.5468248  0.54662937 0.5464415 ]
     
611. target_tensor data sample: [0. 0. 0. 0. 0.]
     
612.   pred_logits min/max: -18.813835/9.649268
     
613.   pred_logits sample: [-3.7702944 -2.8623612 -3.6244416 -3.1673868 -3.6599746]
     
614.   target_tensor stats: total_pixels=196608, positive_pixels=62786, negative_pixels=133822, positive_ratio=0.319346
     
615.   Total grad_norm: 6.757914e+00, avg: 7.679448e-02
     
616.   Zero gradients: 16/88 (18.2%)
     
617.   Loss: 0.161846
     
618. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
619. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
620. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
621. onTrainOneIter: total_pixels=196608, positive_pixels=57118, negative_pixels=139490, positive_ratio=0.290517
     
622. === TRAIN FUNCTION CALLED ===
     
623. input_tensor id: 2127117536288, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
624. target_tensor id: 2127117540320, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
625. input_tensor min/max: 0.000000/1.000000
     
626. target_tensor min/max: 0.000000/1.000000
     
627. input_tensor data sample: [1. 1. 1. 1. 1.]
     
628. target_tensor data sample: [0. 0. 0. 0. 0.]
     
629.   pred_logits min/max: -18.241982/8.196582
     
630.   pred_logits sample: [-5.855565  -1.6025399 -6.074429  -6.626479  -5.5634313]
     
631.   target_tensor stats: total_pixels=196608, positive_pixels=57118, negative_pixels=139490, positive_ratio=0.290517
     
632.   Total grad_norm: 1.313770e+01, avg: 1.492920e-01
     
633.   Zero gradients: 15/88 (17.0%)
     
634.   Loss: 0.252659
     
635. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.108599/1.000000
     
636. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
637. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
638. onTrainOneIter: total_pixels=196608, positive_pixels=66934, negative_pixels=129674, positive_ratio=0.340444
     
639. === TRAIN FUNCTION CALLED ===
     
640. input_tensor id: 2127117536288, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
641. target_tensor id: 2127117537008, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
642. input_tensor min/max: 0.108599/1.000000
     
643. target_tensor min/max: 0.000000/1.000000
     
644. input_tensor data sample: [0.31287837 0.3132637  0.3129761  0.3132492  0.3135593 ]
     
645. target_tensor data sample: [0. 0. 0. 0. 0.]
     
646.   pred_logits min/max: -15.678344/8.656833
     
647.   pred_logits sample: [-2.1171982 -2.3288114 -2.4293106 -4.0685472 -3.7923076]
     
648.   target_tensor stats: total_pixels=196608, positive_pixels=66934, negative_pixels=129674, positive_ratio=0.340444
     
649.   Total grad_norm: 9.281142e+00, avg: 1.054675e-01
     
650.   Zero gradients: 16/88 (18.2%)
     
651.   Loss: 0.215952
     
652. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.007893/1.000000
     
653. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
654. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
655. onTrainOneIter: total_pixels=196608, positive_pixels=63652, negative_pixels=132956, positive_ratio=0.323751
     
656. === TRAIN FUNCTION CALLED ===
     
657. input_tensor id: 2127117536288, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
658. target_tensor id: 2127117540752, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
659. input_tensor min/max: 0.007893/1.000000
     
660. target_tensor min/max: 0.000000/1.000000
     
661. input_tensor data sample: [0.46717143 0.46717167 0.46717227 0.4692511  0.4536056 ]
     
662. target_tensor data sample: [0. 0. 0. 0. 0.]
     
663.   pred_logits min/max: -19.933399/10.427228
     
664.   pred_logits sample: [-3.3781598 -3.1043682 -4.1953983 -3.8783824 -4.5258155]
     
665.   target_tensor stats: total_pixels=196608, positive_pixels=63652, negative_pixels=132956, positive_ratio=0.323751
     
666.   Total grad_norm: 1.586424e+01, avg: 1.802754e-01
     
667.   Zero gradients: 16/88 (18.2%)
     
668.   Loss: 0.279551
     
669. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
670. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
671. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
672. onTrainOneIter: total_pixels=196608, positive_pixels=62347, negative_pixels=134261, positive_ratio=0.317113
     
673. === TRAIN FUNCTION CALLED ===
     
674. input_tensor id: 2127117536288, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
675. target_tensor id: 2127117535136, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
676. input_tensor min/max: 0.000000/1.000000
     
677. target_tensor min/max: 0.000000/1.000000
     
678. input_tensor data sample: [1. 1. 1. 1. 1.]
     
679. target_tensor data sample: [0. 0. 0. 0. 0.]
     
680.   pred_logits min/max: -18.501469/7.382880
     
681.   pred_logits sample: [-5.885928  -1.9822763 -5.7978067 -6.279719  -5.901813 ]
     
682.   target_tensor stats: total_pixels=196608, positive_pixels=62347, negative_pixels=134261, positive_ratio=0.317113
     
683.   Total grad_norm: 6.419612e+00, avg: 7.295014e-02
     
684.   Zero gradients: 17/88 (19.3%)
     
685.   Loss: 0.146821
     
686. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000958/1.000000
     
687. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
688. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
689. onTrainOneIter: total_pixels=196608, positive_pixels=65981, negative_pixels=130627, positive_ratio=0.335597
     
690. === TRAIN FUNCTION CALLED ===
     
691. input_tensor id: 2127117535136, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
692. target_tensor id: 2127117539168, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
693. input_tensor min/max: 0.000958/1.000000
     
694. target_tensor min/max: 0.000000/1.000000
     
695. input_tensor data sample: [1. 1. 1. 1. 1.]
     
696. target_tensor data sample: [0. 0. 0. 0. 0.]
     
697.   pred_logits min/max: -15.779600/9.060302
     
698.   pred_logits sample: [-6.6180816 -2.8671558 -7.646466  -7.6430163 -8.011188 ]
     
699.   target_tensor stats: total_pixels=196608, positive_pixels=65981, negative_pixels=130627, positive_ratio=0.335597
     
700.   Total grad_norm: 4.236301e+00, avg: 4.813978e-02
     
701.   Zero gradients: 17/88 (19.3%)
     
702.   Loss: 0.129366
     
703. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.060862/1.000000
     
704. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
705. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
706. onTrainOneIter: total_pixels=196608, positive_pixels=65192, negative_pixels=131416, positive_ratio=0.331584
     
707. === TRAIN FUNCTION CALLED ===
     
708. input_tensor id: 2127117535136, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
709. target_tensor id: 2127117534416, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
710. input_tensor min/max: 0.060862/1.000000
     
711. target_tensor min/max: 0.000000/1.000000
     
712. input_tensor data sample: [0.8448762  0.83817345 0.8356864  0.8310597  0.8385409 ]
     
713. target_tensor data sample: [0. 0. 0. 0. 0.]
     
714.   pred_logits min/max: -19.217228/8.068666
     
715.   pred_logits sample: [-5.399063  -2.5961206 -5.7274303 -5.5696764 -6.552657 ]
     
716.   target_tensor stats: total_pixels=196608, positive_pixels=65192, negative_pixels=131416, positive_ratio=0.331584
     
717.   Total grad_norm: 6.170573e+00, avg: 7.012015e-02
     
718.   Zero gradients: 16/88 (18.2%)
     
719.   Loss: 0.190160
     
720. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
721. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
722. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
723. onTrainOneIter: total_pixels=196608, positive_pixels=59389, negative_pixels=137219, positive_ratio=0.302068
     
724. === TRAIN FUNCTION CALLED ===
     
725. input_tensor id: 2127117535136, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
726. target_tensor id: 2127117542192, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
727. input_tensor min/max: 0.000000/1.000000
     
728. target_tensor min/max: 0.000000/1.000000
     
729. input_tensor data sample: [0.54000276 0.5401727  0.5410493  0.54192585 0.5413445 ]
     
730. target_tensor data sample: [0. 0. 0. 0. 0.]
     
731.   pred_logits min/max: -20.906250/10.136984
     
732.   pred_logits sample: [-5.288657  -0.5613942 -4.808365  -5.148283  -6.4273825]
     
733.   target_tensor stats: total_pixels=196608, positive_pixels=59389, negative_pixels=137219, positive_ratio=0.302068
     
734.   Total grad_norm: 9.717779e+00, avg: 1.104293e-01
     
735.   Zero gradients: 17/88 (19.3%)
     
736.   Loss: 0.207772
     
737. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
738. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
739. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
740. onTrainOneIter: total_pixels=196608, positive_pixels=57964, negative_pixels=138644, positive_ratio=0.294820
     
741. === TRAIN FUNCTION CALLED ===
     
742. input_tensor id: 2127117542192, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
743. target_tensor id: 2127117538016, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
744. input_tensor min/max: 0.000000/1.000000
     
745. target_tensor min/max: 0.000000/1.000000
     
746. input_tensor data sample: [1. 1. 1. 1. 1.]
     
747. target_tensor data sample: [0. 0. 0. 0. 0.]
     
748.   pred_logits min/max: -17.454338/10.083261
     
749.   pred_logits sample: [-7.375121  -2.1423428 -3.6835415 -4.190499  -3.5363479]
     
750.   target_tensor stats: total_pixels=196608, positive_pixels=57964, negative_pixels=138644, positive_ratio=0.294820
     
751.   Total grad_norm: 7.362339e+00, avg: 8.366294e-02
     
752.   Zero gradients: 17/88 (19.3%)
     
753.   Loss: 0.194175
     
754. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
755. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
756. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
757. onTrainOneIter: total_pixels=196608, positive_pixels=64318, negative_pixels=132290, positive_ratio=0.327138
     
758. === TRAIN FUNCTION CALLED ===
     
759. input_tensor id: 2127117542192, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
760. target_tensor id: 2127117536144, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
761. input_tensor min/max: 0.000000/1.000000
     
762. target_tensor min/max: 0.000000/1.000000
     
763. input_tensor data sample: [0.26033032 0.26178974 0.2643577  0.26182985 0.29657215]
     
764. target_tensor data sample: [0. 0. 0. 0. 0.]
     
765.   pred_logits min/max: -17.451447/8.749518
     
766.   pred_logits sample: [-3.1394143 -2.3746457 -2.5780208 -2.6517417 -3.294534 ]
     
767.   target_tensor stats: total_pixels=196608, positive_pixels=64318, negative_pixels=132290, positive_ratio=0.327138
     
768.   Total grad_norm: 6.025170e+00, avg: 6.846784e-02
     
769.   Zero gradients: 16/88 (18.2%)
     
770.   Loss: 0.198316
     
771. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.130840/1.000000
     
772. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
773. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
774. onTrainOneIter: total_pixels=196608, positive_pixels=56826, negative_pixels=139782, positive_ratio=0.289032
     
775. === TRAIN FUNCTION CALLED ===
     
776. input_tensor id: 2127117542192, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
777. target_tensor id: 2127117533408, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
778. input_tensor min/max: 0.130840/1.000000
     
779. target_tensor min/max: 0.000000/1.000000
     
780. input_tensor data sample: [0.5569104  0.5471595  0.5624075  0.61679864 0.66409314]
     
781. target_tensor data sample: [0. 0. 0. 0. 0.]
     
782.   pred_logits min/max: -17.332588/8.201318
     
783.   pred_logits sample: [-4.1926503 -2.3131046 -1.782809  -4.220693  -4.3083854]
     
784.   target_tensor stats: total_pixels=196608, positive_pixels=56826, negative_pixels=139782, positive_ratio=0.289032
     
785.   Total grad_norm: 1.092631e+01, avg: 1.241626e-01
     
786.   Zero gradients: 16/88 (18.2%)
     
787.   Loss: 0.251830
     
788. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
789. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
790. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
791. onTrainOneIter: total_pixels=196608, positive_pixels=65594, negative_pixels=131014, positive_ratio=0.333628
     
792. === TRAIN FUNCTION CALLED ===
     
793. input_tensor id: 2127117542192, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
794. target_tensor id: 2127117537872, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
795. input_tensor min/max: 0.000000/1.000000
     
796. target_tensor min/max: 0.000000/1.000000
     
797. input_tensor data sample: [0.30872753 0.30795506 0.3072505  0.30659503 0.30607665]
     
798. target_tensor data sample: [0. 0. 0. 0. 0.]
     
799.   pred_logits min/max: -15.688857/9.875670
     
800.   pred_logits sample: [-2.7347832 -2.9977772 -2.4747734 -3.9203231 -3.9806335]
     
801.   target_tensor stats: total_pixels=196608, positive_pixels=65594, negative_pixels=131014, positive_ratio=0.333628
     
802.   Total grad_norm: 7.661690e+00, avg: 8.706466e-02
     
803.   Zero gradients: 16/88 (18.2%)
     
804.   Loss: 0.203311
     
805. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
806. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
807. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
808. onTrainOneIter: total_pixels=196608, positive_pixels=60903, negative_pixels=135705, positive_ratio=0.309769
     
809. === TRAIN FUNCTION CALLED ===
     
810. input_tensor id: 2127117537872, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
811. target_tensor id: 2127117540896, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
812. input_tensor min/max: 0.000000/1.000000
     
813. target_tensor min/max: 0.000000/1.000000
     
814. input_tensor data sample: [0.94509804 0.94509804 0.9490196  0.94509804 0.93333334]
     
815. target_tensor data sample: [0. 0. 0. 0. 0.]
     
816.   pred_logits min/max: -18.976940/10.455793
     
817.   pred_logits sample: [-4.853873  -1.8235066 -6.3976054 -3.7020407 -7.453571 ]
     
818.   target_tensor stats: total_pixels=196608, positive_pixels=60903, negative_pixels=135705, positive_ratio=0.309769
     
819.   Total grad_norm: 6.020146e+00, avg: 6.841075e-02
     
820.   Zero gradients: 17/88 (19.3%)
     
821.   Loss: 0.161017
     
822. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.013700/1.000000
     
823. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
824. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
825. onTrainOneIter: total_pixels=196608, positive_pixels=63809, negative_pixels=132799, positive_ratio=0.324549
     
826. === TRAIN FUNCTION CALLED ===
     
827. input_tensor id: 2127117540896, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
828. target_tensor id: 2127117534560, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
829. input_tensor min/max: 0.013700/1.000000
     
830. target_tensor min/max: 0.000000/1.000000
     
831. input_tensor data sample: [1. 1. 1. 1. 1.]
     
832. target_tensor data sample: [0. 0. 0. 0. 0.]
     
833.   pred_logits min/max: -15.595974/8.512142
     
834.   pred_logits sample: [-5.638851  -2.199562  -5.5557685 -5.906612  -6.0124063]
     
835.   target_tensor stats: total_pixels=196608, positive_pixels=63809, negative_pixels=132799, positive_ratio=0.324549
     
836.   Total grad_norm: 5.905108e+00, avg: 6.710350e-02
     
837.   Zero gradients: 16/88 (18.2%)
     
838.   Loss: 0.152975
     
839. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
840. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
841. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
842. onTrainOneIter: total_pixels=196608, positive_pixels=58898, negative_pixels=137710, positive_ratio=0.299571
     
843. === TRAIN FUNCTION CALLED ===
     
844. input_tensor id: 2127117534560, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
845. target_tensor id: 2127117541184, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
846. input_tensor min/max: 0.000000/1.000000
     
847. target_tensor min/max: 0.000000/1.000000
     
848. input_tensor data sample: [1. 1. 1. 1. 1.]
     
849. target_tensor data sample: [0. 0. 0. 0. 0.]
     
850.   pred_logits min/max: -14.539489/9.890100
     
851.   pred_logits sample: [-6.6866727 -1.7504314 -6.607692  -6.8713923 -6.3957305]
     
852.   target_tensor stats: total_pixels=196608, positive_pixels=58898, negative_pixels=137710, positive_ratio=0.299571
     
853.   Total grad_norm: 9.108993e+00, avg: 1.035113e-01
     
854.   Zero gradients: 16/88 (18.2%)
     
855.   Loss: 0.167791
     
856. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.005656/1.000000
     
857. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
858. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
859. onTrainOneIter: total_pixels=196608, positive_pixels=59339, negative_pixels=137269, positive_ratio=0.301814
     
860. === TRAIN FUNCTION CALLED ===
     
861. input_tensor id: 2127117541184, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
862. target_tensor id: 2127117540320, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
863. input_tensor min/max: 0.005656/1.000000
     
864. target_tensor min/max: 0.000000/1.000000
     
865. input_tensor data sample: [0.9796097  0.9801199  0.97778463 0.972542   0.9675691 ]
     
866. target_tensor data sample: [0. 0. 0. 0. 0.]
     
867.   pred_logits min/max: -18.201971/9.204210
     
868.   pred_logits sample: [-5.128061  -1.8479347 -6.7860436 -7.2832127 -7.4964623]
     
869.   target_tensor stats: total_pixels=196608, positive_pixels=59339, negative_pixels=137269, positive_ratio=0.301814
     
870.   Total grad_norm: 6.003107e+00, avg: 6.821712e-02
     
871.   Zero gradients: 16/88 (18.2%)
     
872.   Loss: 0.156269
     
873. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
874. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
875. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
876. onTrainOneIter: total_pixels=196608, positive_pixels=63789, negative_pixels=132819, positive_ratio=0.324448
     
877. === TRAIN FUNCTION CALLED ===
     
878. input_tensor id: 2127117541184, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
879. target_tensor id: 2127117537008, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
880. input_tensor min/max: 0.000000/1.000000
     
881. target_tensor min/max: 0.000000/1.000000
     
882. input_tensor data sample: [0.62337327 0.62494206 0.62651056 0.6280789  0.62964696]
     
883. target_tensor data sample: [0. 0. 0. 0. 0.]
     
884.   pred_logits min/max: -17.127758/10.331607
     
885.   pred_logits sample: [-3.3882873 -2.0255425 -3.4521842 -3.2518628 -4.02957  ]
     
886.   target_tensor stats: total_pixels=196608, positive_pixels=63789, negative_pixels=132819, positive_ratio=0.324448
     
887.   Total grad_norm: 1.447163e+01, avg: 1.644503e-01
     
888.   Zero gradients: 15/88 (17.0%)
     
889.   Loss: 0.302376
     
890. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000198/1.000000
     
891. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
892. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
893. onTrainOneIter: total_pixels=196608, positive_pixels=63291, negative_pixels=133317, positive_ratio=0.321915
     
894. === TRAIN FUNCTION CALLED ===
     
895. input_tensor id: 2127117537008, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
896. target_tensor id: 2127117540752, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
897. input_tensor min/max: 0.000198/1.000000
     
898. target_tensor min/max: 0.000000/1.000000
     
899. input_tensor data sample: [1. 1. 1. 1. 1.]
     
900. target_tensor data sample: [0. 0. 0. 0. 0.]
     
901.   pred_logits min/max: -13.293324/9.662045
     
902.   pred_logits sample: [-6.2140203 -1.8821324 -5.8295836 -5.2483406 -6.1711583]
     
903.   target_tensor stats: total_pixels=196608, positive_pixels=63291, negative_pixels=133317, positive_ratio=0.321915
     
904.   Total grad_norm: 6.417795e+00, avg: 7.292949e-02
     
905.   Zero gradients: 16/88 (18.2%)
     
906.   Loss: 0.174915
     
907. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
908. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
909. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
910. onTrainOneIter: total_pixels=196608, positive_pixels=62321, negative_pixels=134287, positive_ratio=0.316981
     
911. === TRAIN FUNCTION CALLED ===
     
912. input_tensor id: 2127117537008, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
913. target_tensor id: 2127117542336, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
914. input_tensor min/max: 0.000000/1.000000
     
915. target_tensor min/max: 0.000000/1.000000
     
916. input_tensor data sample: [0.6966744  0.6974533  0.69881487 0.6997276  0.7007255 ]
     
917. target_tensor data sample: [0. 0. 0. 0. 0.]
     
918.   pred_logits min/max: -17.328581/8.967299
     
919.   pred_logits sample: [-3.3396568 -2.373651  -2.51014   -3.644245  -4.3425913]
     
920.   target_tensor stats: total_pixels=196608, positive_pixels=62321, negative_pixels=134287, positive_ratio=0.316981
     
921.   Total grad_norm: 5.134469e+00, avg: 5.834624e-02
     
922.   Zero gradients: 17/88 (19.3%)
     
923.   Loss: 0.141017
     
924. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
925. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
926. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
927. onTrainOneIter: total_pixels=196608, positive_pixels=62264, negative_pixels=134344, positive_ratio=0.316691
     
928. === TRAIN FUNCTION CALLED ===
     
929. input_tensor id: 2127117542336, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
930. target_tensor id: 2127117539168, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
931. input_tensor min/max: 0.000000/1.000000
     
932. target_tensor min/max: 0.000000/1.000000
     
933. input_tensor data sample: [0.14117648 0.14117648 0.14117648 0.14117648 0.14117648]
     
934. target_tensor data sample: [0. 0. 0. 0. 0.]
     
935.   pred_logits min/max: -19.326881/8.288991
     
936.   pred_logits sample: [-2.2640986 -2.780824  -2.2204611 -2.32738   -2.5331128]
     
937.   target_tensor stats: total_pixels=196608, positive_pixels=62264, negative_pixels=134344, positive_ratio=0.316691
     
938.   Total grad_norm: 6.556057e+00, avg: 7.450065e-02
     
939.   Zero gradients: 16/88 (18.2%)
     
940.   Loss: 0.158988
     
941. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
942. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
943. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
944. onTrainOneIter: total_pixels=196608, positive_pixels=63798, negative_pixels=132810, positive_ratio=0.324493
     
945. === TRAIN FUNCTION CALLED ===
     
946. input_tensor id: 2127117542336, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
947. target_tensor id: 2127117534416, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
948. input_tensor min/max: 0.000000/1.000000
     
949. target_tensor min/max: 0.000000/1.000000
     
950. input_tensor data sample: [0.3963111  0.39642578 0.3965332  0.39688694 0.39699322]
     
951. target_tensor data sample: [0. 0. 0. 0. 0.]
     
952.   pred_logits min/max: -17.139322/9.320560
     
953.   pred_logits sample: [-2.8763387 -2.7789052 -2.4400127 -2.121132  -2.5112505]
     
954.   target_tensor stats: total_pixels=196608, positive_pixels=63798, negative_pixels=132810, positive_ratio=0.324493
     
955.   Total grad_norm: 6.363823e+00, avg: 7.231617e-02
     
956.   Zero gradients: 17/88 (19.3%)
     
957.   Loss: 0.142143
     
958. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
959. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
960. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
961. onTrainOneIter: total_pixels=196608, positive_pixels=62744, negative_pixels=133864, positive_ratio=0.319132
     
962. === TRAIN FUNCTION CALLED ===
     
963. input_tensor id: 2127117542336, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
964. target_tensor id: 2127117536720, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
965. input_tensor min/max: 0.000000/1.000000
     
966. target_tensor min/max: 0.000000/1.000000
     
967. input_tensor data sample: [0.23519292 0.23541868 0.23526305 0.2343759  0.23503873]
     
968. target_tensor data sample: [0. 0. 0. 0. 0.]
     
969.   pred_logits min/max: -20.914375/9.933438
     
970.   pred_logits sample: [-2.569855  -2.9163275 -2.526288  -3.1561742 -3.4568253]
     
971.   target_tensor stats: total_pixels=196608, positive_pixels=62744, negative_pixels=133864, positive_ratio=0.319132
     
972.   Total grad_norm: 8.607569e+00, avg: 9.781328e-02
     
973.   Zero gradients: 16/88 (18.2%)
     
974.   Loss: 0.164255
     
975. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
976. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
977. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
978. onTrainOneIter: total_pixels=196608, positive_pixels=56452, negative_pixels=140156, positive_ratio=0.287130
     
979. === TRAIN FUNCTION CALLED ===
     
980. input_tensor id: 2127117542336, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
981. target_tensor id: 2127117538016, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
982. input_tensor min/max: 0.000000/1.000000
     
983. target_tensor min/max: 0.000000/1.000000
     
984. input_tensor data sample: [0.29676616 0.29803535 0.29982874 0.3017616  0.30329573]
     
985. target_tensor data sample: [0. 0. 0. 0. 0.]
     
986.   pred_logits min/max: -17.376493/9.682109
     
987.   pred_logits sample: [-3.00241   -1.7597618 -1.970201  -3.122077  -4.0058064]
     
988.   target_tensor stats: total_pixels=196608, positive_pixels=56452, negative_pixels=140156, positive_ratio=0.287130
     
989.   Total grad_norm: 6.598444e+00, avg: 7.498232e-02
     
990.   Zero gradients: 17/88 (19.3%)
     
991.   Loss: 0.152219
     
992. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.006333/0.979446
     
993. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
     
994. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
     
995. onTrainOneIter: total_pixels=196608, positive_pixels=63946, negative_pixels=132662, positive_ratio=0.325246
     
996. === TRAIN FUNCTION CALLED ===
     
997. input_tensor id: 2127117542336, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
     
998. target_tensor id: 2127117536144, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
     
999. input_tensor min/max: 0.006333/0.979446
     
1000. target_tensor min/max: 0.000000/1.000000
      
1001. input_tensor data sample: [0.36447188 0.35725468 0.34968525 0.34103572 0.3392299 ]
      
1002. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1003.   pred_logits min/max: -18.276306/10.350447
      
1004.   pred_logits sample: [-3.110876  -3.76414   -2.9791148 -3.91517   -4.0277023]
      
1005.   target_tensor stats: total_pixels=196608, positive_pixels=63946, negative_pixels=132662, positive_ratio=0.325246
      
1006.   Total grad_norm: 8.193271e+00, avg: 9.310535e-02
      
1007.   Zero gradients: 17/88 (19.3%)
      
1008.   Loss: 0.187463
      
1009. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1010. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1011. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1012. onTrainOneIter: total_pixels=196608, positive_pixels=63232, negative_pixels=133376, positive_ratio=0.321615
      
1013. === TRAIN FUNCTION CALLED ===
      
1014. input_tensor id: 2127117542336, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1015. target_tensor id: 2127117533408, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1016. input_tensor min/max: 0.000000/1.000000
      
1017. target_tensor min/max: 0.000000/1.000000
      
1018. input_tensor data sample: [0.29525822 0.2924292  0.2893951  0.28596255 0.28239384]
      
1019. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1020.   pred_logits min/max: -17.121223/9.996688
      
1021.   pred_logits sample: [-2.8377306 -2.0156505 -2.0348973 -2.8039515 -3.2783327]
      
1022.   target_tensor stats: total_pixels=196608, positive_pixels=63232, negative_pixels=133376, positive_ratio=0.321615
      
1023.   Total grad_norm: 1.346201e+01, avg: 1.529773e-01
      
1024.   Zero gradients: 16/88 (18.2%)
      
1025.   Loss: 0.234369
      
1026. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1027. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1028. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1029. onTrainOneIter: total_pixels=196608, positive_pixels=59097, negative_pixels=137511, positive_ratio=0.300583
      
1030. === TRAIN FUNCTION CALLED ===
      
1031. input_tensor id: 2127117533408, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1032. target_tensor id: 2127117540032, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1033. input_tensor min/max: 0.000000/1.000000
      
1034. target_tensor min/max: 0.000000/1.000000
      
1035. input_tensor data sample: [0.9394344  0.93835163 0.9425199  0.9438328  0.9514084 ]
      
1036. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1037.   pred_logits min/max: -14.167475/10.221355
      
1038.   pred_logits sample: [-4.0194135 -2.2372093 -5.091132  -5.08241   -5.291009 ]
      
1039.   target_tensor stats: total_pixels=196608, positive_pixels=59097, negative_pixels=137511, positive_ratio=0.300583
      
1040.   Total grad_norm: 1.541523e+01, avg: 1.751731e-01
      
1041.   Zero gradients: 15/88 (17.0%)
      
1042.   Loss: 0.262680
      
1043. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1044. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1045. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1046. onTrainOneIter: total_pixels=196608, positive_pixels=60720, negative_pixels=135888, positive_ratio=0.308838
      
1047. === TRAIN FUNCTION CALLED ===
      
1048. input_tensor id: 2127117533408, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1049. target_tensor id: 2127117535856, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1050. input_tensor min/max: 0.000000/1.000000
      
1051. target_tensor min/max: 0.000000/1.000000
      
1052. input_tensor data sample: [0.43924886 0.43925828 0.4392669  0.43927485 0.43928218]
      
1053. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1054.   pred_logits min/max: -19.034407/11.550397
      
1055.   pred_logits sample: [-3.435771  -2.3059251 -2.2554433 -3.8883805 -4.0604706]
      
1056.   target_tensor stats: total_pixels=196608, positive_pixels=60720, negative_pixels=135888, positive_ratio=0.308838
      
1057.   Total grad_norm: 9.875311e+00, avg: 1.122194e-01
      
1058.   Zero gradients: 16/88 (18.2%)
      
1059.   Loss: 0.209333
      
1060. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1061. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1062. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1063. onTrainOneIter: total_pixels=196608, positive_pixels=59897, negative_pixels=136711, positive_ratio=0.304652
      
1064. === TRAIN FUNCTION CALLED ===
      
1065. input_tensor id: 2127117535856, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1066. target_tensor id: 2127117538160, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1067. input_tensor min/max: 0.000000/1.000000
      
1068. target_tensor min/max: 0.000000/1.000000
      
1069. input_tensor data sample: [0.07971865 0.07947554 0.08606026 0.08255988 0.07539687]
      
1070. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1071.   pred_logits min/max: -16.871040/11.374657
      
1072.   pred_logits sample: [-2.901302  -2.5952477 -3.2703307 -3.2710183 -3.9213095]
      
1073.   target_tensor stats: total_pixels=196608, positive_pixels=59897, negative_pixels=136711, positive_ratio=0.304652
      
1074.   Total grad_norm: 9.135140e+00, avg: 1.038084e-01
      
1075.   Zero gradients: 16/88 (18.2%)
      
1076.   Loss: 0.205695
      
1077. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1078. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1079. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1080. onTrainOneIter: total_pixels=196608, positive_pixels=63878, negative_pixels=132730, positive_ratio=0.324900
      
1081. === TRAIN FUNCTION CALLED ===
      
1082. input_tensor id: 2127117535856, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1083. target_tensor id: 2127117541616, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1084. input_tensor min/max: 0.000000/1.000000
      
1085. target_tensor min/max: 0.000000/1.000000
      
1086. input_tensor data sample: [0.48235947 0.48235294 0.48235294 0.48235294 0.48235294]
      
1087. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1088.   pred_logits min/max: -18.115614/11.373105
      
1089.   pred_logits sample: [-4.37724   -3.8814015 -5.0797167 -3.786492  -4.254979 ]
      
1090.   target_tensor stats: total_pixels=196608, positive_pixels=63878, negative_pixels=132730, positive_ratio=0.324900
      
1091.   Total grad_norm: 6.408868e+00, avg: 7.282804e-02
      
1092.   Zero gradients: 16/88 (18.2%)
      
1093.   Loss: 0.154247
      
1094. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/0.982491
      
1095. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1096. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1097. onTrainOneIter: total_pixels=196608, positive_pixels=63475, negative_pixels=133133, positive_ratio=0.322851
      
1098. === TRAIN FUNCTION CALLED ===
      
1099. input_tensor id: 2127117541616, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1100. target_tensor id: 2127117540320, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1101. input_tensor min/max: 0.000000/0.982491
      
1102. target_tensor min/max: 0.000000/1.000000
      
1103. input_tensor data sample: [0.4153527  0.41499883 0.41089106 0.40683705 0.3995743 ]
      
1104. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1105.   pred_logits min/max: -18.218029/10.478106
      
1106.   pred_logits sample: [-4.087691  -2.7718482 -3.3362787 -3.961798  -4.8355126]
      
1107.   target_tensor stats: total_pixels=196608, positive_pixels=63475, negative_pixels=133133, positive_ratio=0.322851
      
1108.   Total grad_norm: 4.474280e+00, avg: 5.084409e-02
      
1109.   Zero gradients: 16/88 (18.2%)
      
1110.   Loss: 0.113150
      
1111. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1112. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1113. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1114. onTrainOneIter: total_pixels=196608, positive_pixels=61833, negative_pixels=134775, positive_ratio=0.314499
      
1115. === TRAIN FUNCTION CALLED ===
      
1116. input_tensor id: 2127117541616, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1117. target_tensor id: 2127117535712, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1118. input_tensor min/max: 0.000000/1.000000
      
1119. target_tensor min/max: 0.000000/1.000000
      
1120. input_tensor data sample: [0.78351593 0.7835667  0.7836641  0.7838259  0.78577363]
      
1121. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1122.   pred_logits min/max: -15.482409/10.918242
      
1123.   pred_logits sample: [-4.941795  -0.6911363 -3.8678432 -4.8598986 -4.3533115]
      
1124.   target_tensor stats: total_pixels=196608, positive_pixels=61833, negative_pixels=134775, positive_ratio=0.314499
      
1125.   Total grad_norm: 1.374920e+01, avg: 1.562409e-01
      
1126.   Zero gradients: 15/88 (17.0%)
      
1127.   Loss: 0.281596
      
1128. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1129. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1130. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1131. onTrainOneIter: total_pixels=196608, positive_pixels=59375, negative_pixels=137233, positive_ratio=0.301997
      
1132. === TRAIN FUNCTION CALLED ===
      
1133. input_tensor id: 2127117541616, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1134. target_tensor id: 2127117540752, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1135. input_tensor min/max: 0.000000/1.000000
      
1136. target_tensor min/max: 0.000000/1.000000
      
1137. input_tensor data sample: [0.5256015  0.52060986 0.5186995  0.52241004 0.52920365]
      
1138. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1139.   pred_logits min/max: -22.547131/11.043216
      
1140.   pred_logits sample: [-4.611463  -1.901287  -3.9827065 -4.63077   -3.9644232]
      
1141.   target_tensor stats: total_pixels=196608, positive_pixels=59375, negative_pixels=137233, positive_ratio=0.301997
      
1142.   Total grad_norm: 1.129185e+01, avg: 1.283165e-01
      
1143.   Zero gradients: 15/88 (17.0%)
      
1144.   Loss: 0.258179
      
1145. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1146. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1147. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1148. onTrainOneIter: total_pixels=196608, positive_pixels=63426, negative_pixels=133182, positive_ratio=0.322601
      
1149. === TRAIN FUNCTION CALLED ===
      
1150. input_tensor id: 2127117541616, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1151. target_tensor id: 2127117540176, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1152. input_tensor min/max: 0.000000/1.000000
      
1153. target_tensor min/max: 0.000000/1.000000
      
1154. input_tensor data sample: [0.35548508 0.3568854  0.35450715 0.3524825  0.35229024]
      
1155. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1156.   pred_logits min/max: -18.693195/11.868806
      
1157.   pred_logits sample: [-4.249918  -2.3057885 -1.9677954 -4.7093353 -4.9494405]
      
1158.   target_tensor stats: total_pixels=196608, positive_pixels=63426, negative_pixels=133182, positive_ratio=0.322601
      
1159.   Total grad_norm: 5.823107e+00, avg: 6.617167e-02
      
1160.   Zero gradients: 17/88 (19.3%)
      
1161.   Loss: 0.141586
      
1162. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1163. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1164. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1165. onTrainOneIter: total_pixels=196608, positive_pixels=58416, negative_pixels=138192, positive_ratio=0.297119
      
1166. === TRAIN FUNCTION CALLED ===
      
1167. input_tensor id: 2127117541616, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1168. target_tensor id: 2127117539168, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1169. input_tensor min/max: 0.000000/1.000000
      
1170. target_tensor min/max: 0.000000/1.000000
      
1171. input_tensor data sample: [0.43878824 0.43494838 0.4271439  0.42232224 0.4230855 ]
      
1172. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1173.   pred_logits min/max: -27.498243/10.141516
      
1174.   pred_logits sample: [-3.1762288 -2.664098  -3.2043517 -3.190942  -3.2147143]
      
1175.   target_tensor stats: total_pixels=196608, positive_pixels=58416, negative_pixels=138192, positive_ratio=0.297119
      
1176.   Total grad_norm: 6.051132e+00, avg: 6.876286e-02
      
1177.   Zero gradients: 17/88 (19.3%)
      
1178.   Loss: 0.130991
      
1179. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1180. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1181. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1182. onTrainOneIter: total_pixels=196608, positive_pixels=61145, negative_pixels=135463, positive_ratio=0.311000
      
1183. === TRAIN FUNCTION CALLED ===
      
1184. input_tensor id: 2127117541616, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1185. target_tensor id: 2127117534416, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1186. input_tensor min/max: 0.000000/1.000000
      
1187. target_tensor min/max: 0.000000/1.000000
      
1188. input_tensor data sample: [0.3110924  0.30823323 0.30512977 0.30235988 0.3002512 ]
      
1189. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1190.   pred_logits min/max: -22.577290/11.847639
      
1191.   pred_logits sample: [-2.85167   -3.0088475 -1.9843479 -2.9361854 -3.4338117]
      
1192.   target_tensor stats: total_pixels=196608, positive_pixels=61145, negative_pixels=135463, positive_ratio=0.311000
      
1193.   Total grad_norm: 4.517135e+00, avg: 5.133108e-02
      
1194.   Zero gradients: 17/88 (19.3%)
      
1195.   Loss: 0.116914
      
1196. onTrainOneIter: image_tensor shape: torch.Size([3, 3, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1197. onTrainOneIter: mask_tensor shape: torch.Size([3, 1, 256, 256]), device: cuda:0, min/max: 0.000000/1.000000
      
1198. onTrainOneIter: mask_tensor sample: [0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
      
1199. onTrainOneIter: total_pixels=196608, positive_pixels=69023, negative_pixels=127585, positive_ratio=0.351069
      
1200. === TRAIN FUNCTION CALLED ===
      
1201. input_tensor id: 2127117534416, shape: torch.Size([3, 3, 256, 256]), dtype: torch.float32, device: cuda:0
      
1202. target_tensor id: 2127117536720, shape: torch.Size([3, 1, 256, 256]), dtype: torch.float32, device: cuda:0
      
1203. input_tensor min/max: 0.000000/1.000000
      
1204. target_tensor min/max: 0.000000/1.000000
      
1205. input_tensor data sample: [0.87912726 0.87912726 0.87912726 0.87912726 0.87912726]
      
1206. target_tensor data sample: [0. 0. 0. 0. 0.]
      
1207.   pred_logits min/max: -15.272973/9.378516
      
1208.   pred_logits sample: [-4.9603224 -1.193005  -5.6421976 -6.114594  -6.0347657]
      
1209.   target_tensor stats: total_pixels=196608, positive_pixels=69023, negative_pixels=127585, positive_ratio=0.351069
      
1210.   Total grad_norm: 4.873687e+00, avg: 5.538280e-02
      

复制代码

gaoyifei

day270010678 发表于 2026-1-27 16:43
别提了，昨天晚上心血来潮，重构了他整体框架， 今天调试，  调试了半天还是有错误，都懒得弄了。
...

那个包哥不是重构了一个框架吗？提升了遮罩融合度什么的

丸丸

加油楼主！

狗蛋

gaoyifei 发表于 2026-1-27 23:33
那个包哥不是重构了一个框架吗？提升了遮罩融合度什么的

那是吹逼的你也信。。。营销懂吧

gaoyifei

狗蛋 发表于 2026-1-28 09:21
那是吹逼的你也信。。。营销懂吧

差点就心动买了...

unclejay1197

狗蛋 发表于 2026-1-28 09:21
那是吹逼的你也信。。。营销懂吧

哪个包哥 发来看看。。

main9dog

我覺得最需要的是手動上遮照這塊，全世界還沒有高手研究出來怎麼寫嗎？

每次搞這個搞超久....

就是 Add 萬用遮照，常常還有有一些無法遮到，必須人工 Xseg Editor 去手動畫幾張（超花時間.....）

我有想法可是我不知道程式怎麼寫

我的想法就是，竟然 Add 萬用遮照能遮出範圍（我猜範圍座標應該寫在每張照片裡面）是不是能讀取這些座標，然後有一隻程式沿著這些座標走一圈，每10公分標記一個手動畫的點，這樣能快速將指定的照片，上人工遮照，基本上可能微調拉一下就好了，有想法不知道怎麼寫