新人求助训练SAEHD一直MemoryError【已解决】

l14171216 · 发表于 2022-2-7 21:16:16

星级打分

1
2
3
4
5

平均分:NAN 参与人数:0 我的评分:未评

本帖最后由 l14171216 于 2022-2-8 09:22 编辑

已解决，感谢各位大佬指点，把虚拟内存设置成100G放了ssd上之后正常可用了，感谢感谢
------------------------------------------------------------------------------------------------------------------------------------------------------
内存64G，RTX3080Ti的显卡，虚拟内存设置成最小30G最大50G，参数调成最低了还是报错，求大佬指点

首次运行模型.

选择一个或多个 GPU 编号（以逗号分隔）.

[CPU] : CPU
  [0] : NVIDIA GeForce RTX 3080 Ti

[0] 选择哪些 GPU 索引? :
0

[0] 几个小时备份一次 ( 0..24  按英文?获取帮助信息 ) :
0
[n] 保存预览历史记录 ( y/n  按英文?获取帮助信息 ) :
n
[100000] 目标迭代次数 :
100000
[n] 随机翻转SRC人脸 ( y/n  按英文?获取帮助信息 ) :
n
[y] 随机翻转DST人脸 ( y/n  按英文?获取帮助信息 ) :
y
[4] 批量大小 (  按英文?获取帮助信息 ) :
4
[64] 分辨率 resolution ( 64-640  按英文?获取帮助信息 ) :
64
[wf] 脸类型 face_type ( h/mf/f/wf/head  按英文?获取帮助信息 ) :
wf
[liae-ud] AE架构 AE architecture (  按英文?获取帮助信息 ) :
liae-ud
[32] 自动编码器大小 AutoEncoder dimensions ( 32-1024  按英文?获取帮助信息 ) :
32
[16] 编码器大小 Encoder dimensions ( 16-256  按英文?获取帮助信息 ) :
16
[16] 解码器大小 Decoder dimensions ( 16-256  按英文?获取帮助信息 ) :
16
[16] 遮罩解码器大小 Decoder mask dimensions ( 16-256  按英文?获取帮助信息 ) :
16
[y] 训练遮罩内的人脸 Masked training ( y/n  按英文?获取帮助信息 ) :
y
[n] 眼睛和嘴巴优先 Eyes and mouth priority ( y/n  按英文?获取帮助信息 ) :
n
[n] 侧脸优化 Uniform yaw distribution of samples ( y/n  按英文?获取帮助信息 ) :
n
[n] 模糊遮罩边缘 Blur out mask ( y/n  按英文?获取帮助信息 ) :
n
[y] 将模型和优化器放在GPU上运行 Place models and optimizer on GPU ( y/n  按英文?获取帮助信息 ) :
y
[y] 使用AdaBelief优化器? Use AdaBelief optimizer? ( y/n  按英文?获取帮助信息 ) :
y
[n] 使用学习率下降 Use learning rate dropout ( n/y/cpu  按英文?获取帮助信息 ) :
n
[y] 随机扭曲 Enable random warp of samples ( y/n  按英文?获取帮助信息 ) :
y
[0.0] 随机色调/饱和度/光强度 ( 0.0 .. 0.3  按英文?获取帮助信息 ) :
0.0
[0.0] GAN强度 GAN power ( 0.0 .. 5.0  按英文?获取帮助信息 ) :
0.0
[0.0] 人脸风格强度 Face style power ( 0.0..100.0  按英文?获取帮助信息 ) :
0.0
[0.0] 背景风格强度 Background style power ( 0.0..100.0  按英文?获取帮助信息 ) :
0.0
[none] 颜色转换模式 Color transfer for src faceset ( none/rct/lct/mkl/idt/sot  按英文?获取帮助信息 ) :
none
[n] 启用梯度剪裁 Enable gradient clipping ( y/n  按英文?获取帮助信息 ) :
n
[y] 启用预训练 ( y/n  按英文?获取帮助信息 ) : n
Initializing models: 100%|###############################################################| 5/5 [00:00<00:00, 22.64it/s]
加载样本中...: 100%|################################################################| 818/818 [00:01<00:00, 477.13it/s]
加载样本中...: 100%|##############################################################| 1619/1619 [00:03<00:00, 498.30it/s]
Process Process-35:
Traceback (most recent call last):
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\DeepFaceLab\samplelib\SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\DeepFaceLab\samplelib\SampleProcessor.py", line 145, in process
img = get_eyes_mouth_mask()*mask
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\DeepFaceLab\samplelib\SampleProcessor.py", line 80, in get_eyes_mouth_mask
return np.clip(mask, 0, 1)
  File "<__array_function__ internals>", line 6, in clip
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\python-3.6.8\lib\site-packages\numpy\core\fromnumeric.py", line 2097, in clip
return _wrapfunc(a, 'clip', a_min, a_max, out=out, **kwargs)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\python-3.6.8\lib\site-packages\numpy\core\fromnumeric.py", line 58, in _wrapfunc
return bound(*args, **kwds)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\python-3.6.8\lib\site-packages\numpy\core\_methods.py", line 141, in _clip
um.clip, a, min, max, out=out, casting=casting, **kwargs)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\python-3.6.8\lib\site-packages\numpy\core\_methods.py", line 94, in _clip_dep_invoke_with_casting
return ufunc(*args, out=out, **kwargs)
MemoryError: Unable to allocate 1.00 MiB for an array with shape (512, 512, 1) and data type float32

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "multiprocessing\process.py", line 258, in _bootstrap
  File "multiprocessing\process.py", line 93, in run
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\DeepFaceLab\core\joblib\SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\DeepFaceLab\samplelib\SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("示例 %s 中出现异常。错误：%s" % (sample.filename, traceback.format_exc() ) )
Exception: 示例 D:\tool\DFL_maozhihanhua_RTX3000\workspace\data_dst\aligned\00405_0.jpg 中出现异常。错误：Traceback (most recent call last):
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\DeepFaceLab\samplelib\SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\DeepFaceLab\samplelib\SampleProcessor.py", line 145, in process
img = get_eyes_mouth_mask()*mask
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\DeepFaceLab\samplelib\SampleProcessor.py", line 80, in get_eyes_mouth_mask
return np.clip(mask, 0, 1)
  File "<__array_function__ internals>", line 6, in clip
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\python-3.6.8\lib\site-packages\numpy\core\fromnumeric.py", line 2097, in clip
return _wrapfunc(a, 'clip', a_min, a_max, out=out, **kwargs)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\python-3.6.8\lib\site-packages\numpy\core\fromnumeric.py", line 58, in _wrapfunc
return bound(*args, **kwds)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\python-3.6.8\lib\site-packages\numpy\core\_methods.py", line 141, in _clip
um.clip, a, min, max, out=out, casting=casting, **kwargs)
  File "D:\tool\DFL_maozhihanhua_RTX3000\_internal\python-3.6.8\lib\site-packages\numpy\core\_methods.py", line 94, in _clip_dep_invoke_with_casting
return ufunc(*args, out=out, **kwargs)
MemoryError: Unable to allocate 1.00 MiB for an array with shape (512, 512, 1) and data type float32