|
星级打分
平均分:NAN 参与人数:0 我的评分:未评
Running trainer.
Choose one of saved models, or enter a name to create a new model.
[r] : rename
[d] : delete
[0] : new - latest
:
0
Loading new_SAEHD model...
Choose one or several GPU idxs (separated by comma).
[CPU] : CPU
[0] : NVIDIA GeForce GTX 770
[0] Which GPU indexes to choose? :
0
Initializing models: 100%|#######################| 5/5 [00:02<00:00, 2.41it/s]
Loading samples: 100%|####################| 5788/5788 [00:32<00:00, 176.42it/s]
Loading samples: 100%|######################| 300/300 [00:02<00:00, 127.44it/s]
================== Model Summary ==================
== ==
== Model name: new_SAEHD ==
== ==
== Current iteration: 44073 ==
== ==
==---------------- Model Options ----------------==
== ==
== resolution: 128 ==
== face_type: f ==
== models_opt_on_gpu: True ==
== archi: df ==
== ae_dims: 256 ==
== e_dims: 64 ==
== d_dims: 64 ==
== d_mask_dims: 22 ==
== masked_training: True ==
== eyes_prio: False ==
== uniform_yaw: False ==
== lr_dropout: n ==
== random_warp: True ==
== gan_power: 0.0 ==
== true_face_power: 0.0 ==
== face_style_power: 0.0 ==
== bg_style_power: 0.0 ==
== ct_mode: none ==
== clipgrad: False ==
== pretrain: False ==
== autobackup_hour: 0 ==
== write_preview_history: False ==
== target_iter: 0 ==
== random_flip: True ==
== batch_size: 4 ==
== ==
==----------------- Running On ------------------==
== ==
== Device index: 0 ==
== Name: NVIDIA GeForce GTX 770 ==
== VRAM: 4.00GB ==
== ==
===================================================
Starting. Press "Enter" to stop training and save model.
[15:32:25][#044484][1172ms][0.5405][0.6002]
2022-07-22 15:47:43.122029: E tensorflow/stream_executor/cuda/cuda_event.cc:48]
Error polling for event status: failed to query event: CUDA_ERROR_LAUNCH_TIMEOUT
: the launch timed out and was terminated
2022-07-22 15:47:43.122029: E tensorflow/stream_executor/cuda/cuda_driver.cc:101
1] could not synchronize on CUDA context: CUDA_ERROR_LAUNCH_TIMEOUT: the launch
timed out and was terminated ::
2022-07-22 15:47:43.705062: F tensorflow/core/common_runtime/gpu/gpu_event_mgr.c
c:274] Unexpected Event status: 1
请按任意键继续. . .
刚开始可以训练一定时间以后就闪退然后就这样了
|
|