200万（BS19）448预训练木兰丹

天光脑震荡

星级打分

• 1
• 2
• 3
• 4
• 5

平均分:5  参与人数:1  我的评分:未评

上次是用一张2080ti22GB来练，只能开BS11，练到100万迭代，这次用两张2080ti22GB，BS开到19，练了差不多两个月,在今天早上，200万迭代达成，又可以拿出来献丑了！鉴于上次免费分享只收到区区一百几十灵石，造成无法买到最新版镇坛之宝的尴尬局面，这次就不免费了……
本次100万练到200万，吸取了两位坛友（jinyuan37和丸丸）的宝贵意见，用了ice1.818软件开扭曲加上遮罩练了80万左右，开嘴眼又练了20万左右，出来的效果如何连本人也不清楚，最近比较忙，等有时间自己再测试一下。
本丹建议在显存20GB以上显卡上使用，16GB估计也行。




模型摘要                 模型选项 | 超参数      权重约束 | 优化器    素材增强 | 模型增强
-----------------------  ---------------------  -------------------  ---------------------
模型名称:阁下有病_MULAN  模型类型:MULAN         启用注意力           随机翻转src
当前迭代:2001839         训练算法:ICE-V71       - eye+mouth          随机翻转dst
模型格式:me-model        架构选项:df-dpu        启用遮罩约束         ────────────────
创建时间:2024-02-20      人脸类型:wf            遮罩边沿平滑
训练时长:2274.14h        分辨率:448             ────────────────
模型版本:23.17           AE编码器长度:448       启用 RO-1 优化器
模型作者:阁下有病        编码器维度:96          启用 AdamW 优化器
                         解码器维度:96          优化器放置于GPU
训练状态:预训练          掩码层维度:22          启用梯度剪裁
                         ─────────────────────
                         批量大小BS:19
                         学习率 lr = 5e-05


自动备份:5
0-NVIDIA GeForce RTX 2080 Ti (19.67gb/19.67gb)  1-NVIDIA GeForce RTX 2080 Ti (19.67gb/19.67gb)




[12:23:52][i:01856715][3622ms]-[0.43183][0.43168] [save]
[13:21:47][i:01857468][4213ms]-[0.43171][0.43094] [save]
[14:21:50][i:01858248][3761ms]-[0.43002][0.43001] [save]
[15:21:50][i:01859028][3844ms]-[0.43041][0.42986] [save]
[16:21:57][i:01859808][3864ms]-[0.42955][0.43140] [save]
[17:21:47][i:01860583][3830ms]-[0.42988][0.42797] [save]
[18:21:49][i:01861358][3952ms]-[0.43118][0.42802] [save]
[19:21:50][i:01862130][3944ms]-[0.42850][0.42740] [save]
[20:21:48][i:01862899][4074ms]-[0.42659][0.42884] [save]
[21:21:55][i:01863671][4012ms]-[0.43069][0.42802] [save]
[22:21:50][i:01864444][3846ms]-[0.42556][0.42654] [save]
[23:21:47][i:01865215][3960ms]-[0.42818][0.42826] [save]
[00:21:48][i:01865985][3916ms]-[0.42744][0.42501] [save]
[01:21:48][i:01866757][3924ms]-[0.42607][0.42591] [save]
[02:21:53][i:01867532][3750ms]-[0.42744][0.42628] [save]
[03:21:49][i:01868304][3970ms]-[0.42630][0.42555] [save]
[04:21:48][i:01869076][4208ms]-[0.42816][0.42725] [save]
[05:21:48][i:01869847][4015ms]-[0.42508][0.42567] [save]
[06:21:49][i:01870629][3666ms]-[0.42708][0.42538] [save]
[07:21:53][i:01871412][3820ms]-[0.42644][0.42481] [save]
[08:21:47][i:01872192][3936ms]-[0.42583][0.42437] [save]
[09:21:48][i:01872974][3852ms]-[0.42624][0.42319] [save]
[11:30:41][i:01874529][3920ms]-[0.42164][0.42403] [save]
[12:28:30][i:01875281][4134ms]-[0.42594][0.42505] [save]
[13:28:30][i:01876058][3884ms]-[0.42440][0.42414] [save]
[14:28:33][i:01876835][3985ms]-[0.42256][0.42079] [save]
[15:28:36][i:01877613][4067ms]-[0.42321][0.42516] [save]
[16:28:32][i:01878388][3880ms]-[0.42344][0.42195] [save]
[17:28:31][i:01879161][3916ms]-[0.42159][0.41953] [save]
[18:28:33][i:01879932][3969ms]-[0.42216][0.42215] [save]
[19:28:32][i:01880704][3939ms]-[0.42397][0.42346] [save]
[20:28:35][i:01881477][3852ms]-[0.42031][0.41898] [save]
[21:28:31][i:01882249][4236ms]-[0.42184][0.42007] [save]
[22:28:32][i:01883022][3877ms]-[0.42183][0.42086] [save]
[23:28:34][i:01883794][3896ms]-[0.42234][0.41982] [save]
[00:28:32][i:01884566][3904ms]-[0.42151][0.42274] [save]
[01:28:34][i:01885338][4068ms]-[0.41910][0.41926] [save]
[02:28:30][i:01886111][3986ms]-[0.42056][0.41918] [save]
[03:28:32][i:01886885][3931ms]-[0.42099][0.41963] [save]
[04:28:30][i:01887657][3988ms]-[0.42023][0.42030] [save]
[05:28:32][i:01888431][3990ms]-[0.42143][0.41753] [save]
[06:28:34][i:01889204][3797ms]-[0.41864][0.42017] [save]
[07:28:31][i:01889976][4026ms]-[0.42052][0.41790] [save]
[08:28:30][i:01890748][3786ms]-[0.42076][0.41774] [save]
[09:28:31][i:01891518][3903ms]-[0.41809][0.41996] [save]
[10:28:30][i:01892298][3814ms]-[0.41754][0.41793] [save]
[11:28:36][i:01893076][3820ms]-[0.41936][0.41703] [save]
[12:28:34][i:01893855][4026ms]-[0.41962][0.41773] [save]
[13:28:30][i:01894628][4162ms]-[0.41830][0.41620] [save]
[14:28:32][i:01895402][3864ms]-[0.41981][0.41739] [save]
[15:28:30][i:01896131][4091ms]-[0.41963][0.41733] [save]
[16:28:35][i:01896860][3985ms]-[0.41698][0.41654] [save]
[17:28:32][i:01897633][3849ms]-[0.41729][0.41556] [save]
[18:28:32][i:01898389][3808ms]-[0.41722][0.41763] [save]
[19:28:30][i:01899164][3724ms]-[0.41743][0.41653] [save]
[20:28:33][i:01899944][3922ms]-[0.41674][0.41458] [save]
[21:28:36][i:01900712][3918ms]-[0.41554][0.41541] [save]
[22:28:31][i:01901487][3964ms]-[0.41553][0.41599] [save]
[23:28:34][i:01902253][3876ms]-[0.41767][0.41608] [save]
[00:28:30][i:01903026][3813ms]-[0.41698][0.41381] [save]
[01:28:32][i:01903793][3805ms]-[0.41493][0.41387] [save]
[02:28:39][i:01904560][4005ms]-[0.41737][0.41560] [save]
[03:28:34][i:01905327][3853ms]-[0.41424][0.41527] [save]
[04:28:33][i:01906094][4019ms]-[0.41528][0.41515] [save]
[05:28:31][i:01906863][4035ms]-[0.41344][0.41116] [save]
[06:28:32][i:01907637][3760ms]-[0.41553][0.41455] [save]
[07:28:34][i:01908394][4049ms]-[0.41533][0.41592] [save]
[08:28:34][i:01909147][3852ms]-[0.41282][0.41497] [save]
[09:28:34][i:01909910][4002ms]-[0.41348][0.41351] [save]
[10:28:31][i:01910683][3998ms]-[0.41481][0.41426] [save]
[11:28:33][i:01911449][3867ms]-[0.41216][0.41193] [save]
[12:28:38][i:01912210][3836ms]-[0.41409][0.41143] [save]
[13:28:34][i:01912981][4020ms]-[0.41257][0.41258] [save]
[14:28:33][i:01913754][3808ms]-[0.41515][0.41304] [save]
[15:28:31][i:01914525][3831ms]-[0.41110][0.41397] [save]
[16:28:32][i:01915297][4105ms]-[0.41311][0.41113] [save]
[17:28:38][i:01916069][4096ms]-[0.41497][0.41077] [save]
[18:28:32][i:01916839][3974ms]-[0.41109][0.41149] [save]
[19:28:32][i:01917609][4222ms]-[0.41203][0.41132] [save]
[20:28:31][i:01918382][3945ms]-[0.41418][0.41220] [save]
[21:28:33][i:01919155][3726ms]-[0.41266][0.41166] [save]
[22:28:34][i:01919927][3954ms]-[0.41049][0.41137] [save]
[23:28:30][i:01920699][3997ms]-[0.41134][0.40892] [save]
[00:28:33][i:01921471][3956ms]-[0.41016][0.41034] [save]
[01:28:33][i:01922245][3982ms]-[0.41016][0.41109] [save]
[02:28:33][i:01923018][3770ms]-[0.41173][0.40977] [save]
[03:28:35][i:01923790][4018ms]-[0.41249][0.40912] [save]
[04:28:32][i:01924561][3917ms]-[0.40974][0.40971] [save]
[05:28:31][i:01925333][4035ms]-[0.41188][0.41012] [save]
[06:28:31][i:01926105][3923ms]-[0.41166][0.41021] [save]
[07:28:31][i:01926879][3926ms]-[0.40883][0.40774] [save]
[08:28:35][i:01927651][3873ms]-[0.40836][0.40893] [save]
[09:28:31][i:01928422][3925ms]-[0.41171][0.40898] [save]
[10:28:30][i:01929194][4022ms]-[0.40936][0.41069] [save]
[11:28:31][i:01929967][3836ms]-[0.40980][0.40926] [save]
[12:28:32][i:01930741][4003ms]-[0.41035][0.40733] [save]
[13:28:35][i:01931516][4035ms]-[0.40760][0.40764] [save]
[14:28:31][i:01932288][3871ms]-[0.40919][0.40740] [save]
[15:28:31][i:01933060][3911ms]-[0.40973][0.40804] [save]
[16:28:33][i:01933833][4023ms]-[0.40827][0.40874] [save]
[17:28:32][i:01934605][3792ms]-[0.41117][0.40735] [save]
[18:28:35][i:01935376][3981ms]-[0.40859][0.40593] [save]
[19:28:30][i:01936146][3750ms]-[0.40636][0.40843] [save]
[20:28:34][i:01936917][4035ms]-[0.40970][0.40691] [save]
[21:28:33][i:01937688][3954ms]-[0.40901][0.40844] [save]
[22:28:31][i:01938459][3889ms]-[0.40799][0.40538] [save]
[23:28:36][i:01939232][3896ms]-[0.40728][0.40608] [save]
[00:28:33][i:01940002][4047ms]-[0.40801][0.40747] [save]
[01:28:32][i:01940774][4135ms]-[0.40857][0.40644] [save]
[02:28:31][i:01941546][4012ms]-[0.40606][0.40538] [save]
[03:28:31][i:01942319][3848ms]-[0.40733][0.40673] [save]
[04:28:40][i:01943091][4082ms]-[0.40639][0.40533] [save]
[05:28:34][i:01943861][3892ms]-[0.40363][0.40509] [save]
[06:28:34][i:01944632][3877ms]-[0.40791][0.40735] [save]
[07:28:34][i:01945404][4039ms]-[0.40750][0.40568] [save]
[08:28:31][i:01946177][4152ms]-[0.40676][0.40415] [save]
[09:28:37][i:01946953][3977ms]-[0.40605][0.40477] [save]
[10:28:30][i:01947723][3855ms]-[0.40780][0.40385] [save]
[11:28:35][i:01948467][4454ms]-[0.40518][0.40576] [save]
[12:28:33][i:01949234][3926ms]-[0.40470][0.40560] [save]
[13:28:30][i:01950007][3788ms]-[0.40636][0.40501] [save]
[14:28:39][i:01950759][3908ms]-[0.40560][0.40597] [save]
[15:28:33][i:01951517][3759ms]-[0.40421][0.40473] [save]
[16:28:32][i:01952276][4090ms]-[0.40460][0.40274] [save]
[17:28:31][i:01953044][4138ms]-[0.40539][0.40349] [save]
[18:28:33][i:01953820][3803ms]-[0.40387][0.40330] [save]
[19:28:37][i:01954587][3774ms]-[0.40363][0.40391] [save]
[20:28:33][i:01955346][3835ms]-[0.40393][0.40452] [save]
[21:28:34][i:01956110][3954ms]-[0.40533][0.40311] [save]
[22:28:34][i:01956886][3773ms]-[0.40455][0.40517] [save]
[23:28:30][i:01957662][3865ms]-[0.40384][0.40319] [save]
[00:28:36][i:01958440][3934ms]-[0.40326][0.40219] [save]
[01:28:32][i:01959216][3865ms]-[0.40388][0.40413] [save]
[02:28:32][i:01959991][4012ms]-[0.40484][0.40090] [save]
[03:28:34][i:01960765][3931ms]-[0.40294][0.40255] [save]
[04:28:31][i:01961537][3858ms]-[0.40226][0.40177] [save]
[05:28:38][i:01962313][4003ms]-[0.40298][0.40015] [save]
[06:28:31][i:01963086][4193ms]-[0.40266][0.40324] [save]
[07:28:32][i:01963861][4171ms]-[0.40047][0.40106] [save]
[08:28:32][i:01964620][3809ms]-[0.40349][0.40103] [save]
[09:28:31][i:01965396][3945ms]-[0.40257][0.40414] [save]
[10:28:36][i:01966169][3879ms]-[0.40111][0.40094] [save]
[11:28:32][i:01966943][3721ms]-[0.40400][0.40259] [save]
[12:28:31][i:01967712][3884ms]-[0.40377][0.40031] [save]
[13:28:30][i:01968486][3906ms]-[0.40037][0.40040] [save]
[14:28:32][i:01969258][3928ms]-[0.40354][0.40214] [save]
[15:28:34][i:01970029][3883ms]-[0.40058][0.40031] [save]
[16:28:33][i:01970803][3835ms]-[0.40096][0.40259] [save]
[17:28:34][i:01971578][4093ms]-[0.40007][0.39863] [save]
[18:28:33][i:01972351][3927ms]-[0.40237][0.40089] [save]
[19:28:31][i:01973122][3900ms]-[0.40153][0.39990] [save]
[20:28:39][i:01973895][3899ms]-[0.39925][0.39891] [save]
[21:28:30][i:01974667][3802ms]-[0.40136][0.40144] [save]
[22:28:34][i:01975444][4090ms]-[0.40035][0.39958] [save]
[23:28:33][i:01976217][3896ms]-[0.40040][0.39833] [save]
[00:28:31][i:01976989][4017ms]-[0.39942][0.40114] [save]
[01:28:37][i:01977762][4072ms]-[0.40222][0.39759] [save]
[02:28:33][i:01978534][3961ms]-[0.39905][0.40076] [save]
[03:28:32][i:01979309][3960ms]-[0.40122][0.40139] [save]
[04:28:32][i:01980082][3942ms]-[0.40059][0.39844] [save]
[05:28:33][i:01980855][3934ms]-[0.39878][0.39911] [save]
[06:28:36][i:01981625][4166ms]-[0.40099][0.39784] [save]
[07:28:31][i:01982396][3992ms]-[0.40088][0.40007] [save]
[08:28:31][i:01983170][3756ms]-[0.39807][0.39900] [save]
[09:28:32][i:01983942][3903ms]-[0.39963][0.39766] [save]
[10:28:32][i:01984714][4105ms]-[0.40033][0.39781] [save]
[11:28:35][i:01985480][3754ms]-[0.40024][0.39982] [save]
[12:28:35][i:01986252][4160ms]-[0.39956][0.39637] [save]
[13:28:33][i:01987024][3909ms]-[0.39777][0.39758] [save]
[14:28:34][i:01987797][3962ms]-[0.39946][0.39894] [save]
[15:28:34][i:01988568][3809ms]-[0.39675][0.39963] [save]
[16:28:35][i:01989338][4004ms]-[0.39802][0.39881] [save]
[17:28:32][i:01990110][3936ms]-[0.39859][0.39719] [save]
[18:28:32][i:01990884][3849ms]-[0.39980][0.39829] [save]
[19:28:34][i:01991657][3950ms]-[0.39619][0.39502] [save]
[20:28:31][i:01992430][4041ms]-[0.39812][0.39719] [save]
[21:28:38][i:01993204][3922ms]-[0.39785][0.39582] [save]
[22:28:31][i:01993974][3866ms]-[0.39778][0.39611] [save]
[23:28:35][i:01994748][3911ms]-[0.39978][0.39716] [save]
[00:28:35][i:01995521][3979ms]-[0.39873][0.39918] [save]
[01:28:34][i:01996292][4004ms]-[0.39652][0.39645] [save]
[02:28:38][i:01997064][3965ms]-[0.39566][0.39723] [save]
[03:28:32][i:01997834][3946ms]-[0.39580][0.39601] [save]
[04:28:34][i:01998607][3917ms]-[0.39884][0.39472] [save]
[05:28:32][i:01999380][4062ms]-[0.39700][0.39450] [save]
[06:28:32][i:02000152][3978ms]-[0.39644][0.39497] [save]
[07:28:37][i:02000924][3806ms]-[0.39645][0.39591] [save]
[08:28:34][i:02001695][4104ms]-[0.39732][0.39542] [save]

970977470

ICE版本的,  应该不兼容原版和 MVE版本

Deannnn

你这还有很大的下降空间啊，跑预训练不用开注意力的

天光脑震荡

Deannnn 发表于 2024-6-10 14:17
你这还有很大的下降空间啊，跑预训练不用开注意力的

如果不开注意力的话，loss直接可以降到0.2左右

Deannnn

天光脑震荡 发表于 2024-6-10 14:35
如果不开注意力的话，loss直接可以降到0.2左右

把最下面那张图的鱼尾纹褶子跑出来就差不多了

天光脑震荡

970977470 发表于 2024-6-10 13:11
ICE版本的,  应该不兼容原版和 MVE版本

是的，所以连ICE也一起打包了

天光脑震荡

Deannnn 发表于 2024-6-10 14:44
把最下面那张图的鱼尾纹褶子跑出来就差不多了

再跑个100万都不知道行不行

Deannnn

天光脑震荡 发表于 2024-6-10 14:54
再跑个100万都不知道行不行

你多少张图跑？

天光脑震荡

Deannnn 发表于 2024-6-10 15:29
你多少张图跑？

34400

Deannnn

天光脑震荡 发表于 2024-6-10 15:33
34400

你可以分批跑