【科普】RTT和RTM是什么

Lau9

星级打分

• 1
• 2
• 3
• 4
• 5

平均分:NAN  参与人数:0  我的评分:未评

RTT (Ready To Train) is, simply put, a model that has been reused many times, with different dst or src, or in some cases even both. To start out a RTT, you can simply copy the pretrain set .pak file from _internal dir into your aligned folders. U can use a pretrained model but since this is basically the same as pretraining, it's not necessary. Train until the faces look well defined (teeth are sharp), 1m should be fine, there wont be any high frequency detail yet. No reason to change any of the default settings during this state in my experience. After this, you can save a copy of it, and train any dst or src and it will very quickly adapt to anything you throw at it, sometimes even better than a regular trained model would. Many say this can't be done with DF-UD but it definitely can, I've done it. It probably does work better with LIAE, though. What's really cool is, the model has a very far back memory that can reimplement knowledge that it learned from many training sessions ago, for example if you wanted to return back to a dst/src that you used previously with the same model, it will be faster if the model has already seen that face in the past. Eventually, the model can forget things as I hear.

RTM (Ready To Merge) is a model that has been trained specifically to apply one specific face to very wide variety of possible dsts. To do this, a large src set is used, encompassing as many angles/lighting/expressions as physically possible, and its trained against a RTM or pretrain faceset. This allows the model learn how to apply the face to any type of face shape, expression or lighting condition. Creating one of these can at times allow you to completely skip the training step, because you can just plop it into your model file, and merge to an extracted dst without a src set or training required. This is because it already knows exactly how to approximate the face in dst because of its prior knowledge. This should be done with LIAE models as they have better architecture for adapting to different face shapes/lighting but I suppose theoretically it could be done with DF. These can also be used like RTM and retrained with a new DST to basically instantaneously adapt, if for example there's a few kinks that need to be worked with merging without training. PS, this is a major oversimplification , refer to the official DFL guide for step by step instructions on how to perfect an RTM model.

简单地说，RTT（Ready To Train）是一个被多次重用的模型，具有不同的dst或src，或者在某些情况下甚至两者兼而有之。要开始 RTT，您只需将预训练集 .pak 文件从_internal目录复制到对齐的文件夹中即可。您可以使用预训练模型，但由于这与预训练基本相同，因此没有必要。训练直到脸部看起来很清晰（牙齿很锋利），1m 应该没问题，不会有任何高频细节。根据我的经验，没有理由在此状态下更改任何默认设置。在此之后，您可以保存它的副本，并训练任何 dst 或 src，它会很快适应您扔给它的任何东西，有时甚至比常规训练的模型更好。许多人说DF-UD无法做到这一点，但它绝对可以，我已经做到了。不过，它可能确实与 LIAE 配合得更好。真正酷的是，该模型具有非常遥远的回溯记忆，可以重新实现它从许多训练课程中学到的知识，例如，如果您想返回到以前使用同一模型的 dst/src，如果模型过去已经看到过那张脸，它会更快。最终，模型可能会忘记我听到的东西。

RTM（Ready To Merge）是一个经过专门训练的模型，可以将一个特定的面应用于各种可能的DST。为此，使用了一个大型 src 集，其中包含尽可能多的物理角度/光照/表情，并针对 RTM 或预训练面集进行训练。这使模型能够学习如何将面部应用于任何类型的面部形状、表情或照明条件。创建其中一个有时可以让你完全跳过训练步骤，因为你只需将其放入模型文件中，然后合并到提取的 dst，而无需 src 集或训练。这是因为由于其先验知识，它已经确切地知道如何在 dst 中近似面部。这应该用 LIAE 模型来完成，因为它们有更好的架构来适应不同的脸型/照明，但我想理论上可以用 DF 来完成。这些也可以像 RTM 一样使用，并使用新的 DST 进行重新训练，以基本上立即适应，例如，如果有一些扭结需要在没有训练的情况下进行合并。PS，这是一个重大的过度简化，有关如何完善RTM模型的分步说明，请参阅官方DFL指南。