A training trick that may make fine-tuning faster

Neural network training can be changed by splitting each weight vector into two parts: its size and its direction. The goal is to make fine-tuning simpler and faster when adapting a model to a new task.

Instead of adjusting each learned number group as one piece, the method treats how large it is and where it points as separate controls. No concrete speedup, cost saving, model size, or benchmark result is provided in the available details.