What if every dial had its own step size?

One step size for every dial An AI learns by turning dials. A network has millions of them, its weights, and training nudges every one a little at a time. To know which way to turn a dial, training measures how the mistake changes when that dial moves. That measurement is the dial's push, the gradient. It points downhill, toward a smaller mistake. Plain gradient descent moves every dial with the…