The “Kaiming” initialization method has proved to be a mathematically solid way to initialize weights for deep neural networks.

Rectified Linear Unit (ReLU), is one of several keys to the recent success of deep networks, and studying them has yielded important results.

The architecture used by Kaiming et al. is similar to VGG-19, with a few minor differences.

The “Xavier” initialization method was an earlier method proven to be well suited to deeper layered networks. The “Kaiming” init method is here contrasted with it.