To exploit the sparsity of activations we store our encoded sparse weight matrix W in a variation of compressed sparse column (CSC) format [24]. For each column Wj of matrix W we store a vector v that contains the non-zero weights, and a second, equallength vector z that encodes the number of zeros before the corresponding entry in v. Each entry of v and z is represented by a four-bit value. If more than 15 zeros appear before a non-zero entry we add a zero in vector v. For example, we encode the following column