Distributed Leading Non-Zero Detection. Input activations are hierarchically distributed to each PE. To take advantage of the input vector sparsity, we use leading nonzero detection logic to select the first non-zero result. Each group of 4 PEs does a local leading non-zero detection on their input activation. The result is sent to a Leading Nonzero Detection Node (LNZD Node) illustrated in Figure 4. Each LNZD node finds the next non-zero activation across its four children and sends this result up the quadtree. The quadtree is arranged so that wire lengths remain constant as we add PEs. At the root LNZD Node, the selected non-zero activation is broadcast back to all the PEs via a separate wire placed in an H-tree.