Queue Depth. The activation FIFO queue deals with load imbalance between the PEs. A deeper FIFO queue can better decouple producer and consumer, but with diminishing returns, as shown in our experiment in Figure 8. We varied the FIFO queue depth from from 1 to 256 in powers of 2 across 9 benchmarks using 64 PEs, and measured the load balance efficiency. This efficiency is defined as: 1 − bubble cycles (due to starvation) divided by total computation cycles. At FIFO size = 1, around half of the total cycles are idle and the accelerator suffers from severe load imbalance. Load imbalance is reduced as FIFO depth is increased but with diminishing returns beyond a depth of 8. Thus, we choose 8 as the optimal queue depth.