A Three-Wafer-Stacked Hybrid 15-MPixel CIS + 1-MPixel EVS With 4.6-GEvent/s Readout, In-Pixel TDC, and On-Chip ISP and ESP Function

We introduce a hybrid 4096 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 3680 CMOS image sensor (CIS) with an embedded 1032 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 928 event-based vision sensor (EVS). Within a four-by-four cluster of CIS pixels, one color channel is substituted to provide a photocurrent for an EVS pixel. The EVS readout circuit is allocated on a second wafer and wrapped behind the four-by-four pixel cluster. A third wafer is used to integrate all peripheral readout circuitry for the EVS and CIS functions while achieving a compact sensor footprint. The EVS pixel achieves sub-1 Hz noise rates and 3% contrast non-uniformity (CTNU) at a linear nominal contrast threshold (NCT) of 15%. The row and column readout circuitry bypasses elements without events in a fraction of a clock cycle. Furthermore, a skip function is realized that reads only bordering pixels of a connected set of events. This achieves an event rate of up to 4.6 GEvents/s (GEps). The sensor captures in-pixel time stamps through sampling of a gray code timer realizing a time-to-digital conversion (TDC). Image and event signal processors (ISP & ESP) are integrated on-chip. The ESP provides global ambient and global activity monitoring functions. The sensor supports various flicker mitigation approaches such as adjustment of contrast threshold, in-pixel bandpass filter function and region-of-interest (ROI), and deterministic or random subsampling. The hybrid sensor mitigates challenges in synchronization and parallax error compared with a two-sensor solution using a dedicated CIS and dedicated EVS sensor. Furthermore, only one package and lens are required reducing footprint and cost.