Introduction
CONFOCAL laser scanning microscopy (CLSM) is universally used in biomedical research to investigate molecular mechanisms underlying vital biological functions. CLSM primarily owes its widespread use to its capacity to produce sharp images of structures in vivo. This is achieved through a special arrangement of optical elements, which focus the laser beam in a diffraction-limited volume of about 1 fL, depending on the excitation wavelength, and detect fluorescence from an even smaller volume by filtering the emitted light through a pinhole, a circular aperture of few tens of micrometers in diameter that is placed in front of the detector. Through this special optical arrangement, a fluorescence signal is detected only from molecules that are confined in this small, so-called confocal volume. Fluorescence emission from outer molecules is strongly attenuated by the pinhole, thus enabling the selective observation of a subset of fluorophores at a high signal-to-noise ratio [1].
An important feature of confocal microscopy is the possibility to visualize the three-dimensional spatial distribution of molecules of interest within the investigated specimen. This is achieved by raster-scanning the confocal volume either by steering the laser beam using fast galvanometric scanners and acousto-optic deflectors, or by moving the sample using nanopositioning piezoelectric microscope stages.
Confocal laser scanning microscopes achieve image acquisitions at rates of about 30 frames per second (fps) or more
in fast scanning modes [2]. Under these operating conditions, a relatively
high illumination intensity is needed since the dwell time per pixel is 10
These limitations were overcome by constructing multifocal microscopes, where multiple confocal volumes are simultaneously scanned over the sample. Indeed, the use of massively parallel confocal arrangements permits to acquire a full frame at the acquisition time of a single pixel in classical CLSM. Hence, fast acquisition rates are achieved without increasing the illumination power as a result of the longer dwell times per pixel. Line-scanning and spinning disk confocal systems achieve up to 1000 fps [1], [4] and two-dimensional detector arrays such as electron-multiplying CCD cameras or CMOS sensors are commonly used to measure the fluorescence signal. Fast frame-rate microscopy techniques have a huge potential for biological investigations. An improvement of the frame acquisition speed up to 10–100 kfps would allow the characterization of molecular diffusion processes.
A standard tool to investigate the mobility of molecules in living cells is fluorescence correlation spectroscopy (FCS). In FCS, temporal autocorrelation analysis is applied to detect nonrandomness in the fluctuations of the fluorescence signal. This technique is therefore able to monitor all processes that lead to fluorescence intensity fluctuations at the temporal scale between few tens of nanoseconds up to seconds or longer e.g. formation of triplet and dark states, Brownian motion, protein-protein interactions and liquid flow [5] . On the other hand, classical FCS experimental setups are mostly limited to the observation of a single confocal volume and they cannot investigate simultaneously multiple regions in the sample. The construction of a multifocal microscopy setup which is capable of fast frame-rates above 10 kfps is, therefore, of major scientific interest. Not only the intensity of the fluorescence emission signal would be observed, but also its fast fluctuations as a function of the position within the sample, enabling parallel FCS studies across a cell.
The design of suitable bidimensional photodetectors plays a key role for the implementation of the described multifocal system. The acquisition speed is not the only important parameter, but also the photon detection efficiency (PDE), dark signal and saturation levels are limiting factors. High-gain solid-state detectors as the single photon avalanche photodiode (SPAD) have all these properties. A fair PDE above 40% and few thousands dark counts per second (cps) are commonly specified [6]–[8]. Saturation rates in the order of many millions of photons per second are possible [7]. Additionally, SPADs are not affected by any read-out noise, in contrast to CCD or CMOS sensors, which is a major advantage for combining high frame-rate imaging and FCS. In fact, the frames acquired at 10 to 100 kfps, which are required for FCS, can be binned over time, e.g. few milliseconds, to visualize the spatial distribution of the measured fluorophores. This binning operation, which is performed during post-processing of the data, does not degrade the signal-to-noise ratio due to the absence of read-out noise.
Several pioneering works investigated the use of multifocal FCS experimental setups more than 10 years ago [9]–[11]. The number of SPADs and confocal volumes was not sufficiently large to allow for the reconstruction of images, although the main experimental concepts were already developed. More recent works used the next generation SPAD imagers featuring 1024 photodetectors on the same silicon chip [12]–[15].
We present a new 32
Camera Design
The camera design optimized for multifocal microscopy was based on an array of 1024 independent SPADs produced in
standard CMOS technology [7], [19],
[20]. The detectors were organized in a 32
The SPAD [21] is a reverse-biased p-n junction, which is operated well above its breakdown voltage. Under this biasing condition the absorption of a single photon, causes the generation of an electron-hole pair, which is accelerated by the electric field across the junction. The energy of the charge carriers is eventually sufficient to trigger a self-sustained macroscopic avalanche current of few milliamperes through the device.
A quenching circuit based on a time-varying active load (variable-load quenching circuit, VLQC) [22] has been integrated for sensing the SPAD ignition, quenching the avalanche and resetting the detector to its initial condition. Compared to quenching circuits based on passive loads, the VLQC has the major advantage of speeding up the quenching action, thus minimizing the charge amount which flows through the SPAD after ignition. Moreover, a fixed dead-time, i.e. the minimum time interval between two detection events, of several tens of nanoseconds is externally set.
The use of SPADs as photodetectors has major advantages for imaging applications concerning the signal-to-noise
ratio. No analog measurement of voltage or current is needed, since the detectors act as a digital Geiger-like
counter. Hence, no read-out noise is added to the measurement process. This is a very important advantage for
high-frame rate microscopy imaging, since the probability of detecting a single photon per frame and per pixel is
usually low (
The dominant noise processes for the described pixels were dark counts generation and afterpulsing [19]. The former was below 4000 cps for more than 75% of the total number of pixels at room temperature and at +5 V excess bias. The remaining ones showed values between tens to several hundredths thousands CPSs.
Afterpulsing depends strongly on the overvoltage and dead-time values. In this study, the dead time was set to 200 ns, and the afterpulsing probability achieved a maximum of 5% over the whole array. It increased above 20% if the dead-time was set to 50 ns, which is the lower limit for the current hardware design. On the other hand, the PDE at the defined overvoltage was above 40% at 450 nm, and it decreased to about 27% at 550 nm.
Fig. 1(a) provides a schematic view of the pixel architecture. The VLQC
output, which is synchronous with the avalanche sensing, triggers the processing electronics and an 8-bit
Linear-Feedback Shift-Register (LFSR) counts the detection events. Routing electronics is then implemented on the same
chip to read-out the counter values and to transfer them to the off-chip electronics. This design allows the
measurements of 50 000 to 100 000 fps. The presented architecture additionally allows gating of the LFSR
counters for a very short time between 1.5 and 20 ns. Hence, the trigger signal by the VLQC circuit increments the
LFSR counter only when the signal GATE is asserted (logic level ‘1’). Otherwise, the detected photons are
not counted (logic level ‘0’). To reach gated photon counting within each pixel, the output pulse from the
quenching circuit (
In order to keep the architecture as flexible as possible, the generation of the GATE signal was performed by an external Field Programmable Gate Array (FPGA) (Spartan 6, XC6SLX45-2FGG484 Xilinx, San Jose, CA, USA). A Xem6010 (Opal Kelly, Portland, OR, USA) development board, which incorporates both the Spartan 6 FPGA and a high speed USB 2.0 interface, was used to control the read-out of the chip and to transfer the measured images to the host computer.
Fig. 1(b) shows the architecture based on the internal delay locked
loops (DLLs) of the FPGA device, which were used to generate fast gate signals. FPGA devices require DLLs to de-skew
the internal digital paths and to fine-tune the sampling time of fast serial communication lines. They are designed to
produce a precise phase shift between 10 and 40 ps, which can be dynamically controlled during operation. The update
of the DLL shift requires few tens of clock cycles in the worst case, i.e. few microseconds depending on the used FPGA
family. This dynamic phase shift is, therefore, well suited to generate periodic sequences of pulses.
Fig. 1(d) shows how de-phased clock signals are combined to generate pulses
of variable width. The 50 to 100 MHz board clock (
This logic design was implemented to set up a time-gated FLIM detection system
[18]. The board clock is frequency doubled by a digital clock manager (DCM) and used as a trigger signal for a
pulsed laser diode (
In summary, the presented camera architecture is a tradeoff between accurate time measurements on the timescale of few nanoseconds and fast processing of the measured signals up to the limit of millions of photons per second and pixel. Any detector that is suitable for both FLIM and FCS experiments must fulfill these requirements.
Multifocal Microscopy Setup
The multifocal FLIM/FCS setup was built on a standard Axio Observer D1 inverted microscope (Zeiss, Jena, Germany)
equipped with a C-Apochromat 63
The DOE is a glass hologram designed to diffract a single laser beam into 1024 beams at different angles. The diffraction angles and the intensity of the zeroth order diffraction (transmitted beam) depends on the incident wavelength. The zero order beam, although not negligible, did not affect the performance of the system.
The sharpness of the spots projected on the image plane was adjusted by moving lens L3 along the optical axis. Other
two micrometer stages were used to center the position of the array perpendicularly to the optical axis. The distance
between the DOE position and the lens L3 was fine-tuned to match the pitch of the illumination spots and the active
areas of the SPAD camera. The small diameter of the SPAD acts as a spatial filter and no additional pinholes are
required in front of the detectors, in contrast with standard confocal microscopes. The previously described 32
The acquisition of FLIM images required the optimization of the camera firmware to generate fast gate signals and the synchronization pulses to trigger the laser diode.
Analysis of the Data
The analysis of FLIM data acquired by time-gated techniques has been a subject of several works [25], [26] and more recently of a specialized review focusing on solid-state imaging sensors [24]. The data analysis method depends strongly on the gating scheme used, i.e. the selected values of gate shift and gate width as a function of the lifetime of the excited state of the observed fluorophores. The method described in [26] provides results close to the optimum for the gating scheme described in Section II. Indeed, it is a maximum likelihood (ML) approach and it provides an unbiased estimation of the model parameter even for very low numbers of photons per pixel. Compared to other approaches like the least square technique, ML estimation is both more precise and accurate, as experimentally verified by Maus et al. [27]. This approach has only one drawback. It does not account for the uncorrelated noise present in the decay traces, e.g. due to room light or dark counts of the SPAD detectors. Therefore, background subtraction has to be applied before estimating the lifetimes.
The adopted method searches for the lifetime value
While eq. (1) can be solved in less than 1 s for a whole FLIM image
using MATLAB (The MathWorks Inc., Natick, MA, USA), the FCS data analysis requires more optimized computational
methods to be executed within few seconds. The standard algorithm to calculate the autocorrelation curves (ACCs),
which is usually known as Multi-
Eq. (4) can be solved analytically for simple geometries and
processes as the translational motion of freely diffusing fluorophores in solution
[31]. The solution for this special case yields the autocorrelation function (ACF):
We developed dedicated software both to communicate efficiently with the camera and to calculate the ACC for each
pixel from sets of 130 000 images by massive parallelization of the calculation using a NVIDIA GeForce GTX 780
(NVIDIA corporation, Santa Clara, CA, USA) Graphical Processing Unit (GPU). This GPU board supports the CUDA parallel
computing platform and it is capable of running tens of thousands of threads concurrently. Thereby, the computational
time of the calculation of 1024 complete ACCs decreased to about 4 s, compared to the about 200 s execution time that
was needed when using a single CPU. The parameters of the ACF [eq. (5)
] were calculated for each pixel.
It has been shown that both the estimation of lifetimes and the processing required for FCS data analysis can be embedded in the acquisition electronics [13], [32], [33]. These methods, thought excellent, are absolutely needed for real-time and high throughput applications, which are outside the scope of the current work. Future developments will focus on implementing similar algorithms for the described SPAD array architecture.
Experimental Results
We applied the multifocal microscope in several model experiments to show the performance of the system. The results are divided in three sections concerning the microscopy setup, FLIM and FCS.
A. Microscopy Setup
A first important test was to measure the uniformity of the field of view after illumination by the DOE and detection using the SPAD camera. The uniformity of the experimental setup was measured by imaging an aqueous solution of quantum dots (QD, 525 ITK™ Molecular Probes, Darmstadt Germany). The selected QDs emit around 525 nm after excitation at 488 nm, and the diameter of the nanocrystals is approximately 20 nm according to the manufacturer’s specifications.
Several parameters influenced the uniformity of the detected signal over the field of view, which depend both on the optical coupling of the laser to the microscope, the alignment of the detectors, and the variation of the detection efficiency of the SPADs of the matrix. This parameter is, therefore, the product between the uniformity of the excitation intensity obtained by the DOE, the coupling efficiency between the excitation volumes and the SPAD, and the PDE of the SPADs. Fig. 3 shows the measured uniformity of the system normalized by its mean value after dark counts subtraction. More than 70% of the pixels have a uniformity within 15% of the mean value. The largest deviations of about 30%, which were caused by an uneven illumination of the DOE by the Gaussian laser beam, were obtained at the outer rim of the array. The measured overall uniformity was sufficient for the proposed applications.
Additionally, we tested the linearity of the gate width (Fig. 4), i.e. how precisely gate width could be set by the FPGA device. A stabilized LED was placed in front of the sensor, and image sequences at variable gate widths between 1 and 20 ns were acquired. Considering that the illumination intensity was constant, a signal dependent on the width of the gate was measured. Fig. 4 shows the estimated gate width as a function of the expected value programmed by the control software. One can observe that the width of the gate is well approximated by a linear model over a large temporal range. Below 1.5 ns and above 18 ns, deviations from linearity were observed. Indeed, the gate pulses become too short to efficiently enable and disable the LFSR counters. The obtained gate width range is definitely sufficient for most FLIM experiments.
B. Fluorescence Lifetime Imaging Microscopy
The multifocal FLIM-FCS setup was applied to measure the lifetime of known fluorophores both in solution and in live
cells and compared to previously published values. In order to estimate the lifetime [
The measured average lifetime over all the pixels was 3.8 ns
Afterwards, we investigated living Human Embryonal Kidney HEK293-T cells expressing the eGFP
[35]. The cells were plated in 35 mm petri dishes with a 150
C. Fluorescence Correlation Spectroscopy
The multifocal microscope was applied to measure the diffusion of single fluorophores in solution. We have chosen QD
because of their brighter emission and slower diffusion time compared to Rhodamine 6G, which is typically used for the
calibration of FCS setups. The probe was prepared as described in section V-A.
The sample was illuminated by an average laser power of about 25
The concentration of the fluorophores in various runs was slightly different, but always between 0.05 nM and 2 nM, as verified by FCS using a conventional instrument (ConfoCor 3, Zeiss, Jena, Germany).
Fig. 6(a) and (b) show the values of the estimated
Fig. 6(c) shows the ACC curves of selected pixels. The green and blue
traces show the typical ACC of freely diffusing fluorophores in solution. The measured ACC do not substantially differ
from those measured by the commercial ConfoCor 3 microscope, although the shortest lag time for the SPAD camera has a
much longer duration [Fig. 6(f)]. One should observe that all the ACC of
Fig. 6(b) have a peak at lag times below 100
At present, the dominant noise sources in the FCS measurements shown in Fig. 6
are the afterpulsing and dark-counts generation processes, which systematically affect the absolute value of
Conclusion
We presented a 32
Future improvements of the current experimental setup will focus on the observation of single molecules in solutions and in live cells, and on the real-time processing of FCS and FLIM data by the acquisition electronics. Additionally, further investigations are required to improve the photodetectors by reducing the dark-count rate and afterpulsing probability.
Acknowledgment
The authors thank N. Tavraz for the preparation of the HEK293-T cell cultures; G. Simmerle and A. Veronese for the excellent technical assistance. COST is acknowledged for support in the framework of the MP1205 action.