Imaging a periodic moving/state-changed object with Hadamard-based computational ghost imaging

Hui Guo(郭辉) Le Wang(王乐) and Sheng-Mei Zhao(赵生妹)

1Institute of Signal Processing and Transmission,Nanjing University of Posts and Telecommunications,Nanjing 210003,China

2College of Information Engineering,Fuyang Normal University,Fuyang 236037,China

Keywords: ghost imaging,periodic moving object,periodic state-changed object,Hadamard matrix

1. Introduction

Ghost imaging (GI), also called single-pixel imaging, is a novel optical imaging technique, in which the object is illuminated by speckle patterns sequentially and collected by a bucket detector recording as one-dimensional intensity sequences. The object image is recovered by correlating the illumination patterns and the corresponding single-pixel intensity sequences.[1–4]The GI can be achieved by quantum entangled photons, pseudo-thermal light source, and thermal light source.[5–7]The computational ghost imaging(CGI)simplifies the configuration of GI and further expands the application range of GI.[8–12]Compared with traditional imaging techniques, GI has the ability to reconstruct the image of an object in various optically-harsh or noisy environments.[13,14]

As we all know, the relative motion between an imaging system and an object of interest is one of the common causes of imaging degradation. When the speed of the moving object is too fast, it is hard to image the object with a good signal-to-noise ratio via a conventional pixelated sensor. CGI,a nonlocal imaging method, exhibits substantial advantages

where traditional camera technology is limited.[15,16]To that end, many scholars used CGI to study the imaging of moving objects.[17–26]The imaging condition of lensless GI for a moving object was given in Ref.[17]. Later,GI with relative motion between the imaging system and the object such as shaking,[18]moving in the tangential direction[19]and moving in the axial direction[20]were discussed successively. Meanwhile, compressive sensing (CS) theory and complementary modulation technique were used to reconstruct background subtracted images.[21]In Ref.[22],the displacement obtained by calculating the cross-correlation of the unclear rough images at the corresponding time was used to improve the imaging quality of moving object. When the type of the object’s motion was known,the illumination patterns were transformed according to the motion of the moving object to effectively reconstruct the moving image.[23,24]Recently, to overcome the limited rate of the structured illumination in CGI, lightemitting diode(LED)array with high frame rate was designed to capture images of dynamic scene.[25]In Ref. [26], LED array was further designed to realize colorful moving object high-speed imaging in low-light-level scenarios. However,all previous methods require a bucket detector with a high sampling rate. The moving/state-changed object is always taken as relatively static for the fast bucket detector.

On the other hand,traditional CGI that uses random patterns to obtain a nice reconstruction requires more thanMmeasurements of anM-pixels image.[27]Recently,Hadamard patterns, as the orthogonal patterns instead of the random speckle patterns,were used to illuminate the object to improve the image quality and reduce the number of the measurements toM.[28,29]

In this work,an attempt is made to image a fast periodic moving/state-changed object by using Hadamard-based CGI with a slow bucket detector,which is named as PO-HCGI.In the scheme, if we want to getNframes of the moving/statechanged object,we divide each row of a Hadamard matrix intoNgroups. Each group of matrix elements is modulated as a speckle pattern to illuminate the moving/state-changed object one by one(the cycleTof the periodic moving/state-changed object is known). Then, a slow bucket detector measures the total intensity reflected from the object irradiated byNgroups of speckle patterns as a bucket detector signal. With the periodic movement or change of the object, we get a series of bucket detector signals. Finally, we retrieveNframes of the moving/state-changed object by using the second-order correlation algorithm based on the Hadamard matrix and bucket detector signals. The advantage of our scheme is that although the motion speed of the periodic moving/state-changed object is faster than the sampling rate of the bucket detector,our scheme can still image it.

The organization of the paper is as follows. In Section 2,PO-HCGI will be analyzed. In Section 3,the performance of the PO-HCGI is verified by simulations and experiments. Finally,Section 4 concludes the paper.

2. Scheme description

The schematic diagram of the proposed PO-HCGI is illustrated in Fig. 1. To further eliminate the effect of noise from the bucket detector on the imaging of the periodic moving/state-changed object,differential speckle patterns are constructed according to Refs.[21,30]. The light source illuminates a digital micro-mirror device (DMD), which is controlled by a computer, to produce pairs of speckle patterns.Each pair includes speckle patternsHki(x,y) and its inverse patterns ～Hki(x,y).Hki(x,y) and ～Hki(x,y), which are square matrix with elements±1, the resolution of the speckle patterns areNx×NyandHki(x,y) =−～Hki(x,y). If we want to restoreNframes of the periodic moving/state-changed object, we divide each row of the natural order Hadamard matrixHintoNgroups. Then, the matrix elements in rowigroupkare used to create the speckle patternHki(x,y)(k=1,2,...,N −1,N). When we understand the cycle timeTof the periodic moving/state-changed object, speckle patternsH1i(x,y),H2i(x,y),...,Hki(x,y),...,HN−1i(x,y),HNi(x,y)are modulated and projected on the DMD to irradiate each frame of the object successively. Then, a slow bucket detector is used to collect the total energy reflected from the object as one measurement and recorded asBi. Meanwhile, ～Biare obtained for speckle patterns ～Hki(x,y)(k=1,2,...,N −1,N),and we set differential signalsDiasBi −～Bi. After 2×Mmeasurements, a set of differential signals{Di}Mi=1are achieved,and then we can getNframes of the periodic moving/statechanged object by the second-order correlation algorithm.

Fig.1. A schematic diagram of the PO-HCGI scheme.

We use picture frame to represent each state of the periodic moving/state-changed object. The individual states of the targetTN(x,y) can be expressed asT(x,y,k),k=1,2,...,N,

whereNis the number of the frames we want to obtain. The speckle patternsHki(x,y)illuminating thekth frame of the object can be expressed asHki(x,y)T(x,y,k). The slow bucket detector get the total energyBireflected from the object, expressed as

whereηis the bucket detector responsivity,andnis the noise from environmental illuminations.Similarly,corresponding to～Hki(x,y),k=1,2,...,N, ～Bican be expressed as

3. Result and discussion

Fig. 2. A schematic diagram of the PO-HCGI scheme experimental system.

To prove the validity of PO-HCGI, the mean square error(MSE)is used as an objective evaluation,which is defined as[30]

whereT(x,y)and ˆT(x,y)are the intensity values of the original and the recovered image,respectively.NxandNyrepresent the horizontal and vertical dimensions of the object.

At first,we verify the feasibility of PO-HCGI by imaging the periodic moving object in numerical simulations. The results are compared with those results of Hadamard-based CGI with a fast bucket detector (HF-CGI) and Hadamard-based CGI with a slow bucket detector(HS-CGI),which are shown in Fig. 3. Two different categories of object images, including a two-grayscale jumping “square” logo, a two-grayscale“T”logo, and an eight-grayscale“spacecraft”are assumed to be the original objects (each 64×64-pixels). Taking the periodic moving object “T” logo as an example, the object is displayed continuously and repeatedly through the DMD-2 with cycle timeT. In PO-HCGI, if we want to obtain four 64×64-pixels frames during the movement of the object, a 16384×16384 Hadamard matrix should be generated. Then,we split each row of the Hadamard matrix into four groups and prepare speckle patterns with 64×64-pixels,respectively.Four groups speckle patterns in the same row of the Hadamard matrix are taken to illuminate the moving object in one cycleT, and then recorded by a slow bucket detector as a single measurement. We will get 32768 bucket detector values via the original speckle patterns and their inverse, and then calculate 16384 differential signals. In HF-CGI, to image four 64×64-pixels frames, four groups of identical 4096 64×64-pixels Hadamard speckle patterns need to be generated. We choose the same speckle patterns in each group to illuminate the moving object in one cycleT, and a fast bucket detector needs to make a detection at each timeT/4. Finally, we will get 32768 bucket detector values by original speckle patterns and its inverse speckle patterns,and calculate 16384 differential signals. In HS-CGI, we also need to generate four groups of identical 4096 64×64-pixels Hadamard speckle patterns. The same speckle patterns in each group are taken to illuminate the moving object in one cycleTand the reflected light is detected by a slow detector at each timeT. The slow detector will collect 8192 bucket detector values by original speckle patterns and its inverse speckle patterns and we will get 4096 differential signals.The results show that four frames of the moving “T” logo can be recovered by PO-HCGI and HF-CGI at the number of measurements at 32768(16384 differential signals)and the quality are all completely faithful to the object.Although the imaging quality and the number of the measurements of PO-HCGI and HF-CGI schemes are almost the same,the sampling rate of the fast bucket detector is four times that of the slow bucket detector.Because HS-CGI uses a slow bucket detector,it can only reconstruct one picture frame with overlapping states of the moving object. It can also be seen that the operation trajectory of the“square”logo jumping within the object plane and the“spacecraft”are complex,and our scheme can still reconstruct their frames with high quality.The simulation results of the two-grayscale jumping“square”logo and the eight-grayscale “spacecraft” further validate the effectiveness of the PO-HCGI. As can be seen from Fig. 3,the imaging quality of PO-HCGI using a slow detector is almost the same as that of HF-CGI using a fast bucket detector,but HS-CGI using a slow bucket detector can only obtain one picture frame with overlapping states.

Fig.3. The simulation results of the periodic moving objects using different schemes,where MSE is presented.

Furthermore, the results of imaging the periodic statechanged object using PO-HCGI, HF-CGI, and HS-CGI are shown in Fig. 4. In Fig. 4, we use a two-grayscale “running man” logo and an eight-grayscale “Marilyn Monroe” video with 64×64-pixels as the original objects. Taking the periodic state-changed object “running man” logo as an example. In PO-HCGI, if we want to image eight frames of the periodic state-changed object, we need to generate a 32768×32768 Hadamard matrix. Then, each row of the Hadamard matrix is split into eight groups and each group is prepared as speckle pattern with 64×64-pixels. Both HF-CGI and HSCGI need to generate eight groups of identical 4096 64×64-pixels Hadamard speckle patterns. The response speed of the fast bucket detector in the HF-CGI is eight times that of the slow bucket detector in the PO-HCGI and the HS-CGI.Finally,both PO-HCGI and HF-CGI will collect 65536 bucket detector values and calculate 32768 differential signals.In HS-CGI,it will collect 8192 bucket detector values and calculate 4096 differential signals. Taking “running man” logo as an example, it can be seen that eight frames of the object can be recovered by PO-HCGI and HF-CGI with a low MSE in Fig.4.The HS-CGI can only get one picture frame with eight overlapping states.The reconstructed results of“Marilyn Monroe”video also further certify that PO-HCGI is effective to image eight-grayscale periodic state-changed object by using a slow bucket detector.

In addition, we perform the experiments on the random jumping “square” logo with four frames chosen from Fig. 3 as the objects. Four 1024×768-pixels jumping“square”logo are displayed by DMD-2 with cycle timeTas 1.6 ms. We use DMD-1 to modulate each speckle pattern with 8×8-pixels. In PO-HCGI,to get four frames of the object,we need to set four groups speckle patterns in each cycle. We use computer-1 to generate a 256×256 Hadamard matrix.Both HF-CGI and HSCGI need to generate four groups of identical 64 8×8-pixels Hadamard speckle patterns. The response speed of the fast bucket detector in HF-CGI is four times that of the slow bucket detector in PO-HCGI and HS-CGI. Two experimental results of PO-HCGI,HF-CGI,and HS-CGI are shown in Fig.5. The experimental results-1 of PO-HCGI and HF-CGI are similar to the simulated results as shown in Fig.3.

Fig.4. The simulation results of the periodic state-changed objects using different schemes,where MSE is presented.

Fig.5. The experimental results using different schemes to reconstruct four frames of jumping“square”logo.

Fig.6. The experimental results using different schemes to reconstruct eight frames of jumping“square”logo.

Since DMD-2 takes a certain time to refresh each state of the moving object,we also get the intermediate states as the results of the experimental results-2.The experimental results of HS-CGI are a picture frame with four overlapping states.If we want to get eight frames of the jumping“square”logo in each cycleT. In the PO-HCGI, we need to generate a 512×512 Hadamard matrix and divide each row of the matrix into eight groups. In HF-CGI and HS-CGI, we need to generate eight groups of identical 64 8×8-pixels Hadamard speckle patterns.The response speed of the fast bucket detector is set to be eight times that of the slow bucket detector. We will get experimental results as shown in Fig.6. From Fig.6,it can be seen that four frames of the moving object with four intermediate states will be got by PO-HCGI and HF-CGI, and the HS-CGI still get the same picture frame as shown in Fig.5.

We further use two frames“T”logo,chosen from Fig.3,at various cycle timesTto study the effectiveness of POHCGI. Two frames 1024×768-pixels “T” logo are displayed by DMD-2 with cycle timeTas 0.4 ms, 0.6 ms, and 0.8 ms,respectively. Here, we use DMD-1 to modulate each speckle pattern with 16×16-pixels. If we want to get two frames of the periodic moving “T” logo. In PO-HCGI, a 512×512 Hadamard matrix will be generated and each row of the matrix will be divided into two groups. In HF-CGI and HS-CGI,we will set two groups of identical 256 16×16-pixels Hadamard speckle patterns.The experimental results are shown in Fig.7.Both PO-HCGI and HF-CGI can image the“T”logo. As can be seen from Fig. 7, the longer the cycle timeT, that is, the longer the duration of each state of the moving object,the better the imaging effect of PO-HCGI. According to Nyquist–Shannon sampling theorem,for the imaging of two frames of a moving object with two groups of speckle patterns in each cycle, undersampling will also lead to some experimental results that can only be the intermediate states of the moving object. Moreover, we further compare the experimental results of PO-HCGI,HF-CGI,and HS-CGI for getting four frames of the moving“T”logo with cycle timeTas 0.4 ms,0.6 ms,and 0.8 ms. As shown in Fig.8,both PO-HCGI and HF-CGI can image the “T” logo with four frames include two frames of the moving object and two frames of the intermediate states.The HS-CGI is consistently not able to accurately image the moving objects by using a slow bucket detector. Therefore,the effectiveness of the PO-HCGI is further validated.

Fig.7. The experimental results using different schemes with different cycle T to reconstruct two frames of“T”logo.

Fig.8. The experimental results using different schemes with different cycle T to reconstruct four frames of“T”logo.

4. Conclusion

We have proposed the PO-HCGI to image a periodic moving/state-changed object with Hadamard-based CGI. We have compared the performance of PO-HCGI, HF-CGI, and HS-CGI by simulations and experiments. The results show that PO-HCGI is able to image a fast periodic moving/statechanged object with a slow bucket detector, which provides more application flexibility of CGI for imaging fast periodic moving/state-changed object.

Acknowledgements

Project supported by the National Natural Science Foundation of China (Grant Nos. 61871234 and 62001249), the University Talent Project of Anhui Province, China (Grant No.gxyq2020102),and the Scientific Research Project of College of Information Engineering, Fuyang Normal University(Grant No.FXG2021ZZ02).