Passive detection of rank-one signals with a multiantenna reference channel

In this work we consider a two-channel passive detection problem, in which there is a surveillance array where the presence/absence of a target signal is to be detected, and a reference array that provides a noise-contaminated version of the target signal. We assume that the transmitted signal is an unknown rank-one signal, and that the noises are uncorrelated between the two channels, but each one having an unknown and arbitrary spatial covariance matrix. We show that the generalized likelihood ratio test (GLRT) for this problem rejects the null hypothesis when the largest canonical correlation of the sample coherence matrix between the surveillance and the reference channels exceeds a threshold. Further, based on recent results from random matrix theory, we provide an approximation for the null distribution of the test statistic.


INTRODUCTION
In this work we consider a passive detection problem in which there is a surveillance channel where the presence/absence of a target signal is to be detected, and a reference channel that provides a noise-contaminated version of the target signal, and hence assists the surveillance channel in the detection process. This problem is of interest in applications such as passive radar, passive sensing or communications, among others. In passive radar, for instance, commercial RF signals are used as non-cooperative transmitters that illuminate potential targets of interest [1][2][3]. The reference channel acquires a version of the transmitted signal through a direct path (i.e., transmitter-receiver), whereas the surveillance channel acquires a different version of the transmitted signal through a target path (i.e., transmitter-target-receiver). The signal paths for the reference and the surveillance channels are typically separated by digital beamforming using directional antennas. In communications, passive detection might have application in cognitive radio [4], or to enhance the transmission opportunities of small cells in Heterogeneous Networks (HetNets) [5].
Passive target detection with a noisy reference channel has recently been considered in [1,6] for an unknown deterministic signal The work of I. Santamaria was supported by the Spanish Government through grants PRX14/0028 (Estancias de Movilidad de Profesores, Ministerio de Educación) and by project RACHEL (TEC2013-47141-C4-3-R) funded by the Ministerio de Economa y Competitividad (MINECO). The work of L. Scharf and D. Cochran was supported in part by a sub-contract with Matrix Research for research sponsored by the Air Force Research Laboratory under contract FA8650-14-D-1722. and single-antenna receivers. The multi-antena case, for deterministic signals and under spatially white noise of known variance, has been addressed in [2]. In this paper, we address the passive detection problem in a multivariate normal model when the surveillance and reference channels are equipped with M antennas, the transmitted signal is an unknown rank-one signal, and the noises at surveillance and reference channels are uncorrelated between them, but each having an unknown and arbitrary spatial covariance matrix. This a problem of testing the covariance structure in a two-channel multivariate normal model. Detection of a rank-one signal under white noise of unknown level has been addressed in [7], and extensions to diagonal noise covariance matrices and rank-P signals can be found in [8], and [9,10], respectively. Other variants of this problem have been considered in [11][12][13][14][15][16]. Differently from these single-channel detection, the model considered in this paper cannot be solved without the assistance of a reference channel.
In this paper, we show that the generalized likelihood ratio test (GLRT) for our problem rejects the null hypothesis when the largest canonical correlation of the sample coherence matrix between the surveillance and the reference channels exceeds a threshold. Further, we provide an approximation for the null distribution that allows us to set the threshold for a given probability of false alarm.

Notation
The superscripts (⋅) T and (⋅) H denote transpose and Hermitian, respectively. The determinant, trace and Frobenius norm of a matrix A will be denoted, respectively, as det(A), tr(A) and ||A|| F . I M is the identity matrix of dimensions M × M , and 0 M denotes either a column vector with M zeros, or the zero matrix of dimensions M × M (the difference should be clear from the context). We use A 1/2 (A −H/2 ) to denote the square root matrix of the Hermitian matrix A (A −1 ). The expectation operator will be denoted by E [⋅], and x ∼ CN (0, R) indicates that x is a complex circular Gaussian random vector of zero mean and covariance R. Finally, diag M (A) is a block-diagonal matrix formed by M × M blocks on the diagonal.

PROBLEM FORMULATION
We consider a passive network consisting of a reference channel and a surveillance channel, both equipped with M antennas. Denoting the signal transmitted by the non-cooperative transmitter as s[n], the detection problem can be formulated as follows: In this work, we make the following assumptions: (A1) The propagation delay difference between the surveillance and reference channels, as well as the Doppler shift in the case of moving targets, have been previously estimated and compensated via cross-correlation processing, like in bistatic radar systems [17].
(A3) The noise is modeled as a temporally white complex Gaussian vector with a 2M × 2M spatial noise covariance matrix given by where Σss and Σrr are arbitrary full-rank positive definite (pd) matrices.
With these assumptions, the detection problem (1) amounts to testing between two different structures for the composite covariance matrix of x[n] ∼ CN (0 2M , R). More specifically, where and We now consider N consecutive array snapshots The likelihood can be written as where S = 1 N XX H is the sample covariance matrix, which can be partitioned as where Sss is the sample covariance matrix of the surveillance channel and the other blocks are defined similarly. Remark: Notice that it is not possible to solve the considered detection problem using only the surveillance channel and exploiting the rank-deficient structure of the transmitted signal as in [7][8][9][10]12]. The reason is that, since Σss is an arbitrary positive definite matrix, both hypotheses have the same set of admissible covariance matrices and the hypotheses are thus indistinguishable.

Problem Invariances
Let us first consider the problem invariances.

Proposition 1. The detection problem (3) is invariant to the transformation group
with group action TRxxT H .
Proof. If we apply an arbitrary block-diagonal transformation T to the vector of observations x, it is clear that the structure of the covariance matrices under each hypothesis remains unchanged after the transformation. As stated in [18], since the distributions remain in the same family (zero-mean complex Gaussians) and the parameter spaces are preserved, we can conclude that the detection problem (3) is invariant to block-diagonal transformations.
Now, it is well-known that the canonical correlations between the surveillance and the reference channels are a complete set of invariants to block-diagonal transformations [19], [20]. Therefore, any invariant detector must be only a function of them. The sample canonical correlations are the singular values of the sample coherence matrix between the surveillance and reference channels C = Sss −1/2 SsrSrr −H/2 . More specifically, from the singular value de- In summary, any invariant detector for our problem must be a function of the canonical correlations. This function, however, will depend in general on the unknown parameters, and therefore the uniformly most powerful invariant (UMPI) test is unlikely to exist.

Derivation of the GLR test
A conventional approach when the UMPI test does not exist, is the generalized likelihood ratio test (GLRT), in which the unknown parameters are replaced by their maximum likelihood estimates under each hypothesis where R0 and R1 are matrices with the block structure shown in (4) and (5), respectively. It is known that the GLRT is invariant to transformations for which the detection problem itself is invariant [21]. This means that the GLRT must be also a function of the canonical correlations. More specifically, we have the following result, which is the main contribution of this paper.
Theorem 1. The GLRT for H0 ∶ R = R0 versus H1 ∶ R = R1 can be written as follows where k1 is the maximum sample canonical correlation between the surveillance and reference channels, and η is a suitable threshold.
Proof. Under H0, the composite covariance R0 in (4) has a blockdiagonal structure where the blocks have no particular structure apart from being positive definite matrices. Then, it is well known that its ML estimate isR0 = diag M (S), and the compressed likelihood under H0 reduces to On the other hand, to find the ML estimate of R1 amounts to solving the following optimization problem where G denotes the set of pd matrices with the required structure: arbitrary positive definite blocks along the main diagonal and rankone blocks along the antidiagonal, as in (5). We can now apply block diagonal transformations to S andR1 such that (14) where K = diag (k1, ⋯, k M ) contains the sample canonical correlations, and J = diag (σ1, 0, ⋯, 0) is a diagonal matrix with a single non-zero element on its diagonal satisfying 0 < σ1 ≤ 1. Notice also that Ls = S 1/2 ss F and Lr = S 1/2 rr G, where F and G contain the left and right singular vectors of C.
Substituting (13) and (14) into (12), we obtain the following equivalent optimization problem where T is the transformation group defined in (8). Defining P H = T −1 L, which also belongs to the transformation group T , problem (15) can be reformulated as After some algebra we find that Λ −1 1 can be written as where D = diag (− σ 1 (1−σ 2 1 ) , 0, ⋯, 0). Plugging (17) into (16) yields the following problem where ps and pr are the first columns of Ps and Pr, respectively. Solving (18) with respect to Ps and Pr, it is easy to see that they should be unitary matrices. The optimization with respect to σ1 shows that we have to take Ps = Pr =I M and σ1 = k1, thus making the last term in (18) It is now easy to check that the compressed likelihood under H1 reduces to and, finally, the likelihood ratio is which proves Theorem 1.
In comparison to previous works on detecting correlation in multi-channel time series, the main difference of our work lies on the fact that the anti-diagonal blocks of the composite matrix under the alternative for our model,Rsr andRrs, are rank-one matrices. Otherwise, if the covariance matrix under H1 were an arbitrary pd matrix, the GLR statistic would be the following generalized Hadamard ratio [22], [23] Notice also that 1 − H is in fact the Generalized Coherence (GC) originally defined in [24], and widely applied to multi-channel detection problems since then.

Distribution under the null hypothesis
Under H0, the joint density of the squared canonical correlations was derived in 1939 by Hsu, [25]. However, to obtain the marginal distribution of k 2 1 from the joint density is not an easy task in general, and no simple closed-form expression exists. In this work, we exploit recent results from random matrix theory that provide the asymptotic distribution of the largest squared canonical correlation when N → ∞, M → ∞ with M /N → p. Specifically, we have the following result adapted from [26].
The accuracy of the Tracy-Widom approximation is verified in Fig. 1, which shows the normalized histogram of 10 5 realizations of the random variable

SIMULATION RESULTS
In this section we evaluate the performance of the GLR detector by means of Monte Carlo simulations. According to our model, the additive noise follows a Gaussian distribution with an arbitrary, randomly generated, covariance matrices (Σss, Σrr). For each realization we scale the channels so that the signal-to-noise-ratio (SNR), defined as SNRi = 10 log 10 (h H i hi/ tr(Σii)), i = {s, r}, is constant. For comparison, we have also included the performance of three other detectors: 1. Generalized Coherence (GC) detector [22]- [24].
2. Covariance matching detector: this detector uses structured estimates for R0 and R1 that minimize the Frobenius norm between the sample covariance and the estimate: ||S − R|| 2 F . 3. Cross-correlation (CC) detector, which uses | tr(S H sr Ssr)| as statistic. This is an extension to the multiantenna case of the CC detector typically used in passive radar systems [3].
For the considered scenario, the entries of hs and hr are randomly drawn from a zero-mean Gaussian distribution with unit variance, and scaled to achieve the desired SNR. The probability of false alarm for all detectors is fixed to P fa = 1e − 3. The threshold for the GLRT that achieves the required P fa is obtained using the Tracy-Widom approximation described in subsection 3.3, whereas for the GC detector we use the null distribution derived in [23]. For the covariance matching and the cross-correlation detectors the threshold is determined empirically using 10 5 random realizations under H0. Moreover, for each SNR and for each realization of the unknowns (hs, hr, Σss, Σrr), the probability of detection is estimated from 2000 independent simulations.
The first example involves a scenario with M = 4 antennas and N = 100 snapshots. Fig. 2 depicts the probability of detection P d versus the SNR (we assume that SNR is the same for both channels) for the considered detectors. The GLRT performs only slightly better than the GC detector, and both clearly improve on the covariance matching and the CC detectors. Fig. 3 shows the results obtained for a scenario with M = 8 antennas and N = 250 snapshots, where we can observe that the relative advantage of the GLRT, which exploits the rank-one structure of the covariance matrix, with respect to the GC detector becomes more important as the number of antennas increases.

CONCLUSIONS
This paper addressed the problem of detecting a rank-one signal in a two-channel passive network when the noise covariance matrices at both the surveillance and the reference channels are arbitrary positive definite matrices. The GLRT for this problem is given by the largest empirical canonical correlation between the surveillance and the reference channels. Recent results from random matrix theory show that the limiting distribution of the test statistic under the null converges to a Tracy-Widom law of order 2, which allowed us to set the threshold for a given false alarm probability.