공분산 행렬의 반전이 왜 랜덤 변수 사이의 부분 상관을 생성합니까?

공분산 행렬을 반전시키고 결과 정밀도 행렬에서 적절한 셀을 가져 와서 무작위 변수 간의 부분 상관 관계를 찾을 수 있다고 들었습니다 (이 사실은 http://en.wikipedia.org/wiki/Partial_correlation 에 있지만 증거는 없습니다) .

왜 그런가요?

— 매스
소스

다른 모든 변수에 대해 제어 된 셀에서 부분 상관 관계를 얻으려는 경우 여기 의 마지막 단락에 빛이 비칠 수 있습니다.

— ttnphns

답변:

다변량 랜덤 변수 $(X_1,X_2,\ldots,X_n)$ 에 비 변성 공분산 행렬 $\mathbb{C} = (\gamma_{ij}) = (\text{Cov}(X_i,X_j))$ , 모든 실제 선형 조합 세트 $X_i$ 형성 $n$ 기준으로 실제 차원 벡터 공간 및 비 분해성 내적 $E=(X_1,X_2,\ldots, X_n)$

⟨ X_{i}, X_{j} ⟩ = γ_{i j} .

$\langle X_i,X_j \rangle = \gamma_{ij}\ .$

그 이중 기준 이 내적에 대하여 , 고유의 관계에 의해 정의되고 $E^{*} = (X_1^{*},X_2^{*}, \ldots, X_n^{*})$

⟨ X_{i}^{*}, X_{j} ⟩ = δ_{i j},

$\langle X_i^{*}, X_j \rangle = \delta_{ij}\ ,$

크로네 커 델타 ( 및 경우 과 동일) $1$ $i=j$ $0$ 기타).

의 부분 상관 관계 때문에 이중으로 여기에 관심있는 와 부분 사이의 상관 관계로 얻을 수있다 공간으로 투사가 다른 모든 벡터에 의해 스팬 후 남아 (의는 단순히 전화를하자 그 " 잔차 ", ) 및 의 비교 가능한 부분 , 그의 잔류 . 그러나 또한 모든 벡터에 직교하는 벡터이며 및 긍정적 내적 갖는 어디서 $X_i$ $X_j$ $X_i$ $X_{i\circ}$ $X_j$ $X_{j\circ}$ $X_i^{*}$ $X_i$ $X_i$ 의 음이 아닌 배수 여야하며 경우에도 마찬가지입니다. 그러므로 우리가 쓰자 $X_{i\circ}$ $X_i^{*}$ $X_j$

X_{i \circ} = λ_{i} X_{i}^{*}, X_{j \circ} = λ_{j} X_{j}^{*}

$X_{i\circ} = \lambda_i X_i^{*},\ X_{j\circ} = \lambda_j X_j^{*}$

양의 실수 번호에 대한 및 . $\lambda_i$ $\lambda_j$

부분 상관은 잔차의 정규화 된 내적이며, 크기 조정에 의해 변경되지 않습니다.

ρ_{i j \circ} = \frac{⟨ X_{i \circ}, X_{j \circ} ⟩}{\sqrt{⟨ X_{i \circ}, X_{i \circ} ⟩ ⟨ X_{j \circ}, X_{j \circ} ⟩}} = \frac{λ_{i} λ_{j} ⟨ X_{i}^{*}, X_{j}^{*} ⟩}{\sqrt{λ_{i}^{2} ⟨ X_{i}^{*}, X_{i}^{*} ⟩ λ_{j}^{2} ⟨ X_{j}^{*}, X_{j}^{*} ⟩}} = \frac{⟨ X_{i}^{*}, X_{j}^{*} ⟩}{\sqrt{⟨ X_{i}^{*}, X_{i}^{*} ⟩ ⟨ X_{j}^{*}, X_{j}^{*} ⟩}} .

$\rho_{ij\circ} = \frac{\langle X_{i\circ}, X_{j\circ} \rangle}{\sqrt{\langle X_{i\circ}, X_{i\circ} \rangle\langle X_{j\circ}, X_{j\circ} \rangle}} = \frac{\lambda_i\lambda_j\langle X_{i}^{*}, X_{j}^{*} \rangle}{\sqrt{\lambda_i^2\langle X_{i}^{*}, X_{i}^{*} \rangle\lambda_j^2\langle X_{j}^{*}, X_{j}^{*} \rangle}} = \frac{\langle X_{i}^{*}, X_{j}^{*} \rangle}{\sqrt{\langle X_{i}^{*}, X_{i}^{*} \rangle\langle X_{j}^{*}, X_{j}^{*} \rangle}}\ .$

(두 경우 모두 잔차가 0이 아닌지 여부에 관계없이 잔차가 직교 할 때마다 부분 상관 관계는 0이됩니다.)

이중 기본 요소의 내부 제품을 찾아야합니다. 이를 위해 원래 기준 이중 기본 요소를 확장하십시오 . $E$

X_{i}^{*} = \sum_{j = 1}^{n} β_{i j} X_{j} .

$X_i^{*} = \sum_{j=1}^n \beta_{ij} X_j\ .$

그런 다음 정의에 따라

δ_{i k} = ⟨ X_{i}^{*}, X_{k} ⟩ = \sum_{j = 1}^{n} β_{i j} ⟨ X_{j}, X_{k} ⟩ = \sum_{j = 1}^{n} β_{i j} γ_{j k} .

$\delta_{ik} = \langle X_i^{*}, X_k \rangle = \sum_{j=1}^n \beta_{ij}\langle X_j, X_k \rangle = \sum_{j=1}^n \beta_{ij}\gamma_{jk}\ .$

의 항등 행렬과 의 행렬 변화 표기법을 사용하여 행렬 표기법에서 $\mathbb{I} = (\delta_{ij})$ $\mathbb{B} = (\beta_{ij})$

I = B C .

$\mathbb{I} = \mathbb{BC}\ .$

That is, $\mathbb{B} = \mathbb{C}^{-1}$ , which is exactly what the Wikipedia article is asserting. The previous formula for the partial correlation gives

ρ_{i j \cdot} = \frac{β_{i j}}{\sqrt{β_{i i} β_{j j}}} = \frac{C_{i j}^{- 1}}{\sqrt{C_{i i}^{- 1} C_{j j}^{- 1}}} .

$\rho_{ij\cdot} = \frac{\beta_{ij}}{\sqrt{\beta_{ii} \beta_{jj}}} = \frac{\mathbb{C}^{-1}_{ij}}{\sqrt{\mathbb{C}^{-1}_{ii} \mathbb{C}^{-1}_{jj}}}\ .$

— whuber
소스

+1, great answer. But why do you call this dual basis "dual basis with respect to this inner product" -- what does "with respect to this inner product" exactly mean? It seems that you use the term "dual basis" as defined here mathworld.wolfram.com/DualVectorSpace.html in the second paragraph ("Given a vector space basis

v_{1}, . . ., v_{n}

$v_1, ..., v_n$ for

V

$V$ there exists a dual basis...") or here en.wikipedia.org/wiki/Dual_basis, and it's independent of any scalar product.

— amoeba says Reinstate Monica

@amoeba There are two kinds of duals. The (natural) dual of any vector space

V

$V$ over a field

R

$R$ is the set of linear functions

ϕ : V \to R

$\phi:V\to R$ , called

V^{*}

$V^*$ . There is no canonical way to identify

V^{*}

$V^*$ with

V

$V$ , even though they have the same dimension when

V

$V$ is finite-dimensional. Any inner product

γ

$\gamma$ corresponds to such a map

g : V \to V^{*}

$g:V\to V^*$ , and vice versa, via

g (v) (w) = γ (v, w) .

$g(v)(w)=\gamma(v,w).$ (Nondegeneracy of

γ

$\gamma$ ensures

g

$g$ is a vector space isomorphism.) This gives a way to view elements of

V

$V$ as if they were elements of the dual

V^{*}

$V^*$ --but it depends on

γ

$\gamma$ .

— whuber

@mpettis Those dots were hard to notice. I have replaced them with small open circles to make the notation easier to read. Thanks for pointing this out.

— whuber

@Andy Ron Christensen's Plane Answers to Complex Questions might be the sort of thing you are looking for. Unfortunately, his approach makes (IMHO) undue reliance on coordinate arguments and calculations. In the original introduction (see p. xiii), Christensen explains that's for pedagogical reasons.

— whuber

@whuber, Your proof is awesome. I wonder whether any book or article contains such a proof so that I can cite.

— Harry

Here is a proof with just matrix calculations.

I appreciate the answer by whuber. It is very insightful on the math behind the scene. However, it is still not so trivial how to use his answer to obtain the minus sign in the formula stated in the wikipediaPartial_correlation#Using_matrix_inversion.

ρ_{X_{i} X_{j} \cdot V ∖ {X_{i}, X_{j}}} = - \frac{p_{i j}}{\sqrt{p_{i i} p_{j j}}}

$\rho_{X_iX_j\cdot \mathbf{V} \setminus \{X_i,X_j\}} = - \frac{p_{ij}}{\sqrt{p_{ii}p_{jj}}}$

To get this minus sign, here is a different proof I found in "Graphical Models Lauriten 1995 Page 130". It is simply done by some matrix calculations.

The key is the following matrix identity:

{(\begin{matrix} A & B \\ C & D \end{matrix})}^{- 1} = (\begin{matrix} E^{- 1} & - E^{- 1} G \\ - F E^{- 1} & D^{- 1} + F E^{- 1} G \end{matrix})

$\begin{pmatrix} A & B \\ C & D \end{pmatrix}^{-1} = \begin{pmatrix} E^{-1} & -E^{-1}G \\ -FE^{-1} & D^{-1}+FE^{-1}G \end{pmatrix}$ where

E = A - B D^{- 1} C

$E = A - BD^{-1}C$ ,

F = D^{- 1} C

$F = D^{-1}C$ and

G = B D^{- 1}

$G = BD^{-1}$ .

Write down the covariance matrix as

Ω = (\begin{matrix} Ω_{11} & Ω_{12} \\ Ω_{21} & Ω_{22} \end{matrix})

$\Omega = \begin{pmatrix} \Omega_{11} & \Omega_{12} \\ \Omega_{21} & \Omega_{22} \end{pmatrix}$ where

Ω_{11}

$\Omega_{11}$ is covariance matrix of

(X_{i}, X_{j})

$(X_i, X_j)$ and

Ω_{22}

$\Omega_{22}$ is covariance matrix of

V ∖ {X_{i}, X_{j}}

$\mathbf{V} \setminus \{X_i, X_j \}$ .

Let $P = \Omega^{-1}$ . Similarly, write down $P$ as

P = (\begin{matrix} P_{11} & P_{12} \\ P_{21} & P_{22} \end{matrix})

$P = \begin{pmatrix} P_{11} & P_{12} \\ P_{21} & P_{22} \end{pmatrix}$

By the key matrix identity,

P_{11}^{- 1} = Ω_{11} - Ω_{12} Ω_{22}^{- 1} Ω_{21}

$P_{11}^{-1} = \Omega_{11} - \Omega_{12}\Omega_{22}^{-1}\Omega_{21}$

We also know that $\Omega_{11} - \Omega_{12}\Omega_{22}^{-1}\Omega_{21}$ is the covariance matrix of $(X_i, X_j) | \mathbf{V} \setminus \{X_i, X_j\}$ (from Multivariate_normal_distribution#Conditional_distributions). The partial correlation is therefore

ρ_{X_{i} X_{j} \cdot V ∖ {X_{i}, X_{j}}} = \frac{[P_{11}^{- 1}]_{12}}{\sqrt{[P_{11}^{- 1}]_{11} [P_{11}^{- 1}]_{22}}} .

$\rho_{X_iX_j\cdot \mathbf{V} \setminus \{X_i,X_j\}} = \frac{[P_{11}^{-1}]_{12}}{\sqrt{[P_{11}^{-1}]_{11}[P_{11}^{-1}]_{22}}}.$ I use the notation that the

(k, l)

$(k,l)$ th entry of the matrix

M

$M$ is denoted by

[M]_{k l}

$[M]_{kl}$ .

Just simple inversion formula of 2-by-2 matrix,

(\begin{matrix} [P_{11}^{- 1}]_{11} & [P_{11}^{- 1}]_{12} \\ [P_{11}^{- 1}]_{21} & [P_{11}^{- 1}]_{22} \end{matrix}) = P_{11}^{- 1} = \frac{1}{det P_{11}} (\begin{matrix} [P_{11}]_{22} & - [P_{11}]_{12} \\ - [P_{11}]_{21} & [P_{11}]_{11} \end{matrix})

$\begin{pmatrix} [P_{11}^{-1}]_{11} & [P_{11}^{-1}]_{12} \\ [P_{11}^{-1}]_{21} & [P_{11}^{-1}]_{22} \\ \end{pmatrix} = P_{11}^{-1} = \frac{1}{\text{det} P_{11}} \begin{pmatrix} [P_{11}]_{22} & -[P_{11}]_{12} \\ -[P_{11}]_{21} & [P_{11}]_{11} \\ \end{pmatrix}$

Therefore,

ρ_{X_{i} X_{j} \cdot V ∖ {X_{i}, X_{j}}} = \frac{[P_{11}^{- 1}]_{12}}{\sqrt{[P_{11}^{- 1}]_{11} [P_{11}^{- 1}]_{22}}} = \frac{- \frac{1}{det P_{11}} [P_{11}]_{12}}{\sqrt{\frac{1}{det P_{11}} [P_{11}]_{22} \frac{1}{det P_{11}} [P_{11}]_{11}}} = \frac{- [P_{11}]_{12}}{\sqrt{[P_{11}]_{22} [P_{11}]_{11}}}

$\rho_{X_iX_j\cdot \mathbf{V} \setminus \{X_i,X_j\}} = \frac{[P_{11}^{-1}]_{12}}{\sqrt{[P_{11}^{-1}]_{11}[P_{11}^{-1}]_{22}}} = \frac{- \frac{1}{\text{det}P_{11}}[P_{11}]_{12}}{\sqrt{\frac{1}{\text{det}P_{11}}[P_{11}]_{22}\frac{1}{\text{det}P_{11}}[P_{11}]_{11}}} = \frac{-[P_{11}]_{12}}{\sqrt{[P_{11}]_{22}[P_{11}]_{11}}}$ which is exactly what the Wikipedia article is asserting.

— Po C.
소스

If we let i=j, then rho_ii V\{X_i, X_i} = -1, How do we interpret those diagonal elements in the precision matrix?

— Jason

Good point. The formula should be only valid for i=/=j. From the proof, the minus sign comes from the 2-by-2 matrix inversion. It would not happen if i=j.

— Po C.

So the diagonal numbers can't be associated with partial correlation. What do they represent? They are not just inverses of the variances, are they?

— Jason

This formula is valid for i=/=j. It is meaningless for i=j.

— Po C.

Note that the sign of the answer actually depends on how you define partial correlation. There is a difference between regressing $X_i$ and $X_j$ on the other $n - 1$ variables separately vs. regressing $X_i$ and $X_j$ on the other $n - 2$ variables together. Under the second definition, let the correlation between residuals $\epsilon_i$ and $\epsilon_j$ be $\rho$ . Then the partial correlation of the two (regressing $\epsilon_i$ on $\epsilon_j$ and vice versa) is $-\rho$ .

This explains the confusion in the comments above, as well as on Wikipedia. The second definition is used universally from what I can tell, so there should be a negative sign.

I originally posted an edit to the other answer, but made a mistake - sorry about that!

— Johnny Ho
소스