SEM-slide-eng-part0-IV.knit

background-image: url("../pic/slide-front-page.jpg")
class: center,middle
count: false

# Advanced Econometrics III

## (高级计量经济学III 全英文)

### Hu Huaping (胡华平 )

### NWAFU (西北农林科技大学)

### School of Economics and Management (经济管理学院)

### huhuaping01 at hotmail.com

### 2023-04-06

<div>
<style type="text/css">.xaringan-extra-logo {
width: 110px;
height: 70px;
z-index: 0;
background-image: url(../pic/logo/nwafu-logo-circle-wb.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
top:0.2em;left:1em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('div')
          logo.classList = 'xaringan-extra-logo'
          logo.href = null
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

???
Good evening everyone. Welcome to my class.

I am teacher Hu Huaping.

Here is my email.

You can contact me when you have any questions with our course.

In this part, we will learn Simultaneous Equation Models together by using almost eight lessons.

---
count: false
class: center, middle, duke-orange,hide_logo

# Part 2：Simultaneous Equation Models (SEM)

Chapter 18. Why Should We Concern SEM ?

Chapter 19. What is the Identification Problem ?

Chapter 20. How to Estimate SEM ?

]

???

As you see this part contains four chapters.

Firstly, we will go through Chapter 17. Regressor Endogeneity problems and instrumental Variables solutions will be discussed in this chapter. Anyway, this chapter will be a good start for learning SEM.

The next three chapters focus closely on SEM. We will answer three important questions in turn.

In Chapter 18, we will know SEM is important in social science and it also brings new challenges to us.

Large SEM system always contains lots of parameters need to be solved and will face with the identification problems. We will give you guides and rules to check the SEM identification status in Chapter 19.

Finally, we will discuss different SEM estimation approaches in Chapter 20, including 2SLS, Three-stage least squares (3SLS) and full information maximum likelihood (FIML)  method.

---
layout: false
class: center, middle, duke-softblue,hide_logo
name: chapter17

# Chapter 17. Endogeneity and Instumental Variables

[17.1 Definition and source of endogeneity](#definition)

[17.2 Estimation problem with endogeneity](#problem)

[17.3 IV and choices](#IV)

[17.4 Two-stage least squares method](#TSLS)

[17.5 Testing instrument validity](#validity)

[17.6 Testing regressor endogeneity](#endogeneity)

???
[source](https://web.sgh.waw.pl/~mrubas/AdvEcon/pdf/T2_Endogeneity.pdf)

So let us start the first chapter.

In this chapter:

- You will see how the method of **instrumental variables** (IV) can be used to solve the problem of **endogeneity** due to one or more regressors.

- Also we will learn the method of **two stage least squares** in section 17.4. 2SLS method is second in popularity only to ordinary least squares for estimating linear equations in econometrics.

- And some useful testing techniques will be introduced to check instrument validity and regressor endogeneity. These content will be uncovered in the last two sections.

---
layout: false
class: center, middle, duke-softblue,hide_logo
name: definition

## 17.1 Definition and sources of endogeneity

???
In this section we will talk about the definition of endogeneity and its main sources.

The story begins with the stochastic regressors.

---
layout: true
  
<div class="my-header-h2"></div>
<div class="watermark1"></div>
<div class="watermark2"></div>
<div class="watermark3"></div>

<div class="my-footer"><span>huhuaping@  <a href="#chapter17"> Chapter 17. Endogeneity and Instumental Variables  |</a>
&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;
<a href="#definition"> 17.1 Definition and source of endogeneity </a> </span></div>

---

### Review: the CLRM assumptions

Let's revise the classic linear regression model   assumptions(CLRM):

- **A1**: The true model is 
`$\boldsymbol{y= X \beta+\epsilon}$`.

- **A2**: 
`$\forall i, E(\epsilon_i|X) = 0$` (conditional zero mean)  more about this later.

- **A3**: 
`$Var(\epsilon|X) = E(\epsilon \epsilon^{\prime}|X) = \sigma^2I$` (identical conditional variance ).

- **A4**: 
`$\boldsymbol{X}$` has full column rank.

- **A5**: (for inference purposes, 
`$\boldsymbol{\epsilon} \sim  N(\boldsymbol{0}, σ^2\boldsymbol{I}))$`.

Under **A1-A4**（also namely **CLRM**）, the OLS is **Gauss-Markov** efficient.

Under **A1-A5**, we donote **N-CLRM**.

???
we have discussed these assumptions before the class. And we denote the Gauss-Markov model assumption system as classical linear regression model (CLRM).

- **Assumption 1**: The  model is assumed to be true and be linear with parameters.

- **Assumption 2**: the disturbance term has conditional zero expectation. And we will relax this assumption later.

- **Assumption 3**: this means the conditional variance of disturcances is identity given X. the other way to express assumption 3 is that the  disturbance term is homoscedasticity['hɔməusi,dæs'tisəti].

- **Assumption 4**: The regressor matrix X has full column rank. Which means that none of the regressors is the perfect linear function on the rest of the regressors.

- **Assumption 5**: We will bring in the norm distribution assumption on the disburtance. This assumption aims for hypothesis testing purposes.

Joint with A2, A3, and A5, the disturbance is assumed to be normally identical independent distribution. We just write i.i.d for shortly.

With assumptions **A1-A4**, We note **CLRM**. Under CLRM the OLS is **Gauss-Markov** efficient. Which means under CLRM, when we use OLS method we will obtain the Best-Linear-unbiased estimators on the true parameters. [hand writing]

While, with assumptions **A1-A5**, we used to denote as N-CLRM. we can conduct kinds of inference and tests based on N-CLRM.

Now let us focus on assumption 2.

---

### Review: the A2 assumptions

For the population regression model(PRM):

`$$\begin{align}
Y_i &=  \beta_1 +\beta_2X_i + u_i && \text{(PRM)}
\end{align}$$`

- **CLRM A2** assumes that X is fixed (given) or independent of the error term. The regressors 
`$X$` **are not** random variables. At this point, we can use the OLS method and get **BLUE**(Best Linear Unbiased Estimator).

`$$\begin{align}
Cov(X_i, u_i)= 0; \quad
E(X_i u_i)= 0
\end{align}$$`

- If the above A2 assumption is violated, the independent variable 
`$X$` is related to the random disturbance term. In this case, OLS estimation will no longer get **BLUE**, and **instrumental variable method** (IV) should be used for estimation.

`$$\begin{align}
Cov(X_i, u_i) \neq 0 ; \quad
E(X_i u_i) \neq 0
\end{align}$$`

???

If A2 is violated, then the endogeneity problem arise.

Now we go head to define the endogenous variable.

事实上，无论
`$X_i$`与
`$u_i$`是否相关，我们都可以采用**IV法**得到**BLUE**。

---

###  Good model: random experiment

**Randomized controlled experiment **: Ideally, the value of the independent variable 
`$X$` is randomly changed (refer to the **reason**), and then we look at the change in the dependent variable 
`$Y$` (refer to the **result**).

`$$\begin{equation}
\boldsymbol{y= X \beta+u}
\end{equation}$$`

- If
`$Y_i$` and
`$X_i$` does exist systematic relationship (linear relationship), then change
`$X_i$` causes the corresponding change of 
`$Y_i$`.

- Any other random factors will be added to the random disturbance
`$u_i$`. The effect of the random disturbance on the change of `$Y_i$` should be **independent** to the effect of 
`$X_i$` on the change of 
`$Y_i$`.

---

###  Good model: exogenous regressors

**Exogenous regressors**: If independent variables
`$X_i$` is **perfectly random** (randomly assigned) as mentioned above , then they are called **exogenous regressor**. More precisely, they can be defined as:

**Strictly Exogeneity**：

`$$E\left(u_{i} \mid X_{1}, \ldots, X_{N}\right)=E\left(u_{i} \mid \mathbf{x}\right)=0$$`

]

???

因为在**随机控制实验**，给定样本
`$i$`和样本
`$j$`，自变量的取值分别为
`$X_i$`和
`$X_j$`，它们应该是相互独立的。因此可以把上述假设进一步简化为：

> **同期外生性假设**（contemporaneously exogeneity）：
`$E(u_i|X_{i})=0, \text{for } i =1, \ldots,N$`。

**同期**表示的是在同样的时间点上，或所有的截面观测单位在某个时间上获得的观测。

大样本情况下OLS方法

在大样本情形下，上述**严格外生性假设**可以进一步转换为**同期不相关假设**：

- `$E(u_i)=0$`，而且

- `$\operatorname{cov}(X_i, u_i)=0$`

因为我们可以证明（证明略），在大样本情况OLS方法下：

- `$E(u_i|X_{i})=0 \quad \Rightarrow \quad E(u_i)=0$`

- `$E(u_i|X_{i})=0 \quad \Rightarrow \quad \operatorname{cov}(X_i, u_i)=0$`

---

### Endogeneity: definition

We use the term **endogeneity** frequently in econometrics.

Also this concept is used broadly to describe any situation where a regressor is **correlated** with the error term.

- Assume that we have the bivariate linear model

`$$\begin{equation}
Y_{i}=\beta_{0}+\beta_{1} X_{i}+\epsilon_{i}
\end{equation}$$`

- The explanatory variable 
`$X$` is said to be **.red[En]dogenous** if it is correlated
with 
`$\epsilon$`.

`$$\begin{align}
Cov(X_i, \epsilon_i) \neq 0 ; \quad
E(X_i \epsilon_i) \neq 0
\end{align}$$`

- And if 
`$X$` is **uncorrelated** with 
`$\epsilon$`, it is said to be **.red[Ex]ogenous**.

???
Next, we should ask what the sources of endogeneity come from.

---

### Endogeneity: sources

In applied econometrics, endogeneity usually arises in one of four ways:

- **Omitted variables**: when the model is set incorrectly.

- **Measurement errors** in the regressors.

- **Autocorrelation** of the error term in autoregressive models.

- **Simultaneity**: when 
`$\boldsymbol{Y}$` and 
`$\boldsymbol{X}$` are simultaneously determined, as in the supply/demand model (we will go to explain it in the next three chapter).

???

So, let us go through all of these four situations quickly.

---

### Source 1: Omitted variables

Suppose that the **"assumed true model"** for wage determination is:

`$$\begin{align}
Wage_{i}=\beta_{1}+\beta_{2} Edu_{i}+\beta_{3} Abl_{i}+\epsilon_{i}  \quad \text{(the assumed true model)}
\end{align}$$`

However, because the individual's **ability variable**(
`$Abl$`) is often not directly observed, so we often can't put it into the model, and build a **mis-specified model**.

`$$\begin{align}
Wage_{i}=\alpha_{1}+\alpha_{2} Edu_{i}+v_{i}  \quad \text{(the mis-specified model)}
\end{align}$$`

- Where **ability variable**(
`$Abl$`) is included in the new disturbance
`$v_i$`, and
`$v_{i}=\beta_{3} abl_{i}+u_{i}$`

- Obviously, in the mis-specified model, we ignore the **ability variable**(
`$Abl$`), while  variable **years of education**(
`$Edu$`) is actually related to it.

- So in the mis-specified  model, 
`$cov(Edu_i, v_i) \neq 0$`, thus **years of education**(
`$Edu$`) may cause the endogenous problem.

???

why the variables are Omitted?  Because people run a model without considering the necessary  or important variables we should.

We will explain later that **Omission** of relevant variable will result in **inconsistent estimates** because **A2** is not hold.

Now, let us talk about the second source of endogeneity. the Measurement error in the regressors.

---

### Source 1：Omitted variables (demo1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-04.png)

---

### Source 1：Omitted variables (demo1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-03.png)

---

### Source 1：Omitted variables (demo1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-02.png)

---

### Source 1：Omitted variables (demo1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-01.png)

---

### Source 1：Omitted variables  (demo2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-part-05.png)

---

### Source 1：Omitted variables  (demo2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-part-04.png)

---

### Source 1：Omitted variables  (demo2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-part-03.png)

---

### Source 1：Omitted variables  (demo2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-part-02.png)

---

### Source 1：Omitted variables  (demo2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ommit-sequence/chpt11-ommit-sequence-part-01.png)

---

### Source 2: Measurement errors

Again, let's consider the **"assumed true model"**:

`$$\begin{align}
Wage_{i}=\beta_{1}+\beta_{2} Edu_{i}+\beta_{3} Abl_{i}+\epsilon_{i}  \quad \text{(the assumed true model)}
\end{align}$$`

It is hard to observe individual's **ability variable**(
`$Abl$`), and somebody will instead to use the variable **IQ score**(
`$IQ_i$`), and construct the  mis-specified **"proxy variable" model**:

`$$\begin {align} 
Wage_i=\alpha_{1}+\alpha_{2} Edu_i+\alpha_{3} IQ_i+v_i \quad \text{(the mis-specified model)}
\end {align}$$`

- It should exist stuffs (
`$Abl\_other_i$`) which the model does not include (due to the measurement error). So the measurement errors (
`$Abl\_other_i$`) will go to the disturbance term 
`$v_i$` in the mis-specified model.

- And we know that measurement errors (
`$Abl\_other_i$`) will be correlated with the education variable. Thus `$cov(Edu_i, v_i) \neq 0$`, and the **education variable**(
`$Edu$`) may cause the endogenouse problem.

???

In practice, the observation value of a regressor is not accurate, in most time is only the "approximation".So there is **measurement error** in the regressor in our model.

---

### Source 2: Measurement errors (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-04.png)

---

### Source 2: Measurement errors (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-03.png)

---

### Source 2: Measurement errors (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-02.png)

---

### Source 2: Measurement errors (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-01.png)

---

### Source 2: Measurement errors (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-part-06.png)

---

### Source 2: Measurement errors (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-part-05.png)

---

### Source 2: Measurement errors (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-part-04.png)

---

### Source 2: Measurement errors (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-part-03.png)

---

### Source 2: Measurement errors (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-part-02.png)

---

### Source 2: Measurement errors (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-measure-sequence/chpt11-measure-sequence-part-01.png)

---

### Source 3: Autocorrelation

**Autoregressive model**: Lag variable of dependent variable(
`$Y_{t-1}, \ldots, Y_{t-p},\ldots$`) appears in the model as regressors.

`$$\begin {align} 
Y_t=\beta_{1}+\beta_{2} Y_{t-1}+\beta_{3}X_t+u_t 
\end {align}$$`

If the disturbance term determined following a first-order autocorrelation AR(1):

`$$\begin {align} 
u_t=\rho u_{t-1}+ \epsilon_t 
\end {align}$$`

Then, it is obvious that
`$cov(Y_{t-1}, u_{t-1}) \neq 0$` and 
`$cov(Y_{t-1}, u_{t}) \neq 0$`.

Thus the **lag dependent variable**(
`$Y_{t-1}$`)  will cause the endogeneity problem in the **Autoregressive model**.

---

### Source 3: Autocorrelation (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-04.png)

---

### Source 3: Autocorrelation (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-03.png)

---

### Source 3: Autocorrelation (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-02.png)

---

### Source 3: Autocorrelation (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-01.png)

---

### Source 3: Autocorrelation (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-part2-05.png)

---

### Source 3: Autocorrelation (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-part2-04.png)

---

### Source 3: Autocorrelation (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-part2-03.png)

---

### Source 3: Autocorrelation (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-part2-02.png)

---

### Source 3: Autocorrelation (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-ar-sequence/chpt11-ar-sequence-part2-01.png)

---

### Source 4: Simultaneity

For the equations system of supply and demand:

`$$\begin{cases}
  \begin{align}
    \text { Demand: } & Q_{i}=\alpha_{1}+\alpha_{2} P_{i}+u_{d i} \\ 
    \text { Supply: } & Q_{i}=\beta_{1}+\beta_{2} P_{i} + u_{s i}
  \end{align}
\end{cases}$$`

As we all know, because of the price
`$P_i$` will both affect supply and the demand `$Q_i$`, And vice versa. There is a feedback cycle mechanism in this system.

So, we can get
`$cov(P_i, u_{di}) \neq 0$`, and
`$cov(P_i, u_{si}) \neq 0$`, which will cause the endogenous problem finally.

???
/ˌsɪmltəˈneɪət/

---

### Source 4: Simultaneity (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-04.png)

---

### Source 4: Simultaneity (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-03.png)

---

### Source 4: Simultaneity (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-02.png)

---

### Source 4: Simultaneity (demo 1)

Here we show a visual demonstration on this situation:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-01.png)

---

### Source 4: Simultaneity (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-part-05.png)

---

### Source 4: Simultaneity (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-part-04.png)

---

### Source 4: Simultaneity (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-part-03.png)

---

### Source 4: Simultaneity (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-part-02.png)

---

### Source 4: Simultaneity (demo 2)

An intuitive demonstration is show as follows:

![](pic/chpt11-endogeneity-sem-sequence/chpt11-sem-sequence-part-01.png)

---

## 17.2 Estimation problem with endogeneity

???
The question is what will happen on OLS estimators with model contains endogenous regressors.

So, let us illustrate these estimation problems in detail in this section.

we will discuss along two different situations.

---
layout: true
  
<div class="my-header-h2"></div>
<div class="watermark1"></div>
<div class="watermark2"></div>
<div class="watermark3"></div>

<div class="my-footer"><span>huhuaping@   <a href="#chapter17"> Chapter 17. Endogeneity and Instumental Variables  |</a> &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; <a href="#problem"> 17.2 Estimation problem with endogeneity </a></span></div>

---

### Inconsistent estimates with measurement error

Consider the simple regression model:

`$$\begin{align}
Y_{i}=\beta_{0}+\beta_{1} X_{i}+\epsilon_{i} \quad \text{(1)}
\end{align}$$`

We would like to measure the effect of the variable 
`$X$` on 
`$Y$`, but we
can observe only an imperfect measure of it (i.e. a **proxy variable**),
which is

`$$\begin{align}
X^{\ast}_{i}=X_{i} - v_{i} \quad \text{(2)}
\end{align}$$`

Where 
`$v_i$` is a random disturbance with mean 0 and variance 
`$\sigma_{v}^{2}$`.

Further, let's assume that 
`$X_i, \epsilon_i$` and 
`$v_i$` are **pairwise independent**.

???

In the second situation, OLS estimators will be also inconsistent with measurement error on regressors.

---

### Inconsistent estimates with measurement error

Given the assumed true model (1):

`$$\begin{align}
Y_{i} &=\beta_{0}+\beta_{1} X_{i}+\epsilon_{i} && \text{ eq(1) assumed true model } 
\end{align}$$`

with the proxy variable 
`$X^{\ast}_i$`, we may use the error specified model (4):

`$$\begin{align}
X^{\ast}_i &= X_i - v_i && \text{ eq(2) proxy variable }\\
X_i &= X^{\ast}_i + v_i && \text{ eq(3)}\\
Y_{i} &=\beta_{0}+\beta_{1} X^{\ast}_{i}+u_{i} && \text{ eq(4) error specified model} 
\end{align}$$`

We can substitute eq (3) into the model (1) to obtain eq(5)

`$$\begin{align}
Y_{i} =\beta_{0}+\beta_{1} X_{i}^{*}+\epsilon_{i}  
=\beta_{0}+\beta_{1}\left(X^{\ast}_{i} + v_{i}\right)+\epsilon_{i}  
=\beta_{0}+\beta_{1} X^{\ast}_{i}+\left(\epsilon_{i} + \beta_{1} v_{i}\right)  \quad \text{eq(5)}
\end{align}$$`

which means
`$u_{i}=\left(\epsilon_{i}+\beta_{1} v_{i}\right)$` in the error specified model. As we know, the OLS consistent estimator of 
`$\beta_{1}$` in the last equation 
 .red[**requires**]
`$\operatorname{Cov}\left(X^{\ast}_{i}, u_{i}\right)= 0$`.

???
For simple, we just replace the `$X_i$` in the assumed true model 1, with the proxy variable `$X^{\ast}_i$`, and get the error specified model 4.

Meanwhile the measurement error will fall into the composite error term `$u_i$`. As you see.

---

### Inconsistent estimates with measurement error

Note that 
`$E\left(u_{i}\right)=E\left(\epsilon_{i} +\beta_{1} v_{i}\right)=E\left(\epsilon_{i}\right)+\beta_{1} E\left(v_{i}\right)=0$`

However,

`$$\begin{aligned} 
  \operatorname{Cov}\left(X^{\ast}_{i}, u_{i}\right) &=E\left[\left(X^{\ast}_{i}-E\left(X^{\ast}_{i}\right)\right)\left(u_{i}-E\left(u_{i}\right)\right)\right] \\ &=E\left(X^{\ast}_{i} u_{i}\right) \\ 
  &=E\left[\left(X_{i}-v_{i}\right)\left(\epsilon_{i} +\beta_{1} v_{i}\right)\right] \\ 
  &=E\left[X_{i} \epsilon_{i}+\beta_{1} X_{i} v_{i}- v_{i}\epsilon_{i}- \beta_{1} v_{i}^{2}\right] 
  \leftarrow \quad \text{(pairwise independent)}  \\ 
  &=-E\left(\beta_{1} v_{i}^{2}\right) \\ 
  &=-\beta_{1} \operatorname{Var}\left(v_{i}\right) \\ 
  &=-\beta_{1} \sigma_{v_i}^{2} \neq 0 
\end{aligned}$$`

Thus, 
`$X$` in model (4) is .red[**endogenous**] and we expect the OLS estimator of 
`$\beta_{1}$` to be **inconsistent**.

???

So far as we know, both omitted variable(s) and error measurement will cause endogeneity, and result in violation of A2.

Let us sum up all these cases in general.

---

### OLS estimation: Violation of A2

In general, when **A2** is violated, we expect OLS estimates to be **biased**:

The OLS estimators of `$\hat{\beta}$` is

`$$\begin{align}
\widehat{\beta} &=\beta+\left(X^{\prime} X\right)^{-1} X^{\prime} \epsilon && \text{(6)}
\end{align}$$`

and we can take expectation on both sides.

`$$\begin{equation}
  \begin{aligned}
  E(\widehat{\beta}) &=\beta+E\left(\left(X^{\prime} X\right)^{-1} X^{\prime} \epsilon\right) \\
  &=\beta+E\left(E\left(\left(X^{\prime} X\right)^{-1} X^{\prime} \epsilon | X\right)\right) \\
  &=\beta+E\left(\left(X^{\prime} X\right)^{-1} X^{\prime} E(\epsilon | X)\right) \neq \beta
  \end{aligned}
\end{equation}$$`

If **A2** 
`$E(\epsilon | X) = 0$` is violated, which means 
`$E(\epsilon | X) \neq 0$`, the OLS estimator is **biased**.

???

The result will show that the OLS estimator is not equal to its true parameter if **A2** is violated.

> Here, you should know that the value take expectation on conditional expectation is equal to unconditional expectation.

---

### OLS estimation: consistency

Let's see under what conditions we can establish consistency.

`$$\begin{equation}
  \begin{aligned}
  p \lim \widehat{\beta} &=\beta+p \lim \left(\left(X^{\prime} X\right)^{-1} X^{\prime} \epsilon\right) 
  =\beta+p \lim \left(\left(\frac{1}{n} X^{\prime} X\right)^{-1} \frac{1}{n} X^{\prime} \epsilon\right) \\
  &=\beta+p \lim \left(\frac{1}{n} X^{\prime} X\right)^{-1} \times p \lim \left(\frac{1}{n} X^{\prime} \epsilon\right)
  \end{aligned}
\end{equation}$$`

By the WLLN (Weak Law of Large Numbers)

`$$\begin{equation}
\frac{1}{n} X^{\prime} \epsilon=\frac{1}{n} \sum_{i=1}^{n} X_{i} \epsilon_{i} \xrightarrow{p} E\left(X_{i} \epsilon_{i}\right)
\end{equation}$$`

Hence 
`$\widehat{\beta}$` is consistent if 
`$E\left(X_{i} \epsilon_{i}\right)=0$` for all `$i$`. The condition 
`$E\left(X_{i} \epsilon_{i}\right)=0$` is more likely to be satisfied than A2 
`$E(\epsilon | X) = 0$`. Thus, a large class of estimators that cannot be proved to be **unbiased** are **consistent**.

???

Let's see under what conditions we can establish consistency.

We can take the probability limits on both sides of equation (6).

we will derive that the probability limits of this part will equal to expectation of `$X_i$` times `$\epsilon_i$`

We have dived deeply with theory evidence.

Now, I will show you an example.

---

### Wage example: the origin model

Consider the following "error specified" wage model:

`$$\begin{align}
lwage_i = \beta_1 +\beta_2educ_i + \beta_3exper_i +\beta_4expersq_i + e_i
\end{align}$$`

The difficulty with this model is that the error term may include some unobserved attributes, such as personal **ability**, that determine both wage and education.

In other words, the independent variable  **educ**  is correlated with the error term. And it is endogenous variable.

> **Note**:

> We will use **years of schooling** as the proxy variable of **educ** in practice, and it surely bring in error measurement issues as we have mentioned.

???

Consider the following error specified wage model, which the logarithmic (ˌlɒɡəˈrɪðmɪk) wage is the function of education,labor market experience and its quadratic term.

As what we have discussed before. The difficulty with this model is that the error term may include some unobserved attributes, such as personal **ability**, that determine both wage and education.

> There is one more thing we should remind. We will use **years of schooling** as the proxy variable of **educ** in practice, and it surely bring in error measurement issues as we have mentioned.

Let us show all variables of our dataset.

---

### Wage example: All variables in dataset

With data set `wooldridge::mroz`, researchers were interest in the return to education for married women.

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-59ed014aaffec66b2ad5" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-59ed014aaffec66b2ad5">{"x":{"filter":"none","vertical":false,"caption":"<caption>variables and labels<\/caption>","data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22"],[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22],["educ","exper","expersq","fatheduc","lwage","motheduc","wage","age","city","faminc","hours","husage","huseduc","hushrs","huswage","inlf","kidsge6","kidslt6","mtr","nwifeinc","repwage","unem"],[" years of schooling"," actual labor mkt exper"," exper^2"," father's years of schooling"," log(wage)"," mother's years of schooling"," est. wage from earn, hrs"," woman's age in yrs"," =1 if live in SMSA"," family income, 1975"," hours worked, 1975"," husband's age"," husband's years of schooling"," hours worked by husband, 1975"," husband's hourly wage, 1975"," =1 if in lab frce, 1975","  kids 6-18","  kids  under 6 years"," fed. marg. tax rte facing woman"," (faminc - wage*hours)/1000"," rep. wage at interview in 1976"," unem. rate in county of resid."]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>index<\/th>\n      <th>vars<\/th>\n      <th>labels<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"tip","columnDefs":[{"className":"dt-center","targets":"_all"},{"visible":false,"targets":0},{"orderable":false,"targets":0}],"pageLength":7,"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[7,10,25,50,100]}},"evals":[],"jsHooks":[]}</script>

???

Other variables include father education, mother education, which are both measured by schooling years, etc (ɪt ˈsetərə).

---

### Wage example: Raw dataset

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-eb89a6e592b4a29d1a67" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-eb89a6e592b4a29d1a67">{"x":{"filter":"none","vertical":false,"caption":"<caption>dataset n=(428)<\/caption>","data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180","181","182","183","184","185","186","187","188","189","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204","205","206","207","208","209","210","211","212","213","214","215","216","217","218","219","220","221","222","223","224","225","226","227","228","229","230","231","232","233","234","235","236","237","238","239","240","241","242","243","244","245","246","247","248","249","250","251","252","253","254","255","256","257","258","259","260","261","262","263","264","265","266","267","268","269","270","271","272","273","274","275","276","277","278","279","280","281","282","283","284","285","286","287","288","289","290","291","292","293","294","295","296","297","298","299","300","301","302","303","304","305","306","307","308","309","310","311","312","313","314","315","316","317","318","319","320","321","322","323","324","325","326","327","328","329","330","331","332","333","334","335","336","337","338","339","340","341","342","343","344","345","346","347","348","349","350","351","352","353","354","355","356","357","358","359","360","361","362","363","364","365","366","367","368","369","370","371","372","373","374","375","376","377","378","379","380","381","382","383","384","385","386","387","388","389","390","391","392","393","394","395","396","397","398","399","400","401","402","403","404","405","406","407","408","409","410","411","412","413","414","415","416","417","418","419","420","421","422","423","424","425","426","427","428"],[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428],[1.2101536989212,0.328512102365494,1.51413774490356,0.0921233221888542,1.52427220344543,1.55648005008698,2.12025952339172,2.0596342086792,0.754336357116699,1.54489934444427,1.4019216299057,1.52427220344543,0.733953237533569,0.818369090557098,1.30283117294312,0.298028379678726,1.16760957241058,1.64383935928345,0.6931471824646,2.02193164825439,1.25424754619598,1.27295768260956,1.1786550283432,1.1786550283432,0.767558693885803,1.33181178569794,1.3862943649292,1.55326962471008,1.98181486129761,1.76936042308807,0.430807888507843,0.899754822254181,1.76662969589233,1.27295768260956,1.33678889274597,0.901704847812653,0.865123689174652,1.51184713840485,1.72602915763855,2.68314242362976,0.985294282436371,1.36593854427338,0.945033669471741,1.51237618923187,0.6931471824646,1.24478840827942,0.701164901256561,1.5198632478714,0.820968568325043,0.969831526279449,0.828508198261261,0.0943096429109573,0.162543892860413,0.470003634691238,0.629248440265656,1.39716017246246,2.26544380187988,2.0845410823822,1.52583885192871,0.762160062789917,1.48160457611084,1.26282644271851,0.999675571918488,1.83258152008057,2.47930765151978,1.27901530265808,1.93793559074402,1.0704528093338,1.12392258644104,1.32175588607788,1.74499976634979,1.30174362659454,1.64186644554138,2.10702013969421,1.46706759929657,1.60581135749817,-1.02973937988281,1.08768618106842,0,0.938208699226379,-0.150590375065804,0,1.07367050647736,1.26584839820862,0.486368983983994,2.12025952339172,1.12985253334045,0.993251800537109,1.65862798690796,0.347412198781967,1.56832420825958,0.510845601558685,0.114845432341099,-0.6931471824646,-0.336452275514603,1.02822554111481,1.58068859577179,0.555894613265991,0.901420712471008,0.884304583072662,0.428204596042633,1.05841505527496,0.87833958864212,1.65490829944611,1.32175588607788,0.328512102365494,1.3862943649292,1.17288458347321,1.22418713569641,0.287657082080841,2.23026180267334,1.50407743453979,1.53115200996399,1.37515759468079,1.76026880741119,-0.6931471824646,1.40648913383484,1.7917594909668,1.29929208755493,1.35100388526917,1.01628088951111,1.0753436088562,1.47896468639374,1.68948674201965,2.28859782218933,-1.82263112068176,-0.960765182971954,1.29099416732788,0.864871144294739,1.54045212268829,0.616212129592896,1.64865863323212,1.19349813461304,2.14397621154785,0.724403560161591,0.941607534885406,0.782759368419647,1.83258152008057,1.20396280288696,1.49164485931396,1.89213263988495,2.13089489936829,1.48060405254364,0.894331336021423,0.202532544732094,0.485507816076279,1.0986123085022,1.55326962471008,0.121597968041897,2.00180435180664,1.49503660202026,0.905229806900024,0.632547557353973,1.3862943649292,2.10291385650635,1.95964395999908,0.510845601558685,1.23692393302917,1.44331252574921,1.02165925502777,0.63615345954895,1.61645328998566,0.223143547773361,1.04980707168579,1.41505193710327,0.575376629829407,2.60668158531189,1.51791453361511,0.755041599273682,1.09497237205505,0.942114353179932,1.72494280338287,1.03154611587524,0.474369078874588,0.810930192470551,0.709266602993011,1.71054947376251,0.460268884897232,1.33181178569794,1.0986123085022,2.15799856185913,1.43758130073547,1.54489934444427,1.41059672832489,3.21887588500977,0.968161880970001,1.7917594909668,1.68872952461243,-0.409171968698502,0.223143547773361,0.822155833244324,1.24170196056366,1.42712438106537,1.49709749221802,0.559615790843964,1.30002820491791,1.88442981243134,0.955511391162872,1.58208727836609,1.7556140422821,1.51310324668884,2.25189161300659,2.3644323348999,0.105350479483604,1.39972877502441,0.988462507724762,1.09064733982086,1.15461444854736,1.26694762706757,2.88519167900085,1.22888004779816,1.20396280288696,1.35738027095795,0.837723612785339,0.536961138248444,0.748723804950714,2.29587268829346,1.10780322551727,0.620845258235931,-2.05416369438171,1.89201200008392,1.72972452640533,0.469378411769867,0.98084169626236,2.06949234008789,1.67518818378448,1.3862943649292,1.79921495914459,1.83258152008057,1.09064733982086,1.44312357902527,1.25036013126373,1.60231256484985,1.01855850219727,1.29705321788788,1.68519449234009,-0.420984894037247,1.56209468841553,2.14652752876282,2.34746289253235,0.969831526279449,1.9241464138031,1.62672758102417,-0.0392607264220715,1.46014869213104,1.95539355278015,0.926359891891479,2.06619167327881,1.42284321784973,2.10103178024292,2.26146101951599,0.70131379365921,2.03101253509521,1.16236925125122,0.470003634691238,1.41059672832489,0.393055111169815,1.29099416732788,0,0.95712548494339,0.559615790843964,1.56861591339111,1.7101879119873,1.41059672832489,0.223143547773361,0.510845601558685,1.33239245414734,0.860185861587524,2.32277989387512,1.91959547996521,1.97610676288605,0.895434737205505,0.18123759329319,0.495305836200714,0.577792406082153,1.07881772518158,1.60319852828979,0.620845258235931,2.08389401435852,1.37916910648346,1.11238372325897,1.06712162494659,1.11880695819855,1.58854103088379,1.3903112411499,1.71480643749237,0.20106153190136,0.987271010875702,0.98350065946579,2.23317074775696,1.14361751079559,-0.611382901668549,2.15305209159851,1.29983735084534,0.840920448303223,1.05848443508148,1.15265846252441,1.29357588291168,1.83258152008057,2.32718014717102,1.16614627838135,2.03499317169189,0.679251074790955,1.54713690280914,0.75301855802536,0.847283601760864,0.871125996112823,0.228250473737717,0.0896578282117844,1.32175588607788,1.19610190391541,1.63611876964569,1.89201200008392,1.51830899715424,2.47215914726257,1.32175588607788,1.47364103794098,1.36947882175446,1.20396280288696,1.19872915744781,1.27020990848541,0.470003634691238,0.799981653690338,1.56594562530518,1.75897800922394,0.858025848865509,0.6931471824646,0.641853868961334,1.63374018669128,1.70374763011932,1.84400403499603,1.96611881256104,0.864997446537018,0.933305203914642,0.779233157634735,0.955511391162872,1.31624734401703,1.4759064912796,1.49139726161957,1.45575046539307,0.510845601558685,1.18043804168701,1.68848943710327,0.790727496147156,1.40179860591888,-0.433556020259857,1.68317151069641,-1.76667666435242,3.15559506416321,2.25952100753784,1.30692636966705,0.798497676849365,0.559044182300568,0.147902622818947,1.94449484348297,1.37833786010742,3.0647451877594,-0.741917312145233,0.765700399875641,0.619392991065979,1.46545207500458,2.18925952911377,1.02165925502777,0.977009475231171,0.916290760040283,2.90509605407715,-0.199671193957329,0.6931471824646,2.73339295387268,1.86833465099335,2.12025952339172,1.51519322395325,0.914609313011169,1.49955606460571,0.803077220916748,0.728031635284424,0.516409993171692,1.22644829750061,0.916290760040283,1.37647128105164,1.8289749622345,1.36828315258026,1.06471073627472,1.40648913383484,1.04731893539429,1.94809341430664,1.07800137996674,0.653938472270966,1.92789161205292,1.36102783679962,0.6931471824646,1.60468661785126,0.183903649449348,3.11351537704468,1.92682921886444,1.2701256275177,0.68269270658493,1.68106997013092,0.556295990943909,1.62822043895721,0.916290760040283,1.3415584564209,0,1.12223124504089,0.540170788764954,1.3915057182312,1.69717395305634,3.21887588500977,0.871167778968811,1.16732954978943,1.21698772907257,0.575376629829407,1.15161573886871,0.994251251220703,0.526324927806854,-1.5431821346283,1.91204309463501,0.554287314414978,0.916290760040283,1.50093913078308,0.944683790206909,1.24126863479614,1.56498432159424,0.838026463985443,1.66885709762573,1.7694286108017,1.22644829750061,1.40648913383484],[12,12,12,12,14,12,16,12,12,12,12,11,12,12,10,11,12,12,12,12,16,12,13,12,12,17,12,12,17,12,11,16,13,12,16,11,12,10,14,17,12,12,16,12,12,12,16,12,12,12,12,12,12,8,10,16,14,17,14,12,14,12,8,12,12,8,17,12,12,12,12,12,9,10,12,12,12,17,15,12,6,14,12,14,9,17,13,9,15,12,12,12,12,12,12,12,12,13,12,13,12,12,12,16,12,13,11,12,12,12,17,14,16,17,12,11,12,12,17,10,13,11,12,16,17,12,16,12,16,8,12,12,12,13,11,12,12,14,12,12,12,17,14,12,9,12,12,12,14,16,17,15,12,16,17,17,12,16,13,12,11,16,14,16,12,9,17,14,12,12,11,12,12,10,12,5,17,11,12,12,14,11,12,14,12,10,16,13,12,12,12,11,12,9,13,12,12,12,13,16,12,16,17,12,12,9,12,12,13,12,12,12,12,10,12,16,12,11,12,10,12,12,12,12,16,17,12,17,12,12,12,8,12,13,12,12,8,12,17,17,12,13,12,12,12,12,9,10,12,16,13,8,16,13,12,11,13,12,12,10,12,17,15,16,10,11,12,12,14,16,14,8,7,12,12,14,12,12,12,14,16,12,12,12,13,13,10,12,12,12,12,14,17,10,9,12,12,16,12,17,12,17,11,16,11,13,11,8,11,12,10,17,12,12,17,14,12,12,12,12,12,12,9,10,12,12,12,12,12,17,12,17,12,10,12,12,12,12,12,12,16,13,13,12,16,17,12,14,12,17,12,14,12,12,17,16,16,12,9,12,12,16,14,12,12,11,12,16,17,17,14,12,14,12,10,12,13,16,12,7,16,14,12,10,12,16,10,12,14,12,6,15,12,17,14,13,6,16,14,15,14,8,14,12,12,12,12,12,12,8,12,17,12,12,14,13,17,8,12,11,12,12,17,10,12,13,12,12],[14,5,15,6,7,33,11,35,24,21,15,14,0,14,6,9,20,6,23,9,5,11,18,15,4,21,31,9,7,7,32,11,16,14,27,0,17,28,24,11,1,14,6,10,6,4,10,22,16,6,12,32,15,17,34,9,37,10,35,6,19,10,11,15,12,12,14,11,9,24,12,13,29,11,13,19,2,24,9,6,22,30,10,6,29,29,36,19,8,13,16,11,15,6,13,22,24,2,6,2,2,14,9,11,9,6,19,26,19,3,7,28,13,9,15,20,29,9,1,8,19,23,3,13,8,17,4,15,11,7,0,0,10,8,2,4,6,18,3,22,33,28,23,27,11,6,11,14,17,17,14,11,7,8,6,8,4,25,24,11,19,9,19,14,22,6,23,15,6,11,2,22,10,14,12,9,13,18,8,11,9,9,14,9,2,12,15,11,7,9,19,11,8,13,4,7,19,14,14,3,9,7,7,14,29,19,14,16,10,12,24,6,9,14,26,7,4,15,23,1,29,9,6,11,17,6,7,2,24,4,11,25,11,2,19,7,2,20,10,19,17,12,11,6,10,4,2,13,21,9,4,2,19,4,9,14,6,24,1,13,3,10,16,9,19,4,10,5,7,3,38,16,13,1,7,15,10,2,19,25,25,7,15,11,25,19,4,14,19,18,14,11,4,29,21,24,19,31,28,15,27,13,4,10,8,4,18,3,11,8,10,33,19,35,21,7,18,4,12,16,14,3,1,27,12,6,9,2,6,9,16,22,26,11,11,15,13,6,20,17,8,13,15,14,14,6,24,10,2,9,23,12,8,16,10,7,19,2,9,14,9,16,7,6,22,9,9,14,17,12,13,8,10,16,1,6,4,8,4,15,7,14,16,15,23,19,4,12,12,25,14,14,11,7,18,4,37,13,14,17,5,2,0,3,21,20,19,4,19,11,14,8,13,24,1,1,3,4,21,10,13,9,14,2,21,22,14,7],[196,25,225,36,49,1089,121,1225,576,441,225,196,0,196,36,81,400,36,529,81,25,121,324,225,16,441,961,81,49,49,1024,121,256,196,729,0,289,784,576,121,1,196,36,100,36,16,100,484,256,36,144,1024,225,289,1156,81,1369,100,1225,36,361,100,121,225,144,144,196,121,81,576,144,169,841,121,169,361,4,576,81,36,484,900,100,36,841,841,1296,361,64,169,256,121,225,36,169,484,576,4,36,4,4,196,81,121,81,36,361,676,361,9,49,784,169,81,225,400,841,81,1,64,361,529,9,169,64,289,16,225,121,49,0,0,100,64,4,16,36,324,9,484,1089,784,529,729,121,36,121,196,289,289,196,121,49,64,36,64,16,625,576,121,361,81,361,196,484,36,529,225,36,121,4,484,100,196,144,81,169,324,64,121,81,81,196,81,4,144,225,121,49,81,361,121,64,169,16,49,361,196,196,9,81,49,49,196,841,361,196,256,100,144,576,36,81,196,676,49,16,225,529,1,841,81,36,121,289,36,49,4,576,16,121,625,121,4,361,49,4,400,100,361,289,144,121,36,100,16,4,169,441,81,16,4,361,16,81,196,36,576,1,169,9,100,256,81,361,16,100,25,49,9,1444,256,169,1,49,225,100,4,361,625,625,49,225,121,625,361,16,196,361,324,196,121,16,841,441,576,361,961,784,225,729,169,16,100,64,16,324,9,121,64,100,1089,361,1225,441,49,324,16,144,256,196,9,1,729,144,36,81,4,36,81,256,484,676,121,121,225,169,36,400,289,64,169,225,196,196,36,576,100,4,81,529,144,64,256,100,49,361,4,81,196,81,256,49,36,484,81,81,196,289,144,169,64,100,256,1,36,16,64,16,225,49,196,256,225,529,361,16,144,144,625,196,196,121,49,324,16,1369,169,196,289,25,4,0,9,441,400,361,16,361,121,196,64,169,576,1,1,9,16,441,100,169,81,196,4,441,484,196,49],[7,7,7,7,14,7,7,3,7,7,3,7,16,10,7,10,7,12,7,7,16,10,3,7,7,14,7,7,12,12,7,3,10,14,12,3,3,3,7,17,12,9,16,3,7,7,16,10,7,7,7,3,7,7,3,12,7,17,7,7,3,12,7,7,7,12,16,7,7,7,12,10,9,0,10,14,7,3,12,12,7,17,3,7,7,12,7,7,12,10,0,12,10,7,7,7,3,12,7,12,7,7,10,14,7,12,7,7,10,7,12,7,7,17,7,7,7,10,10,12,7,12,7,10,7,10,7,7,7,7,7,7,16,12,7,3,7,7,7,12,7,12,12,14,7,7,7,12,12,14,10,12,7,16,7,17,3,10,9,7,3,16,12,7,7,7,12,3,7,7,7,7,10,10,7,12,17,10,7,7,12,7,12,7,7,7,7,14,7,12,7,7,12,3,7,12,12,7,7,14,12,17,17,7,7,10,7,7,7,3,0,7,12,7,7,12,7,7,12,7,7,3,7,7,10,12,7,12,7,7,7,7,12,7,7,7,7,7,14,17,7,10,7,7,12,7,7,9,7,14,7,3,16,3,16,7,16,12,7,7,12,12,12,16,7,9,7,12,12,10,7,12,7,7,3,10,17,7,3,3,12,7,7,7,10,7,10,7,9,9,12,12,12,7,9,12,7,12,7,12,12,14,7,14,10,12,7,7,12,7,7,12,12,12,12,14,10,7,7,7,7,7,7,7,7,7,12,12,7,14,10,12,7,7,12,7,10,12,10,7,14,7,12,12,7,16,0,12,7,17,7,12,3,7,14,7,12,12,7,7,14,7,12,12,7,7,7,7,3,12,7,7,14,10,10,10,10,12,3,7,16,7,7,0,7,7,10,7,12,7,7,7,7,16,12,10,7,7,16,7,7,7,12,10,10,10,7,10,12,12,7,17,12,16,10,16,7,7,9,3,7,7,7,7,7,7,16,12],[12,7,12,7,12,14,14,3,7,7,12,14,16,10,7,16,10,12,7,12,10,12,7,7,12,16,3,3,12,12,7,3,12,7,12,10,3,10,7,14,12,9,14,3,12,12,14,10,7,12,7,7,12,7,7,12,7,17,17,12,14,12,7,7,7,12,12,12,7,12,12,10,7,0,7,12,7,3,10,7,12,12,7,7,7,7,7,7,7,10,7,12,10,12,7,7,7,14,7,12,12,7,7,14,12,10,7,7,7,7,12,7,12,10,10,7,7,7,12,7,7,12,14,12,7,10,7,7,12,10,7,7,12,10,7,12,7,7,7,7,3,12,16,7,3,12,7,12,12,16,12,12,7,14,7,10,7,14,7,7,12,12,17,7,7,3,12,7,7,7,3,7,10,10,12,7,14,10,7,7,10,12,12,7,7,7,12,7,12,12,12,10,12,10,12,12,12,7,12,12,12,12,16,7,16,7,7,10,12,10,0,7,12,12,10,12,3,7,12,10,7,7,7,7,12,12,7,12,7,10,10,7,12,17,7,7,7,7,12,14,7,12,7,7,16,7,10,12,7,16,10,3,16,7,12,7,7,7,12,12,7,10,14,16,7,10,7,14,14,12,7,7,3,7,7,7,12,10,7,3,12,7,12,7,10,7,0,7,10,9,12,12,12,3,9,12,12,14,7,12,12,12,7,12,10,12,7,7,3,12,7,16,12,12,7,14,7,7,12,10,7,3,7,7,10,7,7,12,12,12,10,14,7,7,14,10,10,7,7,7,14,7,12,14,14,14,0,16,7,12,7,10,7,10,10,14,7,12,7,7,12,14,12,7,7,7,12,16,3,16,7,16,7,10,10,10,10,12,10,7,16,7,7,7,7,7,10,10,12,7,7,7,7,14,14,7,7,7,16,12,7,7,12,12,12,10,3,12,12,12,7,16,12,10,10,12,7,7,7,7,7,7,7,7,7,7,12,12]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>id<\/th>\n      <th>lwage<\/th>\n      <th>educ<\/th>\n      <th>exper<\/th>\n      <th>expersq<\/th>\n      <th>fatheduc<\/th>\n      <th>motheduc<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"tip","columnDefs":[{"targets":2,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\", null);\n  }"},{"className":"dt-center","targets":"_all"},{"visible":false,"targets":0},{"orderable":false,"targets":0}],"pageLength":8,"scrollX":true,"scrollCollapse":true,"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[8,10,25,50,100]}},"evals":["options.columnDefs.0.render"],"jsHooks":[]}</script>

???

We will use 428 observations totally after delete rows with missing wage value.

---

### Wage example: the scatter

---

### Wage example: use OLS method directly

Of course, you can conduct the OLS regression directly without considering problems due to endogeneity, and may obtain the inconsistent estimators (as we have proved).

```r
mod_origin <- formula(lwage ~ educ +exper+expersq)
ols_origin <- lm(formula = mod_origin, data = mroz)
```

The OLS regression resuts is

`$$\begin{equation} \begin{alignedat}{999} &\widehat{lwage}=&&-0.52&&+0.11educ&&+0.04exper&&-0.00expersq\\ &\text{(t)}&&(-2.6282)&&(7.5983)&&(3.1549)&&(-2.0628)\\&\text{(se)}&&(0.1986)&&(0.0141)&&(0.0132)&&(0.0004)\\&\text{(fitness)}&& R^2=0.1568;&& \bar{R^2}=0.1509\\& && F^{\ast}=26.29;&& p=0.0000 \end{alignedat} \end{equation}$$`

This looks good, but we know it is not reliable due to endogeneity behind this "error specified" model.

???

You can type and run these R codes and then get the following tidy OLS results.

You may see that:

- The t statistic of estimators are given in this row.

- And this row shows the standard error of the estimators.

- The t test of all coefficients are significant with its abslute t value larger than 2.

- And the F test is also significant with a small p value.

This looks good, but we know it is not reliable due to endogeneity behind this "error specified" model.

---
layout: false
class: center, middle, duke-softblue,hide_logo
name: IV

## 17.3 IV and the choices

???

How to handle the endogenous variable problems?

There is a way to go ahead, and we will resort to use instrumental variables in this section.

---
layout: true
  
<div class="my-header-h2"></div>
<div class="watermark1"></div>
<div class="watermark2"></div>
<div class="watermark3"></div>

<div class="my-footer"><span>huhuaping@  <a href="#chapter17"> Chapter 17. Endogeneity and Instumental Variables  |</a> &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;
<a href="#IV"> 17.3 IV and the choices </a> </span></div>

---

### IV: Motivation

We have seen that OLS estimator of 
`$\beta$` is inconsistent when one or more regressors is endogenous .

The **problems** of OLS arise because we imposed 
`$E(X_i\epsilon_i)=0$`, which means we believe the sample data with

`$$\boldsymbol{X^{\prime} {e}=0}$$`

When in fact error terms and regressors are correlated 
`$E(X_i\epsilon_i) \neq 0$`.

---

### IV: Motivation

Suppose we can find a set of **explanatory variables** 
`$\boldsymbol{Z}$` satisfying two
conditions:

- **Relevance**: 
`$\boldsymbol{Z}$` is correlated with 
`$\boldsymbol{X}$`

- **Exogeneity**: 
`$\boldsymbol{Z}$` is not correlated with 
`$\boldsymbol{\epsilon}$`

These variables (
`$\boldsymbol{Z}$`, in matrix form) can be used for consistent estimation and are known as .red[**Instrumental Variables (IV)**] .

---

### IV: Estimators

Our instrumental variable estimator, 
`$\hat{\beta}_{IV}$` is defined in terms of the following "**normal equation**" (**moment condition**, to be more precise)

`$$\begin{align}
\boldsymbol{Z^{\prime} \hat{\epsilon}=Z^{\prime}\left(y-X \hat{\beta}_{IV}\right)=0}
\end{align}$$`

and thus, provided that 
`$Z^{\prime} X$` is **square** and **non singular**,

`$$\begin{align}
\boldsymbol{\hat{\beta}_{IV}=\left(Z^{\prime} X\right)^{-1} Z^{\prime} y}
\end{align}$$`

The condition that 
`$\boldsymbol{Z^{\prime} X}$` is square and non singular, intuitively, is satisfied when we have as many instruments as regressors (a situation that is called **exact identification**).

However 
`$\boldsymbol{\hat{\beta}_{IV}}$` is generally **biased** in **finite sample**, but we can show that it is still **consistent**.

???

So let us prove this point.

---

### IV: Consistency

`$\boldsymbol{\hat{\beta}_{IV}}$` is consistent. Since:

`$$\begin{align}
\boldsymbol{\hat{\beta}_{IV}
=\left(Z^{\prime} X\right)^{-1} Z^{\prime} y
=\left(Z^{\prime} X\right)^{-1} Z^{\prime} (X\beta +\epsilon)
=\beta+\left(Z^{\prime} X\right)^{-1} Z^{\prime} \epsilon}
\end{align}$$`

`$$\begin{align}
  p \lim \boldsymbol{{\hat{\beta}_{IV}}} 
  &=\boldsymbol{\beta+p \lim \left(\left(Z^{\prime} X\right)^{-1} Z^{\prime} \epsilon\right)} \\
  &=\boldsymbol{\beta+\left(p \lim \left(\frac{1}{n} Z^{\prime} X\right)\right)^{-1} \operatorname{plim}\left(\frac{1}{n} Z^{\prime} \epsilon\right) 
  =\beta}
\end{align}$$`

- Relevance guarantees

`$$\begin{align}
  p \lim \left(\frac{1}{n} \boldsymbol{Z^{\prime} X}\right) 
  &=p \lim \left(\frac{1}{n} \sum z_{i} X_{i}^{\prime}\right) \\
  &=E\left(Z_{i} X_{i}^{\prime}\right) \neq 0
\end{align}$$`

]

- Exogeneity ensures

`$$\begin{align}
  p \lim \left(\frac{1}{n} \boldsymbol{Z^{\prime} \epsilon}\right) 
  &=p \lim \left(\frac{1}{n} \sum Z_{i} \epsilon_{i}\right) \\
  &=E\left(Z_{i} \epsilon_{i}\right)=0
\end{align}$$`
]

???
The IV estimator `$\boldsymbol{\hat{\beta}_{IV}}$` is consistent, Since the two instrument conditions are guaranteed .

For simple, the instrument relevance condition guarantees correlation between `$Z_i$` and `$X_i$`, and the instrument exogeneity condition ensures uncorrelations between `$Z_i$` and error term `$\epsilon_i$`.

Thus, the IV estimator is consistent under probability limits.

---

### IV: Inference

The natural estimator for 
`$\sigma^{2}$` is

`$$\begin{align}
\hat{\sigma}_{I V}^{2}
=\frac{\sum e_{i}^{2}}{n-k}
=\frac{\boldsymbol{\left(y-X \hat{\beta}_{IV}\right)^{\prime}\left(y-X \hat{\beta}_{I V}\right)}}{n-k}
\end{align}$$`

can be shown to be consistent (not proved here).

Thus, we can perform hypothesis testing based on IV estimators 
`$\boldsymbol{\hat{\beta}_{IV}}$`.

???

For the purpose of inference, we need to get the consistent estimate of the variance of disturbance `$\sigma^2$`.

The natural estimator for the variance of stochastic disturbance `$\sigma^{2}$` is `$\hat{\sigma}_{I V}^{2}$` as show below.

And we don't try to prove it here.

After that, we can perform hypothesis testing based on IV estimators `$\boldsymbol{\hat{\beta}_{IV}}$`.

---

### Choice of instruments

However, finding **valid instruments** is the most difficult part of IV estimation in practice.

> Good instruments need not only to be exogenous, but also need be highly correlative with the regressors.

> **Joke**: If you can find a valid instrumental variable, you can get  PhD from MIT.

Without a proof, we say that the **asymptotic variance** of 
`$\hat{\beta}_{I V}$` is

`$$\begin{align}
\operatorname{Var}\left(\boldsymbol{\hat{\beta}_{I V}}\right)=\sigma^{2}\boldsymbol{\left(Z^{\prime} X\right)^{-1}\left(Z^{\prime} Z\right)\left(X^{\prime} Z\right)^{-1}}
\end{align}$$`

Where 
`$\boldsymbol{X^{\prime} Z}$` is the matrix of covariances between instruments and regressors.

If such correlation is low, 
`$\boldsymbol{X^{\prime} Z}$` will have elements close to zero and hence 
`$\boldsymbol{\left(X^{\prime} Z\right)^{-1}}$` will have huge elements. Thus, 
`$\operatorname{Var}\left(\boldsymbol{\hat{\beta}_{IV}}\right)$` will be very large.

???

So, what is the real challenge?

___

Without a proof, we say that the **asymptotic ('æsɪmp,toʊtɪk) variance** of `$\hat{\beta}_{I V}$` takes following output.

If such correlation is low, this covariances matrix `$\boldsymbol{X^{\prime} Z}$` will have elements close to zero and hence its inverse matrix `$\boldsymbol{\left(X^{\prime} Z\right)^{-1}}$` will have huge elements.

And then, the asymptotic variance  of IV estimator `$\operatorname{Var}\left(\boldsymbol{\hat{\beta}_{IV}}\right)$` will be very large.

This means the IV estimation will be useless.

---

### Choice of instruments

The **common strategy** is  to construct 
`$\boldsymbol{Z}=(X_{ex}, X^{\ast})$` generally.

- Variables 
`$X_{ex}$` in 
`$\boldsymbol{X}=(X_{ex}, X_{en})$` are the assumed **exogenous** variables included in 
model.

- Other exogenous variables
`$\boldsymbol{X^{\ast}}$`  are "close" to the model, but do not enter the model explicitly.

If 
`$X$` can be shown to be exogenous, then
`$X=Z$` and Gauss-Markov efficiency is recovered.

> **Instrumental Variable Estimators** .red[do not] have any efficiency properties .

> We can only talk about **relative efficiency**. It means that we can only choose the optimal set of instruments. Such that our estimator is the best we can obtain within all the class of possible instrumental variable estimators.

???

So, it is really big challenge to find the valid instruments and control the low level of asymptotic variance.

While we can find a solution if we are luckly.

___

We should remind one other thing.

___

So we will ask that with what conditions we can excute this common strategy?

---

### Too many available instruments

In case there are more **instruments** than **.red[en]dogenous variables** (**over-identification**), we want to choose those instruments that have the highest correlation with 
`$X$` and hence give the lowest possible variance.

The best choice is obtained by using the fitted values of an OLS regression of each column of 
`$\boldsymbol{X}$` on all instruments
`$\boldsymbol{Z}$`, that is (after running 
`$k$` regressions, one for each column of 
`$\boldsymbol{X}$`)

`$$\begin{align}
\boldsymbol{\hat{X}=Z\left(Z^{\prime} Z\right)^{-1} Z^{\prime} X=ZF}
\end{align}$$`

We now use 
`$\boldsymbol{\hat{X}}$` as instrument, which is
`$\boldsymbol{\hat{\beta}_{I V}=\left(\hat{X}^{\prime} X\right)^{-1} \hat{X}^{\prime} y}$`

We notice that (try to prove this):

`$$\begin{align}
  \boldsymbol{\hat{\beta}_{I V} =\left(\hat{X}^{\prime} X\right)^{-1} \hat{X}^{\prime} y=\left(X^{\prime} Z\left(Z^{\prime} Z\right)^{-1} Z^{\prime} X\right)^{-1} X^{\prime} Z\left(Z^{\prime} Z\right)^{-1} Z^{\prime} y }
\end{align}$$`

???

The clue is show below, and you may try to prove it.

The last results show that when we have more instruments than endogenous variables `$\boldsymbol{\hat{\beta}_{IV}}$` can be computed in 2 steps (2SLS).

---

### IV solution with omitted variables

Let's go back to our example of the wage equation. Assume we are modeling wage as a function of **education** and **ability**.

`$$Wage_{i}=\beta_{0}+\beta_{1} Edu_{i}+\beta_{2} Abl_{i}+\epsilon_{i}$$`

However, individual's ability is clearly something that is not observed or measured and hence cannot be included in the model. since, ability is not included in the model it is included in the error term.

`$$Wage_{i}=\beta_{0}+\beta_{1} Edu_{i}+e_{i}, \quad \text {where} \quad e_{i}=\beta_{2} Abl_{i}+\epsilon_{i}$$`

The problem is that ability not only affects wages but the more able individuals may spend more years in school, causing a positive correlation between the error term and education, 
`$\operatorname{cov}(Edu_i, e_i)>0$` .

Thus, 
`$Educ$` is an **endogenous variable**.

---

### IV solution to omitted variables

If we can find a valid instrument for Educ we can estimate 
`$\beta_{1}$` using IV method. Suppose that we have a variable 
`$Z$` that satisfies the following conditions

- `$Z$` does .red[not] directly affect wages

- `$Z$` is .red[uncorrelated] with 
`$e$` (exogeneity),  i.e.

`$$\operatorname{Cov}(e, z)=0  \quad \text{(4)}$$`

> since 
`$e_{i}=\beta_{2} Abl_{i}+\epsilon_{i}$`, 
`$Z$` must be uncorrelated with ability.
]

.pull-right[
- 
`$Z$` is (at least partially) .red[correlated] with the endogenous variable,i.e. Education (relevance),

`$$\operatorname{Cov}(Z, Edu) \neq 0 \quad \text{(5)}$$`

> Such condition can be tested(
`$\alpha_2$`) by using a simple regression$:

`$$Edu_{i}=\alpha_{1}+\alpha_{2} Z_{i}+u_{i}$$`

]

Then, 
`$Z$` is a valid instrument for 
`$Educ_i$`. We showed earlier that the IV estimator of 
`$\beta_{1}$` is consistent.

???

And the instrumental variable `$Z$` should satisfies both relevance condition and exogenous condition as we have mentioned before.

So, what are these appropriate instruments for education?

---

### IV solution to omitted variables

Several economists have used **family background variables** as IVs for education.

- For example, **mother's education** is positively correlated with child's education, so it satisfies condition of **Relevance**.

> The problem is that mother's education might also be correlated with child's ability, in which case the condition of  **Exogeneity** fails.

- Another IV for education that has been used by economists is the number of `$siblings$` while growing up.

> Typically, having more siblings is associated with lower average levels of education and it should be uncorrelated with innate ability.

???

Anyway, it is worth to try using mother education as the instrument.

---
exclude: true

### IV solution to measurement error

Also, let us consider the case with measurement error in the independent variable.

For some reason, a variable 
`$X_i$` cannot be directly observed (data is not available), so it is common to find a **imperfect measure**(the **proxy variable**) for it.

The **"real model"** for wage is assumed to be:

`$$\begin {align} 
\log (Wage_i)=\beta_{0}+\beta_{1} Edu_i+\beta_{2} Abl_i+u_i
\end {align}$$`

However, since the ability variable (
`$Abl_i$`) cannot be observed, we often replace it with the **IQ level** variable (
`$IQ_i$`) and construct the following **"proxy variable model"** :

`$$\begin {align} 
\log (Wage_i)=\beta_{0}+\beta_{1} Edu_i+\beta_{2} IQ_i+u_i^{\ast}
\end {align}$$`

At this point, **intelligence level** 
`$IQ_i$` can be considered as a potential instrument for **ability**
`$abil_i$`.

???
NOT show because this case may cause confuse.

Now we have got some available instruments, and the following question is how to process the IV estimation.

In the next section, we will talk about Two-stage least squares method.

---
layout: false
class: center, middle, duke-softblue,hide_logo
name: TSLS

## 17.4 Two-stage least squares method

???
In this section, we will discuss how to perform Two-stage least squares estimation procedure.

---
layout: true
  
<div class="my-header-h2"></div>
<div class="watermark1"></div>
<div class="watermark2"></div>
<div class="watermark3"></div>

<div class="my-footer"><span>huhuaping@   <a href="#chapter17"> Chapter 17. Endogeneity and Instumental Variables  |</a> &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; <a href="#TSLS"> 17.4 Two-stage least squares method </a></span></div>

---

### Two-stage least squares: glance

When we have **more** instruments than endogenous variables, 
`$\boldsymbol{\hat{\beta}_{IV}}$` can be computed in 2 steps:

- **Step 1**: Regress each column of 
`$X$` on all the instruments (
`$Z$` ,in matrix form ). For each column of 
`$X$`, get the fitted values and combine them into the matrix 
`$\hat{X}$`.

- **Step 2**: Regress 
`${Y}$` on 
`$\hat{X}$`

And, this procedure is named **two-stage least squares** or **2SLS** or **TSLS**.

---

### Two-stage least squares: indentification

Consider the model setting

`$$\begin{align}
Y_{i}=\beta_{0}+\sum_{j=1}^{k} \beta_{j} X_{j i}+\sum_{s=1}^{r} \beta_{k+s} W_{ri}+\epsilon_{i}
\end{align}$$`

where 
`$\left(X_{1 i}, \ldots, X_{k i}\right)$` are **endogenous regressors**, 
`$\left(W_{1 i}, \ldots, W_{r i}\right)$` are **exogenous regressors** and there are 
`$m$` **instrumental variables** 
`$\left(Z_{1 i}, \ldots, Z_{m i}\right)$` satisfying instrument relevance and instrument exogeneity conditions.

- When 
`$m=k$` ,the coefficients are **exactly identified**.

- When 
`$m>k$` ,the coefficients are **overidentified**.

- When 
`$m<k$`, the coefficients are **underidentified**.

- Finnaly, coefficients can be identified only when 
`$m \geq k$`.

???

Because the model identification is the most important thing before applying the estimation procedure.

So, We should overview the model status explicitly.

We will denote the general model format as below.

---

### Two-stage least squares: the procedure

- **Stage 1**: Regress 
`$X_{1i}$` on constant, all the instruments 
`$\left(Z_{1i}, \ldots, Z_{m i}\right)$` and all exogenous regressors 
`$\left(W_{1i}, \ldots, W_{ri}\right)$` using OLS and obtain the fitted values 
`$\hat{X}_{1 i}$` . Repeat this to get 
`$\left(\hat{X}_{1 i}, \ldots, \hat{X}_{k i}\right)$`

- **Stage 2**: Regress 
`$Y_{i}$` on constant, 
`$\left(\hat{X}_{1 i}, \ldots, \hat{X}_{k i}\right)$` and `$\left(W_{1 i}, \ldots, W_{r i}\right)$` using
OLS to obtain 
`$\left(\hat{\beta}_{0}^{IV}, \hat{\beta}_{1}^{IV}, \ldots, \hat{\beta}_{k+r}^{IV}\right)$`

???

So, in case with “exactly identification” and “over-identification”, we can go ahead with the **Two-Stage Least Squares** as a “whole” solution for IV estimation.

---

### Two-stage least squares: the solutions

We can conduct the **2SLS** procedure with following two solutions:

- use the **"Step-by-Step solution"** methods without variance correction.

- use the  **"Integrated solution"** with variance correction.

**Notice**:

DO NOT use **"Step-by-Step solution"** solution in your paper! It is only for teaching purpose here.

In `R` ecosystem, we have two packages to execute the  **Integrated solution**:

- We can use `systemfit` package function `systemfit::systemfit()`.

- Or we may use `ARE` package function `ARE::ivreg()`.

]

???

Let us apply these solutions to the empirical wage examples.

---

### Step-by-step solution: stage 1 model

First, let's try to use `$matheduc$` as instrument of endogenous variable `$educ$`.

**Stage 1 of 2SLS**: with mother education as instrument

we can obtain the fitted variable `$\widehat{educ}$` by conduct the following **step 1** OLS regression

`$$\begin{align}
\widehat{educ} = \hat{\gamma}_1 +\hat{\gamma}_2exper + \hat{\gamma}_3expersq +\hat{\gamma}_4mothereduc 
\end{align}$$`

???

Again, let us do the demo of two-stage least squares procedure based on the wage example.

In the firs stage, we can obtain the fitted variable `$\widehat{educ}$` by conduct the following OLS regression.

---

### Step-by-step solution: stage 1 OLS estimate

Here we obtain the OLS results of **Stage 1 of 2SLS**:

```r
mod_step1 <- formula(educ~exper + expersq + motheduc)  # modle setting
ols_step1 <- lm(formula = mod_step1, data = mroz)  # OLS estimation
```

`$$\begin{equation}
\begin{alignedat}{999}
&\widehat{educ}=&&+9.78&&+0.05exper_i&&-0.00expersq_i&&+0.27motheduc_i\\ 
&(s)&&(0.4239)&&(0.0417)&&(0.0012)&&(0.0311)\\ 
&(t)&&(+23.06)&&(+1.17)&&(-1.03)&&(+8.60)\\ 
&(fit)&&R^2=0.1527&&\bar{R}^2=0.1467 && &&\\ 
&(Ftest)&&F^*=25.47&&p=0.0000 && &&
\end{alignedat}
\end{equation}$$`

The t -value for coefficient of `$mothereduc$`  is so large (larger than 2), indicating a strong correlation between this instrument and the endogenous variable  `$educ$`  even after controlling for other variables.

???
we should note that the t-value for coefficient of `$mothereduc$`  is larger than 2. and the t test is significant. This  means there is a strong correlation between the instrument `$motheduc$` and the endogenous variable  `$educ$`  even when we control all other variables.

---

### Step-by-step solution: stage 1 OLS predicted values

Along with the regression of **Stage 1 of 2SLS**, we will extract the fitted value `$\widehat{educ}$` and add them into new data set.

```r
mroz_add <- mroz %>% mutate(educHat = fitted(ols_step1)) # add fitted educ to data set
```

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-510e4e9925695c9a3cd3" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-510e4e9925695c9a3cd3">{"x":{"filter":"none","vertical":false,"data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180","181","182","183","184","185","186","187","188","189","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204","205","206","207","208","209","210","211","212","213","214","215","216","217","218","219","220","221","222","223","224","225","226","227","228","229","230","231","232","233","234","235","236","237","238","239","240","241","242","243","244","245","246","247","248","249","250","251","252","253","254","255","256","257","258","259","260","261","262","263","264","265","266","267","268","269","270","271","272","273","274","275","276","277","278","279","280","281","282","283","284","285","286","287","288","289","290","291","292","293","294","295","296","297","298","299","300","301","302","303","304","305","306","307","308","309","310","311","312","313","314","315","316","317","318","319","320","321","322","323","324","325","326","327","328","329","330","331","332","333","334","335","336","337","338","339","340","341","342","343","344","345","346","347","348","349","350","351","352","353","354","355","356","357","358","359","360","361","362","363","364","365","366","367","368","369","370","371","372","373","374","375","376","377","378","379","380","381","382","383","384","385","386","387","388","389","390","391","392","393","394","395","396","397","398","399","400","401","402","403","404","405","406","407","408","409","410","411","412","413","414","415","416","417","418","419","420","421","422","423","424","425","426","427","428"],[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428],[1.2101536989212,0.328512102365494,1.51413774490356,0.0921233221888542,1.52427220344543,1.55648005008698,2.12025952339172,2.0596342086792,0.754336357116699,1.54489934444427,1.4019216299057,1.52427220344543,0.733953237533569,0.818369090557098,1.30283117294312,0.298028379678726,1.16760957241058,1.64383935928345,0.6931471824646,2.02193164825439,1.25424754619598,1.27295768260956,1.1786550283432,1.1786550283432,0.767558693885803,1.33181178569794,1.3862943649292,1.55326962471008,1.98181486129761,1.76936042308807,0.430807888507843,0.899754822254181,1.76662969589233,1.27295768260956,1.33678889274597,0.901704847812653,0.865123689174652,1.51184713840485,1.72602915763855,2.68314242362976,0.985294282436371,1.36593854427338,0.945033669471741,1.51237618923187,0.6931471824646,1.24478840827942,0.701164901256561,1.5198632478714,0.820968568325043,0.969831526279449,0.828508198261261,0.0943096429109573,0.162543892860413,0.470003634691238,0.629248440265656,1.39716017246246,2.26544380187988,2.0845410823822,1.52583885192871,0.762160062789917,1.48160457611084,1.26282644271851,0.999675571918488,1.83258152008057,2.47930765151978,1.27901530265808,1.93793559074402,1.0704528093338,1.12392258644104,1.32175588607788,1.74499976634979,1.30174362659454,1.64186644554138,2.10702013969421,1.46706759929657,1.60581135749817,-1.02973937988281,1.08768618106842,0,0.938208699226379,-0.150590375065804,0,1.07367050647736,1.26584839820862,0.486368983983994,2.12025952339172,1.12985253334045,0.993251800537109,1.65862798690796,0.347412198781967,1.56832420825958,0.510845601558685,0.114845432341099,-0.6931471824646,-0.336452275514603,1.02822554111481,1.58068859577179,0.555894613265991,0.901420712471008,0.884304583072662,0.428204596042633,1.05841505527496,0.87833958864212,1.65490829944611,1.32175588607788,0.328512102365494,1.3862943649292,1.17288458347321,1.22418713569641,0.287657082080841,2.23026180267334,1.50407743453979,1.53115200996399,1.37515759468079,1.76026880741119,-0.6931471824646,1.40648913383484,1.7917594909668,1.29929208755493,1.35100388526917,1.01628088951111,1.0753436088562,1.47896468639374,1.68948674201965,2.28859782218933,-1.82263112068176,-0.960765182971954,1.29099416732788,0.864871144294739,1.54045212268829,0.616212129592896,1.64865863323212,1.19349813461304,2.14397621154785,0.724403560161591,0.941607534885406,0.782759368419647,1.83258152008057,1.20396280288696,1.49164485931396,1.89213263988495,2.13089489936829,1.48060405254364,0.894331336021423,0.202532544732094,0.485507816076279,1.0986123085022,1.55326962471008,0.121597968041897,2.00180435180664,1.49503660202026,0.905229806900024,0.632547557353973,1.3862943649292,2.10291385650635,1.95964395999908,0.510845601558685,1.23692393302917,1.44331252574921,1.02165925502777,0.63615345954895,1.61645328998566,0.223143547773361,1.04980707168579,1.41505193710327,0.575376629829407,2.60668158531189,1.51791453361511,0.755041599273682,1.09497237205505,0.942114353179932,1.72494280338287,1.03154611587524,0.474369078874588,0.810930192470551,0.709266602993011,1.71054947376251,0.460268884897232,1.33181178569794,1.0986123085022,2.15799856185913,1.43758130073547,1.54489934444427,1.41059672832489,3.21887588500977,0.968161880970001,1.7917594909668,1.68872952461243,-0.409171968698502,0.223143547773361,0.822155833244324,1.24170196056366,1.42712438106537,1.49709749221802,0.559615790843964,1.30002820491791,1.88442981243134,0.955511391162872,1.58208727836609,1.7556140422821,1.51310324668884,2.25189161300659,2.3644323348999,0.105350479483604,1.39972877502441,0.988462507724762,1.09064733982086,1.15461444854736,1.26694762706757,2.88519167900085,1.22888004779816,1.20396280288696,1.35738027095795,0.837723612785339,0.536961138248444,0.748723804950714,2.29587268829346,1.10780322551727,0.620845258235931,-2.05416369438171,1.89201200008392,1.72972452640533,0.469378411769867,0.98084169626236,2.06949234008789,1.67518818378448,1.3862943649292,1.79921495914459,1.83258152008057,1.09064733982086,1.44312357902527,1.25036013126373,1.60231256484985,1.01855850219727,1.29705321788788,1.68519449234009,-0.420984894037247,1.56209468841553,2.14652752876282,2.34746289253235,0.969831526279449,1.9241464138031,1.62672758102417,-0.0392607264220715,1.46014869213104,1.95539355278015,0.926359891891479,2.06619167327881,1.42284321784973,2.10103178024292,2.26146101951599,0.70131379365921,2.03101253509521,1.16236925125122,0.470003634691238,1.41059672832489,0.393055111169815,1.29099416732788,0,0.95712548494339,0.559615790843964,1.56861591339111,1.7101879119873,1.41059672832489,0.223143547773361,0.510845601558685,1.33239245414734,0.860185861587524,2.32277989387512,1.91959547996521,1.97610676288605,0.895434737205505,0.18123759329319,0.495305836200714,0.577792406082153,1.07881772518158,1.60319852828979,0.620845258235931,2.08389401435852,1.37916910648346,1.11238372325897,1.06712162494659,1.11880695819855,1.58854103088379,1.3903112411499,1.71480643749237,0.20106153190136,0.987271010875702,0.98350065946579,2.23317074775696,1.14361751079559,-0.611382901668549,2.15305209159851,1.29983735084534,0.840920448303223,1.05848443508148,1.15265846252441,1.29357588291168,1.83258152008057,2.32718014717102,1.16614627838135,2.03499317169189,0.679251074790955,1.54713690280914,0.75301855802536,0.847283601760864,0.871125996112823,0.228250473737717,0.0896578282117844,1.32175588607788,1.19610190391541,1.63611876964569,1.89201200008392,1.51830899715424,2.47215914726257,1.32175588607788,1.47364103794098,1.36947882175446,1.20396280288696,1.19872915744781,1.27020990848541,0.470003634691238,0.799981653690338,1.56594562530518,1.75897800922394,0.858025848865509,0.6931471824646,0.641853868961334,1.63374018669128,1.70374763011932,1.84400403499603,1.96611881256104,0.864997446537018,0.933305203914642,0.779233157634735,0.955511391162872,1.31624734401703,1.4759064912796,1.49139726161957,1.45575046539307,0.510845601558685,1.18043804168701,1.68848943710327,0.790727496147156,1.40179860591888,-0.433556020259857,1.68317151069641,-1.76667666435242,3.15559506416321,2.25952100753784,1.30692636966705,0.798497676849365,0.559044182300568,0.147902622818947,1.94449484348297,1.37833786010742,3.0647451877594,-0.741917312145233,0.765700399875641,0.619392991065979,1.46545207500458,2.18925952911377,1.02165925502777,0.977009475231171,0.916290760040283,2.90509605407715,-0.199671193957329,0.6931471824646,2.73339295387268,1.86833465099335,2.12025952339172,1.51519322395325,0.914609313011169,1.49955606460571,0.803077220916748,0.728031635284424,0.516409993171692,1.22644829750061,0.916290760040283,1.37647128105164,1.8289749622345,1.36828315258026,1.06471073627472,1.40648913383484,1.04731893539429,1.94809341430664,1.07800137996674,0.653938472270966,1.92789161205292,1.36102783679962,0.6931471824646,1.60468661785126,0.183903649449348,3.11351537704468,1.92682921886444,1.2701256275177,0.68269270658493,1.68106997013092,0.556295990943909,1.62822043895721,0.916290760040283,1.3415584564209,0,1.12223124504089,0.540170788764954,1.3915057182312,1.69717395305634,3.21887588500977,0.871167778968811,1.16732954978943,1.21698772907257,0.575376629829407,1.15161573886871,0.994251251220703,0.526324927806854,-1.5431821346283,1.91204309463501,0.554287314414978,0.916290760040283,1.50093913078308,0.944683790206909,1.24126863479614,1.56498432159424,0.838026463985443,1.66885709762573,1.7694286108017,1.22644829750061,1.40648913383484],[12,12,12,12,14,12,16,12,12,12,12,11,12,12,10,11,12,12,12,12,16,12,13,12,12,17,12,12,17,12,11,16,13,12,16,11,12,10,14,17,12,12,16,12,12,12,16,12,12,12,12,12,12,8,10,16,14,17,14,12,14,12,8,12,12,8,17,12,12,12,12,12,9,10,12,12,12,17,15,12,6,14,12,14,9,17,13,9,15,12,12,12,12,12,12,12,12,13,12,13,12,12,12,16,12,13,11,12,12,12,17,14,16,17,12,11,12,12,17,10,13,11,12,16,17,12,16,12,16,8,12,12,12,13,11,12,12,14,12,12,12,17,14,12,9,12,12,12,14,16,17,15,12,16,17,17,12,16,13,12,11,16,14,16,12,9,17,14,12,12,11,12,12,10,12,5,17,11,12,12,14,11,12,14,12,10,16,13,12,12,12,11,12,9,13,12,12,12,13,16,12,16,17,12,12,9,12,12,13,12,12,12,12,10,12,16,12,11,12,10,12,12,12,12,16,17,12,17,12,12,12,8,12,13,12,12,8,12,17,17,12,13,12,12,12,12,9,10,12,16,13,8,16,13,12,11,13,12,12,10,12,17,15,16,10,11,12,12,14,16,14,8,7,12,12,14,12,12,12,14,16,12,12,12,13,13,10,12,12,12,12,14,17,10,9,12,12,16,12,17,12,17,11,16,11,13,11,8,11,12,10,17,12,12,17,14,12,12,12,12,12,12,9,10,12,12,12,12,12,17,12,17,12,10,12,12,12,12,12,12,16,13,13,12,16,17,12,14,12,17,12,14,12,12,17,16,16,12,9,12,12,16,14,12,12,11,12,16,17,17,14,12,14,12,10,12,13,16,12,7,16,14,12,10,12,16,10,12,14,12,6,15,12,17,14,13,6,16,14,15,14,8,14,12,12,12,12,12,12,8,12,17,12,12,14,13,17,8,12,11,12,12,17,10,12,13,12,12],[14,5,15,6,7,33,11,35,24,21,15,14,0,14,6,9,20,6,23,9,5,11,18,15,4,21,31,9,7,7,32,11,16,14,27,0,17,28,24,11,1,14,6,10,6,4,10,22,16,6,12,32,15,17,34,9,37,10,35,6,19,10,11,15,12,12,14,11,9,24,12,13,29,11,13,19,2,24,9,6,22,30,10,6,29,29,36,19,8,13,16,11,15,6,13,22,24,2,6,2,2,14,9,11,9,6,19,26,19,3,7,28,13,9,15,20,29,9,1,8,19,23,3,13,8,17,4,15,11,7,0,0,10,8,2,4,6,18,3,22,33,28,23,27,11,6,11,14,17,17,14,11,7,8,6,8,4,25,24,11,19,9,19,14,22,6,23,15,6,11,2,22,10,14,12,9,13,18,8,11,9,9,14,9,2,12,15,11,7,9,19,11,8,13,4,7,19,14,14,3,9,7,7,14,29,19,14,16,10,12,24,6,9,14,26,7,4,15,23,1,29,9,6,11,17,6,7,2,24,4,11,25,11,2,19,7,2,20,10,19,17,12,11,6,10,4,2,13,21,9,4,2,19,4,9,14,6,24,1,13,3,10,16,9,19,4,10,5,7,3,38,16,13,1,7,15,10,2,19,25,25,7,15,11,25,19,4,14,19,18,14,11,4,29,21,24,19,31,28,15,27,13,4,10,8,4,18,3,11,8,10,33,19,35,21,7,18,4,12,16,14,3,1,27,12,6,9,2,6,9,16,22,26,11,11,15,13,6,20,17,8,13,15,14,14,6,24,10,2,9,23,12,8,16,10,7,19,2,9,14,9,16,7,6,22,9,9,14,17,12,13,8,10,16,1,6,4,8,4,15,7,14,16,15,23,19,4,12,12,25,14,14,11,7,18,4,37,13,14,17,5,2,0,3,21,20,19,4,19,11,14,8,13,24,1,1,3,4,21,10,13,9,14,2,21,22,14,7],[196,25,225,36,49,1089,121,1225,576,441,225,196,0,196,36,81,400,36,529,81,25,121,324,225,16,441,961,81,49,49,1024,121,256,196,729,0,289,784,576,121,1,196,36,100,36,16,100,484,256,36,144,1024,225,289,1156,81,1369,100,1225,36,361,100,121,225,144,144,196,121,81,576,144,169,841,121,169,361,4,576,81,36,484,900,100,36,841,841,1296,361,64,169,256,121,225,36,169,484,576,4,36,4,4,196,81,121,81,36,361,676,361,9,49,784,169,81,225,400,841,81,1,64,361,529,9,169,64,289,16,225,121,49,0,0,100,64,4,16,36,324,9,484,1089,784,529,729,121,36,121,196,289,289,196,121,49,64,36,64,16,625,576,121,361,81,361,196,484,36,529,225,36,121,4,484,100,196,144,81,169,324,64,121,81,81,196,81,4,144,225,121,49,81,361,121,64,169,16,49,361,196,196,9,81,49,49,196,841,361,196,256,100,144,576,36,81,196,676,49,16,225,529,1,841,81,36,121,289,36,49,4,576,16,121,625,121,4,361,49,4,400,100,361,289,144,121,36,100,16,4,169,441,81,16,4,361,16,81,196,36,576,1,169,9,100,256,81,361,16,100,25,49,9,1444,256,169,1,49,225,100,4,361,625,625,49,225,121,625,361,16,196,361,324,196,121,16,841,441,576,361,961,784,225,729,169,16,100,64,16,324,9,121,64,100,1089,361,1225,441,49,324,16,144,256,196,9,1,729,144,36,81,4,36,81,256,484,676,121,121,225,169,36,400,289,64,169,225,196,196,36,576,100,4,81,529,144,64,256,100,49,361,4,81,196,81,256,49,36,484,81,81,196,289,144,169,64,100,256,1,36,16,64,16,225,49,196,256,225,529,361,16,144,144,625,196,196,121,49,324,16,1369,169,196,289,25,4,0,9,441,400,361,16,361,121,196,64,169,576,1,1,9,16,441,100,169,81,196,4,441,484,196,49],[7,7,7,7,14,7,7,3,7,7,3,7,16,10,7,10,7,12,7,7,16,10,3,7,7,14,7,7,12,12,7,3,10,14,12,3,3,3,7,17,12,9,16,3,7,7,16,10,7,7,7,3,7,7,3,12,7,17,7,7,3,12,7,7,7,12,16,7,7,7,12,10,9,0,10,14,7,3,12,12,7,17,3,7,7,12,7,7,12,10,0,12,10,7,7,7,3,12,7,12,7,7,10,14,7,12,7,7,10,7,12,7,7,17,7,7,7,10,10,12,7,12,7,10,7,10,7,7,7,7,7,7,16,12,7,3,7,7,7,12,7,12,12,14,7,7,7,12,12,14,10,12,7,16,7,17,3,10,9,7,3,16,12,7,7,7,12,3,7,7,7,7,10,10,7,12,17,10,7,7,12,7,12,7,7,7,7,14,7,12,7,7,12,3,7,12,12,7,7,14,12,17,17,7,7,10,7,7,7,3,0,7,12,7,7,12,7,7,12,7,7,3,7,7,10,12,7,12,7,7,7,7,12,7,7,7,7,7,14,17,7,10,7,7,12,7,7,9,7,14,7,3,16,3,16,7,16,12,7,7,12,12,12,16,7,9,7,12,12,10,7,12,7,7,3,10,17,7,3,3,12,7,7,7,10,7,10,7,9,9,12,12,12,7,9,12,7,12,7,12,12,14,7,14,10,12,7,7,12,7,7,12,12,12,12,14,10,7,7,7,7,7,7,7,7,7,12,12,7,14,10,12,7,7,12,7,10,12,10,7,14,7,12,12,7,16,0,12,7,17,7,12,3,7,14,7,12,12,7,7,14,7,12,12,7,7,7,7,3,12,7,7,14,10,10,10,10,12,3,7,16,7,7,0,7,7,10,7,12,7,7,7,7,16,12,10,7,7,16,7,7,7,12,10,10,10,7,10,12,12,7,17,12,16,10,16,7,7,9,3,7,7,7,7,7,7,16,12],[12,7,12,7,12,14,14,3,7,7,12,14,16,10,7,16,10,12,7,12,10,12,7,7,12,16,3,3,12,12,7,3,12,7,12,10,3,10,7,14,12,9,14,3,12,12,14,10,7,12,7,7,12,7,7,12,7,17,17,12,14,12,7,7,7,12,12,12,7,12,12,10,7,0,7,12,7,3,10,7,12,12,7,7,7,7,7,7,7,10,7,12,10,12,7,7,7,14,7,12,12,7,7,14,12,10,7,7,7,7,12,7,12,10,10,7,7,7,12,7,7,12,14,12,7,10,7,7,12,10,7,7,12,10,7,12,7,7,7,7,3,12,16,7,3,12,7,12,12,16,12,12,7,14,7,10,7,14,7,7,12,12,17,7,7,3,12,7,7,7,3,7,10,10,12,7,14,10,7,7,10,12,12,7,7,7,12,7,12,12,12,10,12,10,12,12,12,7,12,12,12,12,16,7,16,7,7,10,12,10,0,7,12,12,10,12,3,7,12,10,7,7,7,7,12,12,7,12,7,10,10,7,12,17,7,7,7,7,12,14,7,12,7,7,16,7,10,12,7,16,10,3,16,7,12,7,7,7,12,12,7,10,14,16,7,10,7,14,14,12,7,7,3,7,7,7,12,10,7,3,12,7,12,7,10,7,0,7,10,9,12,12,12,3,9,12,12,14,7,12,12,12,7,12,10,12,7,7,3,12,7,16,12,12,7,14,7,7,12,10,7,3,7,7,10,7,7,12,12,12,10,14,7,7,14,10,10,7,7,7,14,7,12,14,14,14,0,16,7,12,7,10,7,10,10,14,7,12,7,7,12,14,12,7,7,7,12,16,3,16,7,16,7,10,10,10,10,12,10,7,16,7,7,7,7,7,10,10,12,7,7,7,7,14,14,7,7,7,16,12,7,7,12,12,12,10,3,12,12,12,7,16,12,10,10,12,7,7,7,7,7,7,7,7,7,7,12,12],[13.4203646654842,11.8612192298621,13.4320752813252,11.8959890152209,13.2666507160944,13.7401237638273,13.9052416564652,10.7190230275878,12.0837209308513,12.1100802020396,13.4320752813252,13.955746283668,14.0581556357017,12.8849830472993,11.8959890152209,14.3941428734491,12.9168147931529,13.2344430606819,12.0950694845271,13.3233796370804,12.6642916571386,13.3698600382808,12.1133803037106,12.0936212358642,13.1623413600177,14.5192974838693,10.8617781802533,10.9141623552506,13.2666507160944,13.2666507160944,11.9006958233752,10.9606427564511,13.4412237672203,12.0819106200227,13.3727565356065,12.4520107811485,11.0385928413394,12.8157778439608,12.0837209308513,13.9052416564652,13.0349728344237,12.6172922382071,13.7698246788663,10.938683620824,13.2344430606819,13.1623413600177,13.8832825208381,12.9069283355331,12.1027697217594,13.2344430606819,12.0508029985005,11.9006958233752,13.4320752813252,12.1093560777082,11.8293182470425,13.3233796370804,11.7030359079455,14.6863549481147,14.4666943548784,13.2344430606819,13.988678063412,13.3479009026538,12.0314059928198,12.0936212358642,12.0508029985005,13.3892570439614,13.4203646654836,13.3698600382808,11.9849255916194,13.4221749763122,13.3892570439614,12.8707103015113,11.9885462132765,10.1575703291745,12.0676378742348,13.4532964452276,11.7415370941071,11.0129576944825,12.787998018896,11.8959890152209,13.4423099537175,13.3002789253834,12.0094468571928,11.8959890152209,11.9885462132765,11.9885462132765,11.7476921509242,12.1148423997666,11.9578421960996,12.8707103015113,12.1027697217594,13.3698600382808,12.8966936631408,13.2344430606819,12.0676378742348,12.1038559082565,12.0837209308513,13.6153727577524,11.8959890152209,13.0799911395681,13.0799911395681,12.0819106200227,11.9849255916194,13.9052416564652,13.3233796370804,12.6990614424975,12.1148423997666,12.0533374336605,12.1148423997666,11.7839932693051,13.2666507160944,12.0127054166842,13.4060919196957,12.787998018896,12.8966936631408,12.1137423658763,11.9885462132765,11.9849255916194,13.0349728344237,11.9578421960996,12.1148423997666,13.433523529988,13.6578289329505,13.4060919196957,11.9578421960996,12.9124285049847,11.8238873145568,12.0936212358642,13.3698600382808,12.73126909791,11.6489383538719,11.6489383538719,13.3479009026538,12.7609146233762,11.7415370941071,13.1623413600177,11.8959890152209,12.1133803037106,11.7839932693051,12.1038559082565,10.7955248638133,13.3511594621452,14.5042867663568,12.0343024901455,10.9606427564511,13.2344430606819,12.0314059928198,13.4203646654836,13.4478101231691,14.5185733595379,13.4203646654836,13.3698600382808,11.9281966706335,13.8316778597449,11.8959890152209,12.7609146233762,11.8238873145568,13.9436459108744,12.0837209308513,12.0314059928198,13.4532964452276,13.3233796370804,14.7917504906885,12.0819106200227,12.1038559082565,10.8252257788522,13.433523529988,12.0936212358642,11.8959890152209,12.0314059928198,10.6707738577383,12.1038559082565,12.8125192844694,12.8849830472993,13.3892570439614,11.9849255916194,13.9414735378801,12.9164527309872,11.9578421960996,12.0314059928198,12.787998018896,13.3233796370804,13.4203646654836,11.9849255916194,11.7415370941071,12.0508029985005,13.4320752813252,12.0314059928198,13.2666507160944,13.3233796370804,13.4532964452276,12.8344784200964,13.2962962415606,12.8707103015113,13.1623413600177,13.2666507160944,13.4532964452276,12.0819106200227,13.4203646654836,13.1224473147661,13.3233796370804,13.2666507160944,14.3374139524632,12.0819106200227,14.3977634951062,12.1148423997666,12.0819106200227,12.905842149036,13.3479009026538,12.8538754257771,10.2098852672059,11.8959890152209,13.3233796370804,13.4203646654836,12.8564098609371,13.2666507160944,10.753124078188,12.0936212358642,13.433523529988,12.4995912162393,11.9885462132765,11.9849255916194,11.8959890152209,12.0314059928198,13.4478101231691,13.2344430606819,11.9281966706335,13.0799911395681,12.0837209308513,12.6269597418333,12.8344784200964,12.0698102472291,13.3698600382808,14.418445185029,12.1148423997666,11.9281966706335,11.7415370941071,12.1137423658763,13.3479009026538,13.988678063412,12.1093560777082,13.3892570439614,12.0314059928198,11.8959890152209,14.4186641390225,11.8238873145568,12.5446095213837,13.4060919196957,12.1100802020396,14.3941428734491,12.6269597418333,10.6707738577383,14.5240596815963,11.8238873145568,13.3233796370804,12.0819106200227,11.8959890152209,12.0837209308513,13.0349728344237,13.4060919196957,11.7839932693051,12.8125192844694,13.9766053854047,14.3941428734491,12.1148423997666,12.6269597418333,12.0094468571928,13.7350548935074,13.8020323342788,13.1224473147661,11.6558175350205,12.1027697217594,10.996874637866,11.6965187889627,11.9281966706335,12.0936212358642,13.3479009026538,12.5446095213837,12.1148423997666,10.9990470108603,13.40826429269,11.9281966706335,13.4320752813252,12.0314059928198,12.8728826745056,12.1148423997666,9.95005165091144,12.0819106200227,12.9179148270432,12.648761921895,13.4203646654836,13.3698600382808,13.1623413600177,10.9177829769078,12.645461820224,13.4221749763122,13.4532964452276,13.8063770802673,12.0127054166842,13.4320752813252,13.3727565356065,13.4060919196957,11.8238873145568,13.3479009026538,12.7609146233762,13.1623413600177,12.1133803037106,11.7839932693051,10.9606427564511,13.2962962415606,12.0094468571928,14.2755053820117,13.4532964452276,13.1282403094175,12.1100802020396,13.8020323342788,12.1133803037106,11.8238873145568,13.3892570439614,12.905842149036,12.0819106200227,10.7132300329364,11.6965187889627,12.0343024901455,12.8538754257771,11.8959890152209,11.9849255916194,13.0799911395681,13.2344430606819,13.3233796370804,12.905842149036,13.9776915719019,12.0533374336605,12.0314059928198,13.9052416564652,12.8966936631408,12.8707103015113,11.8959890152209,12.1137423658763,12.1093560777082,13.8316778597449,12.0676378742348,13.4320752813252,13.955746283668,13.955746283668,13.7698246788663,10.2098852672059,14.4186641390225,11.7415370941071,13.3233796370804,12.0950694845271,12.8538754257771,11.9578421960996,12.905842149036,12.8125192844694,13.8020323342788,12.1148423997666,13.0799911395681,11.9849255916194,12.0819106200227,13.3233796370804,13.9766053854047,13.2666507160944,11.8959890152209,12.1038559082565,11.9849255916194,13.3233796370804,14.4911279018524,11.0385928413394,14.4600202803302,12.0676378742348,14.3670594779293,12.0094468571928,12.905842149036,12.4995912162393,12.6990614424975,12.6269597418333,13.2962962415606,12.6269597418333,12.0936212358642,14.3374139524632,12.0819106200227,12.1027697217594,12.0936212358642,12.0950694845271,12.1148423997666,12.6269597418333,12.8538754257771,13.3892570439614,12.0698102472291,12.0819106200227,12.0819106200227,12.0314059928198,13.8020323342788,13.9872159673559,11.8238873145568,11.7030359079455,12.0676378742348,14.4911279018524,13.4478101231691,11.861219229862,11.7415370941071,12.9873923993329,13.1224473147661,13.4485342475006,12.9168147931529,11.0440791633979,13.1623413600177,13.4532964452276,13.3698600382808,12.0819106200227,14.3670594779293,13.4060919196957,12.8867933581278,12.4995912162393,13.0349728344237,11.7839932693051,11.8238873145568,12.1100802020396,12.0094468571928,12.0676378742348,11.9849255916194,12.0819106200227,11.7415370941071,12.1100802020396,12.1038559082565,13.4203646654836,13.2666507160944]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>id<\/th>\n      <th>lwage<\/th>\n      <th>educ<\/th>\n      <th>exper<\/th>\n      <th>expersq<\/th>\n      <th>fatheduc<\/th>\n      <th>motheduc<\/th>\n      <th>educHat<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"tip","columnDefs":[{"targets":2,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\", null);\n  }"},{"targets":8,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\", null);\n  }"},{"className":"dt-center","targets":"_all"},{"visible":false,"targets":0},{"orderable":false,"targets":0}],"pageLength":5,"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[5,10,25,50,100]}},"evals":["options.columnDefs.0.render","options.columnDefs.1.render"],"jsHooks":[]}</script>

---

### Step-by-step solution: stage 2 model

**Stage 2 of 2SLS**: with mother education as instrument

In the second stage, we will regress log(wage) on the `$\widehat{educ}$`  from stage 1 and experience and its quadratic term exp square.

`$$\begin{align}
lwage = \hat{\beta}_1 +\hat{\beta}_2\widehat{educ} + \hat{\beta}_3exper +\hat{\beta}_4expersq + \hat{\epsilon}
\end{align}$$`

```r
mod_step2 <- formula(lwage~educHat + exper + expersq)
ols_step2 <- lm(formula = mod_step2, data = mroz_add)
```

---

### Step-by-step solution: stage 2 OLS estimate

By using the new data set (`moroz_add`), the result of the explicit 2SLS procedure are shown as below.

`$$\begin{equation}
\begin{alignedat}{999}
&\widehat{lwage}=&&+0.20&&+0.05educHat_i&&+0.04exper_i&&-0.00expersq_i\\ 
&(s)&&(0.4933)&&(0.0391)&&(0.0142)&&(0.0004)\\ 
&(t)&&(+0.40)&&(+1.26)&&(+3.17)&&(-2.17)\\ 
&(fit)&&R^2=0.0456&&\bar{R}^2=0.0388 && &&\\ 
&(Ftest)&&F^*=6.75&&p=0.0002 && &&
\end{alignedat}
\end{equation}$$`

Keep in mind, however, that the **standard errors** calculated in this way are incorrect (Why?).

]

???
while the t-test on the coefficient of education is not significant because the t statistics is less than the critical value 2. But the model F-test is significant with small p value here.

---

### Integrated solution:  the whole story

We need a **Integrated solution** for following reasons:

- We should obtain the correct estimated error for test and inference.

- We should avoid tedious steps in the former step-by-step routine. When the model contains more than one endogenous regressors and there are lots available instruments, then the step-by-step solution will get extremely tedious.

---

### Integrated solution: the `R` toolbox

In `R` ecosystem, we have two packages to execute the  integrated solution:

- We can use `systemfit` package function `systemfit::systemfit()`.

- Or we may use `ARE` package function `ARE::ivreg()`.

Both of these tools can conduct the integrated solution, and will adjust the variance of estimators automatically.

---
exclude: true

### Rscript: `systemfit` for 2SLS (m)

---

### Rscript: `ARE::ivreg()` for IV (m)

---

### Integrated solution: `motheduc` IV model

In order to get the correct estimated error, we need use the  **"integrated solution"** for 2SLS. And we will process the estimation with proper software and tools.

Firstly, let's consider using `$matheduc$` as the only instrument for `$educ$`.

`$$\begin{cases}
  \begin{align}
  \widehat{educ} &= \hat{\gamma}_1 +\hat{\gamma}_2exper + \hat{\gamma}_3expersq +\hat{\gamma}_4motheduc  && \text{(stage 1)}\\
  lwage & = \hat{\beta}_1 +\hat{\beta}_2\widehat{educ} + \hat{\beta}_3exper +\hat{\beta}_4expersq + \hat{\epsilon}  && \text{(stage 2)}
  \end{align}
\end{cases}$$`

---

### Integrated solution: `motheduc` IV results

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-567ef846d5a2b12a7214" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-567ef846d5a2b12a7214">{"x":{"filter":"none","vertical":false,"caption":"<caption>2SLS result(`motheduc` as instrument)<\/caption>","data":[["1","2","3","4","5","6","7","8"],["eq1","eq1","eq1","eq1","eq2","eq2","eq2","eq2"],["(Intercept)","exper","expersq","motheduc","(Intercept)","educ","exper","expersq"],[9.77510269022664,0.0488615000639512,-0.00128106497318671,0.267690809092189,0.198186056472701,0.0492629533503828,0.0448558478735965,-0.00092207616246943],[0.423888615357072,0.0416692604217384,0.00124490562440028,0.0311297966206036,0.472877229537861,0.0374360256307112,0.0135768173487378,0.000406381308328035],[23.0605454737027,1.17260300685492,-1.02904585542687,8.59918271727529,0.419106787329105,1.31592369970944,3.30385588326167,-2.26899255347916],[0,0.241613422371949,0.304044786477339,0,0.675350330266827,0.188910669909711,0.00103457078695701,0.0237705466653522]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>eq<\/th>\n      <th>vars<\/th>\n      <th>Estimate<\/th>\n      <th>Std. Error<\/th>\n      <th>t value<\/th>\n      <th>Pr(>|t|)<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"t","columnDefs":[{"targets":3,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":4,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":5,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":6,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"className":"dt-center","targets":"_all"},{"visible":false,"targets":0},{"orderable":false,"targets":0}],"pageLength":10,"order":[],"autoWidth":false,"orderClasses":false}},"evals":["options.columnDefs.0.render","options.columnDefs.1.render","options.columnDefs.2.render","options.columnDefs.3.render"],"jsHooks":[]}</script>

- The t-test for variable `educ` is significant (p-value less than 0.05).

**Note** : The corresponding code of `R` programming is in the following slides. The table results use the report from the `systemfit::systemfit()` function.

]

---

### (Supplements) R code (m): `systemfit::systemfit()`

The R code using `systemfit::systemfit()` as follows:

```r
# load pkg
require(systemfit)
# set two models
eq_1 <- educ ~  exper + expersq + motheduc
eq_2 <- lwage ~ educ + exper + expersq 
sys <- list(eq1 = eq_1, eq2 = eq_2)
# specify the instruments
instr <- ~  exper + expersq + motheduc
# fit models 
fit.sys <- systemfit(
  sys, inst=instr, 
  method="2SLS", data = mroz)
# summary of model fit
smry.system_m <- summary(fit.sys) 
```

---

### (Supplements) R report (m): `systemfit::systemfit()`

The following is the 2SLS analysis report using `systemfit::systemfit() `:

```r
smry.system_m
```

```

systemfit results 
method: 2SLS

N  DF  SSR detRCov OLS-R2 McElroy-R2
system 856 848 2085    1.97   0.15      0.112

N  DF  SSR   MSE RMSE    R2 Adj R2
eq1 428 424 1890 4.457 2.11 0.153  0.147
eq2 428 424  196 0.462 0.68 0.123  0.117

The covariance matrix of the residuals
      eq1   eq2
eq1 4.457 0.305
eq2 0.305 0.462

The correlations of the residuals
      eq1   eq2
eq1 1.000 0.212
eq2 0.212 1.000

2SLS estimates for 'eq1' (equation 1)
Model Formula: educ ~ exper + expersq + motheduc
Instruments: ~exper + expersq + motheduc

Estimate Std. Error t value Pr(>|t|)    
(Intercept)  9.77510    0.42389   23.06   <2e-16 ***
exper        0.04886    0.04167    1.17     0.24    
expersq     -0.00128    0.00124   -1.03     0.30    
motheduc     0.26769    0.03113    8.60   <2e-16 ***
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 2.111 on 424 degrees of freedom
Number of observations: 428 Degrees of Freedom: 424 
SSR: 1889.658 MSE: 4.457 Root MSE: 2.111 
Multiple R-Squared: 0.153 Adjusted R-Squared: 0.147

2SLS estimates for 'eq2' (equation 2)
Model Formula: lwage ~ educ + exper + expersq
Instruments: ~exper + expersq + motheduc

Estimate Std. Error t value Pr(>|t|)   
(Intercept)  0.198186   0.472877    0.42    0.675   
educ         0.049263   0.037436    1.32    0.189   
exper        0.044856   0.013577    3.30    0.001 **
expersq     -0.000922   0.000406   -2.27    0.024 * 
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.68 on 424 degrees of freedom
Number of observations: 428 Degrees of Freedom: 424 
SSR: 195.829 MSE: 0.462 Root MSE: 0.68 
Multiple R-Squared: 0.123 Adjusted R-Squared: 0.117 
```

]

**NOTE** : `systemfit::systemfit()` simultaneously reports the analysis results of two equations in 2SLS!

]

---

### (Supplements) R code (m): `ARE::ivreg()`

The R code using `ARE::ivreg()` as follows:

```r
# load pkg 
require(AER)
# specify model
mod_iv_m <- formula(lwage ~ educ + exper + expersq
                     | motheduc + exper + expersq)
# fit model
lm_iv_m <- ivreg(formula = mod_iv_m, data = mroz)
# summary of model fit
smry.ivm <- summary(lm_iv_m)
```

]

---

### (Supplements) R report (m): `ARE::ivreg()`

The following is the 2SLS analysis report using `ARE::ivreg()`:

```r
smry.ivm
```

```

Call:
ivreg(formula = mod_iv_m, data = mroz)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.1080 -0.3263  0.0602  0.3677  2.3435

Coefficients:
             Estimate Std. Error t value Pr(>|t|)   
(Intercept)  0.198186   0.472877    0.42    0.675   
educ         0.049263   0.037436    1.32    0.189   
exper        0.044856   0.013577    3.30    0.001 **
expersq     -0.000922   0.000406   -2.27    0.024 * 
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.68 on 424 degrees of freedom
Multiple R-Squared: 0.123,	Adjusted R-squared: 0.117 
Wald test: 7.35 on 3 and 424 DF,  p-value: 8.23e-05 
```

]

**Note**: `ARE::ivreg()` Only reports the result of the last equation of 2SLS, not include the first equation!

]

???

We can see that the t-test for
`$educ$` is not significant.

We should note that the instruments (motheduc + exper + expersq) are included as whole behind the procedure in this code chunk. But we do not  see these instruments in the output.

We can see that the t-test on the coefficient of education is still not significant.

---
exclude: true

### Rscript: `systemfit` for 2SLS (f)

---

### Rscript: `ARE::ivreg()` for IV (f)

---

### Integrated solution: `fatheduc` IV model

Now let's consider using `$fatheduc$` as the only instrument for `$educ$`.

`$$\begin{cases}
  \begin{align}
  \widehat{educ} &= \hat{\gamma}_1 +\hat{\gamma}_2exper + \hat{\gamma}_3expersq +\hat{\gamma}_4fatheduc  && \text{(stage 1)}\\
  lwage & = \hat{\beta}_1 +\hat{\beta}_2\widehat{educ} + \hat{\beta}_3exper +\hat{\beta}_4expersq + \hat{\epsilon}  && \text{(stage 2)}
  \end{align}
\end{cases}$$`

We will repeat the whole procedure with `R`.

???
We will repeat the whole procedure with `R`.

---

### Integrated solution: `fatheduc` IV results

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-251a17c0b14ab3736aab" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-251a17c0b14ab3736aab">{"x":{"filter":"none","vertical":false,"caption":"<caption>2SLS result(`fatheduc` as instrument)<\/caption>","data":[["1","2","3","4","5","6","7","8"],["eq1","eq1","eq1","eq1","eq2","eq2","eq2","eq2"],["(Intercept)","exper","expersq","fatheduc","(Intercept)","educ","exper","expersq"],[9.88703428977956,0.0468243339408936,-0.00115038254654544,0.270506101172372,-0.0611169333074602,0.070226291272055,0.0436715881293298,-0.000882154958614193],[0.395607787564272,0.041107424252616,0.00122856802818919,0.0288785943434333,0.436446127555986,0.0344426941325624,0.0134001210314073,0.000400917007546146],[24.9920112813079,1.13907243745426,-0.936360478337541,9.36701066386505,-0.140033166635486,2.03893142045653,3.25904430467248,-2.20034306854058],[0,0.255316073334826,0.349620629655963,0,0.88870028062505,0.0420765724763237,0.00120792842331396,0.0283211937528232]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>eq<\/th>\n      <th>vars<\/th>\n      <th>Estimate<\/th>\n      <th>Std. Error<\/th>\n      <th>t value<\/th>\n      <th>Pr(>|t|)<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"t","columnDefs":[{"targets":3,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":4,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":5,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":6,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"className":"dt-center","targets":"_all"},{"visible":false,"targets":0},{"orderable":false,"targets":0}],"pageLength":10,"order":[],"autoWidth":false,"orderClasses":false}},"evals":["options.columnDefs.0.render","options.columnDefs.1.render","options.columnDefs.2.render","options.columnDefs.3.render"],"jsHooks":[]}</script>

- The t-test for variable `educ` is significant (p-value less than 0.05).

**Note** : The corresponding code of `R` programming is in the following slides. The table results use the report from the `systemfit::systemfit()` function.

]

---

### (Supplements) R code (f): `systemfit::systemfit()`

The R code using `systemfit::systemfit()` as follows:

```r
# load pkg
require(systemfit)
# set two models
eq_1 <- educ ~  exper + expersq + fatheduc
eq_2 <- lwage ~ educ + exper + expersq 
sys <- list(eq1 = eq_1, eq2 = eq_2)
# specify the instruments
instr <- ~ exper + expersq + fatheduc
# fit models 
fit.sys <- systemfit(
  sys, inst=instr, 
  method="2SLS", data = mroz)
# summary of model fit
smry.system_f <- summary(fit.sys) 
```

---

### (Supplements) R report (f): `systemfit::systemfit()`

The following is the 2SLS analysis report using `systemfit::systemfit() `:

```r
smry.system_f
```

```

systemfit results 
method: 2SLS

N  DF  SSR detRCov OLS-R2 McElroy-R2
system 856 848 2030    1.92  0.173      0.135

N  DF  SSR   MSE  RMSE    R2 Adj R2
eq1 428 424 1839 4.337 2.082 0.176  0.170
eq2 428 424  191 0.451 0.672 0.143  0.137

The covariance matrix of the residuals
      eq1   eq2
eq1 4.337 0.195
eq2 0.195 0.451

The correlations of the residuals
      eq1   eq2
eq1 1.000 0.139
eq2 0.139 1.000

2SLS estimates for 'eq1' (equation 1)
Model Formula: educ ~ exper + expersq + fatheduc
Instruments: ~exper + expersq + fatheduc

Estimate Std. Error t value Pr(>|t|)    
(Intercept)  9.88703    0.39561   24.99   <2e-16 ***
exper        0.04682    0.04111    1.14     0.26    
expersq     -0.00115    0.00123   -0.94     0.35    
fatheduc     0.27051    0.02888    9.37   <2e-16 ***
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 2.082 on 424 degrees of freedom
Number of observations: 428 Degrees of Freedom: 424 
SSR: 1838.719 MSE: 4.337 Root MSE: 2.082 
Multiple R-Squared: 0.176 Adjusted R-Squared: 0.17

2SLS estimates for 'eq2' (equation 2)
Model Formula: lwage ~ educ + exper + expersq
Instruments: ~exper + expersq + fatheduc

Estimate Std. Error t value Pr(>|t|)   
(Intercept) -0.061117   0.436446   -0.14   0.8887   
educ         0.070226   0.034443    2.04   0.0421 * 
exper        0.043672   0.013400    3.26   0.0012 **
expersq     -0.000882   0.000401   -2.20   0.0283 * 
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.672 on 424 degrees of freedom
Number of observations: 428 Degrees of Freedom: 424 
SSR: 191.387 MSE: 0.451 Root MSE: 0.672 
Multiple R-Squared: 0.143 Adjusted R-Squared: 0.137 
```

]

**NOTE** : `systemfit::systemfit()` simultaneously reports the analysis results of two equations in 2SLS!

]

---

### (Supplements) R code (f): `ARE::ivreg()`

The R code using `ARE::ivreg()` as follows:

```r
# load pkg 
require(AER)
# specify model
mod_iv_f <- formula(lwage ~ educ + exper + expersq
                     | fatheduc + exper + expersq)
# fit model
lm_iv_f <- ivreg(formula = mod_iv_f, data = mroz)
# summary of model fit
smry.ivf <- summary(lm_iv_f)
```

]

???

We can see the insturments (fatheduc + exper + expersq) are included as whole behind the procedure in this code chunk.

While, We can find that the t-test on the coefficient of education is significant now with its p value less than 0.05.

---

### (Supplements) R report (f): `ARE::ivreg()`

The following is the 2SLS analysis report using `ARE::ivreg()`:

```r
smry.ivf
```

```

Call:
ivreg(formula = mod_iv_f, data = mroz)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.0917 -0.3278  0.0501  0.3736  2.3535

Coefficients:
             Estimate Std. Error t value Pr(>|t|)   
(Intercept) -0.061117   0.436446   -0.14   0.8887   
educ         0.070226   0.034443    2.04   0.0421 * 
exper        0.043672   0.013400    3.26   0.0012 **
expersq     -0.000882   0.000401   -2.20   0.0283 * 
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.67 on 424 degrees of freedom
Multiple R-Squared: 0.143,	Adjusted R-squared: 0.137 
Wald test: 8.31 on 3 and 424 DF,  p-value: 2.2e-05 
```

]

**Note**: `ARE::ivreg()` Only reports the result of the last equation of 2SLS, not include the first equation!

]

---
exclude: true

### Rscript: `systemfit` for 2SLS (mf)

---

### Rscript: `ARE::ivreg()` for IV (mf)

---

### Integrated solution: `mothedu` and `fatheduc` IV model

Also, we can use both `$matheduc$` and `$fatheduc$` as instruments for `$educ$`.

`$$\begin{cases}
  \begin{align}
  \widehat{educ} &= \hat{\gamma}_1 +\hat{\gamma}_2exper + \hat{\beta}_3expersq +\hat{\beta}_4motheduc + \hat{\beta}_5fatheduc  && \text{(stage 1)}\\
  lwage & = \hat{\beta}_1 +\hat{\beta}_2\widehat{educ} + \hat{\beta}_3exper +\hat{\beta}_4expersq + \hat{\epsilon}  && \text{(stage 2)}
  \end{align}
\end{cases}$$`

---

### Integrated solution: `mothedu` and `fatheduc` IV results

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-0e2c2a4f3e5b69f029e3" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-0e2c2a4f3e5b69f029e3">{"x":{"filter":"none","vertical":false,"caption":"<caption>2SLS result(`motheduc` and `fatheduc` as instruments)<\/caption>","data":[["1","2","3","4","5","6","7","8","9"],["eq1","eq1","eq1","eq1","eq1","eq2","eq2","eq2","eq2"],["(Intercept)","exper","expersq","motheduc","fatheduc","(Intercept)","educ","exper","expersq"],[9.10264010960015,0.0452254233687072,-0.00100909095717081,0.157597032748592,0.189548410154955,0.0481003069324232,0.0613966286601352,0.0441703929487622,-0.000898969588155513],[0.426561367230798,0.0402507123800771,0.00120334481233516,0.0358941155466901,0.0337564667819247,0.400328077604299,0.0314366956447082,0.0134324755294433,0.00040168561187618],[21.3395792701382,1.12359311660512,-0.838571743383022,4.39060916666395,5.61517327567147,0.120152219200492,1.95302424128887,3.28832856251575,-2.23799300143372],[0,0.261822937931716,0.402183285060695,1.42984036721749e-05,3.56151237390634e-08,0.904419479360811,0.0514741739152211,0.00109183842526983,0.0257400273342565]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>eq<\/th>\n      <th>vars<\/th>\n      <th>Estimate<\/th>\n      <th>Std. Error<\/th>\n      <th>t value<\/th>\n      <th>Pr(>|t|)<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"t","columnDefs":[{"targets":3,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":4,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":5,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":6,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"className":"dt-center","targets":"_all"},{"visible":false,"targets":0},{"orderable":false,"targets":0}],"pageLength":10,"order":[],"autoWidth":false,"orderClasses":false}},"evals":["options.columnDefs.0.render","options.columnDefs.1.render","options.columnDefs.2.render","options.columnDefs.3.render"],"jsHooks":[]}</script>

**Note** : The corresponding code of `R` programming is in the following slides. The table results use the report from the `systemfit::systemfit()` function.

]

???

We can see that the t-test result of the coefficient of variable `educ` is significant (p-value less than 0.1).

The insturments (motheduc +fatheduc + exper + expersq) are included behind the procedure in this code chunk.

And we can find that the t-test on the coefficient of education is significant with its p value less than 0.1.

---

### (Supplements) R code (mf): `systemfit::systemfit()`

The R code using `systemfit::systemfit()` as follows:

```r
# load pkg
require(systemfit)
# set two models
eq_1 <- educ ~ exper + expersq + motheduc + fatheduc
eq_2 <- lwage ~ educ + exper + expersq 
sys <- list(eq1 = eq_1, eq2 = eq_2)
# specify the instruments
instr <- ~ exper + expersq + motheduc + fatheduc
# fit models 
fit.sys <- systemfit(
  sys, inst=instr, 
  method="2SLS", data = mroz)
# summary of model fit
smry.system_mf <- summary(fit.sys) 
```

---

### (Supplements) R report (mf): `systemfit::systemfit()`

The following is the 2SLS analysis report using `systemfit::systemfit() `:

```r
smry.system_mf
```

```

systemfit results 
method: 2SLS

N  DF  SSR detRCov OLS-R2 McElroy-R2
system 856 847 1952    1.83  0.205      0.149

N  DF  SSR   MSE  RMSE    R2 Adj R2
eq1 428 423 1759 4.157 2.039 0.211  0.204
eq2 428 424  193 0.455 0.675 0.136  0.130

The covariance matrix of the residuals
      eq1   eq2
eq1 4.157 0.242
eq2 0.242 0.455

The correlations of the residuals
      eq1   eq2
eq1 1.000 0.176
eq2 0.176 1.000

2SLS estimates for 'eq1' (equation 1)
Model Formula: educ ~ exper + expersq + motheduc + fatheduc
Instruments: ~exper + expersq + motheduc + fatheduc

Estimate Std. Error t value Pr(>|t|)    
(Intercept)  9.10264    0.42656   21.34  < 2e-16 ***
exper        0.04523    0.04025    1.12     0.26    
expersq     -0.00101    0.00120   -0.84     0.40    
motheduc     0.15760    0.03589    4.39  1.4e-05 ***
fatheduc     0.18955    0.03376    5.62  3.6e-08 ***
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 2.039 on 423 degrees of freedom
Number of observations: 428 Degrees of Freedom: 423 
SSR: 1758.575 MSE: 4.157 Root MSE: 2.039 
Multiple R-Squared: 0.211 Adjusted R-Squared: 0.204

2SLS estimates for 'eq2' (equation 2)
Model Formula: lwage ~ educ + exper + expersq
Instruments: ~exper + expersq + motheduc + fatheduc

Estimate Std. Error t value Pr(>|t|)   
(Intercept)  0.048100   0.400328    0.12   0.9044   
educ         0.061397   0.031437    1.95   0.0515 . 
exper        0.044170   0.013432    3.29   0.0011 **
expersq     -0.000899   0.000402   -2.24   0.0257 * 
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.675 on 424 degrees of freedom
Number of observations: 428 Degrees of Freedom: 424 
SSR: 193.02 MSE: 0.455 Root MSE: 0.675 
Multiple R-Squared: 0.136 Adjusted R-Squared: 0.13 
```

]

**NOTE** : `systemfit::systemfit()` simultaneously reports the analysis results of two equations in 2SLS!

]

---

### (Supplements) R code (mf): `ARE::ivreg()`

The R code using `ARE::ivreg()` as follows:

```r
# load pkg 
require(AER)
# specify model
mod_iv_mf <- formula(
  lwage ~ educ + exper + expersq
         | motheduc + fatheduc + exper + expersq)
# fit model
lm_iv_mf <- ivreg(formula = mod_iv_mf, data = mroz)
# summary of model fit
smry.ivmf <- summary(lm_iv_mf)
```

]

---

### (Supplements) R report (mf): `ARE::ivreg()`

The following is the 2SLS analysis report using `ARE::ivreg()`:

```r
smry.ivmf
```

```

Call:
ivreg(formula = mod_iv_mf, data = mroz)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.0986 -0.3196  0.0551  0.3689  2.3493

Coefficients:
             Estimate Std. Error t value Pr(>|t|)   
(Intercept)  0.048100   0.400328    0.12   0.9044   
educ         0.061397   0.031437    1.95   0.0515 . 
exper        0.044170   0.013432    3.29   0.0011 **
expersq     -0.000899   0.000402   -2.24   0.0257 * 
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.68 on 424 degrees of freedom
Multiple R-Squared: 0.136,	Adjusted R-squared: 0.13 
Wald test: 8.14 on 3 and 424 DF,  p-value: 2.79e-05 
```

]

**Note**: `ARE::ivreg()` Only reports the result of the last equation of 2SLS, not include the first equation!

]

---

### Solutions comparison: a glance

Until now, we obtain totally **Five** estimation results with different model settings or solutions:

a. Error specification model with OLS regression directly.

b. (**Step-by-Step solution**) Explicit 2SLS estimation **without** variance correction (IV regression step by step with only `$matheduc$` as instrument).

c. (**Integrated solution**) Dedicated IV estimation **with** variance correction ( using `R` tools of `systemfit::systemfit()` or `ARE::ivreg()`).

- The IV model with only 
`$motheduc$` as instrument for endogenous variable `$edu$`

- The IV model with only 
`$fatheduc$` as instrument for endogenous variable `$edu$`

- The IV model with  both 
`$motheduc$` and 
`$fatheduc$` as instruments

For the purpose of comparison, all results will show in next slide.

???

we use `R` function `ARE::ivreg()` to get the IV estimation **with** variance correction with the last three model considering different instruments.

---

### Solutions comparison: tidy reports (png)

---

### Solutions comparison: tidy reports (html)

<table style="text-align:center"><tr><td colspan="6" style="border-bottom: 1px solid black"></td></tr><tr><td style="text-align:left"></td><td colspan="5">Dependent variable: lwage</td></tr>
<tr><td></td><td colspan="5" style="border-bottom: 1px solid black"></td></tr>
<tr><td style="text-align:left"></td><td>OLS</td><td>explicit 2SLS</td><td>IV mothereduc</td><td>IV fathereduc</td><td>IV mothereduc and fathereduc</td></tr>
<tr><td style="text-align:left"></td><td>(1)</td><td>(2)</td><td>(3)</td><td>(4)</td><td>(5)</td></tr>
<tr><td colspan="6" style="border-bottom: 1px solid black"></td></tr><tr><td style="text-align:left">Constant</td><td>-0.5200<sup>***</sup></td><td>0.2000</td><td>0.2000</td><td>-0.0610</td><td>0.0480</td></tr>
<tr><td style="text-align:left"></td><td>(0.2000)</td><td>(0.4900)</td><td>(0.4700)</td><td>(0.4400)</td><td>(0.4000)</td></tr>
<tr><td style="text-align:left"></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td style="text-align:left">educ</td><td>0.1100<sup>***</sup></td><td></td><td>0.0490</td><td>0.0700<sup>**</sup></td><td>0.0610<sup>*</sup></td></tr>
<tr><td style="text-align:left"></td><td>(0.0140)</td><td></td><td>(0.0370)</td><td>(0.0340)</td><td>(0.0310)</td></tr>
<tr><td style="text-align:left"></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td style="text-align:left">educHat</td><td></td><td>0.0490</td><td></td><td></td><td></td></tr>
<tr><td style="text-align:left"></td><td></td><td>(0.0390)</td><td></td><td></td><td></td></tr>
<tr><td style="text-align:left"></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td style="text-align:left">exper</td><td>0.0420<sup>***</sup></td><td>0.0450<sup>***</sup></td><td>0.0450<sup>***</sup></td><td>0.0440<sup>***</sup></td><td>0.0440<sup>***</sup></td></tr>
<tr><td style="text-align:left"></td><td>(0.0130)</td><td>(0.0140)</td><td>(0.0140)</td><td>(0.0130)</td><td>(0.0130)</td></tr>
<tr><td style="text-align:left"></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td style="text-align:left">expersq</td><td>-0.0008<sup>**</sup></td><td>-0.0009<sup>**</sup></td><td>-0.0009<sup>**</sup></td><td>-0.0009<sup>**</sup></td><td>-0.0009<sup>**</sup></td></tr>
<tr><td style="text-align:left"></td><td>(0.0004)</td><td>(0.0004)</td><td>(0.0004)</td><td>(0.0004)</td><td>(0.0004)</td></tr>
<tr><td style="text-align:left"></td><td></td><td></td><td></td><td></td><td></td></tr>
<tr><td colspan="6" style="border-bottom: 1px solid black"></td></tr><tr><td style="text-align:left">Observations</td><td>428</td><td>428</td><td>428</td><td>428</td><td>428</td></tr>
<tr><td style="text-align:left">R<sup>2</sup></td><td>0.1600</td><td>0.0460</td><td>0.1200</td><td>0.1400</td><td>0.1400</td></tr>
<tr><td style="text-align:left">Adjusted R<sup>2</sup></td><td>0.1500</td><td>0.0390</td><td>0.1200</td><td>0.1400</td><td>0.1300</td></tr>
<tr><td style="text-align:left">Residual Std. Error (df = 424)</td><td>0.6700</td><td>0.7100</td><td>0.6800</td><td>0.6700</td><td>0.6700</td></tr>
<tr><td style="text-align:left">F Statistic (df = 3; 424)</td><td>26.0000<sup>***</sup></td><td>6.8000<sup>***</sup></td><td></td><td></td><td></td></tr>
<tr><td colspan="6" style="border-bottom: 1px solid black"></td></tr></table>

]
]

???

参看在线图书"Principles of Econometrics with R"（10.1 The Instrumental Variables (IV) Method，[免费在线](https://bookdown.org/ccolonescu/RPoE4/random-regressors.html#the-instrumental-variables-iv-method)）

---

### Solutions comparison: report tips

- The second column shows the result of the direct OLS estimation, and the third column shows the result of explicit 2SLS estimation without variance correction.

- While the last three column shows the results of IV solution with variance correction.

- And we should also remind that the  `$educ$`  in the IV model is equivalent to the  `$educHat$`  in 2SLS.

- The value within the bracket is the standard error of the estimator.

---

### Solutions comparison: report insights

So the key points of this comparison including:

- Firstly, the table shows that the importance of education in determining wage decreases in the IV model (3) (4) and (5) with the coefficients 0.049, 0.07, 0.061 respectively. And the standard error also decrease along IV estimation (3) , (4) and (5).

- Secondly, It also shows that the explicit 2SLS model (2) and the IV model with only `$motheduc$`  instrument yield the same coefficients, but the **standard errors** are different. The standard error in explicit 2SLS is 0.039, which is little large than the standard error 0.037 in IV estimation.

- Thirdly, the t-test of the coefficient on education shows no significance when we use `motheduc` as the only instrument for education. You can compare this under the  explicit 2SLS estimation or IV estimation.

- Fourthly, we can fully feel and understand the **relative estimated efficiency** of 2SLS!

---

### Solutions comparison: further thinking

After the empirical comparison, we will be even more confused with these results.

While, new question will arise inside our mind.

- Which estimation is the best?

- How to judge and evaluate different instrument choices?

We will discuss these topics in the next section.

---
layout: false
class: center, middle, duke-softblue,hide_logo
name: validity

## 17.5 Testing Instrument validity

???
As we know, valid instruments should satisfy both relevance condition and exogeneity condition.

So, let us check these conditions in this section.

---
layout: true
  
<div class="my-header-h2"></div>
<div class="watermark1"></div>
<div class="watermark2"></div>
<div class="watermark3"></div>

<div class="my-footer"><span>huhuaping@   <a href="#chapter17"> Chapter 17. Endogeneity and Instumental Variables  |</a> &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; <a href="#validity"> 17.5 Testing Instrument validity  </a></span></div>

---

### Instrument vality: the concept

Consider the general model

`$$\begin{align}
Y_{i}=\beta_{0}+\sum_{j=1}^{k} \beta_{j} X_{j i}+\sum_{s=1}^{r} \beta_{k+s} W_{ri}+\epsilon_{i}
\end{align}$$`

> - `$Y_{i}$` is the dependent variable
- `$\beta_{0}, \ldots, \beta_{k+1}$` are `$1+k+r$` unknown regression coefficients
- `$X_{1 i}, \ldots, X_{k i}$` are `$k$` endogenous regressors
- `$W_{1 i}, \ldots, W_{r i}$` are `$r$` exogenous regressors which are uncorrelated with `$u_{i}$`
- `$u_{i}$` is the error term
- `$Z_{1 i}, \ldots, Z_{m i}$` are `$m$` instrumental variables

**Instrument valid** means satisfy both Relevance and Exogeneity conditions.

`$$E\left(Z_{i} X_{i}^{\prime}\right) \neq 0$$`

]

`$$E\left(Z_{i} \epsilon_{i}\right)=0$$`

]

???

Consider the general model as we have done.

---

### Instrument Relevance: relax condition

In practice, **Instrument Relevance** also means that:

If there are 
`$k$` endogenous variables and 
`$m$` instruments 
`$Z$`, and 
`$m \geq k$`, it must hold that the exogenous vector

`$$\left(\hat{X}_{1 i}^{*}, \ldots, \hat{X}_{k i}^{*}, W_{1 i}, \ldots, W_{r i}, 1\right)$$`

should .red[not] be **perfectly multicollinear**.

> **Where**:
> - `$\hat{X}_{1i}^{\ast}, \ldots, \hat{X}_{ki}^{\ast}$` are the predicted values from the  `$k$` first stage regressions.
> - 1 denotes the constant regressor which equals 1 for all observations.

???

While the concept of **Instrument Relevance** is much tricky.

So, what is the meaning of **Instrument Relevance**?

___

Obviously, the perfect multicollinear is the rare fact and can be get rid with careful inspection.

What we really need to pay attention is the contrary fact which is called **weak instruments**.

---

### Instrument Relevance: Weak instrument

Instruments that explain little variation in the endogenous regressor `$X$`  are called **weak instruments**.

Formally,  When 
`$\operatorname{corr}\left(Z_{i}, X_{i}\right)$` is close to zero,
`$z_{i}$` is called a weak instrument.

- Consider a simple one regressor model
`$Y_{i}=\beta_{0}+\beta_{1} X_{i}+\epsilon_{i}$`

- The IV estimator of 
`$\beta_{1}$` is 
`$\widehat{\beta}_{1}^{IV}=\frac{\sum_{i=1}^{n}\left(Z_{i}-\bar{Z}\right)\left(Y_{i}-\bar{Y}\right)}{\sum_{i=1}^{n}\left(Z_{i}-\bar{Z}\right)\left(X_{i}-\bar{X}\right)}$`

> Note that 
`$\frac{1}{n} \sum_{i=1}^{n}\left(Z_{i}-\bar{Z}\right)\left(Y_{i}-\bar{Y}\right) \xrightarrow{p} \operatorname{Cov}\left(Z_{i}, Y_{i}\right)$`

> and
`$\frac{1}{n} \sum_{i=1}^{n}\left(Z_{i}-\bar{Z}\right)\left(X_{i}-\bar{X}\right) \xrightarrow{p} \operatorname{Cov}\left(Z_{i}, X_{i}\right)$`.

- Thus,if 
`$\operatorname{Cov}\left(Z_{i},X_{i}\right) \approx 0$`, then `$\widehat{\beta}_{1}^{IV}$` is useless.

???

Let me give you an example.

---

### Example: Weak instrument

We want to run a simple regression to assess the effect of smoking on
child birth weight. The model we run is

`$$\begin{align}
\log (\text {bwght})=\beta_{0}+\beta_{1} \text {packs}+\epsilon_{i}
\end{align}$$`

where packs is the number of packs of cigarettes mother smokes per day. We suspect that packs might be endogenous. (So why?) so we use average price of cigarettes in the state of residence, as an instrument. We assume that cigprice is uncorrelated with
`$\epsilon$`.

---

### Example: Weak instrument

However, by regressing `$packs$` on `$cigprice$` in stage 1, we find basically no effect.

`$$\begin{alignedat}{3}
\widehat{packs} &&= &&
0.067 + &&0.0003 \text { cigprice } \\
&& &&(0.103)  &&(0.0008)
\end{alignedat}$$`

If we insist to use `$cigprice$` as an instrument, and conduct the stage 2 regression, we will find

`$$\begin{alignedat}{3}
\log \widehat{(bwght)} &=
& 4.45   + &2.99 \text {packs} \\
& &(0.91) &(8.70)\\
\end{alignedat}$$`

Obviously, this estimate is meaningless (Why?).

The `$cigprice$` behaves as a **weak instrument**, and the problem was already exposed in stage 1 regression.

???

- because there is huge standard error and not significant on coefficient of packs.

- and also it has the wrong sign on coefficient of packs, which should not be positive.

As what we have discussed, the result is unbelievable since the cigprice is a weak instrument for packs.

---

### Weak instrument: the strategy

There are two ways to proceed if instruments are weak:

- Discard the **weak instruments** and/or find **stronger instruments**.

> While the former is only an option if the unknown coefficients remain identified when the weak instruments are discarded, the latter can be difficult and even may require a redesign of the whole study.

- Stick with the weak instruments but use methods that improve upon TSLS.

> Such as **limited information maximum likelihood estimation (LIML)**.

???
So, what should we do if the instruments are weak or some of them are weak?

---

### Weak instrument: restricted F-test (idea)

In case with a **single** endogenous regressor, we can take the  **F-test** to check the **Weak instrument**.

The basic idea of the F-test is very simple:

If the estimated coefficients of **all instruments** in the **first-stage** of a 2SLS estimation are **zero**, the instruments do not explain any of the variation in the 
`$X$` which clearly violates the relevance assumption.

]

---

### Weak instrument: restricted F-test  (procudure)

We may use the following rule of thumb:

- Conduct the **first-stage regression** of a 2SLS estimation

`$$\begin{align}
X_{i}=\hat{\gamma}_{0}+\hat{\gamma}_{1} W_{1 i}+\ldots+\hat{\gamma}_{p} W_{p i}+ \hat{\theta}_{1} Z_{1 i}+\ldots+\hat{\theta}_{q} Z_{q i}+v_{i} \quad \text{(3)}
\end{align}$$`

- Test the restricted joint hypothesis 
`$H_0: \hat{\theta}_1=\ldots=\hat{\theta}_q=0$` by compute the
`$F$`-statistic.

- If the
`$F$`-statistic is less than  critical value, the instruments are **weak**.

The rule of thumb is easily implemented in `R`. Run the first-stage regression using `lm()` and subsequently compute the restricted `$F$`-statistic by `R` function of `car::linearHypothesis()`.

]

???

Also, you may ask that how do you know the instruments are weak or some of them are weak?

We will test this considering with different situations.

---
exclude: true

### R script: three models

---

### Wage example: restricted F-test (models)

For all  three IV model, we can test instrument(s) relevance respectively.

`$$\begin{align}
educ &= \gamma_1 +\gamma_2exper +\gamma_2expersq + \theta_1motheduc  +v 
&& \text{(relevance test 1)}\\
educ &= \gamma_1 +\gamma_2exper +\gamma_2expersq + \theta_2fatheduc +v  
&& \text{(relevance test 2)} \\
educ &= \gamma_1 +\gamma_2exper +\gamma_2expersq + \theta_1motheduc + \theta_2fatheduc +v  
&& \text{(relevance test 3)}
\end{align}$$`

???

And we will test the weak instrument issues by using restricted F test.

---

### Wage example: restricted F-test (model 1)

Consider model 1:

`$$\begin{align}
educ &= \gamma_1 +\gamma_2exper +\gamma_3expersq + \theta_1motheduc  +v 
\end{align}$$`

The restricted F-test' null hypothesis: 
`$H_0: \theta_1  =0$`.

We will test whether `motheduc` are week instruments.

---

### Wage example: restricted F-test (model 1)

The result show that the p-value of 
`$F^{\ast}$` is much smaller than 0.01. Null hypothesis `$H_0$` was rejected. `motheduc` is  **instruments relevance** (exogeneity valid).

```r
# restricted F-test
constrain_test1 <- linearHypothesis(ols_relevance1, c("motheduc=0"))
# obtain F statistics
F_r1 <- constrain_test1$F[[2]]
```

]

```
Linear hypothesis test

Hypothesis:
motheduc = 0

Model 1: restricted model
Model 2: educ ~ exper + expersq + motheduc

Res.Df  RSS Df Sum of Sq  F Pr(>F)    
1    425 2219                           
2    424 1890  1       330 74 <2e-16 ***
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
```

]

---

### Wage example (compare): classic F-test (model 1)

> Note: **Restriced F test** (73.95) is different with the **classical OLS F test**(show bellow 25.47).

`$$\begin{align}
educ &= \gamma_1 +\gamma_2exper +\gamma_2expersq + \theta_1motheduc  +v 
\end{align}$$`

The classic OLS  F-test' null hypothesis: 
`$H_0: \gamma_2 = \gamma_3= \theta_1  =0$`.

The OLS estimation results are:

???
Restricted F test take the Null hypotheis with  coefficients before the instruments all euqal to zero, While the classical F test take the Null hypotheis with coefficients before all regressors equal to zero.

---

### Wage example: restricted F-test (model 2)

Consider model 2:

`$$\begin{align}
educ &= \gamma_1 +\gamma_2exper +\gamma_3expersq + \theta_1fatheduc +v  
&& \text{(relevance test 2)} 
\end{align}$$`

The restricted F-test' null hypothesis: 
`$H_0: \theta_1  =0$`.

We will test whether `fatheduc` are week instruments.

---

### Wage example: restricted F-test (model 2)

The result show that the p-value of 
`$F^{\ast}$` is much smaller than 0.01. Null hypothesis `$H_0$` was rejected. `fatheduc` is  **instruments relevance** (exogeneity valid).

```r
constrain_test2 <- linearHypothesis(ols_relevance2, c("fatheduc=0"))
# obtain F statistics
F_r2 <- constrain_test2$F[[2]]
```

]

```
Linear hypothesis test

Hypothesis:
fatheduc = 0

Model 1: restricted model
Model 2: educ ~ exper + expersq + fatheduc

Res.Df  RSS Df Sum of Sq    F Pr(>F)    
1    425 2219                             
2    424 1839  1       380 87.7 <2e-16 ***
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
```

]

---

### Wage example: restricted F-test (model 3)

Consider model 3:

`$$\begin{align}
educ &= \gamma_1 +\gamma_2exper +\gamma_3expersq + \theta_1motheduc + \theta_2fatheduc +v  
&& \text{(relevance test 3)}
\end{align}$$`

The restricted F-test' null hypothesis: 
`$H_0: \theta_1 = \theta_2 =0$`.

We will test whether `motheduc` and `fatheduc` are week instruments.

---

### Wage example: restricted F-test (model 3)

The result show that the p-value of 
`$F^{\ast}$` is much smaller than 0.01. Null hypothesis `$H_0$` was rejected. `fatheduc` and `motheduc` are  **instruments relevance** (exogeneity valid).

```r
constrain_test3 <- linearHypothesis(ols_relevance3, c("motheduc=0", "fatheduc=0"))
# obtain F statistics
F_r3 <- constrain_test3$F[[2]]
```

]

```
Linear hypothesis test

Hypothesis:
motheduc = 0
fatheduc = 0

Model 1: restricted model
Model 2: educ ~ exper + expersq + motheduc + fatheduc

Res.Df  RSS Df Sum of Sq    F Pr(>F)    
1    425 2219                             
2    423 1759  2       461 55.4 <2e-16 ***
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
```

]

???

In sum, all relecance model test are significant. And we can conclude that the instrumenta mothereducation and father education satisify the relevance condition.

Until now, we show the relevance F-test with situation that contains only one endogenous regressor.

Next, I will give a two endogeous variables example by using Cragg-Donald test.

---

### Weak instrument: Cragg-Donald test

The former test for weak instruments might be unreliable with **more than** one endogenous regressor, though, because there is indeed one
`$F$`-statistic for each endogenous regressor.

An alternative is the **Cragg-Donald test** based on the following statistic:

`$$\begin{align}
F=\frac{N-G-B}{L} \frac{r_{B}^{2}}{1-r_{B}^{2}}
\end{align}$$`

- where:
`$G$` is the number of exogenous regressors;
`$B$` is the number of endogenous regressors;
`$L$` is the number of external instruments;
`$r_B$` is the lowest canonical correlation.

> **Canonical correlation** is a measure of the correlation between the endogenous and the exogenous variables, which can be calculated by the function `cancor()` in  `R`.

???

external (ɪkˈstɜːnl)

Canonical (kəˈnɒnɪkl)

---

### Hour example: backgound

Let us construct another IV model with two endogenous regressors. We assumed the following work hours determination model:

`$$\begin{equation}
hushrs=\beta_{1}+\beta_{2} mtr+\beta_{3} educ+\beta_{4} kidsl6+\beta_{5} nwifeinc+e
\end{equation}$$`

> - `$hushrs$`: work hours of husband, 1975
- `$mtr$`: federal marriage tax rate on woman
- `$kidslt6$`: have kids < 6 years (dummy variable)
- `$nwifeinc$`: wife’s net income

There are:

- Two **endogenous variables**: `$educ$` and `$mtr$` 
- Two **exogenous regressors**:  `$nwifeinc$` and  `$kidslt6$`
- And two external **instruments**: `$motheduc$`  and  `$fatheduc$`.

---

### Hour example: Cragg-Donald test (R code)

The data set is still `mroz`, restricted to women that are in the labor force(
`$inlf=1$`).

```r
# filter samples
mroz1 <- wooldridge::mroz %>%
  filter(wage>0, inlf==1)
# set parameters
N <- nrow(mroz1); G <- 2; B <- 2; L <- 2 
# for endogenous variables
x1 <- resid(lm( mtr ~ kidslt6 + nwifeinc, data = mroz1))
x2 <- resid(lm( educ ~ kidslt6 + nwifeinc, data = mroz1))
# for instruments
z1 <-resid(lm(motheduc ~ kidslt6 + nwifeinc, data = mroz1))
z2 <-resid(lm(fatheduc ~ kidslt6 + nwifeinc, data=mroz1))
# column bind
X <- cbind(x1,x2)
Y <- cbind(z1,z2)
# calculate Canonical correlation
rB <- min(cancor(X,Y)$cor)
# obtain the F statistics
CraggDonaldF <- ((N-G-L)/L)/((1-rB^2)/rB^2)
```

---

### Hour example: Cragg-Donald test (result)

Run these code lines, we can obtain the results:

<table>
<caption>Cragg-Donald test results</caption>
 <thead>
  <tr>
   <th style="text-align:center;"> G </th>
   <th style="text-align:center;"> L </th>
   <th style="text-align:center;"> B </th>
   <th style="text-align:center;"> N </th>
   <th style="text-align:center;"> rb </th>
   <th style="text-align:center;"> CraggDonaldF </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:center;"> 2 </td>
   <td style="text-align:center;"> 2 </td>
   <td style="text-align:center;"> 2 </td>
   <td style="text-align:center;"> 428 </td>
   <td style="text-align:center;"> 0.0218 </td>
   <td style="text-align:center;"> 0.1008 </td>
  </tr>
</tbody>
</table>

The result show the Cragg-Donald  `$F=$` 0.1008 , which is much smaller than **the critical value** `4.58`<sup>[1]</sup>.

This test can not rejects the null hypothesis, thus we may conclude that some of these instruments are **weak**.

.footnote[
[1] The critical value can be found in table 10E.1 at: Hill C, Griffiths W, Lim G. Principles of econometrics[M]. John Wiley & Sons, 2018.
]

???

You can inquire the critical values in Table 10E.1 of the textbook,Hill, Griffiths, and Lim 2011.

---

### Instrument Exogeneity: the difficulty

**Instrument Exogeneity** means all 
`$m$` instruments must be uncorrelated with the error term,

`$$Cov{(Z_{1 i}, \epsilon_{i})}=0; \quad \ldots; \quad Cov{(Z_{mi}, \epsilon_{i})}=0.$$`

- In the context of the simple IV estimator, we will find that the exogeneity requirement **can not** be tested. (Why?)

- However, if we have more instruments than we need, we can effectively test whether **some of** them are uncorrelated with the structural error.

???
As we know , when we call a instrument is validity, we should also check that it satisfy the exogeneity condition.

---

### Instrument Exogeneity: over-identification case

Under **over-identification** 
`$(m>k)$`, consistent IV estimation with (multiple) different combinations of instruments is possible.

> If instruments are exogenous, the obtained estimates should be **similar**.

> If estimates are very **different**, some or all instruments may .red[not] be exogenous.

The **Overidentifying Restrictions Test** (**J test**) formally check this.

- The null hypothesis is Instrument Exogeneity.

`$$H_{0}: E\left(Z_{h i} \epsilon_{i}\right)=0, \text { for all } h=1,2, \dots, m$$`

---

### Instrument Exogeneity: J-test (procedure)

The **overidentifying restrictions test** (also called the  `$J$`-test, or **Sargan test**) is an approach to test the hypothesis that the additional instruments are exogenous.

Procedure of overidentifying restrictions test is:

- **Step 1**: Compute the **IV regression residuals** :

`$$\widehat{\epsilon}_{i}^{IV}=Y_{i}-\left(\hat{\beta}_{0}^{ IV}+\sum_{j=1}^{k} \hat{\beta}_{j}^{IV} X_{j i}+\sum_{s=1}^{r} \hat{\beta}_{k+s}^{IV} W_{s i}\right)$$`

- **Step 2**: Run the **auxiliary regression**: regress the IV residuals on instruments and exogenous regressors. And test the joint hypothesis 
`$H_{0}: \alpha_{1}=0, \ldots, \alpha_{m}=0$`

`$$\widehat{\epsilon}_{i}^{IV}=\theta_{0}+\sum_{h=1}^{m} \theta_{h} Z_{h i}+\sum_{s=1}^{r} \gamma_{s} W_{s i}+v_{i} \quad \text{(2)}$$`

???
auxiliary (ɔːɡˈzɪliəri)

---

### Instrument Exogeneity: J-test (procedure)

- **Step3**: Compute the **J statistic**:
`$J=m F$`

> where 
`$F$` is the F-statistic of the 
`$m$` restrictions 
`$H_0: \theta_{1}=\ldots=\theta_{m}=0$` in eq(2)

Under the **null hypothesis**, 
`$J$` statistic is distributed as 
`$\chi^{2}(m-k)$` approximately for large samples.

`$$\boldsymbol{J} \sim \chi^{2}({m-k})$$`

> IF `$J$` is **less** than **critical value**, it means that all instruments are .red[ex]ogenous.

> IF `$J$` is **larger** than **critical value**, it mean that some of the instruments are .red[en]ogenous.

- We can apply the  `$J$`-test by using `R` function `linearHypothesis()`.

???
approximately (əˈprɒksɪmətli)

---

### Wage example: J-test (models)

Again, we can use both `$matheduc$` and `$fatheduc$` as instruments for `$educ$`.

Thus, the IV model is over-identification, and we can test the exogeneity of both these two instruments by using **J-test**.

The 2SLS model will be set as below.

And the auxiliary regression should be

`$$\begin{align}
  \hat{\epsilon}^{IV} &= \hat{\alpha}_1 +\hat{\alpha}_2exper + \hat{\alpha}_3expersq +\hat{\theta}_1motheduc + \hat{\theta}_2fatheduc  + v && \text{(auxiliary model)}
  \end{align}$$`

---

### Wage example: J-test (R code for 2SLS residuals)

We have done the 2SLS estimation before, here is the `R` code (by using `ivreg::ivreg()` function):

After the 2SLS estimation, we can obtain the IV residuals of the second stage:

```r
# obtain residual of IV regression, add to data set
mroz_resid <- mroz %>%
  mutate(resid_iv_mf = residuals(lm_iv_mf)) 
```

---

### Wage example: J-test (new data set)

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-fa97d02e28c3bcce5788" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-fa97d02e28c3bcce5788">{"x":{"filter":"none","vertical":false,"caption":"<caption>Data set with the 2SLS residuals<\/caption>","data":[["1","2","3","4","5","6","7","8","9","10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180","181","182","183","184","185","186","187","188","189","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204","205","206","207","208","209","210","211","212","213","214","215","216","217","218","219","220","221","222","223","224","225","226","227","228","229","230","231","232","233","234","235","236","237","238","239","240","241","242","243","244","245","246","247","248","249","250","251","252","253","254","255","256","257","258","259","260","261","262","263","264","265","266","267","268","269","270","271","272","273","274","275","276","277","278","279","280","281","282","283","284","285","286","287","288","289","290","291","292","293","294","295","296","297","298","299","300","301","302","303","304","305","306","307","308","309","310","311","312","313","314","315","316","317","318","319","320","321","322","323","324","325","326","327","328","329","330","331","332","333","334","335","336","337","338","339","340","341","342","343","344","345","346","347","348","349","350","351","352","353","354","355","356","357","358","359","360","361","362","363","364","365","366","367","368","369","370","371","372","373","374","375","376","377","378","379","380","381","382","383","384","385","386","387","388","389","390","391","392","393","394","395","396","397","398","399","400","401","402","403","404","405","406","407","408","409","410","411","412","413","414","415","416","417","418","419","420","421","422","423","424","425","426","427","428"],[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428],[1.2101536989212,0.328512102365494,1.51413774490356,0.0921233221888542,1.52427220344543,1.55648005008698,2.12025952339172,2.0596342086792,0.754336357116699,1.54489934444427,1.4019216299057,1.52427220344543,0.733953237533569,0.818369090557098,1.30283117294312,0.298028379678726,1.16760957241058,1.64383935928345,0.6931471824646,2.02193164825439,1.25424754619598,1.27295768260956,1.1786550283432,1.1786550283432,0.767558693885803,1.33181178569794,1.3862943649292,1.55326962471008,1.98181486129761,1.76936042308807,0.430807888507843,0.899754822254181,1.76662969589233,1.27295768260956,1.33678889274597,0.901704847812653,0.865123689174652,1.51184713840485,1.72602915763855,2.68314242362976,0.985294282436371,1.36593854427338,0.945033669471741,1.51237618923187,0.6931471824646,1.24478840827942,0.701164901256561,1.5198632478714,0.820968568325043,0.969831526279449,0.828508198261261,0.0943096429109573,0.162543892860413,0.470003634691238,0.629248440265656,1.39716017246246,2.26544380187988,2.0845410823822,1.52583885192871,0.762160062789917,1.48160457611084,1.26282644271851,0.999675571918488,1.83258152008057,2.47930765151978,1.27901530265808,1.93793559074402,1.0704528093338,1.12392258644104,1.32175588607788,1.74499976634979,1.30174362659454,1.64186644554138,2.10702013969421,1.46706759929657,1.60581135749817,-1.02973937988281,1.08768618106842,0,0.938208699226379,-0.150590375065804,0,1.07367050647736,1.26584839820862,0.486368983983994,2.12025952339172,1.12985253334045,0.993251800537109,1.65862798690796,0.347412198781967,1.56832420825958,0.510845601558685,0.114845432341099,-0.6931471824646,-0.336452275514603,1.02822554111481,1.58068859577179,0.555894613265991,0.901420712471008,0.884304583072662,0.428204596042633,1.05841505527496,0.87833958864212,1.65490829944611,1.32175588607788,0.328512102365494,1.3862943649292,1.17288458347321,1.22418713569641,0.287657082080841,2.23026180267334,1.50407743453979,1.53115200996399,1.37515759468079,1.76026880741119,-0.6931471824646,1.40648913383484,1.7917594909668,1.29929208755493,1.35100388526917,1.01628088951111,1.0753436088562,1.47896468639374,1.68948674201965,2.28859782218933,-1.82263112068176,-0.960765182971954,1.29099416732788,0.864871144294739,1.54045212268829,0.616212129592896,1.64865863323212,1.19349813461304,2.14397621154785,0.724403560161591,0.941607534885406,0.782759368419647,1.83258152008057,1.20396280288696,1.49164485931396,1.89213263988495,2.13089489936829,1.48060405254364,0.894331336021423,0.202532544732094,0.485507816076279,1.0986123085022,1.55326962471008,0.121597968041897,2.00180435180664,1.49503660202026,0.905229806900024,0.632547557353973,1.3862943649292,2.10291385650635,1.95964395999908,0.510845601558685,1.23692393302917,1.44331252574921,1.02165925502777,0.63615345954895,1.61645328998566,0.223143547773361,1.04980707168579,1.41505193710327,0.575376629829407,2.60668158531189,1.51791453361511,0.755041599273682,1.09497237205505,0.942114353179932,1.72494280338287,1.03154611587524,0.474369078874588,0.810930192470551,0.709266602993011,1.71054947376251,0.460268884897232,1.33181178569794,1.0986123085022,2.15799856185913,1.43758130073547,1.54489934444427,1.41059672832489,3.21887588500977,0.968161880970001,1.7917594909668,1.68872952461243,-0.409171968698502,0.223143547773361,0.822155833244324,1.24170196056366,1.42712438106537,1.49709749221802,0.559615790843964,1.30002820491791,1.88442981243134,0.955511391162872,1.58208727836609,1.7556140422821,1.51310324668884,2.25189161300659,2.3644323348999,0.105350479483604,1.39972877502441,0.988462507724762,1.09064733982086,1.15461444854736,1.26694762706757,2.88519167900085,1.22888004779816,1.20396280288696,1.35738027095795,0.837723612785339,0.536961138248444,0.748723804950714,2.29587268829346,1.10780322551727,0.620845258235931,-2.05416369438171,1.89201200008392,1.72972452640533,0.469378411769867,0.98084169626236,2.06949234008789,1.67518818378448,1.3862943649292,1.79921495914459,1.83258152008057,1.09064733982086,1.44312357902527,1.25036013126373,1.60231256484985,1.01855850219727,1.29705321788788,1.68519449234009,-0.420984894037247,1.56209468841553,2.14652752876282,2.34746289253235,0.969831526279449,1.9241464138031,1.62672758102417,-0.0392607264220715,1.46014869213104,1.95539355278015,0.926359891891479,2.06619167327881,1.42284321784973,2.10103178024292,2.26146101951599,0.70131379365921,2.03101253509521,1.16236925125122,0.470003634691238,1.41059672832489,0.393055111169815,1.29099416732788,0,0.95712548494339,0.559615790843964,1.56861591339111,1.7101879119873,1.41059672832489,0.223143547773361,0.510845601558685,1.33239245414734,0.860185861587524,2.32277989387512,1.91959547996521,1.97610676288605,0.895434737205505,0.18123759329319,0.495305836200714,0.577792406082153,1.07881772518158,1.60319852828979,0.620845258235931,2.08389401435852,1.37916910648346,1.11238372325897,1.06712162494659,1.11880695819855,1.58854103088379,1.3903112411499,1.71480643749237,0.20106153190136,0.987271010875702,0.98350065946579,2.23317074775696,1.14361751079559,-0.611382901668549,2.15305209159851,1.29983735084534,0.840920448303223,1.05848443508148,1.15265846252441,1.29357588291168,1.83258152008057,2.32718014717102,1.16614627838135,2.03499317169189,0.679251074790955,1.54713690280914,0.75301855802536,0.847283601760864,0.871125996112823,0.228250473737717,0.0896578282117844,1.32175588607788,1.19610190391541,1.63611876964569,1.89201200008392,1.51830899715424,2.47215914726257,1.32175588607788,1.47364103794098,1.36947882175446,1.20396280288696,1.19872915744781,1.27020990848541,0.470003634691238,0.799981653690338,1.56594562530518,1.75897800922394,0.858025848865509,0.6931471824646,0.641853868961334,1.63374018669128,1.70374763011932,1.84400403499603,1.96611881256104,0.864997446537018,0.933305203914642,0.779233157634735,0.955511391162872,1.31624734401703,1.4759064912796,1.49139726161957,1.45575046539307,0.510845601558685,1.18043804168701,1.68848943710327,0.790727496147156,1.40179860591888,-0.433556020259857,1.68317151069641,-1.76667666435242,3.15559506416321,2.25952100753784,1.30692636966705,0.798497676849365,0.559044182300568,0.147902622818947,1.94449484348297,1.37833786010742,3.0647451877594,-0.741917312145233,0.765700399875641,0.619392991065979,1.46545207500458,2.18925952911377,1.02165925502777,0.977009475231171,0.916290760040283,2.90509605407715,-0.199671193957329,0.6931471824646,2.73339295387268,1.86833465099335,2.12025952339172,1.51519322395325,0.914609313011169,1.49955606460571,0.803077220916748,0.728031635284424,0.516409993171692,1.22644829750061,0.916290760040283,1.37647128105164,1.8289749622345,1.36828315258026,1.06471073627472,1.40648913383484,1.04731893539429,1.94809341430664,1.07800137996674,0.653938472270966,1.92789161205292,1.36102783679962,0.6931471824646,1.60468661785126,0.183903649449348,3.11351537704468,1.92682921886444,1.2701256275177,0.68269270658493,1.68106997013092,0.556295990943909,1.62822043895721,0.916290760040283,1.3415584564209,0,1.12223124504089,0.540170788764954,1.3915057182312,1.69717395305634,3.21887588500977,0.871167778968811,1.16732954978943,1.21698772907257,0.575376629829407,1.15161573886871,0.994251251220703,0.526324927806854,-1.5431821346283,1.91204309463501,0.554287314414978,0.916290760040283,1.50093913078308,0.944683790206909,1.24126863479614,1.56498432159424,0.838026463985443,1.66885709762573,1.7694286108017,1.22644829750061,1.40648913383484],[12,12,12,12,14,12,16,12,12,12,12,11,12,12,10,11,12,12,12,12,16,12,13,12,12,17,12,12,17,12,11,16,13,12,16,11,12,10,14,17,12,12,16,12,12,12,16,12,12,12,12,12,12,8,10,16,14,17,14,12,14,12,8,12,12,8,17,12,12,12,12,12,9,10,12,12,12,17,15,12,6,14,12,14,9,17,13,9,15,12,12,12,12,12,12,12,12,13,12,13,12,12,12,16,12,13,11,12,12,12,17,14,16,17,12,11,12,12,17,10,13,11,12,16,17,12,16,12,16,8,12,12,12,13,11,12,12,14,12,12,12,17,14,12,9,12,12,12,14,16,17,15,12,16,17,17,12,16,13,12,11,16,14,16,12,9,17,14,12,12,11,12,12,10,12,5,17,11,12,12,14,11,12,14,12,10,16,13,12,12,12,11,12,9,13,12,12,12,13,16,12,16,17,12,12,9,12,12,13,12,12,12,12,10,12,16,12,11,12,10,12,12,12,12,16,17,12,17,12,12,12,8,12,13,12,12,8,12,17,17,12,13,12,12,12,12,9,10,12,16,13,8,16,13,12,11,13,12,12,10,12,17,15,16,10,11,12,12,14,16,14,8,7,12,12,14,12,12,12,14,16,12,12,12,13,13,10,12,12,12,12,14,17,10,9,12,12,16,12,17,12,17,11,16,11,13,11,8,11,12,10,17,12,12,17,14,12,12,12,12,12,12,9,10,12,12,12,12,12,17,12,17,12,10,12,12,12,12,12,12,16,13,13,12,16,17,12,14,12,17,12,14,12,12,17,16,16,12,9,12,12,16,14,12,12,11,12,16,17,17,14,12,14,12,10,12,13,16,12,7,16,14,12,10,12,16,10,12,14,12,6,15,12,17,14,13,6,16,14,15,14,8,14,12,12,12,12,12,12,8,12,17,12,12,14,13,17,8,12,11,12,12,17,10,12,13,12,12],[14,5,15,6,7,33,11,35,24,21,15,14,0,14,6,9,20,6,23,9,5,11,18,15,4,21,31,9,7,7,32,11,16,14,27,0,17,28,24,11,1,14,6,10,6,4,10,22,16,6,12,32,15,17,34,9,37,10,35,6,19,10,11,15,12,12,14,11,9,24,12,13,29,11,13,19,2,24,9,6,22,30,10,6,29,29,36,19,8,13,16,11,15,6,13,22,24,2,6,2,2,14,9,11,9,6,19,26,19,3,7,28,13,9,15,20,29,9,1,8,19,23,3,13,8,17,4,15,11,7,0,0,10,8,2,4,6,18,3,22,33,28,23,27,11,6,11,14,17,17,14,11,7,8,6,8,4,25,24,11,19,9,19,14,22,6,23,15,6,11,2,22,10,14,12,9,13,18,8,11,9,9,14,9,2,12,15,11,7,9,19,11,8,13,4,7,19,14,14,3,9,7,7,14,29,19,14,16,10,12,24,6,9,14,26,7,4,15,23,1,29,9,6,11,17,6,7,2,24,4,11,25,11,2,19,7,2,20,10,19,17,12,11,6,10,4,2,13,21,9,4,2,19,4,9,14,6,24,1,13,3,10,16,9,19,4,10,5,7,3,38,16,13,1,7,15,10,2,19,25,25,7,15,11,25,19,4,14,19,18,14,11,4,29,21,24,19,31,28,15,27,13,4,10,8,4,18,3,11,8,10,33,19,35,21,7,18,4,12,16,14,3,1,27,12,6,9,2,6,9,16,22,26,11,11,15,13,6,20,17,8,13,15,14,14,6,24,10,2,9,23,12,8,16,10,7,19,2,9,14,9,16,7,6,22,9,9,14,17,12,13,8,10,16,1,6,4,8,4,15,7,14,16,15,23,19,4,12,12,25,14,14,11,7,18,4,37,13,14,17,5,2,0,3,21,20,19,4,19,11,14,8,13,24,1,1,3,4,21,10,13,9,14,2,21,22,14,7],[196,25,225,36,49,1089,121,1225,576,441,225,196,0,196,36,81,400,36,529,81,25,121,324,225,16,441,961,81,49,49,1024,121,256,196,729,0,289,784,576,121,1,196,36,100,36,16,100,484,256,36,144,1024,225,289,1156,81,1369,100,1225,36,361,100,121,225,144,144,196,121,81,576,144,169,841,121,169,361,4,576,81,36,484,900,100,36,841,841,1296,361,64,169,256,121,225,36,169,484,576,4,36,4,4,196,81,121,81,36,361,676,361,9,49,784,169,81,225,400,841,81,1,64,361,529,9,169,64,289,16,225,121,49,0,0,100,64,4,16,36,324,9,484,1089,784,529,729,121,36,121,196,289,289,196,121,49,64,36,64,16,625,576,121,361,81,361,196,484,36,529,225,36,121,4,484,100,196,144,81,169,324,64,121,81,81,196,81,4,144,225,121,49,81,361,121,64,169,16,49,361,196,196,9,81,49,49,196,841,361,196,256,100,144,576,36,81,196,676,49,16,225,529,1,841,81,36,121,289,36,49,4,576,16,121,625,121,4,361,49,4,400,100,361,289,144,121,36,100,16,4,169,441,81,16,4,361,16,81,196,36,576,1,169,9,100,256,81,361,16,100,25,49,9,1444,256,169,1,49,225,100,4,361,625,625,49,225,121,625,361,16,196,361,324,196,121,16,841,441,576,361,961,784,225,729,169,16,100,64,16,324,9,121,64,100,1089,361,1225,441,49,324,16,144,256,196,9,1,729,144,36,81,4,36,81,256,484,676,121,121,225,169,36,400,289,64,169,225,196,196,36,576,100,4,81,529,144,64,256,100,49,361,4,81,196,81,256,49,36,484,81,81,196,289,144,169,64,100,256,1,36,16,64,16,225,49,196,256,225,529,361,16,144,144,625,196,196,121,49,324,16,1369,169,196,289,25,4,0,9,441,400,361,16,361,121,196,64,169,576,1,1,9,16,441,100,169,81,196,4,441,484,196,49],[7,7,7,7,14,7,7,3,7,7,3,7,16,10,7,10,7,12,7,7,16,10,3,7,7,14,7,7,12,12,7,3,10,14,12,3,3,3,7,17,12,9,16,3,7,7,16,10,7,7,7,3,7,7,3,12,7,17,7,7,3,12,7,7,7,12,16,7,7,7,12,10,9,0,10,14,7,3,12,12,7,17,3,7,7,12,7,7,12,10,0,12,10,7,7,7,3,12,7,12,7,7,10,14,7,12,7,7,10,7,12,7,7,17,7,7,7,10,10,12,7,12,7,10,7,10,7,7,7,7,7,7,16,12,7,3,7,7,7,12,7,12,12,14,7,7,7,12,12,14,10,12,7,16,7,17,3,10,9,7,3,16,12,7,7,7,12,3,7,7,7,7,10,10,7,12,17,10,7,7,12,7,12,7,7,7,7,14,7,12,7,7,12,3,7,12,12,7,7,14,12,17,17,7,7,10,7,7,7,3,0,7,12,7,7,12,7,7,12,7,7,3,7,7,10,12,7,12,7,7,7,7,12,7,7,7,7,7,14,17,7,10,7,7,12,7,7,9,7,14,7,3,16,3,16,7,16,12,7,7,12,12,12,16,7,9,7,12,12,10,7,12,7,7,3,10,17,7,3,3,12,7,7,7,10,7,10,7,9,9,12,12,12,7,9,12,7,12,7,12,12,14,7,14,10,12,7,7,12,7,7,12,12,12,12,14,10,7,7,7,7,7,7,7,7,7,12,12,7,14,10,12,7,7,12,7,10,12,10,7,14,7,12,12,7,16,0,12,7,17,7,12,3,7,14,7,12,12,7,7,14,7,12,12,7,7,7,7,3,12,7,7,14,10,10,10,10,12,3,7,16,7,7,0,7,7,10,7,12,7,7,7,7,16,12,10,7,7,16,7,7,7,12,10,10,10,7,10,12,12,7,17,12,16,10,16,7,7,9,3,7,7,7,7,7,7,16,12],[12,7,12,7,12,14,14,3,7,7,12,14,16,10,7,16,10,12,7,12,10,12,7,7,12,16,3,3,12,12,7,3,12,7,12,10,3,10,7,14,12,9,14,3,12,12,14,10,7,12,7,7,12,7,7,12,7,17,17,12,14,12,7,7,7,12,12,12,7,12,12,10,7,0,7,12,7,3,10,7,12,12,7,7,7,7,7,7,7,10,7,12,10,12,7,7,7,14,7,12,12,7,7,14,12,10,7,7,7,7,12,7,12,10,10,7,7,7,12,7,7,12,14,12,7,10,7,7,12,10,7,7,12,10,7,12,7,7,7,7,3,12,16,7,3,12,7,12,12,16,12,12,7,14,7,10,7,14,7,7,12,12,17,7,7,3,12,7,7,7,3,7,10,10,12,7,14,10,7,7,10,12,12,7,7,7,12,7,12,12,12,10,12,10,12,12,12,7,12,12,12,12,16,7,16,7,7,10,12,10,0,7,12,12,10,12,3,7,12,10,7,7,7,7,12,12,7,12,7,10,10,7,12,17,7,7,7,7,12,14,7,12,7,7,16,7,10,12,7,16,10,3,16,7,12,7,7,7,12,12,7,10,14,16,7,10,7,14,14,12,7,7,3,7,7,7,12,10,7,3,12,7,12,7,10,7,0,7,10,9,12,12,12,3,9,12,12,14,7,12,12,12,7,12,10,12,7,7,3,12,7,16,12,12,7,14,7,7,12,10,7,3,7,7,10,7,7,12,12,12,10,14,7,7,14,10,10,7,7,7,14,7,12,14,14,14,0,16,7,12,7,10,7,10,10,14,7,12,7,7,12,14,12,7,7,7,12,16,3,16,7,16,7,10,10,10,10,12,10,7,16,7,7,7,7,7,10,10,12,7,7,7,7,14,14,7,7,7,16,12,7,7,12,12,12,10,3,12,12,12,7,16,12,10,10,12,7,7,7,7,7,7,7,7,7,7,12,12],[-0.0168936139370186,-0.654725473528458,0.26899015715309,-0.92539598118415,0.351475854449382,0.292975113425143,0.71271415562751,0.830048350108992,-0.572806441730052,0.228906830042817,0.156774042155226,0.358621519247366,-0.0509066133204574,-0.408678222301124,0.408105126890419,-0.750151842413413,-0.141070302156488,0.626320055910443,-0.632076794076698,0.912354797502101,0.0254234556614159,0.11099882948596,-0.171402377686319,-0.0664925594072741,-0.179599215352786,-0.291163872004291,0.0960621068809853,0.443692773957789,0.624828626321093,0.719357331412329,-0.785563049775182,-0.507790545510033,0.443783143765764,0.0459103697513359,-0.230909252599888,0.17824162561878,-0.410830630831395,0.317801699419701,0.276093101471491,1.21420042720539,0.157163008221737,0.138891231415154,-0.318072148541879,0.375709367705771,-0.324372120908405,0.29763049904083,-0.681088434910156,0.198356032811865,-0.440481355141374,-0.0476877770935547,-0.356944747283523,-1.18345792403222,-1.08260369489006,-0.560364170674192,-0.495402669618212,0.041996807069554,0.954175520786238,0.640891117555331,0.173459736038196,-0.255359240583087,0.0592420232341575,0.126159621192405,0.0833032334355052,0.587433932330092,1.29385470597499,0.339148871753913,0.403905134585027,-0.091506043789797,0.0143457356887464,-0.00538691276887016,0.559546820805009,0.0945945278048848,0.516288508792492,1.06785454389092,0.259918500506911,0.306242061941794,-1.89934413828174,-0.5464397610791,-1.29376673673276,-0.0793106041466247,-1.10371781816441,-1.42369226729725,-0.0629963150487454,0.125535837515306,-0.639208952764896,0.503508557361603,-0.141473506079627,-0.122127609038803,0.393749160125322,-0.859736900007692,0.306874284793166,-0.651113251564913,-1.13030215540938,-1.7106664858376,-1.54360137430426,-0.293281673944727,0.253545796925039,-0.375106773793093,-0.116098590901996,-0.0466968039864216,-0.441400162356297,-0.168632257583259,-0.231237262110173,0.247362931681892,0.212179035325587,-0.750403829667664,0.148121698032979,-0.152702042455519,-0.0753821598599633,-0.621623221326074,0.873275567696825,0.0644454809140338,0.0784163965337148,-0.0414023993722772,0.515121219660719,-1.94043042837152,0.0967213111054877,0.682182640214503,0.164177670039528,0.393108201787298,-0.34468503470542,-0.188483739024943,0.569684382986823,0.236751128589379,0.900925738086386,-3.09858544068781,-2.15350960685116,0.0458465795774061,-0.542674223469475,0.736035545653164,-0.168647721261131,0.86379878237809,0.0568313130869358,1.00189064208552,-0.0838045695771855,-0.00555037435318262,-0.234759934953357,0.421127485390892,0.294682499480048,0.170137644254431,0.628627703223116,0.507073059762063,0.0325868186820346,-0.427780294683821,-0.775236422411043,-0.532011487296725,-0.063346544621401,0.326222311851861,-1.27714960928446,0.480263517159978,-0.0389938541387282,-0.440918932204036,-0.417455534321772,0.0600189094864085,0.778411409832574,0.57197187589614,-0.436312307679904,-0.335926263587334,0.0547730982423016,-0.140299598095827,-0.60201920734727,0.261289924592747,-1.19921900510332,-0.422826755813047,0.0935447220437378,-0.257952787563136,0.974474465469823,0.14997368854433,-0.262477704099322,-0.0669864810685445,0.133906223441156,0.40343558832334,-0.105120705650857,-0.629884976663326,-0.374522753074233,0.0294661528617954,0.196417231672084,-0.76699526381198,0.251122844895762,-0.063346544621401,0.925628453786529,0.389401078643334,0.317852031586053,0.178226620252289,2.34927112661084,-0.0944978072544749,0.301025388575706,0.465374042828675,-1.45917506037425,-0.886433302978932,-0.477413462312051,0.141139736100215,0.346435440263194,0.474138279408821,-0.448938747054779,0.250025113242163,0.584860516874961,-0.27153592169535,0.293643336847713,0.600747224234574,0.403526395936549,0.956302006690231,1.00744609992339,-1.12169683337462,0.0899609522950628,-0.126916901851151,-0.13639997303736,-0.106835474919053,0.0688841768813107,1.69973873345607,-0.0982627510485943,0.186443499513959,0.247803420205653,-0.266530442752575,-0.788625487680281,-0.546865801365647,1.34871477905487,-0.0759477335730478,-0.704378718305366,-2.75950171127604,0.582244177354572,0.620147675653041,-0.548140891603137,-0.181117156861239,0.547951505441228,0.350685737110711,0.336291273253454,0.622627057444892,0.505438721233815,0.143489430582273,0.28116472590167,0.168682963928455,0.440353711726255,0.0875571151381817,-0.00251607766849604,0.635191400664343,-1.04500313779556,0.253414813848456,0.702877563935946,0.740910453675204,-0.306122793726597,0.677296839598163,0.464768727900572,-1.05678002979508,0.323481870604941,1.00823564354156,0.240945019473011,0.981835831809458,0.106850703448273,0.745868414850011,1.25290648161725,0.077295549900896,0.485856724898225,0.153814713352478,-0.639573216061055,0.244946044126822,-0.685860820863343,-0.0361486315188702,-0.828131274214634,-0.127230356525961,-0.349664512562952,0.124965948564242,0.264548102540427,0.0554333629319808,-0.953632490462705,-0.37491567901975,0.195725632621238,-0.123051714306428,1.14998354487907,0.764728661917679,0.688090807955304,-0.120428671620295,-0.718928362195699,-0.33282543801392,-0.472210685593592,-0.289123119889203,0.466531706763694,-0.248759500162999,0.784324718802146,-0.0708878328127416,-0.460466473357537,0.0171185332708492,-0.126340629551927,0.426582177760191,0.00165093051385523,0.353840513275842,-0.623303120016922,-0.23977630198252,-0.316068636090585,0.944509970387592,-0.083429802062629,-1.89613501211245,0.898911039059152,0.112862785436294,-0.290882180117774,-0.268658363765269,-0.14691083303196,-0.242242889777148,0.515742823775113,0.775049416119776,-0.155965352323897,0.520860929601466,-0.206510205787481,0.164883566642426,-0.266273754116661,-0.161270936137879,-0.356138152596389,-0.435443315028583,-1.01090439625166,0.241066945275706,0.182228339709612,0.0656306896830887,0.592442704527549,0.288723138584029,0.849183489560345,0.148959537081828,0.184980260571613,0.422320912515866,0.0185098573421789,-0.0627207660186013,0.0431625956271904,-0.439276668715677,0.156040265456166,0.366627251920239,0.573525063679154,-0.159493454507495,-0.416429668287694,-0.227750889437596,0.61622088331828,0.28718763606626,0.582554111529616,0.337628454200732,-0.460589179391707,-0.105860391888648,-0.382725695488863,-0.289636196587602,0.10909824522737,0.458387187906598,0.182717387052496,0.17979614538702,-0.815429853884105,-0.0881076857628009,0.381945220692643,-0.436319816711066,-0.0708352215799537,-1.75805846693363,0.35602871184966,-3.02613674319883,2.28599030576428,0.842961013484778,-0.0182976068742444,-0.509748526015727,-0.521644758501607,-1.11354730064747,0.5008448786561,0.0827482537910611,1.51958937756241,-1.61152207054416,-0.159686564896191,-0.607654321792243,0.355875224252284,0.682223091006738,-0.151137093968282,-0.0405098281418335,-0.405216455019251,1.85691583198501,-1.30924804470962,-0.779486645034238,1.15045549056586,0.375898562147793,0.790317167281757,0.434504283151072,-0.34485076583524,0.238106141139297,0.097739204022422,-0.28948766808858,-0.492144544727051,-0.0998271579421803,-0.0308671491983059,0.438306836601931,0.533385355918136,0.0184425824017311,-0.196739187191697,0.284134803404672,-0.277905041147011,0.40293760410965,0.253636728048455,-0.531514473273818,0.619645409187826,0.0337641548237284,-0.165520358432699,0.193449419012573,-0.97805520367425,1.75652914206816,0.515375184174767,0.261571089618957,-0.137402545227483,0.228334356700646,-0.793544579234621,0.168076232970706,-0.189740073173977,0.717540212662584,-0.907653108174335,0.212950941633978,-0.775821725636505,0.0828258436641294,0.397604657499961,2.27171797577118,-0.428401516587563,0.250957211306446,-0.0100595837856514,-0.812295454273538,-0.0555333599209453,-0.332891547626048,-0.424599603728088,-2.43271003750308,0.695779647927325,-0.147284080182995,-0.399701754361175,0.425668937917134,-0.26246530858275,0.131691784043849,0.0309538654352464,0.091214962906821,0.352864583224274,0.386524767082009,-0.000599015357611865,0.356486042159094]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>id<\/th>\n      <th>lwage<\/th>\n      <th>educ<\/th>\n      <th>exper<\/th>\n      <th>expersq<\/th>\n      <th>fatheduc<\/th>\n      <th>motheduc<\/th>\n      <th>resid_iv_mf<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"dom":"tip","columnDefs":[{"targets":8,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 4, 3, \",\", \".\", null);\n  }"},{"targets":2,"render":"function(data, type, row, meta) {\n    return type !== 'display' ? data : DTWidget.formatRound(data, 2, 3, \",\", \".\", null);\n  }"},{"className":"dt-center","targets":"_all"},{"visible":false,"targets":0},{"orderable":false,"targets":0}],"pageLength":10,"order":[],"autoWidth":false,"orderClasses":false}},"evals":["options.columnDefs.0.render","options.columnDefs.1.render"],"jsHooks":[]}</script>

???

This table shows the  new data set after adding the IV estimation residuals.

---

### Wage example: J-test (run auxiliary regression)

We run the auxiliary regression with `R` code lines:

```r
# set model formula
mod_jtest <- formula(resid_iv_mf ~ exper +expersq +motheduc +fatheduc)
# OLS estimate
lm_jtest <- lm(formula = mod_jtest, data = mroz_resid)
```

Then we can obtain the OLS estimation results.

]

```

Call:
lm(formula = mod_jtest, data = mroz_resid)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.1012 -0.3124  0.0478  0.3602  2.3441

Coefficients:
             Estimate Std. Error t value Pr(>|t|)
(Intercept)  1.10e-02   1.41e-01    0.08     0.94
exper       -1.83e-05   1.33e-02    0.00     1.00
expersq      7.34e-07   3.98e-04    0.00     1.00
motheduc    -6.61e-03   1.19e-02   -0.56     0.58
fatheduc     5.78e-03   1.12e-02    0.52     0.61

Residual standard error: 0.68 on 423 degrees of freedom
Multiple R-squared:  0.000883,	Adjusted R-squared:  -0.00856 
F-statistic: 0.0935 on 4 and 423 DF,  p-value: 0.984
```

]

???

Remind that the model summary gives a F test result, which is differnt with the F statistic in  J-test.

---

### Wage example: J-test (Restricted F-test)

As what we have done before, We conduct the restrict F-test for the auxiliary regression.

We will restrict jointly with `$\theta_1 = \theta_2 =0$`, and using the R function `linearHypothesis()`:

```r
# restricted F-test
restricted_ftest <- linearHypothesis(lm_jtest, c("motheduc = 0", "fatheduc = 0"), test = "F")
# obtain the F statistics
restricted_f <- restricted_ftest$F[[2]]
```

]

```
Linear hypothesis test

Hypothesis:
motheduc = 0
fatheduc = 0

Model 1: restricted model
Model 2: resid_iv_mf ~ exper + expersq + motheduc + fatheduc

Res.Df RSS Df Sum of Sq    F Pr(>F)
1    425 193                         
2    423 193  2     0.171 0.19   0.83
```

]

The restricted F-statistics is 0.1870 (with round digits 4 here ).

???

Please pay attention to the code `c("motheduc = 0", "fatheduc = 0")`

---

### Wage example: J-test (calculate J-statistic by hand)

Finally, We can calculate J-statistic by hand or obtain it by using special tools.

- Calculate J-statistic by hand

```r
# numbers of instruments
m <- 2
# calculate J statistics
(jtest_calc <- m*restricted_f)
```

```
[1] 0.37
```

- The calculated J-statistic is 0.3740 (with round digits 4 here ).

---

### Wage example: J-test (obtain J-statistic with tools)

Also, We can obtain J-statistic by using special tools.

- using tools of  `linearHypothesis(.,  test = "Chisq")`

```r
# chi square test directly
jtest_chitest <- linearHypothesis(
  lm_jtest,  c("motheduc = 0", "fatheduc = 0"), 
  test = "Chisq")
# obtain the chi square value
jtest_chi <- jtest_chitest$Chisq[2]
```

]

- The chi square test result:

```
Linear hypothesis test

Hypothesis:
motheduc = 0
fatheduc = 0

Model 1: restricted model
Model 2: resid_iv_mf ~ exper + expersq + motheduc + fatheduc

Res.Df RSS Df Sum of Sq Chisq Pr(>Chisq)
1    425 193                              
2    423 193  2     0.171  0.37       0.83
```

]

- We obtain the J-statistic 0.3740 (with round digits 4 here ). It's the same as what we have calculated by hand!

???

In `R`, we can use the function `linearHypothesis(.,  test = "Chisq")` by setting argument `test = "Chisq"`.

Please check that the relations between restricted F statistics and the `$\chi^2$` statistics.

---

### Wage example: J-test (adjust the freedoms)

**Caution**: In this case the  
`$p$`-Value reported by `linearHypothesis(., test = "Chisq")` is wrong because the degrees of freedom are set to  2, and the correct freedom should be `$(m-k)=1$`.

]

- We have obtain the J statistics
`${\chi^2}^{\ast} =0.3740$`, and its correct freedom is `$(m-k)=1$`.

- Then we may compute the correct
`$p$`-Value of this the J statistics (by using function `pchisq()` in `R`).

```r
# correct freedoms
f <- m -1
# compute correct p-value for J-statistic
(pchi <- pchisq(jtest_chi, df = f, lower.tail = FALSE))
```

```
[1] 0.54
```
???

This differs from the degree of overidentification (
`$m−k=2−1=1$`). So the 
`$J$`-statistic is 
`$\chi^2(1)$` distributed instead of following a  `$\chi^2(2)$` distribution as assumed defaultly by `linearHypothesis()`.

---

### Wage example: J-test (the conclutions)

Now we can get the conclusions of J-test.

Since the p-value of  J-test(0.5408)is larger than the criteria value 0.1,  we can't reject the null hypothesis that both instruments are exogenous.

This means both instruments(
`motheduc` and `fatheduc`) are **exogenous**.

???

Finally, we go through all instrument validity tests in this section.

the next section we will illustrate how to test regressor endogeneity.

---
layout: false
class: center, middle, duke-softblue,hide_logo
name: endogeneity

## 17.6 Testing Regressor endogeneity

???

In this section, we focus mainly on regressor endogeneity issues.

---
layout: true
  
<div class="my-header-h2"></div>
<div class="watermark1"></div>
<div class="watermark2"></div>
<div class="watermark3"></div>

<div class="my-footer"><span>huhuaping@   <a href="#chapter17"> Chapter 17. Endogeneity and Instumental Variables  |</a> &emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;&emsp; <a href="#endogeneity"> 17.6 Testing Instrument endogeneity </a></span></div>

---

### Regressor Endogeneity: the concepts

How can we test the regressor endogeneity?

Since OLS is in general more efficient than IV (recall that if Gauss-Markov assumptions hold OLS is BLUE), we don't want to use IV when we don't need to get the consistent estimators.

Of course, if we really want to get a consistent estimator, we also need to check whether the endogenous regressors are really **endogenous** in the model.

So we should test following hypothesis:

`$$H_{0}: \operatorname{Cov}(X, \epsilon)=0 \text { vs. } H_{1}: \operatorname{Cov}(X, \epsilon) \neq 0$$`

---

### Regressor Endogeneity: Hausman test

`Hausman` tells us that we should use OLS if we fail to reject 
`$H_{0}$`. And we should use IV estimation if we reject 
`$H_{0}$`

Let's see how to construct a `Hausman test`. While the idea is very simple.

- If 
`$X$` is **.red[ex]ogenous** in fact, then both OLS and IV are consistent, but OLS estimates are more efficient than IV estimates.

- If 
`$X$` is **.red[en]dogenous** in fact, then the results from OLS estimators are different, while results obtained by IV (eg. 2SLS) are consistent.

---

### Hausman test: the idea

We can compare the difference between estimates computed using both OLS and IV.

- If the difference is **small**, we can conjecture that both OLS and IV are consistent and the small difference between the estimates is not systematic. 
- If the difference is **large** this is due to the fact that OLS estimates are not consistent. We should use IV in this case.

---

### Hausman test: the statistics

The **Hausman test** takes the following statistics form

`$$\begin{align}
\hat{H}=n\boldsymbol{\left[\hat{\beta}_{IV}-\hat{\beta}_{\text {OLS}}\right] ^{\prime}\left[\operatorname{Var}\left(\hat{\beta}_{IV}-\hat{\beta}_{\text {OLS}}\right)\right]^{-1}\left[\hat{\beta}_{IV}-\hat{\beta}_{\text {OLS}}\right]} \xrightarrow{d} \chi^{2}(k)
\end{align}$$`

- If 
`$\hat{H}$` is less than the critical 
`$\chi^2$` value, we can not reject the null hypothesis, and the regressor should **not be endogenous**.

- If 
`$\hat{H}$` is **larger** than the critical 
`$\chi^2$` value, 
the null hypothesis is rejected , and the regressor should **be endogenous**.

---

### Wage example: Hausman test (the origin IV model)

Again, we use both `$matheduc$` and `$fatheduc$` as instruments for `$educ$` in our IV model setting.

`$$\begin{cases}
  \begin{align}
  \widehat{educ} &= \hat{\gamma}_1 +\hat{\gamma}_2exper + \hat{\beta}_3expersq +\hat{\beta}_4motheduc + \hat{\beta}_5fatheduc  && \text{(stage 1)}\\
  lwage & = \hat{\alpha}_1 +\hat{\alpha}_2\widehat{educ} + \hat{\alpha}_3exper +\hat{\alpha}_4expersq + \hat{\epsilon}  && \text{(stage 2)}
  \end{align}
\end{cases}$$`

in `R`, we can use IV model diagnose tool to check the Hausman test results.

In fact, `R` function `summary(lm_iv_mf, diagnostics = TRUE)` by setting `diagnostics = TRUE` will give you these results.

]

---

### Wage example: Hausman test (full model diagnose)

```r
summary(lm_iv_mf, diagnostics = TRUE)
```

```

Call:
ivreg(formula = mod_iv_mf, data = mroz)

Residuals:
    Min      1Q  Median      3Q     Max 
-3.0986 -0.3196  0.0551  0.3689  2.3493

Diagnostic tests:
                 df1 df2 statistic p-value    
Weak instruments   2 423     55.40  <2e-16 ***
Wu-Hausman         1 423      2.79   0.095 .  
Sargan             1  NA      0.38   0.539    
---
Signif. codes:  
0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.68 on 424 degrees of freedom
Multiple R-Squared: 0.136,	Adjusted R-squared: 0.13 
Wald test: 8.14 on 3 and 424 DF,  p-value: 2.79e-05 
```

]

---

### Wage example: the diagnosed conclusions

The results for the lwage equation are as follows:

- **(Wu-)Hausman test** for endogeneity: **barely reject** the null that the variable of concern is uncorrelated with the error term, indicating that  `educ`  is marginally endogenous. The Hausman statistics
`$\hat{H}= {\chi^2}^{\ast} = 2.79$`, and its p-value is 0.095.

- **Weak instruments test**: **rejects** the null hypothesis(Weak instruments). At least one of these instruments(`motheduc` or `fatheduc`) is strong. The **restricted F-test** statistics 
`$F^{\ast}_R = 55.4$`, and its p-value is 0.0000.

- **Sargan overidentifying restrictions**(Instruments exogeneity J-test): **does not** reject the null. The extra instruments (`motheduc` and `fatheduc`)  are valid (both are exogenous, and are uncorrelated with the error term).

???

So far, We have finished both the instrument validity test and the regressor endogeneity test.

Now, I will show you two examples. You can download the data set and go through all these test we have discussed.

---

### Summary

- An **instrumental variable** must have two properties:

- (1) it must be exogenous, that is, uncorrelated with the error term of the structural equation; 
    - (2) it must be partially correlated with the endogenous explanatory variable. 
    
> Finding a variable with these two properties is usually challenging.

- Though we can **never** test whether .red[all] IVs are **exogenous**, we can test that at least .red[some of] them are.

- When we have valid instrumental variables, we can test whether an explanatory variable is **endogenous**.

- The method of **two stage least squares**  is used routinely in the empirical social sciences.

>  But when instruments are poor, then 2SLS can be **worse** than OLS.

---

### Exercise example 1: Card (1995)

In Card (1995) education is assumed to be endogenous due to omitted **ability** or **measurement error**. The standard wage function

`$$\ln \left(w a g e_{i}\right)=\beta_{0}+\beta_{1} E d u c_{i}+\sum_{m=1}^{M} \gamma_{m} W_{m i}+\varepsilon_{i}$$`

is estimated by **Two Stage Least Squares** using a **binary instrument**, which takes value 1 if there is an **accredited 4-year public college in the neighborhood** (in the "local labour market"), 0 otherwise.

> It is argued that the presence of a local college decreases the cost of further education (transportation and accommodation costs) and particularly affects the schooling decisions of individuals with poor family backgrounds.

The set of exogenous explanatory regressors 
`$W$` includes variables like race, years of potential labour market experience, region of residence and some family background characteristics.

---

### Exercise example 1:  Card (1995)

The dataset is available online at
http://davidcard.berkeley.edu/data_sets.html and consists of 3010
observations from the National Longitudinal Survey of Young Men.

- **Education** is measured by the years of completed schooling and varies
we between 2 and 18 years.

> To overcome the small sample problem, you might group the years of education into four educational levels: less than high school, high school graduate, some college and post-college education (a modified version of Acemoglu and Autor (2010) education grouping).

- Since the **actual labour market experience** is not available in the dataset, Card (1995) constructs a potential experience as **age-education-6**.

> Since all individuals in the sample are of similar age (24-34), people with the same years of schooling have similar levels of potential experience.

---

### Exercise example 2: Angrist and Krueger (1991)

The data is available online at http://economics.mit.edu/faculty/angrist/data1/data/angkru1991
and consists of observations from 1980 Census documented in Census of Population and Housing, 1980: Public Use Microdata Samples.

The sample consists of men born in the United States between 1930-1949 divided into two cohorts: those born in the 30's (329509 observations) and those born in the 40's (486926 observations).

**Angrist and Krueger** (1991) estimate the conventional linear earnings function

`$$\begin{align}
\ln \left(w a g e_{i}\right)=\beta E d u c_{i}+\sum_{c} \delta_{c} Y_{c i}+\sum_{s=1}^{S} \gamma_{s} W_{s i}+\varepsilon_{i}
\end{align}$$`

for each cohort separately, by 2SLS using the **quarter of birth** as an instrument for (assumed) endogenous **education**.

---

### Exercise example 2: Angrist and Krueger (1991)

- They observe that individuals born earlier in the year (first two quarters) have less schooling than those born later in the year.

> It is a consequence of **the compulsory schooling laws**, as individuals born in the first quarters of the year reach ***the minimum school leaving age*** at the lower grade and might legally leave school with less education.

- The main criticism of Angrist and Krueger (1991) analysis, pointed out by Bound, Jaeger and Baker (1995) is that the quarter of birth is a **weak instrument**.

- A second criticism of Angrist and Krueger (1991) results, discussed by Bound and Jaeger (1996) is that quarter of birth might be **correlated** with unobserved ability and hence does .red[not] satisfy the **instrumental exogeneity condition**.

---
layout:false
background-image: url("../pic/thank-you-gif-funny-little-yellow.gif")
class: inverse,center

# End Of This Chapter

???

So we finished all content of chapter 17.

The next three chapters will focus on SEM closely.

See you in the next class.

if you have questions, please let me know. you can leave messages by QQ or email.

Thanks. Goodbye!