\documentclass[11pt]{article} 
%\usepackage{amsbsy} % for \boldsymbol and \pmb 
\usepackage{graphicx} % To include pdf files!
\usepackage{amsmath}
\usepackage{amsbsy}
\usepackage{amsfonts}
\usepackage[colorlinks=true, pdfstartview=FitV, linkcolor=blue, citecolor=blue, urlcolor=blue]{hyperref} % For links
\usepackage{fullpage}
%\pagestyle{empty} % No page numbers

% Search instrumental and fix up.

\begin{document}

%\enlargethispage*{1000 pt} 


\begin{center}   
{\Large \textbf{STA 431s15 Assignment Seven}}\footnote{This assignment was prepared by  \href{http://www.utstat.toronto.edu/~brunner}{Jerry Brunner},
Department of Statistical Sciences, University of Toronto. It is licensed under a 
\href{http://creativecommons.org/licenses/by-sa/3.0/deed.en_US}
     {Creative Commons Attribution - ShareAlike 3.0 Unported License}. Use any part of it as you like and share the result freely. The \LaTeX~source code is available from the course website:
\href{http://www.utstat.toronto.edu/~brunner/oldclass/431s15} {\small\texttt{http://www.utstat.toronto.edu/$^\sim$brunner/oldclass/431s15}}}
\vspace{1 mm}
\end{center}

\noindent
The non-computer questions on this assignment are practice for Term Test 2 and the final exam; they will not be handed in.  There will be no SAS on Term Test 2.  The SAS part of this assignment (Questions~\ref{morepigs} and~\ref{SASIQ}) are for Quiz Seven on Friday March 20. Please bring your log files and your output files to the quiz. There will be one or more questions about them, and you will be asked to hand printouts in with the quiz. 

\begin{enumerate} 

% Excellent for omitted variables section of Chapter Zero. Include matrix version for the surface model.

    \item Since the error term $\epsilon_i$ in a regression equation represents ``everything else," omission of explanatory variables that are correlated with the explanatory variables in the model will induce a non-zero covariance between the error term and the explanatory variables in the model. Throughout this question, we will set expected values and intercepts aside, and focus on the covariance matrix. Thus, parameters will be called ``identifiable" if and only if they are identifiable from the covariance matrix of the observable data. 
    \begin{enumerate}
        \item Here is an example for simple regression with no measurement error.
\begin{center}
\includegraphics[width=1.5in]{A7path1}
\end{center}
        \begin{enumerate}
            \item Give the model equation in centered form. The centering can be invisible. 
            \item What is the parameter vector $\boldsymbol{\theta}$ for this model?
            \item Does this model pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers.
            \item Calculate the covariance matrix $\boldsymbol{\Sigma}$ of the observable data vector.
            \item Is the parameter $\beta$ identifiable? Answer Yes or No. If the answer is Yes, prove it. If the answer is No, give a simple numerical example of two parameter vectors with \emph{different} $\beta$ values, yielding the same covariance matrix $\boldsymbol{\Sigma}$ of the observable data.
        \end{enumerate}

 \newpage

        \item Now we add an instrumental variable $Z$. 
\begin{center}
\includegraphics[width=2in]{A7path2}
\end{center}
        \begin{enumerate}
            \item Give the model equation in centered form. The centering can be invisible. 
            \item What is the parameter vector $\boldsymbol{\theta}$ for this model?
            \item Does this model pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers.
            \item Calculate the covariance matrix $\boldsymbol{\Sigma}$ of the observable data vector.
            \item Is the entire parameter $\boldsymbol{\theta}$ identifiable in the whole parameter space? Answer Yes or No. If the answer is Yes, prove it. If the answer is No, give the set of points where the parameter vector is not identifiable.
            \item Give a simple numerical example of two parameter vectors with different $\beta$ values, yielding the same covariance matrix $\boldsymbol{\Sigma}$ of the observable data.
            \item Is the entire parameter $\boldsymbol{\theta}$ identifiable where $\phi_{12}\neq 0$? If the answer is Yes, prove it.
            \item Why is it reasonable to assert $\phi_{12}\neq 0$?
        \end{enumerate}

        \item Now suppose $X$ is measured with error as well as being correlated with omitted variables.
\begin{center}
\includegraphics[width=2.2in]{A7path3}
\end{center}
        \begin{enumerate}
            \item Give the model equations in centered form. The centering can be invisible. 
            \item What is the parameter vector $\boldsymbol{\theta}$ for this model?
            \item Does this model pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers.
            \item Calculate the covariance matrix $\boldsymbol{\Sigma}$ of the observable data vector.
            \item Is the parameter $\beta$ identifiable provided $\phi_{12}\neq 0$? Answer Yes or No. If the answer is Yes, prove it. If the answer is No, give a simple numerical example of two parameter vectors with different $\beta$ values, yielding the same covariance matrix $\boldsymbol{\Sigma}$ of the observable data.
        \end{enumerate}

    \end{enumerate}


 \newpage

\item Independently for $i=1, \ldots, n$, let
\begin{eqnarray*} 
   W_{i\mbox{~}} & = &   X_i + e_i            \\  
   Y_{i,1}          & = &  \beta_1 X_i + \epsilon_{i,1}  \\  
   Y_{i,2}          & = & \beta_2 X_i + \epsilon_{i,2}  
\end{eqnarray*}
where $X_i$, $e_i$,  $\epsilon_{i,1}$ and $\epsilon_{i,2}$ are all independent,
$Var(X_i)=\phi$, $Var(e_i)=\omega$, $Var(\epsilon_{i,1})=\psi_1$, $Var(\epsilon_{i,2})=\psi_2$, and all the expected values are zero. The explanatory variable $X_i$ is latent, while $W_i$, $Y_{i,1}$ and $Y_{i,2}$ are observable  

        \begin{enumerate}
            \item Make a path diagram for this model 
            \item What is the parameter vector $\boldsymbol{\theta}$ for this model? 
            \item Does this model pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers.
            \item Calculate the variance-covariance matrix of the observable variables. Show your work.
            \item The parameter of primary interest is $\beta_1$. Is $\beta_1$ identifiable at points in the parameter space where $\beta_1=0$? Why or why not?
            \item Is $\omega$ identifiable where $\beta_1=0$? 
            \item Give a simple numerical example to show that  $\beta_1$ is not identifiable at points in the parameter space where $\beta_1 \neq 0$ and $\beta_2=0$.
            \item Is $\beta_1$ identifiable at points in the parameter space where $\beta_2 \neq 0$? Answer Yes or No and prove your answer. 
            \item Show that the entire parameter vector is identifiable at points in the parameter space where $\beta_1 \neq 0$ and $\beta_2 \neq 0$. 
            \item Recall that an \emph{instrumental variable} is an observable variable that has non-zero covariance with the explanatory variable and zero covariance with the error term in the regression. Under what condition is $Y_2$ an instrumental variable for $X$?
            \item Since the parameter of primary interest is $\beta_1$, it's important to be able to test $H_0: \beta_1=0$. So at points in the parameter space where  $\beta_2 \neq 0$, what \emph{two} equality constraints on the elements of $\boldsymbol{\Sigma}$ are implied by $H_0: \beta_1=0$? If this does not bother you, it should. 
% beta1 may be identifiable when beta1=0, but the entire parameter vector is not.  Fit a reduced model for a likelihood ratio test, and SAS will complain. Probably the Z test will look okay, or will it? I think it's a 2-df test, which should be carried out directly on the covariance matrix. But I don't actually know. Check Wilks' LR test paper. Is identifiability needed under H0? I think it's needed for the asymptotic normality of the MLE. 
            \item Assuming $\beta_1 \neq 0$ and $\beta_2 \neq 0$, you can use the model to deduce more than one testable \emph{inequality} involving the variances and covariances. Give at least one example.
        \end{enumerate}

\newpage

\item  Independently for $i=1, \ldots, n$, let
\begin{eqnarray*} 
   Y_{i,1} & = & \alpha_1 + \beta_1 X_{i,1} + \epsilon_{i,1}    \\ 
   Y_{i,2} & = & \alpha_2 + \beta_2 X_{i,2} + \epsilon_{i,2}    \\
   W_{i,1} & = & \nu_1 +  X_{i,1} + e_{i,1}                     \\  
   W_{i,2} & = & \nu_2 +  X_{i,2} + e_{i,2}                     \\ 
   V_{i,1} & = & \nu_3 +  Y_{i,1} + e_{i,3}                     \\  
   V_{i,2} & = & \nu_4 +  Y_{i,2} + e_{i,4},               
\end{eqnarray*}
where $E(X_{i,j})=\mu_j$, $e_{i,j}$ and  $\epsilon_{i,j}$ are independent of one another and of $X_{i,j}$, $Var(e_{i,j})=\omega_j$, $Var(\epsilon_{i,j})=\psi_j$, and
\begin{displaymath}
V\left(
\begin{array}{c} X_{i,1} \\ X_{i,2} \end{array}
 \right) = \left(
\begin{array}{c c} 
\phi_{11} & \phi_{12} \\
\phi_{12} & \phi_{22}
\end{array} \right).
\end{displaymath} 
In this model, $X_{i,1}$, $X_{i,2}$, $Y_{i,1}$ and $Y_{i,2}$ are latent variables, while $W_{i,1}$, $W_{i,2}$, $V_{i,1}$ and $V_{i,2}$ are observable. 
  \begin{enumerate}
        \item Make a path diagram for this model.
        \item Does this model fit the double measurement design?
        \item Calculate the variance-covariance matrix of the observable variables. Show your work.
            \item Does this problem pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers.
        \item Show that $\beta_1$ and $\beta_2$ are identifiable provided $\phi_{12} \neq 0$.
        \item Are there any other points in the parameter space where $\beta_1$ is identifiable? $\beta_2$?
        \item Give one testable equality constraint (a statement about the $\sigma_{ij}$ quantities) that is implied by the model. Is it still true with $\phi_{12}=0$? $\beta_1 = 0$? $\beta_2 = 0$?
        \item Suppose you wanted to estimate $\beta_1$. Suggest a \emph{statistic} (function of the sample data) to serve as an estimator.
        \item Is your estimator consistent? Under what circumstances? You don't have to prove anything in detail. 
        \item If the primary interest is in $\beta_1$, do we really need the response variable $Y_{i,2}$?
        \item Does any variable in this model qualify as an instrumental variable?
  \end{enumerate}

\newpage  

    \item In this problem, $W_i$ and the $Y_{ij}$ variables are observable, and $X_i$ is latent. The response variable of primary interest is $Y_{i,1}$, while $Y_{i,2}$ and $Y_{i,3}$ are included to help with identifiability. The point of the question is that the error terms need not all be independent for this to work.

Independently for $i=1, \ldots, n$,
\begin{eqnarray} 
    Y_{i,1}  & = &  \beta_{0,1} + \beta_{1,1} X_i + \epsilon_{i,1}  \nonumber  \\
    Y_{i,2}  & = &  \beta_{0,2} + \beta_{1,2} X_i + \epsilon_{i,2}  \nonumber  \\
    Y_{i,3}  & = &  \beta_{0,3} + \beta_{1,3} X_i + \epsilon_{i,3}  \nonumber  \\
    W_{i~}   & = &  X_i + e_i         \nonumber 
\end{eqnarray}
where 
\begin{itemize}
     \item $X_i \sim N(\mu_x,\phi)$ is a latent variable
     \item $e_i \sim N(0,\omega)$
     \item $\boldsymbol{\epsilon}_i = 
           (\epsilon_{i,1},\epsilon_{i,2},\epsilon_{i,3})^\top$
     \item $X_i$, $e_i$ and $\boldsymbol{\epsilon}_i$ are independent of one another
     \item $\boldsymbol{\epsilon}_i$ is multivariate normal with mean zero and covariance matrix
\end{itemize}
    \begin{displaymath}
    \boldsymbol{\Psi} = 
     \left[ \begin{array}{c c c}
                 \psi_{1,1} & \psi_{1,2} & 0 \\
                 \psi_{1,2} & \psi_{2,2} & \psi_{2,3}  \\
                 0          & \psi_{2,3} & \psi_{3,3} 
    \end{array} \right] .
    \end{displaymath}
        \begin{enumerate}
        \item Make a path diagram for this model 
            \item What is the parameter vector $\boldsymbol{\theta}$ for this model? 
            \item How many moment structure equations are there? You do not have to say what they are; just give a number. Don't forget the means.
            \item Does this problem pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers.
            \item Calculate the variance-covariance matrix of the observable variables. Remember that some covariances between errors are non-zero. Show your work.
            \item Solving the complete set of moment structure equations can be 
            done\footnote{Even the intercepts are identifiable from the mean vector $\boldsymbol{\mu}$, because there is no measurement bias term in this model. That's unrealistic, of course.}
            but it's a big chore. The primary interest is in the parameter $\beta_{1,1}$. Show that just this parameter is identifiable.
            \item Does any variable in this model qualify as an instrumental variable?
        \end{enumerate}

\newpage 

\item Here is a model with two instrumental variables. Note that in this case there are omitted variables that affect the observable versions of both explanatory variables, so that the measurement error terms are correlated. This is usually poison. 
\begin{center}
\includegraphics[width=4in]{A7path4}
\end{center}

        \begin{enumerate}
            \item Give the model equations in centered form. The centering can be invisible. 
            \item What is the parameter vector $\boldsymbol{\theta}$ for this model?
            \item Does this model pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers.
            \item Calculate the covariance matrix $\boldsymbol{\Sigma}$ of the observable data vector. When I did it I used the order of observable variables in the path diagram, but it will be easier to check identifiability if you write the observable data vector as $\mathbf{D} = (W_1,W_2,Z_1,Z_2,Y)^\top$.
            \item Are the parameters $\beta_1$ and $\beta_2$ identifiable? Answer Yes or No. If the answer is Yes, prove it. You don't have to finish solving for $\beta_1$ and $\beta_2$. Just give two linear equations in $\beta_1$ and $\beta_2$ as well as a number of $\sigma_{ij}$ quantities.  Presumably it's possible to solve two linear equations in two unknowns. 
        \end{enumerate}

\newpage

    \item \label{IQtheory} Question \ref{SASIQ} (part of the SAS assignment) will use the \emph{Longitudinal IQ Data}. IQ is short for ``Intelligence Quotient," and IQ tests are attempts to measure intelligence. A score of 100 is considered average, while scores above 100 are above average and scores below 100 are below average.  Most IQ tests have many sub-parts, including vocabulary tests, math tests, logical puzzles, tests of spatial reasoning, and so on. What the better tests probably succeed in doing is to measure one \emph{kind} of intelligence --- potential for doing well in school. Of course, they measure it with error.

In the Longitudinal IQ Data, the IQs of adopted children were measured at ages 2, 4, 8 and 13. The birth mother's IQ was assessed at the time of adoption, and the adoptive mother's education (in years) was also recorded. The variables are
        \begin{itemize}
            \item Adoptive mother's education
            \item Birth mother's IQ
            \item IQ at age 2
            \item IQ at age 4
            \item IQ at age 8
            \item IQ at age 13
        \end{itemize}
In our dreams, we wish for a regression model in which the explanatory variables are adoptive mother's actual education (a latent variable), birth mother's true IQ (also latent), and child's IQ at ages 2, 4, 8 and 13 --- all latent. Well, adoptive mother's education has only one measurement and no convincing instrumental variables, so we'll reluctantly set it aside for now.

        \begin{enumerate}
            \item To show you know what's going on, write down a regression model for just the IQ part of the data. My model has 5 latent variables and 5 observable variables. Give all the details. It has been verified many times that IQ scores have a normal distribution, so for once the normal distribution assumption is very reasonable. 
            \item As usual, set the intercepts and expected values aside. Calculate the covariance matrix in terms of the model parameters. 
            \item Does the model pass the test of the parameter count rule? Give the numbers.
            \item To get out of this mess, we re-parameterize, combining the variance of $\epsilon$ and the variance of $e$ into a single parameter for each response variable. This is equivalent to adopting a model with no measurement error in the response variables. So now we have a model that has one explanatory variable measured with error, and 4 response variables measured without error. Write the covariance matrix for this model, which you can mostly just copy from your earlier work. 
            \item Show that the parameters of your model (anyway, those appearing in the covariance matrix) are identifiable. What do you need to assume? What hypotheses would you test about single $\sigma_{ij}$ quantities to verify this?
            \item \label{fitdf} How many degrees of freedom should there be in the likelihood ratio test for model fit? The answer is a number.

\newpage

            \item Suppose you want to test whether all the regression coefficients are equal, using a likelihood ratio test. 
            \begin{enumerate}
                \item \label{eqdf} What are the degrees of freedom for this test?
                \item If you reject $H_0$, what will you conclude about how the birth mother's IQ is related to the child's IQ at various ages?
            \end{enumerate}
        \end{enumerate}
% \end{enumerate}



\item \label{morepigs} We have some unfinished business from the pig study of Assignment 6.

    \begin{enumerate}

        \item You tested for correlated measurement error within questionnaires with two separate $Z$ tests. It was pretty convincing, but conduct a \emph{single} Wald (not likelihood ratio) test of the two null hypotheses simultaneously. The SAS program \texttt{bmi3.sas} has an example of how to do a Wald test. 
        \begin{enumerate}
            \item Give the Wald chi-squared statistic, the degrees of freedom and the $p$-value. What do you conclude? Is there evidence of correlated measurement error, or not? % W =45.81848 , df=2, p < 0.0001  
            \item Find two examples of $Z^2 \sim \chi^2(1)$ from the output for this question.
        \end{enumerate}
        \item The double measurement design allows the measurement error covariance matrices $\boldsymbol{\Omega}_1$ and $\boldsymbol{\Omega}_2$ to be unequal. Carry out a Wald test to see whether the two covariance matrices are equal or not. 
        \begin{enumerate}
            \item Give the Wald chi-squared statistic, the degrees of freedom and the $p$-value. What do you conclude? Is there evidence that the two measurement error covariance matrices are unequal? %  W = 42.06843 , df=3, p < 0.0001
            \item There is evidence that one of the measurements is less accurate on one questionnaire than the other. Which one is it? Give the Wald chi-squared statistic, the degrees of freedom and the $p$-value.
% Number of pigs born is more accurate on questionnaire 2 (estimated error variance = 93.00057 compared to 321.25427): W = 9.66874 , df=1, p = 0.0019
        \end{enumerate}
    \end{enumerate}

\pagebreak

    \item \label{SASIQ}
The longitudinal IQ data described in Question~\ref{IQtheory} are given in the file
\href{http://www.utstat.toronto.edu/~brunner/data/illegal/origIQ.data.txt}
     {\texttt{origIQ.data.txt}}.
These data are taken from \emph{The Statistical Sleuth} by F. Ramsey and D. Schafer, and are reproduced without permission. There is a link on the course web page in case the one in this document does not work. Note there are $n=62$ cases, so please verify that you are reading the correct number of cases.

    \begin{enumerate}
        \item Start by reading the data and then running \texttt{proc~corr} to produce a correlation matrix (with tests) of all the variables, including adoptive mother's education. 
        \item How are the \texttt{proc~corr} results helpful in justifying your identifiability conditions from the Question~\ref{IQtheory}?
        \item Remember your model that has one explanatory variable measured with error, and 4 response variables measured without error? We'll call this the \emph{full model}. Please fit the full model.  
        \item Sticking strictly to the $\alpha=0.05$ significance level, does the full model fit the data adequately? Answer Yes or No, and give a value of $G^2$, the degrees of freedom and the $p$-value. These numbers are all directly on your printout. Do the degrees of freedom agree with your answer to Question~\ref{fitdf}?
        \item Now fit the reduced model in which all the regression coefficients are equal. Using a calculator (or \texttt{proc~IML} if you want to), calculate the likelihood ratio test comparing the full and reduced models. Obtain $G^2$, a number. 
        \item What are the degrees of freedom for this test? Compare your answer to Question~\ref{eqdf}.
        \item Using this table of critical values, do you reject $H_0$ at $\alpha=0.05$? Answer Yes or No. Does birth mother's IQ seem to affect her child's IQ to the same degree at different ages?
\begin{verbatim}
> df = 1:8
> CriticalValue = qchisq(0.95,df)
> round(rbind(df,CriticalValue),3)
               [,1]  [,2]  [,3]  [,4]  [,5]   [,6]   [,7]   [,8]
df            1.000 2.000 3.000 4.000  5.00  6.000  7.000  8.000
CriticalValue 3.841 5.991 7.815 9.488 11.07 12.592 14.067 15.507

\end{verbatim}
To be continued \ldots
    \end{enumerate}

\end{enumerate}



\vspace{30mm}

\noindent Bring your log files and your output files to the quiz. You may be asked for numbers from your printouts, and you may be asked to hand them in. There are lots of \textbf{There must be no error messages, and no notes or warnings about invalid data on your log file.}




\end{document}

% More LongIQ for next time:

        \item That was interesting, but now let's bring in adoptive mother's education. We wonder whether, controlling for birth mother's true IQ, adoptive mother's true education is related to the child's true IQ. Write the model equations. There are now \emph{two} regression coefficients for each response variable. Don't bother with the intercepts.
        \item Calculate the covariance matrix in terms of the model parameters. Does this model pass the test of the parameter count rule?
        \item There is still hope. Your model has a term $\phi_{12}$, representing the covariance between birth mother's true IQ and adoptive mother's true education. But unless the adoption agency acted in a very peculiar way, there is no reason these variables should be related. Furthermore, it's testable without actually fitting the model under consideration. Locate the test on your printout (it's there) and give the $p$-value.
        \item This is exploratory data analysis, so let's tentatively accept the (null) hypothesis $\phi_{12}=0$. Under this assumption, your covariance matrix simplifies quite a bit. Either re-write it, or else circle the terms with $\phi_{12}$, to remind yourself that they equal zero.
        \item The regression coefficients linking adoptive mother's education to child's IQ at various ages are now identifiable, meaning they are identifiable at points in the parameter space where $\phi_{12}=0$. Recover one of them from the covariance matrix just to show you can do it.
        \item Under the null hypothesis that adoptive mother's true education has no effect on child's IQ at any age, four covariances in $\boldsymbol{\Sigma}$ should be zero (assuming $\phi_{12}=0$, of course). Which ones are they?
        \item Give the $p$-values, numbers from your printout. What do you conclude? Do these data support a link between adoptive mother's education and child's IQ?

 
This story could be continued a bit more, but even as it stands, it's pretty good. The lesson is that valid inference about a latent variable model may be possible even when the model parameters cannot be estimated.




% A big question I cut out

    \item \label{twovars} Independently for $i=1, \ldots, n$, let
\begin{eqnarray*}
    W_{i,1} &=& X_{i,1} + e_{i,1}   \\ 
    W_{i,2} &=& X_{i,2} + e_{i,2}   \\ 
    Y_{i,1} &=& \beta_1 X_{i,1} + \epsilon_{i,1}   \\
    Y_{i,2} &=& \beta_2 X_{i,2} + \epsilon_{i,2}   \\
    Y_{i,3} &=& \beta_3 X_{i,1} + \beta_4 X_{i,2} + \epsilon_{i,3}   
\end{eqnarray*}
where 
\begin{itemize}
     \item The $X_{i,j}$ variables are latent, while the $W_{i,j}$ and $Y_{i,j}$ variables are observable. 
     \item $e_{i,1}\sim N(0,\omega_1)$ and $e_{i,2}\sim N(0,\omega_2)$.
     \item $\epsilon_{i,j} \sim N(0,\psi_j)$ for $j=1,2,3$.
     \item $e_{i,j}$ and $\epsilon_{i,j}$ are independent of each other and of $X_{i,j}$.
     \item $X_{i,j}$ have expected value zero and
\end{itemize}
\begin{displaymath}
V\left(
\begin{array}{c} X_{i,1} \\ X_{i,2} \end{array}
 \right) = \left(
\begin{array}{c c} 
\phi_{11} & \phi_{12} \\
\phi_{12} & \phi_{22}
\end{array} \right).
\end{displaymath} 
Denote the vector of observable data by $\mathbf{D}_i = (W_{i,1}, W_{i,2}, Y_{i,1}, Y_{i,2}, Y_{i,3})^\top$, with $V(\mathbf{D}_i) = \boldsymbol{\Sigma} = [\sigma_{ij}]$. 

Among other things, this question illustrates how the search for identifiability can be supported by exploratory data analysis. Hypotheses about \emph{single} covariances, like $H_0: \sigma_{ij}=0$ can be tested without effort by looking at tests of the corresponding correlations. These tests are produced automatically by \texttt{proc~corr}.

\vspace{5mm}

       \begin{enumerate}
            \item What is the parameter vector $\boldsymbol{\theta}$ for this model? 
            \item \label{countingruleQ} Does this problem pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers. % 12 parameters, 5(5+1)/2 = 15 moments.
            \item Calculate the variance-covariance matrix of the observable variables. Show your work.
            \item The parameter $\phi_{12}$ is identifiable. How?
            \item Suppose $\beta_1=0$. Why is the parameter $\beta_1$ identifiable? Of course the same applies to $\beta_2$.
            \item But the idea here is that $Y_1$ and $Y_2$ are instrumental variables, so that $\beta_1 \neq 0$ and $\beta_2 \neq 0$. What hypotheses about \emph{single} covariances would you test to verify this?
            \item From this point on, suppose we have verified $\beta_1 \neq 0$ and $\beta_2 \neq 0$. Under what circumstances (that is, where in the parameter space) can the parameters $\beta_1$ and $\beta_2$ be easily identified?
            \item What hypotheses about \emph{single} covariances would you test to persuade yourself that this is okay?
            \item  Assuming the last step worked out well, give a formula for $\beta_1$ in term of $\sigma_{ij}$ values. 
            \item \label{beta2hat} Suppose you were sure $\phi_{12} \neq 0$, but you were not so sure about normality so you were uncomfortable with maximum likelihood estimation. Suggest a nice estimator of $\beta_2$. Why are you sure it is consistent? Note that even if you were interested in the MLE, this estimate would be an excellent starting value.
            \item Suppose your test for $\phi_{12} = 0$ did not reject the null hypothesis, so  dividing by $\sigma_{12}$ makes you uncomfortable. Show that even if $\phi_{12} = 0$, 
% the parameters $\beta_1$ and $\beta_2$ are still identifiable except on a set of volume zero in the parameter space. 
%           \item What covariance would you test to verify that the true parameter vector is \emph{not} in that volume zero set where $\beta_1$ 
there is another way to identify $\beta_1$. What assumption to you have to make (that is, where in the parameter space does the true parameter vector have to be) for this to work? How would you test it? % \beta1 = sigma35/sigma15, beta2 = sigma45/sigma25
            \item How could you identify $\beta_2$ if $\phi_{12} = 0$? 
            \item In question \ref{beta2hat}, you gave an estimator for $\beta_2$ that is consistent in most of the parameter space. Based on your answer to the preceding question, give a second estimator for $\beta_2$ that is consistent in most of the parameter space. It should be geometrically obvious that except for a set of volume zero in the parameter space, \emph{both} estimators are consistent. % Bringing up the possibility of different asymptotic efficiency depending on there in the parameter space the true parameter is located. Lovely. 
            \item Assuming $\beta_1$ and $\beta_2$ are identifiable one way or the other, now we seek to identify $\phi_{11}$ and $\phi_{22}$. How can this be done? Give the formulas. Also, give a consistent estimator of $\phi_{22}$ that is not the MLE. Why are you sure it's consistent?
            \item Since  $Y_1$ and $Y_2$ are instrumental variables, primary interest is in $\beta_3$ and $\beta_4$, the coefficients linking $Y_3$ to $X_1$ and $X_2$. If our efforts so far have been successful (which they are, except on a set of volume zero in the parameter space), then $\beta_3$ and $\beta_4$ can be identified as the solution to two linear equations in two unknowns. Write these equations \emph{in matrix form}. 
            \item What condition on the $\phi_{ij}$ values ensure a unique solution to the two equations in two unknowns? Is this a lot to ask? 
            \item Now let's back up, and admit that the identification of $\beta_3$ and $\beta_4$ is really the whole point, since they are the parameters of interest. We have seen that $\phi_{12}$ is always identifiable. If $\phi_{12} \neq 0$, it can be used to identify $\beta_1$ and $\beta_2$, and they can be used to identify $\phi_{11}$ and $\phi_{22}$. Then $\beta_3$ and $\beta_4$ can be identified by solving the two equations in two unknowns. Now suppose that $\phi_{12}=0$. In this case $\beta_3$ and $\beta_4$ can be identified without knowing the values of $\phi_{11}$ and $\phi_{22}$, provided $\beta_1$ and $\beta_2$ are non-zero. Show how this can be done. % \beta_3=\sigma_{3,5}/\sigma_{13}, \beta_4=\sigma_{4,5}/\sigma_{2,4}
            \item Assuming that the parameters appearing in the covariances of $\boldsymbol{\Sigma}$ are identifiable, the additional 5 parameters (whch appear only in the variances) may be identified by subtraction. So we see that except on a set of volume zero in the parameter space, all the parameters are identifiable. In that region, how many equality constraints should the model impose on the covariance matrix? Use your answer to Question~\ref{countingruleQ}.
            \item To see what the equality constraints are, note that earlier parts of this question point to two ways of identifying $\beta_1$ and two ways of identifying $\beta_2$. There are also two simple ways to identify $\phi_{12}$. So write down the three constraints. Multiply through by the denominators.
            \item Now you have three equalities involving products of $\sigma_{ij}$ terms. For each one, use your covariance matrix to write both sides in terms of the model parameters. For each equality, does it hold everywhere in the parameter space, or are there some points in the parameter space where it does not hold? If there are points in the parameter space where an equality does not hold, state the set explicitly. 
            \item The idea here is that the three degrees of freedom in the likelihood ratio test of model fit correspond to three equalities involving the covariances, and those equalities are directly testable without the normality assumption\footnote{It's true that I have not told you how to do this yet, but it's not hard.} required by the likelihood ratio test. State the null hypothesis (there's just one) in terms of the $\sigma_{ij}$ quantities.
            \item If the null hypothesis were rejected, what would you conclude about the model?
            \item In ordinary multivariate regression (which has more than one response variable), it is standard to assume the error terms for the response variable may have non-zero covariance. Suppose, then, that $Cov(\epsilon_{i,1},\epsilon_{i,2}) = \psi_{12}$. How would this change the covariance matrix?
            \item Always remembering that $\beta_1$ and $\beta_2$ are non-zero, suppose that $\phi_{12}=0$. Is $\psi_{12}$ identifiable? What if $\phi_{12}\neq 0$?
            \item Well, what if there were non-zero covariances $\psi_{13}$ and $\psi_{23}$ as well? What does the parameter count rule tell you? 
            \item  Again by the parameter count rule, $\phi_{12}\neq 0$ is absolutely necessary to identify the entire parameter if all three $\psi_{ij}$ are added to the model. Why? In this case, are $\psi_{13}$ and $\psi_{23}$ identifiable? Why or why not?
        \end{enumerate}

% Need to work out whether correlation between the EXPLANATORY variables and \epsilon_{i,1},\epsilon_{i,2} would make beta3 and beta4 non-identifiable. 
% In the textbook, discuss why H_0 beta3=beta4=0 is still a 2df test even though it produces 4 constraints on the sigmas, as long as the whole parameter is identifiable.