% 431s23Assignment7.tex Regression with measurement error \documentclass[11pt]{article} %\usepackage{amsbsy} % for \boldsymbol and \pmb \usepackage{graphicx} % To include pdf files! \usepackage{amsmath} \usepackage{amsbsy} \usepackage{amsfonts} \usepackage{comment} \usepackage[colorlinks=true, pdfstartview=FitV, linkcolor=blue, citecolor=blue, urlcolor=blue]{hyperref} % For links \usepackage{fullpage} %\pagestyle{empty} % No page numbers \begin{document} %\enlargethispage*{1000 pt} \begin{center} {\Large \textbf{STA 431s23 Assignment Seven}}\footnote{This assignment was prepared by \href{http://www.utstat.toronto.edu/~brunner}{Jerry Brunner}, Department of Statistical Sciences, University of Toronto. It is licensed under a \href{http://creativecommons.org/licenses/by-sa/3.0/deed.en_US} {Creative Commons Attribution - ShareAlike 3.0 Unported License}. Use any part of it as you like and share the result freely. The \LaTeX~source code is available from the course website: \href{http://www.utstat.toronto.edu/brunner/oldclass/431s23} {\small\texttt{http://www.utstat.toronto.edu/brunner/oldclass/431s23}}} \vspace{1 mm} \end{center} \noindent \emph{For the Quiz on Friday March 17th, please bring a printout of your full R input and output for Question~\ref{Rpigs}. The other problems are not to be handed in. They are practice for the Quiz.} \vspace{2mm} \hrule \begin{enumerate} \item \label{onestage} Here is a one-stage formulation of the double measurement regression model. % See the text for some discussion. Independently for $i=1, \ldots, n$, let \begin{eqnarray*} \mathbf{w}_{i,1} & = & \mathbf{x}_i + \mathbf{e}_{i,1} \\ \mathbf{v}_{i,1} & = & \mathbf{y}_i + \mathbf{e}_{i,2} \nonumber \\ \mathbf{w}_{i,2} & = & \mathbf{x}_i + \mathbf{e}_{i,3}, \nonumber \\ \mathbf{v}_{i,2} & = & \mathbf{y}_i + \mathbf{e}_{i,4}, \nonumber \\ \mathbf{y}_i & = & \boldsymbol{\beta} \mathbf{x}_i + \boldsymbol{\epsilon}_i \nonumber \end{eqnarray*} where \begin{itemize} \item[] $\mathbf{y}_i$ is a $q \times 1$ random vector of latent response variables. Because $q$ can be greater than one, the regression is multivariate. \item[] $\boldsymbol{\beta}$ is a $q \times p$ matrix of unknown constants. These are the regression coefficients, with one row for each response variable and one column for each explanatory variable. \item[] $\mathbf{x}_i$ is a $p \times 1$ random vector of latent explanatory variables, with variance-covariance matrix $\boldsymbol{\Phi}_x$. \item[] $\boldsymbol{\epsilon}_i$ is the error term of the latent regression. It is a $q \times 1$ random vector with variance-covariance matrix $\boldsymbol{\Psi}$. \item[] $\mathbf{w}_{i,1}$ and $\mathbf{w}_{i,2}$ are $p \times 1$ observable random vectors, each representing $\mathbf{x}_i$ plus random error. \item[] $\mathbf{v}_{i,1}$ and $\mathbf{v}_{i,2}$ are $q \times 1$ observable random vectors, each representing $\mathbf{y}_i$ plus random error. \item[] $\mathbf{e}_{i,1}, \ldots, \mathbf{e}_{i,4}$ are the measurement errors in $\mathbf{w}_{i,1}, \mathbf{v}_{i,1}, \mathbf{w}_{i,2}$ and $\mathbf{v}_{i,2}$ respectively. Joining the vectors of measurement errors into a single long vector $\mathbf{e}_i$, its covariance matrix may be written as a partitioned matrix \begin{equation*} cov(\mathbf{e}_i) = cov\left(\begin{array}{c} \mathbf{e}_{i,1} \\ \mathbf{e}_{i,2} \\ \mathbf{e}_{i,3} \\ \mathbf{e}_{i,4} \end{array}\right) = \left( \begin{array}{c|c|c|c} \boldsymbol{\Omega}_{11} & \boldsymbol{\Omega}_{12} & \mathbf{0} & \mathbf{0} \\ \hline \boldsymbol{\Omega}_{12}^\top & \boldsymbol{\Omega}_{22} & \mathbf{0} & \mathbf{0} \\ \hline \mathbf{0} & \mathbf{0} & \boldsymbol{\Omega}_{33} & \boldsymbol{\Omega}_{34} \\ \hline \mathbf{0} & \mathbf{0} & \boldsymbol{\Omega}_{34}^\top & \boldsymbol{\Omega}_{44} \end{array} \right) = \boldsymbol{\Omega}. \end{equation*} \item[] In addition, the matrices of covariances between $\mathbf{x}_i, \boldsymbol{\epsilon}_i$ and $\mathbf{e}_i$ are all zero. \end{itemize} \pagebreak %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Collecting $\mathbf{w}_{i,1}$, $\mathbf{v}_{i,1}$, $\mathbf{w}_{i,2}$ and $\mathbf{v}_{i,2}$ into a single long data vector $\mathbf{d}_i$, we write its variance-covariance matrix as a partitioned matrix: \begin{displaymath} \boldsymbol{\Sigma} = \left( \begin{array}{c|c|c|c} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} & \boldsymbol{\Sigma}_{13} & \boldsymbol{\Sigma}_{14 } \\ \hline & \boldsymbol{\Sigma}_{22} & \boldsymbol{\Sigma}_{23} & \boldsymbol{\Sigma}_{24} \\ \hline & & \boldsymbol{\Sigma}_{33} & \boldsymbol{\Sigma}_{34} \\ \hline & & & \boldsymbol{\Sigma}_{44} \end{array} \right), \end{displaymath} where the covariance matrix of $\mathbf{w}_{i,1}$ is $\boldsymbol{\Sigma}_{11}$, the covariance matrix of $\mathbf{v}_{i,1}$ is $\boldsymbol{\Sigma}_{22}$, the matrix of covariances between $\mathbf{w}_{i,1}$ and $\mathbf{v}_{i,1}$ is $\boldsymbol{\Sigma}_{12}$, and so on. \begin{enumerate} \item Write the elements of the partitioned matrix $\boldsymbol{\Sigma}$ in terms of the parameter matrices of the model. Be able to show your work for each one. \item Prove that all the model parameters are identifiable by solving the covariance structure equations. Once you have solved for a parameter matrix, ou may use it in later solutions. \item Give a Method of Moments estimator of $\boldsymbol{\Phi}_x$. There is more than one reasonable answer. Remember, your estimator cannot be a function of any unknown parameters, or you get a zero. For a particular sample, will your estimate be in the parameter space? Mine is. \item \label{mombetahat} Give a Method of Moments estimator for $\boldsymbol{\beta}$. Remember, your estimator cannot be a function of any unknown parameters, or you get a zero. How do you know your estimator is consistent? You may use $\widehat{\boldsymbol{\Sigma}} \stackrel{p}{\rightarrow} \boldsymbol{\Sigma}$ without proof. \end{enumerate} % that is \emph{not} the MLE added after the question was assigned in 2013. But in 2015 I specified MOM instead. \item \label{nconstr} For the double measurement regression model of Question \ref{onestage}, \begin{enumerate} \item How many unknown parameters appear in the covariance matrix of the observable variables? \item How many unique variances and covariances are there in the covariance matrix of the observable variables? This is also the number of covariance structure equations. \item How many equality constraints does the model impose on the covariance matrix of the observable variables? What are they? \item Does the number of covariance structure equations minus the number of parameters equal the number of constraints? \end{enumerate} \item \label{Rpigs} As part of a much larger study, farmers filled out questionnaires about various aspects of their farms. Some questions were asked twice, on two different questionnaires several months apart. Buried in all the questions were \begin{itemize} \item Number of breeding sows (female pigs) at the farm on June 1st \item Number of sows giving birth later that summer. \end{itemize} There are two readings of these variables, one from each questionnaire. We will assume (maybe incorrectly) that because the questions were buried in a lot of other material and were asked months apart, that errors of measurement are independent between the two questionnaires. However, errors of measurement might be correlated within a questionnaire. The Pig Birth Data are given in the file \href{http://www.utstat.toronto.edu/brunner/openSEM/data/openpigs2.data.txt} {\texttt{openpigs2.data.txt}}. \begin{enumerate} \item Start by reading the data. There are $n=114$ farms; please verify that you are reading the correct number of cases. \item Use the \texttt{var} function to produce a sample covariance matrix of all the observable variables. Don't worry about $n$ versus $n-1$. \item Make a path diagram of the double measurement model for these data. \item Give the details of your model in centered form, supplying any necessary notation. \item Use \texttt{lavaan} to fit your model. Look at \texttt{summary}. If you experience numerical problems you are doing something differently from the way I did it. When I fit a good model everything was fine. When I fit a poor model there was trouble. Just to verify that we are fitting the same model, my estimate of the variance of the latent exogenous variable is 357.145. \item Does your model fit the data adequately? Answer Yes or No and give three numbers: a chi-squared statistic, the degrees of freedom, and a $p$-value. % G^2 = 0.087, df = 1, p = 0.768 Do the degrees of freedom agree with your answer to Question~\ref{nconstr}? \item \label{betahat} If the number of breeding sows present in September increases by one, what happens to the estimated number giving birth that summer? You answer is based on a single number from the output of \texttt{summary}. It is not an integer. % betahat = 0.757 \item Using your answer to Question~\ref{mombetahat} and the output of \texttt{var}, give a method of moments estimate of $\beta$. How does it compare to the MLE? % 0.5*(272.67101+260.02857)/348.52989 = 0.7642093 v.s. MLE of 0.7567. Pretty good! \item %Since maximum likelihood estimates are asymptotically normal, % (approximately normal for large samples), a large-sample confidence interval is $\widehat{\theta} \pm 1.96 se$, where $se$ is the standard error (estimated standard deviation) of $\widehat{\theta}$. Give a large-sample confidence interval for your answer to \ref{betahat}. I used \texttt{parameterEstimates} to do it the easy way. \item \label{rely} Recall that reliability of a measurement is the proportion of its variance that does \emph{not} come from measurement error. What is the estimated reliability of number of breeding sows? There are two numbers, one for questionnaire one and another for questionnaire two. You could do this with a calculator and the output of \texttt{summary}, but I did it with \texttt{:=} in the model string. \item It would be inconvenient at best to get confidence intervals for reliability with a calculator. Obtain confidence intervals for the reliabilities in Question~\ref{rely}. Check \texttt{parameterEstimates}. For the record, the standard errors are calculated using the delta method, described in Appendix~A. \item Is there evidence of correlated measurement error within questionnaires? Answer Yes or No and give some numbers from the output of \texttt{summary} to support your conclusion. \item In the double measurement design, two measurements of a latent variable are equally precise if their error variances are the same. We want to know whether the two measurements of number of breeding sows are equally precise, and also whether the two measurements of the number giving birth are equally precise. \begin{enumerate} \item Give the null hypothesis for testing both comparisons at the same time. \item Carry out the Wald test. This is a question about variances, so you can't trust the normal theory $\widehat{\mathbf{V}}_n$. Bootstrap it. % Display \texttt{summary} just for comparison. \item If the overall test was statistically significant, follow it up with separate tests of the two comparisons. Be able to report the test statistic, degrees of freedom, and $p$-value. Draw directional conclusions if appropriate, being guided by $\alpha=0.05$, but never mentioning it. \end{enumerate} \end{enumerate} % End of R question \item Double measurement is not the only solution to measurement error in regression. Instrumental variables can take care of measurement error and omitted variables at the same time. The following path diagram not only betrays my lack of control over my drawing program, it shows a matrix version of instrumental variables with single measurement of both $\mathbf{x}$ and $\mathbf{y}$ and also massive covariance connecting $\mathbf{x}$ with all the error terms, and all the error terms with one another. These covariances arise from omitted variables. \begin{center} \includegraphics[width=4in]{InstruVar4} \end{center} In centered form (meaning without expected values or intercepts) the model equations are (independently for $i = 1, \ldots, n$), \begin{eqnarray*} \mathbf{y}_i & = & \boldsymbol{\beta} \mathbf{x}_i + \boldsymbol{\epsilon}_i \\ \mathbf{w}_i & = & \mathbf{x}_i + \mathbf{e}_{i,1} \\ \mathbf{v}_i & = & \mathbf{y}_i + \mathbf{e}_{i,2}. \end{eqnarray*} As usual, $\mathbf{x}_i$ is $p \times 1$ and $\mathbf{y}_i$ is $q \times 1$. The $p \times 1$ vector of instrumental variables $\mathbf{z}_i$ has covariance matrix $\boldsymbol{\Phi}_z$. The covariances among the exogenous variables and error terms are given in the partitioned symmetric matrix \begin{displaymath} \mathbf{C} = cov \left( \begin{array}{c} \mathbf{x}_i \\ \hline \boldsymbol{\epsilon}_i \\ \hline \mathbf{e}_{i,1} \\ \hline \mathbf{e}_{i,2} \end{array} \right) = \left( \begin{array}{c|c|c|c} \boldsymbol{\Phi}_x & \mathbf{C}_{12} & \mathbf{C}_{13} & \mathbf{C}_{14} \\ \hline & \boldsymbol{\Psi} & \mathbf{C}_{23} & \mathbf{C}_{24} \\ \hline & & \boldsymbol{\Omega}_1 & \mathbf{C}_{34} \\ \hline & & & \boldsymbol{\Omega}_2 \end{array} \right). \end{displaymath} Finally, the $p \times p$ matrix $\mathbf{K} = cov(\mathbf{z}_i,\mathbf{x}_i)$ has an inverse. \newpage %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{enumerate} \item What are the dimensions of $\mathbf{C}_{23}$ (number of rows and columns)? \item Now we will count unknown parameters. It will help to recall that the number of unique elements in a $k \times k$ symmetric matrix is $k(k+1)/2$. \begin{enumerate} \item How many parameters are in the matrix $\mathbf{C}$? % (2p+2q)(2p+2q+1)/2 = (p+q)(2p+2q+1) \item How many parameters are in the matrix $\boldsymbol{\Phi}_z$? % p(p+1)/2 \item How many parameters are in the matrix $\mathbf{K}$? % p^2 \end{enumerate} \item Letting \begin{displaymath} \boldsymbol{\Sigma} = cov \left( \begin{array}{c} \mathbf{z}_i \\ \hline \mathbf{w}_i \\ \hline \mathbf{v}_i \end{array} \right) = \left( \begin{array}{c|c|c} \boldsymbol{\Sigma}_{11} & \boldsymbol{\Sigma}_{12} & \boldsymbol{\Sigma}_{13} \\ \hline & \boldsymbol{\Sigma}_{22} & \boldsymbol{\Sigma}_{23} \\ \hline & & \boldsymbol{\Sigma}_{33} \end{array} \right), \end{displaymath} how many covariance structure equations are there? % (2p+q)(2p+p+1)/2 \item Is there any way this model passes the test of the parameter count rule? \item The naive model is $\mathbf{v}_i = \boldsymbol{\beta} \mathbf{w}_i + \boldsymbol{\epsilon}_i$, with zero covariance between $\mathbf{w}_i$ and $\boldsymbol{\epsilon}_i$. Ignoring $\mathbf{z}_i$, which is what people would do in practice, give a method of moments estimator of $\boldsymbol{\beta}$ (which is also MLE and least squares), in terms of $\widehat{\boldsymbol{\Sigma}}_{ij}$ matrices. Call it $\widehat{\boldsymbol{\beta}}_{bad}$. \item To what target does $\widehat{\boldsymbol{\beta}}_{bad}$ converge in probability under the true model? My answer is \begin{equation*} \widehat{\boldsymbol{\beta}}_{bad} \stackrel{p}{\rightarrow} \left( \boldsymbol{\beta\Phi}_x + \boldsymbol{\beta}\mathbf{C}_{13} + \mathbf{C}_{21} + \mathbf{C}_{23} + \mathbf{C}_{41} + \mathbf{C}_{43} \right) \left( \boldsymbol{\Phi}_x + \mathbf{C}_{13} + \mathbf{C}_{31} + \boldsymbol{\Omega}_1 \right)^{-1}. \end{equation*} \item We can do better than this. Propose another estimator of $\boldsymbol{\beta}$, and show that it converges in probability to $\boldsymbol{\beta}$ in most of the parameter space. Where does it fail? \item Why have you also just shown that $\boldsymbol{\beta}$ is identifiable except for a set of volume zero in the parameter space? \end{enumerate} \item \label{1extra} When instrumental variables are not available, sometimes identifiability can be obtained by adding more response variables to the model. Independently for $i=1, \ldots, n$, let \begin{eqnarray*} W_{i\mbox{~}} & = & X_i + e_i \\ Y_{i,1} & = & \beta_1 X_i + \epsilon_{i,1} \\ Y_{i,2} & = & \beta_2 X_i + \epsilon_{i,2} \end{eqnarray*} where $X_i$, $e_i$, $\epsilon_{i,1}$ and $\epsilon_{i,2}$ are all independent, $Var(X_i)=\phi>0$, $Var(e_i)=\omega>0$, $Var(\epsilon_{i,1})=\psi_1>0$, $Var(\epsilon_{i,2})=\psi_2>0$, and all the expected values are zero. The explanatory variable $X_i$ is latent, while $W_i$, $Y_{i,1}$ and $Y_{i,2}$ are observable. \begin{enumerate} \item Make a path diagram for this model \item What are the unknown parameters in this model? \item Let $\boldsymbol{\theta}$ denote the vector of Greek-letter unknowns that apear in the covariance matrix of the observable data. What is $\boldsymbol{\theta}$? \item Does this model pass the test of the Parameter Count Rule? Answer Yes or No and give the numbers. \item Calculate the variance-covariance matrix of the observable variables. Show your work. \item The parameter of primary interest is $\beta_1$. Is $\beta_1$ identifiable at points in the parameter space where $\beta_1=0$? Why or why not? \item Is $\omega$ identifiable where $\beta_1=0$? \item Give a simple numerical example to show that $\beta_1$ is not identifiable at points in the parameter space where $\beta_1 \neq 0$ and $\beta_2=0$. \item Is $\beta_1$ identifiable at points in the parameter space where $\beta_2 \neq 0$? Answer Yes or No and prove your answer. \item Show that the entire parameter vector is identifiable at points in the parameter space where $\beta_1 \neq 0$ and $\beta_2 \neq 0$. \item Propose a Method of Moments estimator of $\beta_1$. \item How do you know that your estimator cannot be consistent in a technical sense? \item For what points in the parameter space will your estimator converge in probability to $\beta_1$? \item How do you know that your Method of Moments estimator is also the Maximum Likelihood estimator (assuming normality)? \item Explain why the likelihood rato test of $H_0: \beta_1=0$ will fail. Hint: What will happen when you try to locate $\widehat{\theta}_0$? % \item Recall that an \emph{instrumental variable} is an observable variable that has non-zero covariance with the explanatory variable and zero covariance with the error term in the regression, which in this case is $\epsilon_{i,1}$. Under what condition is $Y_2$ an instrumental variable for $X$? \item Since the parameter of primary interest is $\beta_1$, it's important to be able to test $H_0: \beta_1=0$. So at points in the parameter space where $\beta_2 \neq 0$, what \emph{two} equality constraints on the elements of $\boldsymbol{\Sigma}$ are implied by $H_0: \beta_1=0$? Why is this unexpected? % See discussion in the text. It would be worth simulating a large data set to illustrate this. \item Assuming $\beta_1 \neq 0$ and $\beta_2 \neq 0$, you can use the model to deduce more than one testable \emph{inequality} constraint on the variances and covariances. Give at least one example. \end{enumerate} % The main lesson here is that extra response variables can yield identifiability, but also (a) parameters can be identifiable in some regions and not others, and (b) identifiability when H0 is true needs to be checked separately. % The following is directly an example from the text. I've re-checked it carefully and it's very good. In 2017, the students don't have it yet. In 2023 they do, and I also have a handwritten solution. In 2023 it strikes me as quite difficult, though. \item In the model of Question~\ref{1extra}, suppose that $X$ and the extra response variable $Y_2$ are influenced by common omitted variables, so that there is non-zero covariance between $X$ and $\epsilon_2$. Here is a path diagram. \begin{center} \includegraphics[width=4in]{extra1bex} % In the text \end{center} \begin{enumerate} \item How do you know that the full set of parameters (that is, the ones that appear in $\boldsymbol{\Sigma}$) cannot possibly be identifiable in most of the parameter space? \item Calculate the variance-covariance matrix of the observable variables. How does your answer compare to the one in Question~\ref{1extra}? \item Primary interest is still in $\beta_1$. Propose a Method of Moments estimator of $\beta_1$. Is it the same as the one in Question~\ref{1extra}, or different? \item For what set of points in the current parameter space is $\beta_1$ identifiable? % beta2 and kappa cannot both be zero. \item \label{checkid} If you had data in hand, what null hypothesis could you test about the $\sigma_{ij}$ quantities to verify the identifiability of $\beta_1$? My null hypothesis has two equal signs. % sigma13=sigma23=0 \item Suppose you want to test $H_0:\beta_1=0$, which is likely the main objective. \begin{enumerate} \item If you rejected the null hypothesis in Question~\ref{checkid}, what null hypothesis would you test about the $\sigma_{ij}$ quantities to test $H_0:\beta_1=0$? My null hypothesis has two equal signs. % H0: sigma23=sigma12=0 \item If you failed to reject the null hypothesis in Question~\ref{checkid}, could you still test $H_0:\beta_1=0$? What is the test on $\sigma_{ij}$ quantities in this case? \item If you rejected $H_0:\beta_1=0$, naturally you would want to state whether $\beta_1$ is positive or negative. Is this possible? % Yes, look at the sign of sigma12 and/or sigma23/sigma13. \end{enumerate} \end{enumerate} % But if Cov(epsilon1,epsilon2) ne 0 we are just dead. \end{enumerate} % End of all the questions \vspace{120mm} % \pagebreak \vspace{3mm} \noindent \textbf{Please bring a printout of your full R input and output for Question \ref{Rpigs} to the quiz.} \end{document} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{comment} \begin{enumerate} \item \item \end{enumerate} \end{comment} % Call this random explanatory -- or maybe wait until identifiability. \item Independently for $i=1, \ldots, n$, let $\mathbf{y}_i = \boldsymbol{\beta}_0 + \boldsymbol{\beta}_1 \mathbf{x}_i + \boldsymbol{\epsilon}_i$, where \begin{itemize} \item $\mathbf{y}_i$ is an $q \times 1$ random vector of observable response variables; there are $q$ response variables. \item $\mathbf{x}_i$ is a $p \times 1$ observable random vector; there are $p$ explanatory variables. $E(\mathbf{x}_i) = \boldsymbol{\mu}_x$ and $cov(\mathbf{x}_i) = \boldsymbol{\Phi}_{p \times p}$. The positive definite matrix $\boldsymbol{\Phi}$ is unknown. \item $\boldsymbol{\beta}_0$ is a $q \times 1$ matrix of unknown constants. \item $\boldsymbol{\beta}_1$ is a $q \times p$ matrix of unknown constants. \item $\boldsymbol{\epsilon}_i$ is a $q \times 1$ random vector with expected value zero and unknown positive definite variance-covariance matrix $cov(\boldsymbol{\epsilon}_i) = \boldsymbol{\Psi}_{q \times q}$. \item $\boldsymbol{\epsilon}_i$ is independent of $\mathbf{x}_i$. \end{itemize} Letting $\mathbf{d}_i = \left(\begin{array}{c} \mathbf{x}_i \\ \hline \mathbf{y}_i \end{array} \right)$, we have $cov(\mathbf{d}_i) = \boldsymbol{\Sigma} = \left( \begin{array}{c|c} \boldsymbol{\Sigma}_x & \boldsymbol{\Sigma}_{xy} \\ \hline \boldsymbol{\Sigma}_{yx} & \boldsymbol{\Sigma}_y \end{array} \right)$, and $\widehat{\boldsymbol{\Sigma}} = \left( \begin{array}{c|c} \widehat{\boldsymbol{\Sigma}}_x & \widehat{\boldsymbol{\Sigma}}_{xy} \\ \hline \widehat{\boldsymbol{\Sigma}}_{yx} & \widehat{\boldsymbol{\Sigma}}_y \end{array} \right)$. \begin{enumerate} \item Give the dimensions (number of rows and columns) of the following matrices: \\ $\mathbf{d}_i$, $\boldsymbol{\Sigma}$, $\boldsymbol{\Sigma}_{x}$, $\boldsymbol{\Sigma}_{y}$, $\boldsymbol{\Sigma}_{xy}$, $\boldsymbol{\Sigma}_{yx}$. \item Write the parts of $\boldsymbol{\Sigma}$ in terms of the unknown parameter matrices. \item Give a Method of Moments Estimator for $\boldsymbol{\Phi}$. Just write it down. \item Obtain formulas for the Method of Moments Estimators of $\boldsymbol{\beta}_1$, $\boldsymbol{\beta}_0$ and $\boldsymbol{\Psi}$. Show your work. You may give $\widehat{\boldsymbol{\beta}}_0$ in terms of $\widehat{\boldsymbol{\beta}}_1$, but simplify $\widehat{\boldsymbol{\Psi}}$. \item If the distributions of $\mathbf{x}_i$ and $\boldsymbol{\epsilon}_i$ are multivariate normal, how do you know that your Method of Moments estimates are also the MLEs? \end{enumerate} % End of multivariate regression question