---
title: " HW Question Week 7 "
author: "STA2101F 2021"
date: ''
output:
  pdf_document: default
  html_document: default
fontsize: 12pt
---


```{r setup, include=FALSE}
# knitr::opts_chunk$set(echo = TRUE)
knitr::opts_chunk$set(
  echo = FALSE,
  message = FALSE,
  warning = FALSE,
  error = FALSE, 
  out.width = "100%",
  fig.width = 10,
  fig.height = 6.7, 
  fig.retina = 3,
  cache = FALSE)
```
```{r library}
library(faraway)
data(prostate)
```

## Due November 4 2021 11.59 pm

**Homework to be submitted through Quercus**

A parametric model is a member of the *exponential family* of distributions if its density takes the form
  \begin{equation}
  f(y;\theta) = \exp\{s(y)^T\theta - c(\theta) - h(y)\},
  \end{equation}
  where $\theta = (\theta_1, \dots, \theta_p)$, and $s(y)$ is a $p$-dimensional sufficient statistic. For example, the normal distribution has density
  $$ f(y;\mu, \sigma^2) = \frac{1}{\sqrt{2\pi}\sigma}\exp\{-\frac{1}{2\sigma^2}(y-\mu)^2\} = \exp\{y\frac{\mu}{\sigma^2} +y^2 \frac{-1}{2\sigma^2} -\{\frac{\mu^2}{2\sigma^2} +\frac{1}{2}\log(\sigma^2)\} - \log(\sqrt{2\pi})\},
  $$
  so $s(y) = (y, y^2)$, $\theta = \{\mu/\sigma^2, -1/(2\sigma^2)\}$. The parameter $\theta$ is called the canonical parameter of the exponential family. 
     
  (a) Show that if $y$ is distributed as $Binom(n, p)$ that this is a member of the exponential family, and give expressions for $\theta$ (in terms of $p$) and $c(\theta)$. Show that if $y$ is distributed as $Poisson(\lambda)$ that this is a member of the exponential family and give expressions for $\theta$ (in terms of $\lambda$) and $c(\theta)$. 
  (b) Show that if $y=(y_1, \dots, y_n)$ are independent, and identically distributed, according to (1), that the joint distribution of $y$ is of the same form, with $s(y) = \sum_{i=1}^ns(y_i)$, $c(\theta)$ replaced by $nc(\theta)$. 
  (c) Use this result to argue that the marginal density of $s = (s_1, \dots, s_p)$ has again the same form, i.e. $$f(s;\theta) = \exp\{s^T\theta -nc(\theta)-k(s)\},$$ where $k(s)$ is defined from an appropriate integral. 
  (d) Suppose $y_1, \dots, y_n$ are independent, with $y_i \sim Binom(n_i, p_i)$, where $p_i = \exp(\theta_0 + \theta_1x_{1i} + \theta_2x_{2i})/\{1+\exp(\theta_0 + \theta_1x_{1i} + \theta_2x_{2i})\}$. A slight extension of the argument in (b) shows that the joint density of $y = (y_1, \dots, y_n)$ is a member of the exponential family (you do not need to derive this). What are the sufficient statistics $s$. 
  (e) Show that the maximum likelihood estimate of $\theta$ in (d) is defined by \begin{eqnarray*} \sum_{i=1}^n y_i &=& \sum_{i=1}^n p_i(\hat\theta), \\ \sum_{i=1}^n x_{1i}y_i &=& \sum_{i=1}^n x_{1i}p_i(\hat\theta), \\ 
  \sum_{i=1}^n x_{2i}y_i &=& \sum_{i=1}^n x_{2i}p_i(\hat\theta). 
  \end{eqnarray*}
     
  (f) *Bonus/PhD SM, Problem 10.9*: At each of the doses $x_1 < x_2 < \dots < x_n$ of a drug, $m$ animals are treated. At dose $x_i$, $r_i$ animals have response $1$ and the remaining $m-r_i$ have response $0$. Derive the maximum likelihood equation when  and a *probit* link function is used, and the linear predictor takes the form $\eta = \beta x$; i.e. $r_i \sim Binom(p_i)$ and $p_i = \Phi(x_i\beta)$. Show that if $x_i \equiv x_0$, i.e. only one dose is used, that $$
  \hat\beta = \frac{1}{x_0}\Phi^{-1}(r/m), \quad \text{var} \doteq \frac{\Phi(\beta x_0)\{1-\Phi(\beta x_0)\}}{mx_0^2\{\phi(\beta x_0)\}^2},
  $$
  where $\phi$ and $\Phi$ are the standard normal density and distribution functions. Plot the function $\Phi(\eta)\{1-\Phi(\eta)\}/\phi(\eta^2)$ for $\eta$ in the range $(-3, 3)$ and comment on the implication sfor the choice of $x_0$ if there is some prior knowledge of the likely value of $\beta$. 
     
