Title¶

Exercise: 1 - Normal Distributions and Likelihood

Description¶

The goal of this exercise is to become comfortable with the normal distribution and the idea of the likelihood function. This magnitude of the data is small so that you can focus on the understanding of the concepts.

Instructions¶

Do a few probability and density calculations for a normal distribution
Calculate and plot the likelihood of a sample of just 3 observations.
Determine the Maximum Likelihood Estimates.

Hints:¶

scipy.stats.norm.pdf() : Evaluates the PDF of a normal distribution at a particular value of X

scipy.stats.norm.cdf() : Evaluates the CDF of a normal distribution to find: $$P\left(X\le x\right)$$

In [ ]:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

In [ ]:

from scipy.stats import norm

(a) Let $X\sim N(500,75^2)$. Determine $P(X\geq 600)$.

In [ ]:

### edTest(test_norm_prob) ###
prob = 1-norm.cdf(___,___,___)
prob

(b) Plotting the normal distribution of $X\sim N(500,75^2)$.

In [ ]:

# define parameters
mu = ___
sigma = ___

# the 'dummy' x for plotting
x = np.arange(200,800)

# calculate the normal distribution at each value of x
prob = norm.pdf(___,mu,sigma)

# plot it
plt.plot(___,___);
plt.title(r'$\mathrm{N(\mu=500, \sigma^2=75^2)}$')
plt.ylim((0,0.006))
plt.show()

Question: Does your answer to part (a) makes sense based on this curve?

In [ ]:

### edTest(test_likelihood) ###
# define the data set
x = [3,5,10]

# sigma is known to be 2, an estimate for mu
# is what we need to determine. Consider 
#the values (4, 4.01, 4.02, ..., 7.99).
sigma = 2
mu = np.arange(___,___,0.01)

# calculate the likelihood
like = norm.pdf(x[0],mu,sigma)*___*___

#plot it
plt.plot(mu,like,color="darkred");
plt.title('Likelihood Function')
plt.xlabel(r'$\mu$')
plt.show()

(d) Determine the maximum likelihood estimate for $\mu$.

In [ ]:

### edTest(test_mle) ###
# determine which value of mu aligns with where 
# the maximum of the likelihood function is
mle = ___[np.argmax(__)]
mle

Question: How would you numerically maximize this function if both the mean and variance were unknown? How would you visualize the likelihood function?

In [ ]: