Type 1 and Type 2 Errors Video Lecture | What is Data Analytics? - Data & Analytics

hello and welcome to lecture on type 1
and type 2 errors this is something that
you might have heard and it's a fairly
central and important concept with
respect to hypothesis tests so let's get
into what the core concept of type 1 and
type 2 errors are so the idea is the
following right take any of the
hypothesis tests that you've learnt so
far in this course we've provided you
with a template for how the hypothesis
tests are conducted and in this template
you wouldn't notice the first step mr.
former null and an alternate hypothesis
and your last step is that you
essentially get a p-value associated
with this particular test and based off
of the p-value you either reject the
null hypothesis or you fail to reject
the null hypothesis so this is what we
have characterized in this table okay so
you actually make a null and an
alternate hypothesis and you don't know
which is true right that's why you're
doing this test if you knew so your null
hypothesis was that mu1 equals mu2 so if
you knew if that was true you would not
be doing this test so you don't know if
that's true it that's why you're doing
this test so you might have a null
hypothesis that says mu1 equals four
point eight which is a single sample
case or you might have a null hypothesis
which says that mu1 equals mu2 in either
case you don't know which is true but
let's say there was this all-knowing
world where you knew which was true so
that's what's marked out here in actual
you say actually what is true so here
I'm saying the null hypothesis is true
and in this part I'm saying the
alternate hypothesis is true so here I'm
saying mu1 is equal to let's say mu2 and
here the alternate hypothesis is true mu
1 is not equal to mu2 ok now this is the
truth okay but based off of some data
right you did a test so you might have
done a two-sample z-test or you
might have done a two-sample t-tests and
let's say that you did the test
you actually can compute it the Z or the
T statistic based off of the Z or the T
statistic you calculated a probability
or a p-value and based off of p-value
you took a decision and the core idea
has always been that if the p-value is
too small then you reject the null
hypothesis and the term too small is is
a subjective term and typically people
use some line in the sand some threshold
and that threshold is called alpha so if
the p-value is less than alpha we reject
the null hypothesis typically the value
of alpha tends to be something like 0.05
or 5% okay so based off of this let's
say you've rejected the null hypothesis
that's one decision and here you fail to
reject the null hypothesis for whatever
reason so the the idea is that out here
you reject you would have rejected the
null hypothesis because your p-value
would have been less than your alpha so
p-value was less than alpha so you
rejected the null hypothesis here the
opposite was true P was greater than or
equal to alpha let's say now you don't
have to have the equal to sign also you
can just say greater than and out here
you might want to say less than or equal
to okay so this is what you did now here
is the problem right if the truth was
that the null hypothesis was true okay
the null hypothesis was true which means
that mu 1 was equal to mu 2 right but
you went ahead out here in this quadrant
you went ahead and you rejected the null
hypothesis you said mu 1 is not equal to
MU 2 but in reality mu 1 was equal to MU
2 you did something wrong ok and that
error is what is called is the type 1
error it is when the null hypothesis was
true but you went ahead and you said
then I am rejecting the null hypothesis
ok and we're going to come back to Quon
find that value which is called the type
1 error it's just noteworthy that that's
also called the producer risk in
sometimes the false positive or the
alpha risk we won't go into each of
those terms for instance producer risk
is seen more from a manufacturing
context false positives are seen more
from a medical context but if you kind
of think about it it it definitely makes
sense ok now here is the case out here
where the null hypothesis was true and
you fail to reject the null hypothesis
so that's ok you're happy with that
right reality was that the null
hypothesis was true and you didn't find
any evidence to reject the null
hypothesis so you did the right thing so
you're happy in this quadrant now let's
come to this quadrant out sure here the
null hypothesis in reality because this
is truth so this is truth so here mu 1
was not equal to MU 2 and you correctly
rejected the null hypothesis this is
still the null hypothesis right your
null hypothesis is this ok this mu 1 not
being equal to MU 2 is the alternate
hypothesis but it so happened that the
alternate hypothesis was the reality
that the null hypothesis was wrong and
when the null hypothesis was wrong you
correctly rejected it so you did the
right thing even here and that's a very
good thing that you did you actually
detected that new one was not equal to
mu 2 and you said no I'm rejecting the
null hypothesis so that's a great thing
now come to the final quadrant here the
truth was that mu 1 was not equal to MU
2 but you fail to reject the null
hypothesis you were not able to say I'm
rejecting the idea that mu1 is equal to
mu2 I'm rejecting the null hypothesis
you were not able to reject the null
hypothesis whereas you should have
because mu 1 is not equal to MU 2 and
that's the truth so that's also not a
great thing that you did ok and this
error is called the type 2 error and
again it's called the consumer risk in
the more manufacturing
settings it's called a false negative in
medical settings in a couple of other
settings or the beta is now in really
simple words type 1 error right is the
concept so just just to be really clear
we're right here so the idea is that
you're type 1 error even before you see
any data right even before you do
anything could be as high as you're
alpha and I say after analysis it's
exactly equal to your p-value so let me
explain that a little bit so let's say
you've not collected any data we just
discussed what alpha is alpha is this
idea that you're going to do this
hypothesis test and you're going to get
some p-value and yes you've already said
right if the p-value is too low I'm
going to reject the null hypothesis how
low is too low that's the line in the
sand that you draw and that value is
called alpha so if your p-value is less
than alpha you're going to reject the
null hypothesis what does it mean when
your p-value is less than alpha okay
and first of all you are going to get a
very low p-value let's say you're going
to get a low p-value and this p-value is
less than alpha because you're type 1
error first of all comes about only when
your null hypothesis is true and you go
ahead and reject the null hypothesis all
right so my definition that means your
p-value must have been less than alpha
only then you would have rejected your
null hypothesis now only when you reject
your null hypothesis are you even
putting yourself up for the possibility
of a type 1 error ok so the idea is that
I'm going to tell you that up front your
type 1 error is associated with this
idea that you're rejecting the null
hypothesis and you'll reject the null
hypothesis when your p-value is less
than alpha which means your p-value is
really low now let's take a step back
and think about what the p-value is the
p-value is this idea that if the null
hypothesis is true this is a definition
that we looked at it much earlier even
before we discuss type 1 and type 2
errors p-values by definition express
the idea that if you're not hypothesis
is true this is the probability of
seeing a test statistic as X
dream as this right if your null
hypothesis is actually true so this
probability is really low then
essentially what you're saying is that
you're willing to take the risk and say
hmm the probability of seeing this
statistic is just 1% if the null
hypothesis is true if the null
hypothesis is true the probability of
seeing something so extreme is just so
small it's just 1% then I'm willing to
take the risk and reject the null
hypothesis right so right there by
definition the risk that you took was
that 1% risk it was that p-value right
so whatever your p-value is essentially
you'd if you do the entire calculation
and you calculate the p-value then your
p-value is essentially your type 1 error
that's the risk you're taking that
because that is you've actually
calculated the probability that this
data could actually occur with the null
hypothesis being true so if you're still
going ahead and rejecting the null
hypothesis fully knowing this
probability then that's the risk you're
taking of making a wrong decision now of
course before you even see any data and
even before you calculate the p-value
given that you set yourself up with an
upper threshold of alpha means that you
could get a type 1 error as high as
alpha okay so I hope that makes type 1
errors fairly clear now type 2 errors
are more complicated now and there's a
reason for it it's a function of a
variety of parameters it's a function of
something called Delta which is not the
Delta that we've discussed so far and
the core concept behind Delta is the
following all right type 2 errors when
do they occur they occur in the
situation where your alternate
hypothesis is actually true so mu 1 is
not equal to mu 2 but you failed to
reject the null hypothesis but in order
for me to quantify how likely that is
you need to tell me how much is mu or
not equal to mu 2 right so if I said mu
1 is not equal to MU 2 okay because
you one is equal to mu2 plus a very very
very very very small number so let's say
1 micrometer all right or whatever it is
whatever the metric of MU 1 nu 2 is but
it's a very small number so mu 1 let's
say is 4 but mute to write is 4 point 0
0 0 0 0 1 right then it is true mu 1 is
not equal to MU 2 the alternate
hypothesis is true but the probability
that I am going to reject the null
hypothesis just became very low my
ability to discern between a difference
this difference between 4 & 4 point 0 0
0 0 0 1 is much lower than my ability to
discern between the difference of 4 & 5
all other things being equal so it's
really a function of how different are
they and that's what gets captured in
Delta it's also a function of the sample
size we take which is as the sample size
becomes infinite the uncertainty around
mu 1 and mu 2 becomes smaller and
smaller and theoretically even a small
difference between 4 & 4 point 0 0 0 0 0
1 could potentially be found as long as
the sample size is large enough okay
it's a function of type 1 error because
you're type 1 error tells you how
conservative-liberal you're being in
rejecting the null hypothesis so the at
the more conservative your you are and
type 1 error meaning that you don't want
to make the type 1 error the more likely
you are to make a type 2 error and
similarly the more liberal you are with
the type 1 error meaning you're okay
with some amount of error the better
you're going to be with type 2 error so
it's a little bit of give and take in
terms of the type 1 and type 2 error and
essentially a lot of people are very
interested in understanding the
relationship of beta which is this type
2
okay so it's a bad thing versus Delta
which is something we've discussed here
and for a given sample size and they'll
show it from many different sample sizes
and that's known as an OC curve or an
operational characteristic curve another
word that you might come across this is
known as the power of the test and
that's equal to one minus beta and it's
essentially a very positive thing right
it is one minus a bad thing so this the
higher the power of the test that means
the more strength you have in being able
to detect a difference between mu1 and
mu2 and i've used the concept of nu 1
and mu 2 coming from the two sample
scenarios but all of these core concepts
also apply to single sample tests okay I
hope that clarified and that gave you an
idea of type 1 and type 2 errors thank
you