NPTEL MOOC IDA - Tutorial for Assignment 2 Video Lecture | What is Data Analytics? - Data & Analytics

hello and welcome to our first tutorial
that is intended to help you with the
assignments so this we you the
assignment one the deadline is already
complete and many of you have
successfully submitted the assignment
this tutorial is for assignment two and
again for if if you already worked on
assignment two and you're fairly
confident you don't need this tutorial
it's really meant for those who are
finding the assignments a little
difficult and so it's we're not going to
be giving the answers during this during
this assignment but we are going to help
you with each question to the extent
possible to kind of have you start
thinking in the correct direction okay
so what I have in front of me is a
screen with with you know the it's a
printout essentially of the assignment
paper and I'm going to be going
step-by-step in talking through the
questions so let's let's jump right in
so the first question questions one
through four all pertain to the same
description okay and I'll highlight it
here's your questions that's question
one through three and then question four
okay so questions one through four
that's what we're going to talk about
during this during the first this thing
and they're all putting into a single
graph so this one graph kind of covers
the description for all the four
questions and the important thing to
note here is that it's a continuous
random variable and it is it represents
a cumulative density function that's
very important so it's a CDF and not the
PDF of the distribution okay but even
before we jump into CDF's and pdfs the
important thing is it's a continuous
just distribution right so it's a
continuous random variable so with
continuous random variables you already
know that the probability of a
particular event is zero
and you could if you're asking questions
like the probability between that the
event that if you present a range all
right so if you present the range saying
what is the probability that a failure
will occur between one and three hours
or between two and four hours if you
give it a range then with the PDF you're
really interested in the area under the
curve defined by between the one and
three hours and so you can rephrase that
question as in less than one hour in
which case you're really talking about
the lower limit in which case is zero
hours all the way up to three hours or
you can frame the question as what is
the probability that it will fail after
two or three hours in which case you
you're interested in the area of under
the curve that starts at two hours and
then goes up all the way till the upper
limit in this case it's four hours now
all of this is true for a PDF now the
core principle still holds in that the
probability of a point event is zero so
the probability of if you're not talking
about a range and a range can be defined
as a probable probability that it will
fail in less than two hours the
probability will fail in more than two
hours probability it will fail between
one and three hours for all of those you
can get an exact answer right a nonzero
answer but if you ask the question what
is the probability to fail at exactly a
particular point the answer is zero now
with the PDF you are always interested
in the area under the curve associated
with the range but by definition the CDF
is nothing but the area under the curve
to the left hand side of the PDF for a
given point so if someone is asking you
the question the probability that
something will fail before a particular
point of time in the PDF you might have
been interested in the probability from
the lower limit which could be zero
and all the way to the point under
question so if someone says what is the
probability that that conditioner will
fail before three hours or something in
the PDF you will be interested in the
area under the curve from zero all the
way to three hours but because the CDF
is by definition the area under the
curve of the PDF up to that point you
don't have to look at the area under the
curve you just have to look at that
point you just have to look at the
y-value at that point so if the question
is the probability it will fail before X
look at the value of y at X on the CDF
if the question is the probability that
it will fail after and the fail after
can be asked in many ways that it will
not fail before or in other ways then
you just care about one minus the value
of the CDF at that particular point so
that should in a sense at least help you
answer questions one two three and four
okay so simple rule if given PDFs you
care about area under the curve to
answer probability questions associated
with ranges if given CD f's you just
care about the Y value at a given point
if the question pertains to the
probability from probability of an event
occurring before if it's a probability
of the event occurring after you care
about one minus the probability of what
you will see on the y axis at a given
point y because by definition a CDF is
the area under the curve the PDF from
the starting point which could be which
is zero in this case but it could be
minus infinity in others all the way up
to the point under question the area
under the curve on the PDF from minus
infinity or 0 all the way whatever the
lower limit is up to the point under the
question that area is what is being
plotted in the CDF so if question if the
question has to do is what is the
probability something will happen before
then just take the Y value at the point
if the problem if the question is about
what is the probability that something
will
not happened before or it is going to
happen after then you care about
essentially the area under the curve on
the right-hand side of the PDF which is
the equivalent of saying what is 1 minus
the Y value that you have on the CDF ok
and just also note that how at at the
value for your probability is a hundred
percent right which means the
probability that something will fail
before four hours is a hundred percent
so it's definitely going to have happen
ok ok so that should help you answer the
first question first for questions with
respect to the central limit theorem
it's more of a definition so we won't
really focus on that let's go to the
sixth question for the distribution n of
0 comma 5 square and we usually mention
it as something square just so that it's
very clear that 5 is the standard
deviation so 5 is the standard deviation
so obviously 5 square is the variance we
asked the question what is the
distribution of the sample mean given a
sample size of n and we've discussed
this before in that the sample mean
stays normally distributed so you know
that you also know that the distribution
of the sample mean the the the mean of
the distribution of the sample mean is
the same as the distribution of the data
itself so there's no new bias being
introduced so the mean should not change
and you also know that the standard
deviation gets scaled down by a factor
of square root of n so you could either
think of it as this new standard
deviation should be if the old standard
deviation was Phi the new standard
deviation should be 5 divided by square
root of N or you can think of it as the
the if the old variance was 25 then the
new variance would be 25 divided by n so
teethers this new standard deviation is
5 divided by square root of n or the new
variance is 25 by n ok moving on to
question 7
if the null hypothesis of a statistical
test is that mu is less than 10 then the
alternate hypothesis would be what you
really want to think of the alternate
hypothesis you want to think of the null
hypothesis plus the alternate hypothesis
covering the entire universe so if the
null hypothesis could be 10 or less then
the alternate hypothesis really would be
the rest of the possible Muse that would
not overlap with what the null
hypothesis covers but is but is it's
collectively exhaustive so all the other
possible new values are getting covered
ok and ok so moving on to the next
question question number 8 the HR
department in a company believes that
employees of a specific designation
should be compensated more or less
equally and they are of the belief that
the variance and compensation of all
associate manager should not be more
than 22,000 rupees however they do not
have the compensation information for
all associate managers in this company
so the idea behind saying that is that
if they had the entire data that this
would not be a exercise in inferential
statistics they unfortunately have to
take a sample in order to do this so
they take a sample and what they find is
so they commissioned a survey and
receive data on compensation for 16
associate managers which hypothesis test
would you use to see if the variance in
compensation is less than 22 plus 22
thousand rupees so I mean the main thing
that you need to focus on here is that
this is a test where you are your data
points are ones that tell you the
compensation of each manager but what
you are testing it against is the
variance so it's just really important
for you to
which test is a test to compare variance
and standard deviation and the other
thing is also that because you're
comparing it against a benchmark number
it should be a single sample test okay
good we move on to the next question now
question 9 through 11 pertains to the
following description a teacher
typically grades in class 30% a is 40% B
is 20% C is 6% DS and 4% EPS what is the
probability of getting a be a better for
a randomly selected student the
important thing to recognize out here is
that while you are given a breakdown of
each grade the question that is being
asked of you is only about getting B or
better right so you can essentially you
see this as B or better is you know B
and a is B or better and C D and F is
worse so you just have to break that
into two binary bins and report the
probability there with respect to
question ten in a class of ten students
what is the probability that more than
two students get grades do worse because
it's again presented as D or worse there
are only two possible outcomes either
it's do worse which means you get a D
the probability of DNF or it's the other
way round which is better than D right
so the probability of being a B or C so
there are only two classes and it
becomes kind of like a Bernoulli
distribution but now you are asking the
question is what is the probability that
two people out of ten get worse so the
way you treat this is exactly like a
binomial distribution and they important
hint out here is that you are asking a
question about the probability that two
or more students you not asked see the
it's a binomial distribution and so it's
a discrete distribution so you could get
an answer for what is the probability
that exactly two students get to exactly
two out of ten students get grades D or
worse but that's not what is asked what
is being asked is what
probably that more than two students out
of ten get get this probability and the
answer to this question would become
very hard if you use the binomial
distribution formula and calculated the
probability of two out of ten getting
grades worse than D three out of ten
getting grades worse than D four out of
ten and going all the way to ten out of
ten the easier thing to do would be to
see what is the probability that 0 out
of 10 students gets grades
worse than D one out of 10 students get
gets grades worse than D two out of two
ten students get good grades worse than
D and then add those three probabilities
up and then just do one minus that
number okay that way you are able to
answer the question of what is the
probability that more than two students
get creates D or worse you can you do
that by calculating the probability that
less than 2 or less students in grades
worse than D and doing 1 minus the
probability so that should kind of help
you answer that question the next
question Levin also pertaining to the
same thing is in a class of 50,000
students what is the probability that
more than 5050 students get grades do
worse we just use the normal
approximation to the binomial because
it's a very large number you don't want
to be calculating 50,000 see 5050 or
whatever the simple thing to do would
just be to do the normal approximation
and if you were to do the normal
approximation what would be the mean and
variance of this normal distribution
it's quite simple n which is 50 thousand
times P which is the probability that a
student would get a burst great get a
grade worse than D that is P so n times
P is the mean of the binomial
distribution so this would be a normal
distribution with that mean and
similarly you can look up the variance
of the binomial distribution which is
nothing but n times P times 1 minus P so
if you use those two formulas and then
you know now you know the nature of the
normal distribution it's so to become a
normal distribution with that mean and
that variance what is the probability of
getting a value greater than 50/50 on
that normal distribution
simplest thing to do is just find out
the mean and variance of the normal
distribution and get that p-value by
plugging in like a function that was
shown to you in Excel so you can just
use that kind of norm dist function that
we had and that was shown to you in the
class and and you can plug in those
values and that will give you the answer
to this question so use Excel or use a
text statistics textbook after you
figure out the parameters that is the
mean and variance of the normal
distribution that you have on your hands
and on that normal distribution you want
to know the probability that more than
5050 students get that value so you want
to know the probability of getting a
number greater than 5050 on a normal
distribution with a mean equal to 50,000
times the probability of getting a grade
D or worse and a variance of 50,000
times the probability of getting a D or
worse times 1 minus the probability of
getting a D or worse so that's n P times
1 minus P for the variance so the next
question which is you decide to test the
hypothesis that the average time for a
technician to complete a task is 15
minutes you don't know the population
standard deviation you've collected 14
data points in this regard what test
would you use so the important thing to
think about is you start by kind of
eliminating what are you testing you're
trying to do a test on the population
mean because you're testing hypothesis
is the average time for the technician
so this is not a test on standard
deviation of variance is a test on mean
and so out of these four tests that are
given as choices only two tests are used
to test the means population mean and
with 14 data points if you know the
population variance you use one test if
you don't know the population variance
you say other test so that should help
you eliminate between these two options
and give you one option okay so now
taking the same example from question
tells you calculate the test statistic
for the appropriate test and find the
value to be 1.3 in order to get the
p-value which one of the following would
you calculate
so here's a graph with 1.3 and the on
the question is do you want the
probability of getting a value less than
1.3 in a norm so what is this so I've
said that this is either a standard
normal or a t-distribution so we don't
because it depends on what the test
statistic is right and the important
thing to realize is that once you narrow
down on which test it is on question 12
that should logically tell you which
distribution you want to be looking at
in question 13 okay so if you choose to
do perform a Z test you would be looking
at a normal distribution if you choose a
t-test you'll be looking at a t
distribution and the the couple of other
things that you would need help with is
always remember that with a Z
distribution you don't have anything to
do with degrees of freedom with the T
distribution you have to mention the
degrees of freedom and the degrees of
freedom is always n minus 1 where n is
the number of data points okay the
second thing to realize in terms of
answering choosing between the choices
here is I mean is it a one tailed test
or two tailed test a one tailed test
typically will be framed like what is
the probability the null hypothesis
would be something where you ask the
question such as is the average time
less than or equal to some value is the
average time greater than or equal to
some value those are typically one one
tailed test if you're asking the
question is the probability less than or
equal to some value you'll like to look
at the right-hand side of the
distribution after you calculate the
test statistic for your p-value if your
null hypothesis is one of asking the
question what is the probability of that
this average time is greater than or
something greater than or equal to a
certain value then you like to look at
the left hand side
the distribution if you ask the question
is the average time equal to a certain
value then it becomes a two-tailed test
so you would calculate the test
statistic and the easiest way to make it
a two-tailed test is to multiply by two
so see what question is asked and that
should help you answer whether it's a
normal distribution or a T distribution
how many degrees of freedom are involved
and whether you like you like you like
to get the probability of a value
greater than probability value less than
or the probability of the value which
clearly you cannot exceed 1 the
probability can never be more than 1 so
if you're looking at both sides of the
tail which which which probabilities
would you be interested in ok so
hopefully this these these these this
tutorial kind of helps you answer some
of these questions and we look forward
to most of you successfully completing
assignment 2 and submitting it without
too many errors thank you