Clustering Analysis Video Lecture | What is Data Analytics? - Data & Analytics

hello and welcome to our lecture our
first lecture on clustering analysis
this is the first of two lectures that
we will have on this subject
this one is an introductory lecture it
introduces the basic concepts and
algorithms behind a clustering analysis
and in the next lecture we will actually
go into two specific algorithms and see
how those two algorithms work okay so
diving into the subject
what exactly is clustering clustering is
this idea of dividing data into groups
and we do that because sometimes there
is inherent meaning to doing such a such
an activity and and and in other cases
it it serves as a first step it's fairly
useful to do this now
you might notice that you might have
come across clustering with some other
names as well which so such a
segmentation or partitioning so that's
kind of why we have I have written out
all three for you in this first bullet
point but the core idea is the same it's
it's very interesting it also partly
what what terminology wind up using also
has to do with the context or the
background so typically something like
segmentation is used more in a marketing
sense so when you grouping people in or
customers sometimes people use the word
segmentation partitioning again
interestingly comes more from the
computer science community where you
bring graphs so your partitioning graphs
but again the core idea is that you have
some data and that you take the data and
based on certain properties of this data
you you choose to group it so you can
you can think of it as actually grouping
data objects based on various attributes
associated with this data
the idea behind such an activity is that
the data the data itself is represented
through various attributes or features
and some data points is similar to
others based on these attributes or
features right and you want to group
those that are similar and put them into
one cluster of group and and
differentiate that from other groups or
clusters which are similarly formed
based on some measure of similarity so
the the core idea is to put things that
are similar together and therefore as a
result the different groups are as
dissimilar from each other as possible
so data points within a group are
similar and data points between groups
as a result are dissimilar to each other
now a core idea that clustering often
relates to and and sometimes is often
confused with is classification right I
think an important thing to recognize
here is that clustering is primarily
seen as an unsupervised learning
technique okay but as a classification
is a supervised learning technique the
the idea is that you have some in with
classification you have some input data
and you use the historic input data and
the specific output and in the case of
classification the output actually has
classes right it's it's a categorical
variable and so you used some kind of
supervised learning technique to look at
the relationship between these input
variables and and and the task there is
to make a prediction and assign it a
class in the case of clustering you are
again dividing the input data space in
some sense into groups but the groups
are not based on labels that have been
explicitly given to you so in some sense
there is no output variable
with clustering and what you have is
only the input data space and it's not
even clear that there is another there's
a classification scheme like in the
sense of the in the classification there
was this output variable and this output
variable had one two three or more
categorical states so there was there
were these states that are labels that
was explicitly given to you and you were
now trying to create that relationship
between these labels and the input
variables in the case of clustering not
only do you not have the labels or the
output variable it there might be no
meaning to having such labels so in some
sense with clustering you are really
breaking the data input data data set
based on the inherent relationship
between data points and how similar they
are to each other you don't have an
external label that is given to you
about the data points so if two data
points are very similar across these
attributes then then you group them
together if they're not you group them
apart so in the in the end you might
create a few clusters and you can call
the cluster a cluster B cluster C but
that's not the same thing as
classification which is more of a
supervised learning process okay so why
really do this and and the best way to
see that is this broad there are two
major reasons one is clustering itself
might just be useful to understand the
universe of the data that you're dealing
with and we're going to look at some
examples in in that light the other
reason and and sometimes the clustering
is used is that it serves as a precursor
to further data analysis so it has some
utility from a machine learning sense
itself to do a clustering so now let's
look at the first case which is that in
some sense you get a better
understanding of your data when you do
clustering a fairly common example of
this is in marketing or sales where
businesses for instance collect you know
lots of information about a customer so
a the way you should be thinking about
it is
data object is essentially a customer
and the customer has for instance
various attributes they could be things
like gender age and you know it could
extend all the way to the kinds of
products that the person tends to buy
and so on and so forth
and you you have a full data set of a
lot of customers or potential customers
and you might be interested in grouping
them all right so there's no explicit
output variable but certain types of
customers are very similar to certain
other types of customers they could be
very dissimilar to a third type so
creating groups out there could could
help for instance like a market research
initiative into looking into a
particular group and coming up with
ideas of how to specifically target or
market to them another example you know
another very common example is in terms
of just communicating information right
take a simple example such as googling
or you know searching for a movie on the
internet now there are lots of things
related to a movie for instance they
could be reviews of a movie they could
be trailers and videos of a movie
they could be ratings of a movie and
they could be information on which
theaters the movies running in or where
you can purchase a movie now
a search itself a first search itself
across the web might give you a huge
large bucket of things that could belong
to any of these categories but if you
had an ability to for instance look to
see which pieces of information are
similar to each other you might be
interested in representing one of each
type right so you might have a cluster
that is created which talks about
essentially reviews of a movie and
another cluster that get off of
potential links that talk about where
this movie is playing and so in some
sense you have you could create these
clusters and present the
viewer or present the person who's
searching with a representative of each
cluster or typical value in each cluster
so that so that a person who is
searching for a further review of the
movie does not get twenty links on the
first page that give him trailer give
him or her trailers which would be quite
frustrating so in terms of information
retrieval and communication of the
information it might be fairly useful
clustering is used a lot and biology
mainly in taxonomy where if you just
said the wild universe of mammals or
insects or something like that you you
you kind of want to group animals or
mammals that are similar to each other
and give them some taxonomy that is
different from animals that are
different from each other and here for
instance each animal would be the data
object and the attributes would be you
know various animal related attributes
such as how they feed what what what
their family or genus species is and so
on and so forth it's also used in
climate many times understanding
understanding ocean temperatures ranging
from ocean temperatures to hurricanes to
various other things can be better done
you know if you if you cluster the data
on weather patterns medicine of course
again are certain types of disease
similar to other types of disease are
certain many medical medicines similar
to other medicines it can simplify the
world in some sense and help people
understand how for instance certain
medicines interact with certain disease
and so on and so forth now these give
you an understanding and these are kind
of talk about where clustering is useful
or has been used just to get a better
sense of the data but there is this
completely different and other utility
sometimes to cluster clustering and
that's mainly in the form of serving as
a precursor to further data analysis
the idea here is that clustering for
instance can be used as a great way to
summarize data especially when the
algorithm that a person needs to use
let's say a person was interested in
performing some form of regression or
another more complex supervised or
unsupervised learning task sometimes
with more and more data the task just
becomes computationally harder and
harder and it might be sometimes
beneficial to perform a clustering
analysis on the data which could be in
some cases it's computationally easier
to do the clustering and then just have
a representative from each cluster as a
data point so you could have a
representative or you could have
essentially the cluster Center the the
clusters are representative the serve is
a data point for all the data points in
that cluster and so you're now doing
that same machine learning task be it
regression or factor analysis or
whatever it is you're really doing it on
the prototypes on the representatives
rather than the full data set it's also
used I mean an extension of that is also
how it could be really useful like say
in in a nearest neighbor task right we
in an earlier in earlier lectures we've
spoken about this algorithm called KN n
right K nearest neighbors I mean when
you think about the problem that's
involved with that for any given point
if you if you need to find its nearest
neighbors you need to actually try and
look at the distance between the point
on the question and every other point in
the data set so you need to evaluate the
distance between the point you are
interested in and every other data point
and then pick the K closest neighbors
now that can become computationally very
very very hard and and you know you need
to keep either a memory or you need to
do the computation from scratch a
simpler approach could be that you just
take the cane that you you do a
clustering on the data
and you look to see which cluster Center
is closest and once you identify a
cluster Center that's closest you now
take your point under question and only
evaluate its distance to all the points
in that cluster and the assumption here
is that the points the points under that
cluster are the ones that are going to
be closest to this point because this
this cluster Center was the closest for
instance it's also used in certain other
forms of you know compression of data
specifically something called vector
quantization
which is used a lot in image or sound or
video data where you typically find that
like in in a particular image if you
break it up into pixels there is a whole
host of data points that will look very
very similar to each other
so you take let's say you take a photo
of a person right almost twenty to
thirty percent might be the background
behind the person and that might have
the same color so it could be more
efficient to just acknowledge that
there's a very good chance that the
pixel on all four sides are going to be
the same color as this pixel and so you
kind of summarize the data you reduce
the data to a fewer number of pixels and
memory values and obviously with this
there's some loss of resolution right
there's some loss of information
essentially but that might be acceptable
and the data size itself gets
substantially reduced okay so we spoke
about what clustering is and why use it
now let's just briefly talk a little bit
about the different types of clustering
the major classifications tend to be one
hierarchic hierarchical versus partition
and the the major idea but here is that
hierarchical clustering is essentially a
nested form of cluster
right so think of it this way you can
start with this one mega cluster which
is essentially a single cluster of all
the data points and that's on one end of
the scale and then you go to the other
end of the scale where you have as many
clusters as the number of data points
and each data point is its is a cluster
is its own cluster now somewhere a
hierarchical clustering works on
creating this tree between a single
cluster of all data points to each data
point being its own cluster now either
of these extremes are useless right
because a single cluster of all the data
points is not really clustering you've
just created you've put all the data
points in one group and called it group
one that's not very useful and neither
is it useful to call each data point its
own group right so somewhere in between
these two extremes is the real value add
but hierarchical clustering works on
seek it's some form of nested or kind of
tree structure of clustering where you
start with this one you can eat you can
start at either end but you might start
with one large cluster and then break
that into two and now you have these two
clusters now you go into either of these
two clusters and break that into create
a division there so now you will have
three clusters but the three cluster is
strictly a division of the two clusters
so it's not like you're going to take
some points from cluster D when you have
two clusters let's say you had them
cluster a cluster B it's not like you're
gonna take some points from cluster take
some points from cluster B and call it
cluster C it's in that sense nested it's
in the sense that you you make one split
and very very similar to decision trees
you keep making further splits and this
is contrasted to an approach of
partitioning or clustering partition of
clustering is not this kind of nested
approach it's just that you explicitly
decide on the number of clusters in some
sense and you go and partition the data
it's it's it's simply a division of the
set into non-overlapping sets are
essentially clusters and so in that
sense there is no tree diagram or
anything of that nature with this so for
instance a partition or cluster that if
I decide to do a partition of cluster
and create four clusters and and I
contrast that to a partition or
clustering approach where I had three
clusters the four need not be a further
division of the three they just
completely you know the one that said
decided to break it into four has very
little or no relationship theoretically
to the division the separate exercise or
the separate effort into creating a
partition of cluster with three clusters
whereas the same cannot be said for
hierarchical right because the
hierarchical went in sequence it could
have started top-down or bottom-up
meaning it could have started with all
in hierarchical all the individual
clusters where it's each day some day
you know each cluster is its data point
and then started grouping them or the
other way around but essentially with
hierarchical one is nested into the
other and so on the other major type of
clustering that people talk about is
exclusive versus overlapping versus
fuzzy the idea here is with exclusive
clustering each object is it's assigned
to a single cluster and that object
therefore cannot be assigned to another
cluster and and so there's no there's no
notion that one can simultaneously
belong to multiple clusters
whereas in overlapping in in case in
clustering algorithms that allow for
overlapping you can basically it's not
exclusive a data point can choose to
belong to more than one cluster at a
given point
and the decision on which of these
really depends on the underlying system
right in some cases it might just make
sense to have some data points belonging
to multiple clusters and in some in some
cases that that notion of division is
just nonsensical and finally there you
have the notion of fuzzy clustering
fuzzy clustering kind of takes this
overlapping even further where each data
point is not really assigned to a
cluster but it basically gets a number
between 0 to 1 that that talks about the
weight associated with that data point
belonging to the different clusters so
for a data point each data point gets a
total weight of 1 and it takes that data
that weight of 1 and says I'm going to
assign 0.3 in belonging to cluster a I'm
going to assign point 7 in belonging to
cluster B and I'm going to assign myself
0 to belonging to cluster C so the
constraint is that the sum of its
weights in in terms of belonging to the
different clusters adds up to 1 but it
can use its weight of 1 in any way it
chooses to belong to the different
clusters so that's to give you some idea
between of exclusive versus overlapping
misses for fuzzy the last that's what
mentioning is you you can also have a
complete versus partial question a
complete clustering basic pieces assigns
every object to a cluster whereas a
partial clustering does not it it starts
with the data points chooses to you know
cluster as many points to different
clusters and data points that don't
really help in terms of belonging so
clearly to a given cluster are just not
cluster they just left out and and these
might and the motivation there is that
you're more interested not in kind of
pigeonholing
each data point into a cluster but
you're more interested in the cluster
for me
in itself and and there you don't want
data points that are not so close
terrible that don't really belong so
clearly into one or one of two clusters
to kind of ruin the cluster Center or
ruin that nice division that you've
created okay so these are different
types of clusterings now the algorithms
themselves and these are more
descriptive of the type of algorithm
that goes into it now another area focus
could be the kind of clusters that
you're forming and while this has
everything to do with the algorithm also
here we're just looking at the end
product so the algorithm that you're
using what kind of an end product is it
can it give you so the first one is a
well separated idea the idea with the
well separated is that you want to
create clusters where get the objects
where each object is similar to every
other object in that cluster right so
the way you're defining well separated
clusters are is more from the core idea
that you're very interested in looking
at the relationship between each data
point with respect to each other data
point and you're measuring or you're
quantifying the the clustering itself by
looking at how similar that data point
is to the other data points that are in
that cluster and in there for how
dissimilar this data point is to the
data points that are in the other
clusters and this kind of thinking you
know is very useful especially when the
data itself can be very nicely separated
when the distance between the the
clusters are fairly significant then
this conception becomes very useful the
prototype based approach really talks
more about
how each data point is closed or to its
cluster representative so I the the
commonly used terminology is to the
prototype that defines the cluster and
and and when I describe it I kind of try
to think of it as there's a cluster and
this is this the main representative of
the cluster the the prototype the the
one that kind of signifies that's what
the cluster is and this is the center of
the cluster so in the prototype based
approaches because this representative
is in some sense so central it's also
sometimes called center based clusters
but the idea is the following where each
data point is looked at more from the
perspective of how close this data point
is to the cluster Center to the
prototype and how far this data point is
from the prototypes of the other
clusters or the other representatives
and the classification is done in this
fashion you then have the graph based
clustering and this is really useful if
the data is represented as a graph and
you have nodes which represent each data
point or object and you have these links
or edges which represent some notion of
connection between the data point and
this notion of connection could be the
one that talks about how similar a data
point is to the other data point you
could have some kind of a threshold
value or especially when sometimes your
variable itself is not quantitative
variables you could have some notion
that two data points are connected and
the idea here is to do some graph based
clustering where you're really looking
for a high density of connections
between data points that belong to a
cluster and a very low low level of
connectivity of the data points between
two clusters and for that reason you
know
there's this whole language that comes
from the graph theory community where
you define things called Peaks which
essentially just means that the set of
nodes in the graph are completely
connected to each other and so a lot of
the kind of clustering ideas that go
into graph based clustering are once
that kind of look out for clicks and and
say oh these guys are all connected to
each other so they must be eight a
cluster finally you have density based
clustering the idea behind density based
clustering is that a cluster is
essentially a dense region of objects
that are surrounded by regions of lower
density so the idea here is that because
there are not such well-defined
clusters like in the case of the well
separated idea that you're not really
looking to often do a complete
clustering and there's a lot of noise in
the data and you know the clusters
themselves are irregular or intertwined
and there are lots of outliers and so on
so the idea here is that you acknowledge
all of that but you just I try to
identify spots of extreme density where
once you identify that spot that dense
region becomes a cluster and and so and
so that's fairly useful in defining
clusters where there is a lot of noise
and so on okay with that we will
conclude lecture on lecture that
introduces clustering and the different
types of clusters and where it's useful
and so on in the next lecture we will
look at two basic algorithms one is the
k-means algorithm and that really
belongs to that partition or camp and we
will look at another algorithm called
the hierarchical clustering algorithm
which belongs to the hierarchical camp
thank you