As we begin to discuss different algorithms in data mining, many students experience fear. They’ve been warned that the class is hard and, for many, it is beyond their comfort zone. Because of this, the first algorithm we discuss is multiple linear regression as it is a natural extension of the prerequisite introductory statistics course. It is nonetheless intimidating.
It turns out that virtually every college-bound American student has already been introduced to the framework necessary to intuitively understand regression. So, when introducing it, I draw x and y axes on the whiteboard and a line through the axes and then ask the most scared looking student “what is the formula for a line?” Like clockwork they respond with “y = mx + b” as if it were indelibly written in their mind. I ask what that means and some student will explain how it works (m is the slope, b is the intercept, plug in x and get y). I then tell them that I am going to simply re-write the formula to y = b0 + b1x1, and explain how all I have done is re-arrange and re-name things.
I then ask them “what if we added a third dimension to our x and y – what do you think the formula for this plane, instead of a line, would be?” This is a harder question but always a student will get it, saying something along the lines of “y = b0 + b1x1 + b1z.” I usually need to clarify things a bit here and explain that b1 goes with dimension x1 so our new component is b2x2. “What if we add yet another dimension?” By now, the students are beginning to get it: each new dimension merely adds its own new beta (bn) and its own new variable (xn). I continue on through more dimensions until whoever I call on can give the correct answer.
At this point, I need to tie it back into how we are actually using it in data mining. The ‘y’ is what we are trying to predict and x’s are what we are using to predict. So if I have y = b0 + b1x1 + b2x2 + b3x3 + b4x4, and I am trying to predict the price of a used Toyota Corolla with certain variables, I can write out my formula as:
corolla_price = b0 + b1*mileage + b2*mpg + b3*age + b4*number_of_doors.
When we run our regression algorithm, it will give us the beta weights, which just like our old friend, the formula for a line, we only need to plug in the variable values (e.g., mileage for this car is 100,000, mpg is 27, etc.), multiply them by their weights and finally sum up the products to get a predicted price. So, a complex yet useful algorithm can be rendered a little more tractable by linking it back to students’ eighth grade algebra class. Plus, it helps to ease their transition into a world which will eventually give way to logistic regression and neural networks, both of which are linked back to that little old formula they learned so long ago.