Lottery: A tax on the statistically-challenged.
Linear regression using the LSRL is not the only model for describing data. Some data just are not best described linearly. Non-linear relationships between two quantitative variables can sometimes be changed into linear relationships by transforming one or both variables. Removal of outliers from data may cause a drop in the correlation so that linear no longer does a satisfactory job of describing the data. The bulk of the data may not be linear at all.
Transforming can be thought of as re-expressing the data. We may want to transform either the explanatory variable x, or the response variable y in a scatter plot, or maybe even both. We will call the transformed variable "t" when talking about the transforming in general.
Transforming data amounts to changing the scale. Linear transformations as discussed in Chapter 1 change units by addition or multiplication. Recall, adding a constant amount to each observation does NOT change the spread but does add that constant to the center and quartiles. Multiplying by a constant multiplies both the measure of center and the measure of spread by the same constant. BUT linear transformations of this type CANNOT STRAIGHTEN OUT A CURVED RELATIONSHIP BETWEEN TWO VARIABLES.
Some non-linear functions that have been studied in Algebra include quadratic, logarithmic, reciprocal (negative power), square root, and other power functions.
A monotonic function f(t) moves in ONE direction as its argument t increases.
A monotonic increasing function preserves the ORDER of data. If a>b, then f(a)>f(b).
A monotonic decreasing function reverses the ORDER of data. If a>b, then f(a)<f(b).
The graph of a linear function is a straight line. The graph of a monotonic increasing function is increasing EVERYWHERE. A monotonic decreasing function has a graph that is decreasing EVERYWHERE. A function can be monotonic over some range of t without being everywhere monotonic. For example, the square (quadratic, parabolic) function t2 is monotonic increasing for t > 0. If the range of t includes both positive and negative values, the square function is NOT monotonic since it decreases as t increases for negative values of t and increases as t increases for positive values. Many variables take only 0 or positive values, so we are particularly interested in how functions behave for positive values of t. See board for sketches of graphs.
The increasing monotonic functions are linear, quadratic t > 0, and logarithmic. Even numbered power functions tp are monotonic increasing for t positive. (Logarithms are not even defined for negative or zero values.) Order is preserved. The decreasing monotonic functions are linear, reciprocal, or reciprocal square root. Power functions tp for negative powers are monotonic decreasing and reverse order also.
Non-linear monotonic transformations change data enough to ALTER the shape of distributions and the form of relations between two variables, yet are simple enough to preserve order and allow recovery of the original data.
Strategy for transforming data:
1) if the variable to be transformed takes values that are 0 or negative, first apply a linear transformation to make the values all positive. This can be accomplished by adding a constant to all the observations.
2) next choose a power or logarithmic transformation that simplifies the data...one that approximately straightens a scatter plot. (Note: the use of the word simplifies takes yet another meaning when applied to graphs of data.)
Other things to remember:
1) Power transformations tp for powers p greater than 1 are concave up. They push out the right tail of a distribution and pull in the left tail. This effect gets stronger as the power increases.
2) Power transformations tp for powers p less than 1 (and logarithmic for p = 0) are concave down. They pull in the right tail and push out the left tail. Effect strengthens as p decreases.
There are a variety of transformations available to us depending
on the appearance of the data. Using a "trial and error" or "try it and
see" approach isn't very satisfactory or efficient. It is much more
logical to begin with a theory or mathematical model that we expect to describe
a relationship The transformation needed to make the relationship
linear is then a consequence of the model. One of the most common models
is EXPONENTIAL growth.
A variable grows linearly over time if it ADDS a fixed increment in each equal time period. Exponential growth occurs when a variable is MULTIPLIED by a fixed number in each time period. (Exponential growth increases by a fixed percentage of the previous total). We have studied exponential growth before when dealing with interest problems, bacterial growth, or epidemic situations. See Ex. 4.3 page 204 for a good example.
What does exponential growth look like??? Can we trust our eyes to recognize it??? How can we be sure that we have a correct model???
When exponential growth is suspected from viewing the data, the first step is to calculate the ratios of consecutive terms to see that they are approximately the same. Exponential form is y = abx (variable in the exponent) and we will transform by taking the log of both sides yielding log (y) = log (abx). This statement simplifies using one of the log properties to become log (y) = log (a) + log (bx) which becomes
log (a) + x log (b). When we plot log (y) against x, we should observe a straight line for the transformed data. Apply least-squares regression to the transformed data and check the correlation (r) value and the coefficient of determination (r2) value for suitability.
Remember r2 gives a measure of how successful the regression was in explaining the response.
Even when r2 returns a high value you should ALWAYS inspect the residual plot to further assess the quality of the model. The residual plot should appear totally random showing no pattern. If any part of the residual plot appears recognizable, then perhaps a few data points should be removed to see if the r2 can be improved.
We will do Ex. 4.6 page 209 by hand and using technology.
POWER LAW MODELS (in general y = a xp)
When you order pizza you order by its diameter, like 10 inches...BUT when you eat pizza you eat its AREA or A = Πr2. Notice there is an exponent present but it is not a variable, it is a declared power. Recall that all power functions have a "look" of their own. Odds have ends that go in opposite directions, evens have ends that point in the same direction.
When we are dealing with things of the same general form, whether circles or fish or people, we expect area to go up with the square of a dimension such as diameter or height. Volume should go up with the cube of a linear dimension. Geometry tells us to expect power laws in some settings.
Biologists have found that MANY characteristics of living things are described quite closely by power laws. There are more mice than elephants, and more flies than mice, the abundance of species follows a power law with number of eggs a bird lays, and so on. Sometimes the powers can be predicted from geometry, but sometimes they are mysterious. Why, for example, does the rate at which animals use energy go up as the 3/4 power of their body weight?? This particular relationship is called "Kleiber's law and works from bacteria to whales. Therefore, power laws are a good place to start in simplifying relationships for living things.
Exponential growth models become linear when we apply the logarithm transformation to just the response variable y. Power law models become linear when we apply the logarithm transformation to BOTH variables.
Power law model is y = a xp
Take the log of both sides to get log (y) = log (a) + p log (x)
Taking the log of BOTH variables straightens the scatter plot of y against x.
The power p becomes the slope of the line that links log y to log x.
Follow the steps above to verify that the regression is a good fit using r and r2.
Great, specific directions for accomplishing this appear on page 219 in the technology toolbox.
IF taking logs of both variables makes a scatter plot linear, a power law is a reasonable model for the original data. We can even estimate what power the law involves by regressing log y on log x and using the slope of the regression line as a good estimate of the power. Remember the slope is only an estimate of the p in an underlying power model. The greater the scatter of the points in the scatter plot about the fitted line, the smaller the confidence that this estimate is accurate. See E. 4.8 page 215 for an example