During the last half century, hundreds of papers published in statistical journals have documented general conditions where reliance on least squares regression and Pearson's correlation can result in missing even strong associations between variables. Moreover, highly misleading conclusions can be made, even when the sample size is large. There are, in fact, several fundamental concerns related to non–normality, outliers, heteroscedasticity, and curvature that can result in missing a strong association. Simultaneously, a vast array of new methods has been derived for effectively dealing with these concerns. The paper (i) reviews why least squares regression and classic inferential methods can fail, (ii) provides an overview of the many modern strategies for dealing with known problems, including some recent advances, and (iii) illustrates that modern robust methods can make a practical difference in our understanding of data. Included are some general recommendations regarding how modern methods might be used. Copyright © 2011 John Wiley & Sons, Ltd.