Community Comments on Predicting Stock Returns

The thread on The Stock-Return Predictor at the Vanguard Diehards board is the second-longest (over 650 posts) in the history of that forum. Set forth below are snippets of some of the most helpful observations for the benefit of those who are not sure whether it is worth taking the time to read the entire thread. It is my hope that those who find value in the comments set forth below will be thereby persuaded to make time to read the entire thread.

hocus: “Until publication of the calculator, though, stock investors have not had a means of quantifying the valuation effect and of thereby putting advice to be wary of the effect of valuation changes to significant practical use.”

Probability Investing Russell: “Your calculator seems to show a dip at the 40 year mark …Is it just a peculiarity of the historical data?”

hocus: “Stocks have generally provided higher returns in the post-1921 era. So the predictions based on the larger data set (the 40-year, 50-year, and 60-year results) are slightly lower than what would be obtained from the more recent data only.”

statsguy: “Your calculator provides less pessimistic 10-year returns than the Weigand and Irons model from “Forecasting Stock Returns Using the Market P/E Ratio”, Journal of Portfolio Management (December 2005). The differences are small but significant if I did the calculations correctly.”

ccassell: “We can disagree on what to do with this information, but I don’t think we can debate the merits of this type of forecasting, unless we believe ‘this time it’s different.’ ”

Allan: “I’ve seen absolutely nothing from you [hocus] that I can use in a tangible fashion to formulate an investment plan. Your ideas are so mushy that it’s a complete waste of time to even consider them.”

ccbwc: “For real returns to be as high as your “Worst Possible” scenario over the next 30 years either: (a) Dividend growth will need to be about 50% higher over the next 30 years than it has been historically; or (b) Stocks will need to become more highly valued (as measured by dividend yield) than they are today.”

HockeyMike35: “It also uses data from all the way back to 1870. Does anyone really think earnings were reported in the same way then as they are now?”

JohnDCraig: “I prefer the Shiller general observations and the Gordon Equation to the JWR model. Among other things, the model builds in an historical 7 percent real return, which I don’t believe is supportable.”

cashNcarry: “Snake oil wrapped in a lot of babble and name dropping.”

hocus: “Think of these predictions as you think of weather predictions. When you hear that there is an 80 percent chance of rain tomorrow, do you jump to the conclusion that it is absolutely going to rain? No. But you don’t plan a picnic for a day when there is a prediction that there is an 80 percent chance of rain, right?”

statsguy: “I do not think I would trust the calculator until this research has passed rigorous refereed review.”

Predicting Stock Returns soaring: “No doubt to me that I come away with – valuation at time of retirement has a big impact on the long-term withdrawal rates so as not to run out of money. Now how I deal with that information is my challenge.”

hocus: ‘The difference is that I take this stuff seriously enough to directly state the implications of what we have learned in recent years about how to invest successfully for the long run. I think it is wrong to soften the blow. I think the story should be told straight.”

HockeyMike35: “Shiller with his PE10 data said there was a bubble in 1996. If you acted on that information and sold you would have missed out on ~8 the gains in the market over the last 10 years.”

Trebor: “Do you really think your simple tool is ‘wiser’ than the market? If it was that easy, everybody would be doing it.”

focus: “The expected return of stocks needs to be at least the Treasury Inflation-Protected Securities (TIPS) rate for stock investing to make sense.”

ccbwc: “I have used valuations to adjust my asset allocation for many years with very favorable results. My approach is different and more complex than Rob’s.”

Jim02: “I applaud his effort to inject another piece of objectivity into a very complex, highly subjective topic: making money in the market.”

stevec: “I don’t care if you do or don’t believe that the market will behave similarly as the past. Either way, this is an excellent way to understand what the US market has done in the past.”

hocus: “All of the calculations were done by John Walter Russell. My role in our partnership is to give people who don’t like what the historical stock-return data says about the effect of valuations on long-term returns somebody to yell at on internet discussion boards. (Yes, that’s a joke–sort of.)

Jim02: “I do have a problem with the idea that a dramatic shift in stocks will not be followed by a corresponding shift in bonds. The markets are not disconnected. Investors will continue to demand a risk premium.”

JWR1945: “William Bernstein got it wrong….You can’t have it both ways: a long-term return of 3.5% that lies between 6.5% and 7.0%.”

Know Your Stock Return in Advance

earnabuck: “It really is a shame and indefensible that so many feel the need to jump into it with no interest of posting on the topic but just to disrupt. Are you folks that insecure? I do know one thing, your character has shown loud and clear. Clearly some on the forum have an interest in this topic. If you don’t, stay out!”

JWR1945: “How often do you see confidence limits with financial predictions? Why don’t you see them with the Gordon Model? You should see them on a routine basis. Even if not perfect, they are a whole lot better than having a single number. This is a major shortcoming in today’s financial reporting. Would you really pay much attention to a Gordon Model debate, predicting ten years into the future, with a precision of 0.2% or 0.3%, if you knew the confidence limits? They are in the neighborhood of plus and minus 6%!”

JohnDCraig: “Irrational behavior in investing does follow certain patterns, and this has been well documented in many writings. Just how many experts in behavioral finance believe that such knowledge can be used to predict markets? Basically none, though they do believe such knowledge is generally helpful in investing. Similarly, you model is helpful, but IMO cannot attain the level of predictive value that you seem to claim.”

JWR1945: “I am comfortable using a simple, Gaussian (normal, bell shaped curve) approximation at the 90% (two-sided) confidence level. I believe that you can reasonably increase the precision to 95%, maybe even to 98%, with care, but not more. Higher precision takes you into the realm of what Nassim Taleb and Benoit Mandelbrot do.”

earnabuck: “Many take the SWR studies and consider them of value, though they are based on history. This goes above that to show, based upon that same past history, what the probabilities are for the future at various starting points. If the first has value, than surely this does too.”

JWR1945: “I have limited myself to using the most reliable statistical approximation that has meaning even though doing so limits my claims of statistical precision to 90% (two-sided, 95% one-sided). I have been careful to establish a theoretical basis to support my methodology. I have run a variety of out-of-sample type backtests…. My procedures cannot protect against a new, higher plateau. But it is easy to calculate an adjustment…. I have avoided excessive precision. This leaves open opportunities for others to calculate better numbers. They will need additional assumptions, but most will make sense.”

focus: “One point you made is that plugging in P/E10=20 gives a 10-year return of -3 to +9 . To me this shows that the range of results is so wide that you only get actionable information when the input is at the extremes.”

JWR1945: “I was careful to restrict myself to a single data source, Professor Robert Shiller’s database, because it should be a consistent set of numbers. I restricted the starting point to 1921 with the year 10, 20 and 30 returns specifically to keep us in the modern era. The P/E10 relationships are good at all times except for the single decade 1901-1910. (Gummy isolated this.) However, using more recent data is better than using the entire historical record (data starts in 1871, but P/E10 is not available until 1881). Working with real returns, the 1921-1950 data is reasonably consistent with the post-1950 data. Including 1921-1950 data adds data points without introducing serious errors. By the time that we extend the timeframe to 40, 50 and 60 years, we have to include the days with the Gold Standard and the days before the Federal Reserve…. Another point, highly important: I matched timeframes in my calculations. I used what happens in 10 years to estimate what will happen in 10 years. Similarly with 20 years and so forth. The market’s actual statistics are the standard. At least with the S&P500, failing to match timeframes causes serious errors.”

Robert Shiller's Research

hocus: “I find it revealing and disturbing that there are a number of people who generally describe themselves as “pro-stock” who have reached a point at which they feel that defending the Stocks-for-the-Long-Run Investing Paradigm requires that they argue that long-term stock returns are not predictable. The primary engine of the past bull market was the finding that long-term stock prices are predictable, and the claim that the predictability of long-term prices makes this asset class a good investment choice ‘for the long run’despite its frightening short-term volatility.”

JWR1945: “The most important application of the Stock-Return Predictor may be that it allows you to focus on what has changed, what is different from the past. It takes care of the numbers from the past. You can pay more attention to what is important, especially in terms of your own personal situation…. I agree with Rob Bennett that human (emotional) factors are far more important than numbers in isolation. How do you stick with stocks long enough to get to year 30 if you have lost money at year 10 and if you are barely above water at year 20? Should we really expect people to trust promises of financial success at year 30 if they have to live through that?”

hocus: “It’s not just the old SWR studies that got important numbers wrong, however. All sorts of investment analyses are proven wrong by this calculator.”

JWR1945: “The Gordon Model makes its best predictions 5, 10 and 15 years into the future. It does not do as well at 20 and 30 years. These are new findings. Previously, the Gordon Model had been associated only with the long-term.”

hocus: “It’s only in recent decades that we have gained the calculating tools we need to determine the message of the historical data accurately and effectively and conveniently and with a good bit of completeness. However, we also experienced the longest and strongest bull market in U.S. history during those same decades. So the early uses of these new tools have been efforts in investor self-deception. Most existing “studies” of the historical data are projects aimed not at learning what the data itself says, but projects aimed at rationalizing the absurdities of an extreme bull-market psychology.”

JWR1945: “This opinion is from pages 33 and 34 of “Common Sense on Mutual Funds” by John Bogle — ‘This analysis takes into account my conviction both that the performance of individual securities is unpredictable, and that the performance of portfolios of securities is unpredictable on any short-term basis. While the long-term performance of portfolios is also unpredictable, a careful examination of the past returns can establish some probabilities about the prospective parameters of return, offering intelligent investors a basis for rational expectations about future returns.’ ”

hocus: “There are hundreds of people who contributed to this. I think we all should be very proud of what we have done together. This calculator demonstrates in a compelling way the power of this new internet discussion-board communications medium.”

JWR1945: “You have described what I named “Idiot Switching.” It is about the worst kind of honest switching algorithm that I can imagine: jumping in and out, 100% or 0% stocks….I acknowledge that short-term timing is possible, with difficulty, with much greater sophistication than in the study, but with much lower returns than most studies indicate. Rob Bennett is not even willing to go that far. He questions whether short-term timing is ever feasible, at least for the investing public.”

hocus: “I noted above that the P/E10 that applies today–26–is an extremely high P/E10 level. But look at the most likely 30-year return predicted by the calculator — 5.4 percent real. Is that an “anti-stock” finding? It sure doesn’t seem so to me . It is critically important to show the differences between stocks held for 10-year time-periods and stocks held for 30-year time-periods. There are all sorts of strategic implications that follow from understanding that stocks provide different sorts of returns over different sorts of time-periods.”

JWR1945: “You can use TIPS to coast through an extended bad period even if you have to draw down principal. Your staying power is much better than you might imagine. Typically, you can coast through a decade while retaining 80% of your original buying power while making significant withdrawals.”

hocus: “Go to the calculator and plug in two P/E10 values right next to each other — say a P/E10 of 19 and a P/E10 of 20. The lower P/E10 will give results that are more appealing, but not all that much more appealing. Now take a look at the range of possible returns identified as possibilities by the calculator. The range over 10-year time-periods is large. The combination of these two realities tells you that putting two much stress on small changes in valuation is not likely to produce good results.”

Timing the Market Successfully JWR1945: “When you actually get close to the long-term, the Gordon Model routinely differs by plus and minus 3% to 5% from the historically accurate value, around 6.8% (real, annualized). Those are huge errors.”

hocus: “My understanding (I’d like to be corrected on this point if my current understanding is wrong because many of the strategies of the Valuation-Informed Indexing approach are rooted in this understanding) is that the long-term annualized return of 6.8 percent real does not contain anything speculative in it….If you purchase at moderate valuations, you stand a good chance of getting there within 10 years. If you purchase at today’s valuations, you will likely be getting into the neighborhood of 6.8 percent at the end of 30 years, but it might well take 60 years or longer to get all the way there. If you purchase at times of low valuation, you will likely have a return well in excess of 6.8 percent at the end of 10 years, but you will also likely see your return fall to something close to 6.8 percent by the end of 60 years.”

JohnDCraig: “I think it is very important for you to refine your understanding in this regard, since I am quite certain that neither Bernstein nor Swedroe would agree with your belief that the S&P index is sure to obtain annualized real returns of 6.8%. On the other hand, it seems quite clear that Taylor and Michael do agree with your conclusions on this point. I think what is needed is a full understanding and resolution of these differing views.”

hocus: “The stuff we are talking about here is fundamental stuff. It astounds me that there is so much uncertainty on such basic points. I can tell you what people like Bogle and Bernstein and Burns and Clements say in their books and articles and speeches. I can’t resolve the contradictions between the things they say at some places and times with the things they say at other places and times. The reality is that the big-name investing experts regularly contradict themselves on the most basic points of how stocks work.”

JWR1945: “As for other experts: what is missing are CONFIDENCE LIMITS. Those experts disagree about fine grain adjustments that are small fractions of the actual scatter observed throughout history. In reality, their disagreements involve ‘angels dancing on the head of a pin.'”

hocus: “Has Bernstein ever tried to make sense out of the jumble of contradictions he puts forward in the pages of his book? It is not my intent here to rag on Bernstein. He ain’t the only one doing this sort of thing. I’ve seen Bogle do it. I’ve seen Scott Burns do it. I’ve seen Jonathan Clements do it. The problem they all face is that the now-dominant investing paradigm is flawed to its core. The central premise — that stocks always offer a strong value proposition — is false and has been proven false.”

JWR1945: “With ideal inputs (dividend yield, real earnings growth and P/E10 change), the Gordon Model can cut the uncertainty range at year 10 to plus and minus 3% (at the extremes). This is one-half of the plus and minus 6% confidence limits of the Stock-Return Predictor. But the Stock-Return Predictor requires only the initial value of P/E10. With ideal inputs (dividend yield, real earnings growth and P/E10 change), the Gordon Model can match the uncertainty range of the Stock-Return Predictor at year 20 of plus and minus 2% (at the extremes). But the Stock-Return Predictor requires only the initial value of P/E10. The Stock-Return predictor is more accurate than the Gordon Model at years 30 and beyond.”

The Gordon Equation hocus: “Valuations don’t mean much in the short-term (time-periods of five years or less) because stock prices appear to be unpredictable in the short-term. Valuations don’t mean much in the very long-term (time-periods of over 30 years) because in the very long run there are so many ups and downs that they counter each other and the only thing that matters much is the real economic return (6.8 percent, presuming that stocks continue to perform in the future somewhat as they always have in the past). It is in the time-period from five years out to about 25 years out that valuations are a big deal.”

focus: “I would never invest in anything without having any idea what the expected return is. For instance, I would not walk into a bank and say “I’ll take one CD, please” without asking what rate they are offering.”

hocus: “People set the prices of stocks. So, yes, the P/E10 value is providing information bits about people. It’s a tool that tells you whether people at the moment are properly enthused about stocks, over-enthused about stocks, or under-enthused about stocks. The fact that people today are over-enthused about stocks does not reflect poorly on the asset class. Stocks are fine — at a reasonable price. The problem is that people have bid up the price of stocks so high that there are many circumstances in which buying stocks does not make sense. But that will change. People will come to their senses, and stock prices will return to moderate valuation levels. When they do, the P/E10 value will tell you so.”

JohnDCraig: “This model says that there is a 60% chance of a negative ERP in the next 10 years. Bogle and Bernstein both say that attempting to measure short-term returns (defined by Bernstein as less than a few decades) with any precision will likely do one more harm than good. Bogle describes in his book, Common Sense on Mutual Funds, how humbling an experience it was for him when he attempted to forecast 10 year returns in 1990.”

hocus: “Bogle’s mistake was being too precise. Please note that The Stock-Return Predictor avoids this trap by employing only the level of precision permitted when using a statistically valid examination of the historical stock-return data to generate the numbers.”

JWR1945: “For the first 15 years, you should dollar cost average either entirely into stocks or entirely into TIPS. At today’s valuations, you should dollar cost average entirely into TIPS. At year 15, continue to dollar cost average into your portfolio. You should vary allocations in accordance with P/E10 (i.e., what I call switching). Use the same switching algorithms as for retirement. The reason to begin switching at year 15 is that you have money. Only then does downside risk becomes meaningful.”

hocus: “I’ve seen things said on investing boards that I have never heard said in discussions of any non-investing topic. I think a case can be made that the question of whether valuations affect long-term stock returns is a topic that causes more people more emotional angst than does abortion or impeachment proceedings or the war in Iraq.”

focus: “Just using one factor to forecast returns is probably inadequate. For one thing, we need to know how well your current single factor model, i.e. P/E10, explains the future returns. Something like R-SQUARED needs to be made known. Somebody already linked to the Weigand and Irons paper that showed a six factor model that was superior to a single factor P/E model, so there is already academic work on that subject.”

hocus: “Extreme actions in either direction are a mistake, in my view. It is extreme to go with a 70 percent stock allocation with prices at today’s levels. It is also extreme (in most circumstances) to go with a zero percent stock allocation….The key is that you hold your stocks long enough to escape the 10-year numbers and the 20-year numbers and make it to the green fields of the 30-year numbers.”

The Stock-Return Predictor

JWR1945: “Earlier on this thread, someone pointed me to an academic study which claimed high levels of r-squared (and they were right), but which had much lower levels of r-squared than I routinely report. Let’s understand this number. R-squared simply tells us the strength of a correlation. It doesn’t tell us the size of the data spread. Even if r-squared were zero, the calculator would be right as long as I got the confidence limits right. Since r-squared is not zero, there is a shift in the most likely result. When we include the effect of the effective degrees of freedom, we find that the most likely result is known to about plus and minus 1% (which is just under 20% of [times] the confidence limits) for the Year 10 results. The most likely result varies much more than an adjustment of 1% (added to or subtracted from annualized total return percentage number). Whoever has been spouting off about bad r-squared numbers hasn’t done his homework.”

hocus: “I look forward to a day when there are scores of calculators on the internet that do what the Predictor does. The publication of more calculators will inspire more debate. More debate will teach us new things. Learning new things will help us develop enhancements for the Predictor. It’s a win-win-win.”

Andy71: “How long before you add a reference on your web site taking credit for starting yet another great debate in the investment community? Can it count as a great debate when 80% of the posts are by the thread’s 2 authors?”

hocus: “We are witnessing something phenomenal, Andy. I am totally convinced of that. I believe that we are seeing what Change looks like when we watch it play out in front of us up close and personal and in real time. There’s great stuff on this thread, from all sorts of people. It’s been a wonderful community effort and a wonderful Learning Experience for all who elected to take advantage of it.”

JWR1945: “The standard, normal, bell shaped, Gaussian statistics are best for the kind of thing that I am doing here. Specialized techniques are appropriate when buying and selling $100s of millions in derivatives.”

hocus: “The suggestion that I am expecting to see some pretty darn juicy long-term terms from the stock asset class is on the mark. I am a stock guy, and don’t let anyone tell you different. I think stocks offer the middle-class worker the best opportunity available to him to generate lasting wealth. The key is walking the buy-and-hold walk as well as talking the buy-and-hold talk. I believe that the historical stock-return data lays out a blueprint of what is required to pull that off (not to say that there won’t be any surprises) for the benefit of those willing to tune into its message.”

focus: “I like to think of the more traditional thinking that the stock market always has an expected return of about 7% real as a Zero-factor model. In other words, the expected return is a constant and not dependent on any factors. It is not a bad estimate taking no other information into account. Your model is a 1-factor model–it uses a single factor to explain future returns. I believe that taking that one additional bit of information into account greatly improves the forecasting model. Then you can add more variables like inflation, recent earnings growth rate, recent volatility, etc., like Wiegand and Irons did in their paper and get better forecasting models. Then the debate really becomes which variables should be considered and which excluded. So your model is a good step towards better and more accurate forecast models.”

modern portfolio theory hocus: “There are numerous ways in which the valuation factor is different from the others. One is that valuation is a huge factor. There are circumstances in which the valuation factor alone determines 50 percent of the total return. So ignoring valuations is simply not prudent. Also, we have lots of data on historical valuations, so we have a means of assessing its impact with more accuracy than can be achieved for the other factors. Also, we have control over the valuation level at which we make our stock purchases. So knowing the effect of this factor permits us to enhance our investment results in a way that we can not generally enhance them with knowledge of other factors.”

Sirschnitz: “Just as Investors are interested in the success of their portfolios, Professors are interested in the success of their theories. It’s part of the ‘human thing’.”

hocus: “I used to think that the reason why this assumption was employed was to make things look not so bad for the next 10-year period. It does do that, of course. I always thought that it was an unfortunate practice not to expect valuation changes because it understates the risks that investors face when going with high stock allocations at times of high valuations. I now see a second problem with assuming no valuation changes — it also understates the 30-year return.”

JWR1945: “If valuations were to remain constant, the mathematical requirements of the Gordon Model would be satisfied. The Gordon Model would have been correct throughout history. Examination of the Gordon Model’s 30-year historical performance shows that it has not been accurate. By the time that you reach 30 years, it routinely misses by 3% to 4%, with peak errors near 5%. The problem with the Gordon Model is that valuations change. Reinvested dividends yield something different from the initial yield. Those reinvested dividends earn more (after valuations have fallen) if valuations started out high. Those reinvested dividends earn less (after valuations have risen) if valuations started out at bargain levels.”

hocus: “Bogle and Bernstein have been talking about the effects of valuation changes for a long time. Bogle and Bernstein have failed in that time to translate the insights they have put forward into practical and actionable and reasonable and balanced investing advice. Both things are so.”

JWR1945: “The article looks at P/Ex. The Stock-Return Predictor actually uses 100/[P/E10]. The earnings yield and returns have a linear relationship (theoretically). Price to earnings does not.”

hocus: “Some stock advocates try to plant the suggestion that they are not basing their decisions on predictions of what may happen or expectations as to what may happen. They are either not talking straight or not thinking straight. It is an essential step in the asset allocation process to form some sort of view as to what sort of return it is reasonable to expect from the various investment classes. So the question is not — Should you form expectations re returns or not? It is — Should you look at the historical stock-return data in forming those expectations or not?”

Vanguard Diehards at Morningstar

JWR1945: “In terms of calculating 30-Year Safe Withdrawal Rates, P/E10 is a clear-cut winner. In terms of predicting Year 10 stock market returns, Tobin’s Q is best and 1/Tobin’s Q and 100E10/P are almost the same according to R-squared (with identical variances). However, Tobin’s Q and 1/Tobin’s Q have a problem with outliers. In terms of predicting Year 20 stock market returns, P/E10 is clearly superior.”

hocus: “It’s not possible for those who have come to believe that stocks are always best to accept that valuations always matter. The two beliefs are mutually exclusive. If valuations matter, there is obviously some valuation level at which stocks are not best. The two paradigms cannot be reconciled.”