Thursday, September 16, 2021

Debunking a Bogus Stock Market Prediction

It would be much easier to plan for the future if we knew what stock prices were going to do.  Bank of America has a chart with seemingly solid evidence that stocks will lose a total of about 8% over the next 10 years.  I’m going to show why this evidence is nonsense.  But don’t worry; I’ll do it without making you try to remember any of your high school math.

The Bank of America chart looks intimidating to non-specialists, but I’ll summarize the relevant parts in easy-to-understand language.  The basic idea is that for each month since 1987, they looked at how expensive stocks were that month and compared that to stock market returns over the 10 years following that month.  They found that the more expensive stocks were, the lower the next decade of returns tended to be.  The hope is that we can just use the chart to look up today’s stock prices to see what stock returns we’ll get over the next 10 years.

In the chart below, each dot represents one month from 1987 to 2010.  Notice that the dots are fairly close to forming a straight line.  Statisticians get excited when they see a strong relationship like this.  If the line were perfectly straight, we could just look up how pricey today’s stocks are (using a measure called the P/E or price-to-earnings ratio), and read off the average annual stock return we’ll get over the next 10 years.

The line isn’t perfectly straight, but it’s fairly close.  One measure of how close to a straight line we have is called R-squared.  For our purposes here, we don’t need to know much about R-squared other than 100% means a straight line, and as this percentage drops toward zero, the cloud of dots spreads out.  The chart indicates an R-squared of 79%, which is a strong relationship.

Also indicated on the chart is the prediction that stocks will lose an average of about 0.8% each year over the next decade.  However, if we imagine an oval surrounding the full range of dots, this chart predicts annual stock returns between about -3% and +2%.  If we knew future stock returns really would fall in this range, most people would sell their stocks.  But can we count on stock returns falling in this range?  It turns out that we can’t because the chart is deeply flawed, as I’ll explain below.


The first thing to observe is that this chart is based on about 34 years of stock market data, a little over 3 decades.  Because we’re talking about 10-years returns, you might wonder why there are more than 3 dots on the chart.  The answer is that it uses overlapping periods.  There is a dot for January 1987, then February 1987, and so on.

Consider the ten years of returns starting in January 1987 and compare this to the ten years of returns starting in February 1987.  They are the same in 119 of 120 months.  Each decade of returns starting monthly from 1987 to 2010 overlaps with 119 other decades.  There is a huge amount of redundancy in the chart.  Somehow we went from a 3-dot chart to one with hundreds of dots.

Using overlapping data isn’t always a bad thing, but it is in this case because there is just too little independent data to have any statistical significance.  To show this, I ran some simulations.  I created random stock market data and measured R-squared values.

The method I used for creating this simulated stock market data created returns that ignored stock valuations.  This means that using P/E values to predict stock market returns is futile with this simulated data; the R-squared value of the underlying probability distribution used in the simulations is zero.  To confirm this, I generated a million years of stock market data, and measured the R-squared value.  In a thousand repetitions of this experiment, all R-squared values were less than 0.02%.

However, coincidences are common when you examine very small amounts of data.  I ran simulations of 34 years of stock market data.  I repeated this experiment 100 million times.  Amazingly, in just over one-tenth of the simulations the R-squared value was above 79%, and in 51% of the simulations the R-squared was above 50%.  These seemingly strong correlations are what you get with small amounts of random data, even though the underlying probability distribution has no correlation at all (R-squared equal to zero).

What can we conclude from these experiments?  The data in the Bank of America chart is a meaningless coincidence.  In an earlier article I showed that when we examine stock market data back to 1936, the correlation becomes much weaker (the dots look more like a wide cloud).  We can’t tell what will happen with stocks over the next 10 years with any accuracy just by looking at 34 years of stock returns, and it turns out that going back to 1936 doesn’t help much either.

Why was this error missed?

If this chart is so deeply flawed, why did Bank of America create it and why are so many people spreading it through social media as evidence that stocks will perform poorly?  The answer is that few people are good at probability theory.  Most people who use statistical methods professionally don’t understand the underlying probability theory well enough to use statistics safely.  When we use statistical tools to process data, it’s very easy to lose sight of significant problems, such as having too little data.

All the Bank of America chart is saying is that when stocks were expensive around the year 2000, the market crashed, and now that stock prices are very high again, the stock market may crash again soon.  Or it might not; it’s hard to tell.  The statistics just take this simple idea and dress it up to seem more scientific than it is.

Is this chart a deliberate deception?  Probably not.  Hanlon’s razor applies here: “never attribute to malice that which is adequately explained by stupidity.”  When people who know just enough math to be dangerous, they often fool themselves first and fool others later.

Does this mean stocks will keep going up?

No.  It just means we don’t know what will happen, and the Bank of America chart sheds almost no light on the question.  There are good reasons to believe that a market crash and poor returns over the next decade are more likely today than when stock prices were lower.  But this chart isn’t one of the good reasons.


The crystal ball for viewing future stock returns is still cloudy.  We need to consider a wide range of possible market outcomes in our planning.  It’s important to strike a balance between being positioned to benefit from a rosy future and being protected against a bleak future.  With stock prices so high, I’ve chosen to shift my focus a little more toward protecting myself against poor market returns.

Friday, September 10, 2021

Short Takes: Gen X Wealth, Plug-in Hybrids, and more

Eight months ago, a group of friends including my wife and I took a chance and booked a PEI vacation.  Fortunately, we were able to go, and we’re just back from a great time.  I’m starting to wonder if we were lucky enough to hit a window that’s now closing as the COVID-19 Delta variant continues its exponential spread through Canada.  We just don’t have enough Canadians vaccinated yet to stop the growth without restricting our movements as they were earlier in the pandemic.  It’s been interesting to watch governments at all levels struggle with attempts to encourage people to get vaccinated and to apply lockdown rules only to those who have chosen not to be vaccinated.  All this is a work in progress and teething pains will continue.

Here are some short takes and some weekend reading:

Economist Writing Every Day says U.S. Gen Xers are now 30% wealthier than Boomers were at the same age, and that Millennials are on a similar path.  It would be interesting to see similar data for Canada.

John Robertson dives into the costs of plug-in hybrid vehicles, complete with a spreadsheet for comparing costs.  We’re in a transition period right now when plug-in hybrids make sense.  This will last until battery costs come down enough to make solely electric vehicles the obvious choice.

Scott Ronalds at Steadyhand
reports that  92% of Steadyhand employee financial assets are invested in Steadyhand funds and that employees get no special breaks on fees.  Steadyhand’s situation is one of the rare times that talk of “aligning interests” is true.

Friday, August 27, 2021

Short Takes: Changing Risk Appetite, the 4% Rule, and more

Over the past decade I’ve rented places in Florida through VRBO, and I’ve noticed an interesting change in the ongoing battle between owners and renters.  Early on, when searching for a place, filtering by price worked reasonably well.  You could count on the actual full price to be 20-25% more than the advertised rental price once they added various fees.  Then owners got clever and began to add ever-larger nonsensical fees.  In one extreme case, the added fees doubled the total rental cost.  Filtering by price became useless.  VRBO responded by calculating a full price with all the add-on fees to show to prospective renters during their search.  So, filtering by price works again, except for a new game.  Owners are very cagey about the price of pool and hot tub heat.  Because this is optional, VRBO doesn’t include it in advertised prices.  Owners try to get renters to commit to a rental and then hit them with punitive pool heat prices.  To find a place, we now have to find some suitable rentals and send queries to their owners to find out about pool heat cost.  Owners generally reply, but they avoid giving an actual price. Sometimes they eventually give a figure after a few exchanges, but sometimes they don’t.  In one case, they wanted US$40 per day, which is many times what it costs to heat a much larger pool in Canada.  It pays to understand the game and be wary.

Here are my posts for the past two weeks:

Narrative Economics

Retirement for the Record

Here are some short takes and some weekend reading:

Tom Bradley at Steadyhand
discusses ways to change your risk appetite when it differs from your risk capacity.  I’ve long believed that with the right approach, young people should be able to become comfortable with the gyrations of the stock market with their long-term savings.

The Rational Reminder Podcast
has a recent episode that gathers together expert opinions on the 4% rule for retirement spending from a portfolio.  This includes Moshe Milevsky’s entertaining demonstration of the foolishness of picking a spending level and blindly increasing it by inflation each year without any regard for how your portfolio grows or shrinks.  However, that doesn’t mean William Bengen’s original work on the 4% rule was foolish.  We can decide to adopt a flexible spending plan but choose an initial spending level that isn’t likely to have to be reduced.  We then do an analysis similar to Bengen’s to choose that starting spending level.

Robb Engen at Boomer and Echo say that what we should do about high stock prices is “If you’re invested in a globally diversified and risk appropriate portfolio, the answer is to lower your expected future return assumptions and do nothing else.”  He sees those who act on predictions of market crashes as overconfident.

Monday, August 23, 2021

Retirement for the Record

Just in case you’ve ever wanted to read a book about both retirement planning and the music of the 60s and 70s, Daryl Diamond has you covered with Retirement for the Record: Planning Reliable Income for Your Lifetime … to the Soundtrack of Your Life.  This book’s main focus is the value of an advisor (and his firm in particular) in retirement financial planning.  However, about a third of it is stories about the music most relevant to those born near the peak of the baby boom.

I expected more discussion of how to plan your finances in retirement, but the consistent message is that such planning is difficult, you need help, most advisors aren’t good at it, but Diamond and his firm do it well.  An early chapter tells us that there are no cookie cutter solutions, because the answer to most retirement planning questions is that “it depends” on your particular circumstances.  There is some truth to this, but it would certainly be possible to lay out 6 or 8 examples that illustrate the most important themes of planning a person’s retirement.  Unfortunately, Diamond doesn’t do this.

“I have seen plenty of situations where a DIY investor is trying (trying is the operative word here) to efficiently set up their income streams and in the process ends up sabotaging a preferable outcome. Unfortunately, I meet far too many ‘do-it-yourself investors’ who aren’t doing a good job transitioning to the income years themselves and, realizing this, are seeking my help at this point in their life.”  Of course, he doesn’t see any DIYers who plan their retirement well, because they don’t go to see him.  Given the repeated pitches for advisors and his firm in particular, this whole message just comes off as self-serving.

To show the complexity of retirement financial planning, Diamond lists ten questions that must be answered before it’s possible to tailor a plan to a client’s needs.  Unfortunately, he doesn’t discuss what he would do with the answers.

The Good

Before continuing with criticisms of this book, let’s look at a few parts I liked.  To avoid big tax bills in the future, it often makes sense to start drawing from RRSPs early instead of waiting for mandatory RRIF withdrawals to begin.  “To the best of my knowledge, only one of the Big Five banks allows their planners to illustrate scenarios in which RRSPs are not deferred as long as possible.”  Aside from the problem of possibly giving bad advice to clients, how can planners within big banks be considered professionals if they must do what their bosses tell them to do even if it hurts clients?

Diamond gives a good discussion of the importance of line 23600 (net income) on Canadian tax forms.  This net income figure determines your OAS clawback and the value of your age amount and age credit (for those 65 and older).  Many people don’t realize that if you carry capital losses from previous years forward to offset capital gains in the current year, it reduces your taxes this year, but it doesn’t change line 23600.  So, previous capital losses can’t reduce your OAS clawback or increase your age amount.

There’s nothing wrong with deciding to become more conservative with your investments, but too many investors do this “at precisely the wrong time.”  Investors who get conservative after stocks have crashed “lock in their losses,” “experience lower returns than if they had stayed invested,” “miss the subsequent upturn in the markets,” and “need to decide when to enter back into the markets—and this happens after they have already missed meaningful gains.”

The Not So Good

In a chart showing the “factors for successful investing,” the claimed least important factor is “fees.”  Even a 1% annual fee will consume about one-quarter of the assets you accumulate and decumulate over an average lifetime, so it’s hard to see how fees are unimportant.

Diamond is adamant that most people should take CPP and OAS as early as possible thereby “preserving your personal income-producing assets.”  Unfortunately, his reasons for this mostly come down to focusing on the possibility of dying young.  “I can tell you that over the last eighteen months we have unfortunately had twenty of our clients pass away.”  “Fifteen passed before the age of 71.”  Of course, there are many who are still alive and will live long lives.  Frederick Vettese’s take on when to start CPP in his book Retirement Income for Life (second edition) makes more sense.

One chapter warns what could happen if you’ve delayed CPP and OAS, but one spouse dies.  In this case, that spouse’s CPP and OAS get replaced with a smaller CPP survivor pension.  In one example, a widow sees her income drop 25%, an apparently disastrous outcome.  However, her expenses will drop as well.  It’s important to actually analyze a couple’s spending to see how it would change if one spouse died. Instead, Diamond treats any income drop as a calamity, and declares that almost everyone should take CPP and OAS as early as possible.  In reality, the possibility of living a long life and needing to avoid running out of money have to be considered together with other possible outcomes.

When fees are based on a percentage of client assets, “the better the portfolio performs, the better it is for the client and for the advisor.  There is an alignment of interests in this arrangement, which is why we prefer it and it is the model we use in our firm.”  The alignment of interests is pretty weak with this arrangement.  We get much better alignment when advisors hold the same assets in their personal portfolios as they recommend for their clients.

In a rant about fee disclosure and HST, Diamond writes “While the government contends that it is so interested in fee disclosure and the desire to ‘help’ investors, they are certainly not shy about carving money out of your returns in the form of taxes [HST on management fees and dealer fees].”  I’m not convinced that this HST only affects investor returns.  What if managers and dealers already charge as much as they can get away with, and they would just raise their fees if the HST went away?  Then the HST bites into their fees rather than investor returns.  In reality, this HST affects both sides: investors and managers and dealers.

An entire section of the book devoted to “investing in retirement to generate income” can be summarized as “we use income funds.”  Diamond stresses the importance of “preserving the invested capital” by spending only income and not selling units of the income fund.  However, income fund managers sell capital within the fund to make up part of the promised payments.  This is called return of capital (ROC).  The claim that your “capital can remain invested” is at best misleading.  Diamond views ROC “as a component of this managed investment process that generates regular income.”  This is just double-talk to disguise the fact that income funds don’t really preserve capital.

Diamond advocates a 5% to 5.5% withdrawal rate in retirement.  This is dangerously high, particularly when you’re also paying advisor fees, but has worked out well during the bull run in stocks over the past decade or so.  Who knows what will happen in the coming decade.  Something I didn’t see mentioned is increasing the withdrawal rate with age.  An early retiree certainly should be drawing less than 5%, and an 80-year old can safely draw more than 6%.


Overall, I’m not a fan of the retirement planning part of this book; I found it self-serving for the author.  He could have actually explained his process to help other advisors and maybe some DIYers.  However, it’s clear that Diamond has great passion for the music of his youth.  Readers of the right age (baby boom peak) may enjoy his many interesting music-related stories and trivia questions.  Even though I’m significantly younger than the author, I enjoyed the music discussions more than the retirement planning.

Monday, August 16, 2021

Narrative Economics

In his book Narrative Economics: How Stories Go Viral & Drive Major Economic Events, Nobel Prize-winning economist Robert J. Shiller calls on other economists to incorporate the study of narratives into their predictive models.  He believes the stories we tell each other that go viral are important factors in how economic events unfold.  The book is clearly written and makes its case convincingly, but there isn’t much for individuals to apply to their own lives unless they are economists or politicians.

A simple example of how narratives can cause future events is a viral story about deflation.  At times in the past, people have widely believed that prices were going to fall.  As a result, consumers delayed purchases expecting to buy cheaper later.  This caused spending to fall, and ultimately contributed to sellers lowering their prices.  Narratives can become reality.

Shiller gives many examples of different classes of narratives, including the morality of frugality and conspicuous consumption, the gold standard, automation replacing jobs, market bubbles, evil business, evil unions, and more.

Broadly, the book shows that “popular narratives gone viral have economic consequences.”  Shiller wants “economists to model this relationship to help anticipate economic events.”  He calls on economists to collect data on narratives to facilitate future economic analyses.  He also says “Policymakers should try to create and disseminate counternarratives that establish more rational and more public-spirited economic behavior.”

“When it comes to predicting economic events, one becomes painfully aware that there is no exact science to understanding the impact of narratives on the economy.  But there can be exact research methods that contribute to such an understanding.”  A further challenge is “distinguishing between causation and correlation.  How do we distinguish between narratives that are associated with economic behavior just because they are reporting on the behavior, and narratives that create changes in economic behavior?”

Shiller did a good job of convincing the reader that narratives matter in economics.  However, I have one minor criticism.  In one discussion of bankers and the poor choices they appear to make, Shiller says “It may be best to think of bankers’ behavior at such times as driven by primitive neurological patterns, the same patterns of brain structure that have survived millions of years of Darwinian evolution.”  This unflattering portrayal of bankers’ thinking presupposes that bankers intend to act in the best interests of their banks.  I find this doubtful.  Bankers as individuals all the way up to the CEO have periods of time where they make a lot of money doing things that look good in the short run but hurt the bank in the long run.  This is captured nicely in Charles Prince’s comment, “As long as the music is playing, you’ve got to get up and dance.”  Sometimes, apparently dumb behaviour is actually evil and greedy behaviour.

Overall, Shiller makes a strong case that the spread of narratives should be studied by economists.  However, readers looking for concrete ideas for improving their own investment results won’t find them because this isn’t the book’s focus.