The Biggest Problem With Being A Long-Term Investor

Everybody knows that being a long-term investor is the way to go. The longer you can stay invested in the stock market and real estate markets, the greater your chances of making more money.

There's just one big problem with being long-term investors. We eventually all run out of time. And if we run out of time before enjoying our investment gains, we will have essentially wasted a lot of our time and energy while we were still alive.

A double kick to the groin!

Doing something you're passionate about, like Charlie Munger did until age 99 is amazing. The man was a legend. But dying with an estimated $2.3 billion net worth is not.

S&P 500 Best & Worst Returns Over Various Periods Of Time

Below is a great chart that highlights the returns of the S&P 500 over 1 year, 3 years, 5 years, 7 years, 10 years, 15 years, 20 years, and 30 years. What do you observe?

S&P 500 Best & Worst Returns Over A Various Periods Of Time - The importance of being a long-term investor

Here's what I observe from the S&P 500 returns chart:

  • There are a lot bigger swings on the upside and the downside in the short term
  • As time goes on, the upside and downside swings get shallower
  • The longer you invest, the lower your compound average returns
  • The longer you invest, the greater your percentage chance of not losing money
  • After investing for 15 years, you have never lost money between 1926 – 2022
  • After 30 years of investing in the S&P 500, the lowest compound return percentage was 8% compared to the highest at 13.6%

Based on this chart, our mission should be to invest in the S&P 500 for as long as possible. The minimum investment duration should be 10 years and the ideal investing duration should be 30+ years.

How Old Will You Be In 15 Years?

Investing for the long term is good advice when you're in your 20s, 30s, 40s, and 50s for most people. But what about the people who want to live it up before their traditional retirement age of 65?

Ironically, investing in the long term might be too risky.

In 15 years I will be 61 years old. I will feel sadness because my boy will be 21 and my daughter will be 19. They will most likely have already moved out of the house, leaving my wife and I alone to contemplate all the struggles and good times we had.

I'll never get these 15 years back which is why I'm doing my best to live in the now. Time is too precious to waste it doing things I don't want to do.

However, due to the lack of unlimited funds and a lack of steady income as a fake retiree, I also must invest carefully to at least keep up with inflation. The pressure to provide for my family is strong.

Made All The Mistakes As A Long-Term Investor

Not only am I a long-term investor, I've also invested for a long time.

I've invested in stocks since 1995, my freshman year in college, when my dad opened up an Ameritrade account for me. I've made tons of mistakes over 28 years, including day trading too much, panic selling, and FOMO buying.

Whatever investing mistakes there are, I've done them all!

It was only after my senior Managing Director sat me down and questioned why I was trading so much that I finally settled down and started investing for the long term. This was at age 30, in 2007, at the top of the previous bull market.

By investing in the long term, I then proceeded to lose about 50% of my portfolio's value during the ensuing 2008 global financial crisis! Darn. Should have been a short-term investor and sold everything in 2007!

I'm No Longer Enthusiastic About Investing In The Long-Term

Today, I don't want to invest and not touch my money for 15 years. There's probably a 10-20% chance I won't live until 61. What a shame not to have enjoyed my wealth while I was still alive. But knowing me, I will continue to save and invest for the future.

Below is the average life expectancy chart by race from the CDC. As an Asian person, I'm expected to live to 83.5. But who really knows! Once your health starts deteriorating, things can get bad quick.

With shorter life expectancies post pandemic for all races, one's investing timeline should also be shorter by 2-6 years.

CDC life expectancy chart by age

Protect Your Loved Ones

If you're looking for life insurance due to having debt and/or dependents, check out PolicyGenius. At PolicyGenius, you can get customized life insurance quotes in one place. 

Both my wife and I got matching 20-year term life insurance policies during the pandemic at an affordable price. After we did, we felt tremendous relief knowing that our kids will be financially secure if something were to happen to us. 

It's OK To Stop Investing Once You've Reached Your Goal

I'm all for selling stocks once you've made enough to buy what you want. Same thing goes with selling an investment property or whatever your risk asset of choice.

Even though there's a high probability your stocks and real estate will continue to go up after you sell, unless you sell everything, you will unlikely regret converting funny money into something real or an amazing experience.

Everybody should invest for a reason. If you do, it will make investing much easier in the long term.

Some common reasons to invest include:

  • a primary residence
  • to pay for college
  • to buy a car
  • to fund a traditional retirement
  • to retire early

You could keep renting so your investments can continue to grow, but you may be sacrificing the quality of your life while you wait. You could also keep taking the bus so your investments can continue to grow, but at some point, the inconvenience may no longer be worth it.

Perhaps the grandest goal of investing is to generate enough passive income to retire early. Investing for 20 years so you can live free for the rest of your life sounds like a good trade!

But along the lines of investing for too long, some people will work far beyond what's required to live happily ever after. Finding the balance is tough! I'm still trying.

Your Investing Time Horizon Should Shorten As You Age

One of my strongest beliefs is that it's better to retire by a certain age than a certain financial figure. The reason why is because there's always another dollar to make but never another second of time.

By keeping your retirement target age fixed, your investing time horizon should shorten. Here's an example.

1) You're 22 years old and want to retire at age 50.

Your investment time horizon is 28 years. With such a long time horizon, you are free to take more investment and career risks.

Perhaps you invest 30% of your public stock portfolio in individual companies. You know active investing tends to underperform, but you're also looking for the next multi-bagger stock. After all, you can't outperform the market if you invest everything in the market.

Instead of investing in bonds, you invest in real estate, a bond plus investment. You want more upside when times are good while also being able to take action to protect your investment when times are bad.

2) 13 years later you're 35 years old.

Your retirement age stays the same at 50, therefore, your investment time horizon is 15 years. Phew! After reading this article, you know that if you invest in the S&P 500, 15 years later you are most likely going to come away with a positive return.

Thanks to your financial diligence, your net worth growth rate has surpassed expectations. They certainly surpassed all your peers who spent too much money on cars, eating out, and vacations.

3) Five years later, you're now 40 years old.

At age 40, you're established in your career. You're in your peak earning years, however, you're beginning to tire. You often ask yourself questions such as, “What's the point of working so hard if I'm not enjoying my money?”

As an accredited investor, you're now investing in private funds with 5-10-year time horizons. Although the fees are higher than investing in a S&P 500 index fund, you like diversifying into investments that are staying private for longer in order to capture more of the gains.

You allocate up to 20% of your investments into venture capital, venture debt, and private equity. You're not like the Yale Endowment Fund, with over 70% of its investments in private investments and alternatives. But you see its merits.

4) 10 years later, you're now 50 years old.

Congrats! You've invested for 28 years and have experienced a 14% compound annual return. You are now a multi-millionaire who can retire early if you want to. Investing in private funds that won't return capital for 10 years is now getting a little risky for yourself, but not for your family if you invest through a trust.

One irony you realize is that the longer you invest, the lower the risk of losing money. However, given you want to live it up more now, you are OK with selling off some of your investments and paying capital gains taxes.

You've also discovered something peculiar after 28 years of investing. It's damn hard to spend instead of invest! In addition, due to your frugal habits, you are finding it impossible to decumulate enough to die with little.

Your Target Retirement May Change Several Times

Stick to retiring or doing something new once you reach your target retirement age. It's easier said than done, but you must try. Because if you keep working and investing past what you need to live comfortably, you may look back on your life with regret.

The only problem is your target retirement age might change multiple times in your life.

At age 22, I wanted to retire by age 40. However, due to being able to negotiate a severance that paid for six years of normal living experience, I retired at age 34. To me, the severance package bought me six years of time, which is worth far more than the severance check itself.

I experienced several years of traditional retirement until our son was born in 2017. Then our daughter was born in 2019 and then the pandemic hit in 2020. The pressure to make more money and protect my family increased. Stuck at home, there were fewer things to do, so I decided to make more money online.

Today at 46, I want to re-retire again age 50. This means feeling little-to-no stress about my finances because I truly have enough. But before I try and truly retire by 50, I first need to go back to work!

Growing Bills To Pay

I don't feel stress-free about our finances today because I see two private tuition bills looming in Fall 2024, ever-rising healthcare premiums, and a new house with property taxes and maintenance expenses. My household expenses are growing faster than the rate of inflation.

Although my target retirement age has changed, at least I experienced some truly relaxing years since 2012. Hence, perhaps one of the keys to a happier career is taking sabbaticals throughout your career.

S&P 500 total returns and percentage of time positive over various timeframes durations

Be A Long-Term Investor For Your Children

What keeps me from completely YOLOing away all my money are my children. I'm their secret weapon because they are too young to understand the power of compound growth.

By investing in stocks, real estate, and private growth companies for them today, I'm giving them a head start. I know with 90%+ certainly that in 20 years, they will have wished they could have invested today.

Don't you wish your parents and grandparents bought blue chip stocks and more prime real estate when they were young? You bet your buns of steel you do!

Again, look at the chart above. The best 20-year stretch in the S&P 500 between 1926 – 2022 showed 17.7% compound annual growth rate. Not bad at all!

If you have children and are thoughtful, it's impossible not to be a long-term investor. But if you're single, you may be more inclined to invest for the short-term and live it up more today. Enjoy!

Reader Questions

Are you a long-term investor? If so, how many years do you define as long term? How do you adjust your investment time horizon as you age? If you have children, do you feel a heightened responsibility to be a long-term investor?

If you're looking to invest in the long term, consider diversifying into private growth companies. Private companies are staying private for longer, meaning more of the gains are accruing to private investors.

Check out the Innovation Fund, which invests in AI, modern data infrastructure, development operations, financial technology, and prop tech. Roughly 35% of the Innovation Fund is invested in artificial intelligence, which I'm excited about. In 20 years, I don't want my kids wondering why I didn't invest in AI or work in AI.

The investment minimum is also only $10. Most venture capital funds have a $250,000+ minimum. You can see what the Innovation Fund is holding before deciding to invest and how much.

29 thoughts on “The Biggest Problem With Being A Long-Term Investor”

  1. Thanks, Sam! I needed this post for inspiration. My son (and possible future children) will be happy for the head start I can provide them today. Happy new year!

  2. Sam provides some data for S&P periodic returns, and reaches several conclusions. Are most, or all, of them correct? Maybe not.

    Let’s take his “The longer you invest, the lower your compound average returns.” Not according to my calculations.

    Let’s average the highest and lowest returns for the various time periods.
    For 3-year time periods, the average of +30.8 and -27.3 = +1.75%. Doing the same up through the 30-year period yields +7.8%; +10%; +9.2%; +9.3%; +10.05%; +10.8%. So it seems that the yields sort of flatten, or maybe go up a tiny bit through to the 30-year period. But they definitely do not decrease.

    Don’t like my original numbers – well you can’t dispute them, since they have been posited as being the actual data. Don’t like my simple method of averaging, I don’t care, show me where a simple average calculation is incorrect. You’ll be hard pressed to do so without introducing more numbers (i.e. more information).

    The +10.8% average (expected return) for the 30-year periods is (un)remarkably similar to the long-term S&P returns since 1928, or since 1950.

  3. You said in one of your previous posts: “I once bought an Isaac Asimov book for under $40 in the late 1990s that is now worth over $1,000. If only I had bought 100 copies back then, I’d be rich, happy, and free today.”

    Not to worry. I just bought a new, original, Asimov, published several months ago. Title is “AI, Robot”. What goes around, often comes back around. How could it even go wrong! It might be the future (demise?) of humans.

  4. Hi,

    maybe it would make more sense to do this analysis on a real rather than nominal basis… yes, of course, nominally, if you invest, more often than not, you end up with more dollars. But can those dollars buy the same stuff as before?

    Best

  5. Hi Sam,
    I am ready to retire from my current job after 26 years with some medical benefit. I am 53 years old and planning to get at least a part time different job after I retire. I heard from a good friend that she bought an annuity and the terms were great guaranteed 8% return and also some bonus funds added. What do you think about annuity? Is it something worth buying right now since the rates are high as a retirement guaranteed income. If yes what type of annuity? I am not a high risk investor specially since I will be retiring soon. Thanks.

  6. Sam – always GREAT content. One thought on your Long Term Investor post.

    I am making an assumption or two here – which may not be accurate.

    * Charlie Munger and Warren Buffett have access to the very best estate planning talent available. My guess is that, at the age of 99, that he has previously taken the necessary steps to gift to his kids and take care of family to the extent he is comfortable. Some people do a lot of this and some less so. I also willing to bet that he signed the Giving Pledge – Warren has – to donate a significant amount of his wealth to charity.

    Thus, he lived a non-flamboyant life without obnoxious spending – same with Warren. He will now be able to share this wealth with charitable organizations he deemed worthy. This is a very noble step that is often taken at the $10mm+ level of net worth. Thus, just because he did not draw it down for lifestyle does not mean he did not enjoy the intrinsic joy of helping others – ergo, spending as he saw fit without personal deprivation.

    While many in the retirement space are a bit too frugal in relation to their asset base, spending is what many people do not want to do – they enjoy low/no cost activities or establish a charitable objective. Spending does not equal joy in retirement even if one can afford to do so.

    So, perhaps another approach to a conservative spending mindset.

    Jim

  7. This is definitely a blog that leans toward real estate for wealth accumulation. And I get it. If you do it right your returns will exceed stock market indexing. But I have never owned other than a primary residence and have reached a significant net worth just saving and investing. And no external hassles (hiring people, fixing things, mortgages….), just some internal tuff watching worth drop during market downturns. To me that is an important tradeoff. If you do your studying and young enough just buying opps.

    Its not just about return % between one investment vehicle and another. With each vehicle comes a level of emotional and physical investment. Stock indexing costs you nothing in time and effort and upfront costs – just sit back and watch it grow.

      1. How do you enjoy a rental? It would just be for the investment correct? Just like a stock but you need to maintain. Totally get “enjoy” with the primary residence you live in but people pretty much only do rentals for investment correct?

        1. Sure. There’s the joy of seeing it and knowing it can serve as affordable housing for my children. I see owning rental properties as a way for my kids to earn income doing something and / or a place to stay. I have space in my rental property for when my family visits. It’s only two blocks away and it’s so convenient.

          But of course, there is downside regarding managing it.

          It is due to rental property that I was able to leave work at 34 in 2012.

  8. I’ve enjoyed your posts. Your plans are too light on real estate. Mess around with chat gpt as I did. I asked it what was sp500 appreciation 1990 to 2021 for a 48k investment. I received 764k in 2021. A cagr of 9.37%. Not bad. If instead I instead the 48k as a 20% down payment on a 240k four plex same year will be almost 3x as much taking into account rent increases for inflation and tax shielded rental income. That’s for a orange county or LA county bldg. Takes no account of ability to refi along the way to lower rates, cash out and buy another bldg, or start depreciation schedule all over 1031 which ended recently. Also many more day to day deductions available. Unless you are a very technically savvy options or other stock trader, you’ll get better results elsewhere.

    Good Luck

    CB

    1. Thanks for your wisdom. Do you suggest I take on leverage at this stage in my life to try to make more money? What percentage of my net worth do you recommend I have in real estate?

      At this stage, I am struggling with wanting to make a lot more money, so your perspective on how much is enough how much I should shoot for would be great.

      How did you motivate yourself to spend time making more money once you reached your target net worth?

      1. Sam,
        I’m kinda in coast mode, now 56. If I see a deal I want, I’ll go after it. Real estate phases have longer periodicity than stocks; as such, the variables influencing price tend to have signals sticking around longer than a stock buy, sell signal options traders are familiar with and there’s more options along the way both tax and finance wise. I don’t flip, I buy, fix (have people) and hold. Best to consider these like trees, plant now and harvest (or milk) later. Can’t tell you how much to put in, but if you find say a 4plex with a down payment you’re comfortable with producing decent cash flow, you’re good. Run the numbers as if it was an fha purchase. I buy stuff I’d live in and have very few worries. As with anything, only wager what you can lose. You might read Milt Tanzner’s “Real Estate Investments and How to Make Them”

        You also have incorporating strategies and tax plays here.

  9. I am definitely a long term investor. I invested my savings straight out of university in real estate doing up old houses and renting them out and later in ETFs. My initial plan was to retire at 40 but I decided to retire last year at 37 as I had reached my initial target portfolio amount. But a few weeks from my retirement date, my heart malfunctioned and I was in surgery. Thankfully, I was still in employment and had valid insurance but I still had to pay for some things which made me realize just how expensive healthcare can become. Nearly 2 years on, my heart has recovered well and I do not need all the follow up checkups and scans anymore. I have been back at work for more than a year now, just turned 39 and have started to enjoy spending a little of my money on things I like a new car, a big expensive birthday party with all my friends and family and updating my house to be a bit more comfortable. I have also extended my retirement date to 4 years from now for a few good reasons : the extra medical costs ate some of my portfolio and also I needed a bigger portfolio to cover healthcare costs not covered by insurance. But I am doing the next 4 years in style and enjoying myself a little more just in case.

    1. Glad your heart is doing better! Working for a reason, such as providing extra security to pay for your health, will make working so much more palatable and enjoyable.

      Best of luck!

  10. I’m a long-term investor and five years is the minimum time frame. You’re the first blogger/author I’ve read that talks about decumulation and as a Christian, husband, and father of four (two in college, two in high school), I think the principle is crucial. The biggest adjustment I want to make now is to set an example of generosity for my children and others, which is very difficult as the child of Chinese immigrants – as you know, we are taught frugality from a young age. I’m now learning to spend money on experiences for myself and my kids – something my parents never modeled for me. I don’t feel heightened responsibility as a father to invest; I feel tremendously blessed to live in the bay area and to have done well over the past fifteen years investing in individual tech stock. I do have some regret that I wasn’t more generous in the past and thoughtful about spending. For example, remodeling my house when my kids were younger or perhaps sending my kids to a private high school.

  11. JAMON_GRANDE

    You needed to invest in income producing assets such as commercial real estate. As rents increased, and mortgage rates decreased you’d be a winner. Done right the income is tax free during your lifetime and provides write offs. If you want to surprise yourself, run my friend chat gpt, look at return of a 70k from sp500 1993 to today. Run same simulation as if you put 70k into an L A or OC apt bldg, at 30% down. See the value of that bldg today; you’ve long ago amortized that principal. That 4plex is 1.6, 1.8 million today generating 110, 90k or so after ptax, insurance. Sp500 is round 570k. You could sell that 1,8 million asset into a deferred sales trust then put into stock funds, dividends, ect at say 5% a year, 90k pay effective tax rate,still have principal. Or keep it as real estate and enjoy rent increases, relatively smaller taxes.

    1. Why are you comparing one building to the SP500? You should be comparing a high flying stock vs particular building and see how that comparison works.

  12. While I agree that one should ease off a tad from investing if one has reached their ideal $$$ target with enough finances until 95 (yes, I’m placing 95 yrs for my projections. My mother is 89, father died at 85. I live in Canada.), the hard part is inflation and also long term care facility costs.

    It is strange to have several billion dollars as Munger had left behind in last 24+ hrs. And something slightly sad if he didn’t spend it even ostensibly in past few years, for his loved ones. (in addition to his legacies for them). Here we are sitting, as a bunch of strangers wowing over his fortune….which doesn’t mean much to us. Not really since we can’t access it. For sure he would have donated something.

  13. Been thinking a lot about after tax investing, as I am only invested in tax deferred accounts with about 12 years left until retirement. Wife and I will both have pensions, so 403b accounts seem a bit like play money. That said, in terms of after tax investing (at 51), would it even make sense to start investing in after tax accounts as a high income earner at this stage? Wouldn’t it make more sense to just allocate more to tax deferred accounts (max retirement accounts + what a SEP IRA will allow us to add to that instead of after tax) to take advantage of the current tax benefits and with a relatively short timeline before the funds can be accessed? I feel a bit like the after tax ship has sailed once you are in your 50s.

  14. I am all for being frugal as well saving and investing as much as I can (I’ve been investing 100% of my gross work salary for the past few years and plan to continue this pattern until I retire (thanks to semi-passive rental income on the side for making this possible!) . However, you mentioned “Thanks to your financial diligence, your net worth growth rate has surpassed expectations. They certainly surpassed all your peers who spent too much money on cars, eating out, and vacations.” I’ll never fault people for spending on vacations! What are we all saving for if not for being able to enjoy ourselves with family and friends on vacation?! A disproportionate amount of my best memories are from time traveling abroad in my youth and family vacations as I’ve aged. If you can’t splurge a little bit along the way, it’s a painfully boring journey to FI.

  15. Yes I consider myself a long term investor. It’s funny you ask what we each define as long term because I’ve always treated it like “forever.” My investments are all intended for the way way future when I have no more income coming in and need to start drawing down. But I’ve been saving and investing for so long I think it will feel uncomfortable to go into reverse, ie withdrawing and no longer investing. I’m sure I’ll get used to it but it is strange to think about because my habits are so engrained.

  16. I’m definitely a long term investor. In my post tax account I’ve been buying a S&P index fund weekly since 91. I’ve never sold a share. I will absolutely die with to much money. I live a very comfortable lifestyle but if people knew my net worth they would question my current choices. My house is 20 years old. My vehicles are pretty standard. I don’t have a bunch of toys. I do fly first class and I have no problem spending any amount of money on good food. I love to gamble but my yearly gambling budget is .01 percent of my net worth. I do a couple nice golf and fishing trips a year but nothing extravagant.

    The biggest deterrent to me spending more money is the hassle of owning “stuff”, the hassle of managing people to do “stuff” for me and a lifetime of being conservative managing my money. I don’t see this a problem. My wife and daughter will always be taken care of no matter what happens. My grandkids, if I’m lucky enough to have some, will have many opportunities because of my wealth that the average person doesn’t have, and its quite possible some charities will be given a life alternating sum of money that will help kids I don’t even know. That is enough for me.

  17. No reason to feel sad because the kids grow up and move out (frankly, it would be sadder if they did not). Eventually, grandkids show up and that’s great, you can do fantastic things with them, then give them back to their parents and go do fantastic things with your spouse and friends.

    The chauffer-father in the movie, Sabrina, chose to be a chauffeur because it gave him lots of time to read. I understand that, although my own ambition was a bit too high for that. All the same, I found working in IT to be both lucrative and low stress, giving me the time I wanted to pursue other interests.

    Of course, then you wonder if you didn’t set your goals high enough and apply yourself enough. You have to get past that. It’s not any one person’s sole responsibility to cure cancer, be the first person on Mars, end global warming, or corner all the markets on Wall Street.

    As I’ve advanced our wealth into millions I’ve discovered that our lifestyle doesn’t demand to keep pace, which is great. It just becomes very comfortable. Want an 86’ tv? Order it on a credit card and pay it off on the next statement. Ooh, look, a new laptop and a GoPro? Throw them in too, please. They will be handy in Tahiti. But then we don’t live in a mansion. We drive older and or used cars (as long as it starts and isn’t embarrassing to be seen in, we just don’t care). Sometimes we rent a fancy car when traveling–recently we experienced our first Tesla, despite the fact we’ve had a fair amount of money invested in Tesla for a long time. Then too, my wife favors costume jewelry, mostly, and we buy our clothing online, mostly. We don’t eat out a whole lot either, not because we can’t afford it, but because we like to control what goes into our food, and keep some reins on calories as well.

    Further, seems like the more wealth we accrue, the less we need to spend it. Seems like lots of opportunities come our way simply because we have wealth, not because we are hellbent on spending it.

    If my wife or I should live considerably past Charlie Munger’s time (and that may or may not be unlikely, depending on medical science), we too will likely eventually reach billionaire status (through the power of long term investments—it’s gotten us this far). I would have no problems leaving an estimated $2.3 billion, in 2023 dollars, to my heirs (I would have an awful lot of them by then), because I know I will not have stinted on affording my wife and myself the life experiences we most want.

    But my primary goal when I started was to never be a financial burden on my kids, nor wind up eating cat food, or living alone in a nursing home (public or private). And never, ever, to have to worry about which would run out first, my money or my life.

  18. Pedantic Kevin

    ‘The longer you invest, the lower your compound average returns’….. I’m not sure I understand or agree with this bullet point. Maybe if you are only comparing the best 30yrs against the best 1yrs….. But your average compound annual return over a long period will be the same as the average over a short period, won’t it?

    1. The bar graph is demonstrating standard deviation more than it is probability of returns. In the short term, there’s a wider range of outcomes than in the long term. Even that doesn’t guarantee your portfolio value won’t rise and fall with the market; but during a drop, the older shares are likely to still be in the black.

Leave a Comment

Your email address will not be published. Required fields are marked *