It hit me the other day that I’ve got to get my act together if I plan to retire a second time soon.
The first attempt at retirement lasted for just under a year until I started feeling too sheepish telling anyone I was retired at 34. Although my retirement portfolio was generating about $80,000 a year in passive income at the time, I started itching for more.
Seven years later, I’m running out of steam. I’ve already conducted multiple calls with boutique investment banks, private equity shops, and larger media companies on the potential sale of Financial Samurai after its 10-year anniversary mark in July 2019.
I’ve also tentatively convinced my wife to go back to work once our son turns two years and five months old this Fall. Spending 29 months as a stay at home parent should be long enough to feel like a parent did the best he or she could without feeling too guilty chasing money instead. But we shall see when the time comes.
The final thing I need to do is make sure our after-tax retirement portfolios are generating enough income to cover our desired lifestyle just in case Financial Samurai is sold and my wife can’t get a reasonable job in a field of interest.
I feel blessed to be able to do all the things I love since leaving full-time work in 2012 – coaching high school tennis for the past three years, writing almost daily on Financial Samurai, traveling around the world, and spending time being a stay at home dad since early 2017.
But all good things come to an end. We must frequently adjust in order to keep the good times going for longer.
How To Build A Healthy Retirement Portfolio
Before discussing FIRE (Financial Independence Retire Early) and my retirement portfolio’s latest income figures, I’d like to share five tips for everyone to follow to build their own healthy retirement portfolio.
1) Save until it hurts each month. Most people think that saving for retirement in their 401(k) or IRA is enough, but it is not. In order to have the optionality of retiring early or ensuring a healthy retirement at a more traditional retirement age, it’s important to max out your 401(k) while also contributing at least 20% of your after-401(k), after-tax income to an after-tax investment portfolio.
The after-tax retirement portfolio really is the key to early retirement since most people can’t access their pre-tax retirement accounts without a 10% penalty before age 59.5.
2) Focus on income producing assets. After you’ve had your fill of high octane growth stocks as a young person to build your capital, it’s time to focus on income producing assets as you get closer to retirement. Dividend generating stocks, certificates of deposit, municipal bonds, government treasury bonds, corporate bonds, and real estate should all be considered in your retirement portfolio.
When I was younger, my favorite type of semi-passive income was rental property income because it was a tangible asset that provided reliable income. As I grew older, my interest in rental property waned because I no longer had the patience and time to deal with maintenance issues and tenants. Instead, my interest in REITs and real estate crowdfunding grew since the income generated is 100% passive.
3) Start as soon as possible. Building a large enough early retirement portfolio takes a tremendously long time largely due to declining interest rates since the late 1980s. Gone are the days of making a 5%+ return on a short-term CD or savings account. You need to save early and often to make compounding work most for you.
I knew I didn’t want to work 70 hours a week in finance forever. As a result, I started saving every other paycheck and 100% of my bonus starting my first year out of college in 1999. By the time 2012 rolled around, I was earning enough passive income to negotiate a severance and retire early.
4) Calculate how much retirement income you need. It’s important to have a retirement income goal. Otherwise, it’s too easy to lose motivation and focus. A good goal is to try and generate retirement income to cover all basic living expenses such as food, shelter, transportation, and clothing. Once you hit that goal, focus on covering your wants.
If your annual expense number is $50,000, divide that figure by your expected rate of return or comfortable withdrawal rate to see how much capital you will need to save. If you expect to earn a 4% rate of return, then you would need at least a $1,250,000 after-tax retirement portfolio, and closer to $1,500,000 due to taxes.
5) Make sure you are properly diversified. The first rule of financial independence is to never lose money. We saw a lost decade for tech stocks between 2000 – 2010 after the first dotcom bust. For NASDAQ investors, it took 13 years to get back to even. Then we experienced a housing bust of epic proportions between 2007 – 2010.
You always want to be moving forward on your journey to financial independence. The closer you are to retiring, the more conservative your investments should be. Please do not confuse brains with a bull market.
Financial Samurai Retirement Portfolio Review
Since retiring the first time around in 2012, I have yet to stress test my after-tax retirement portfolios because I received a severance that paid out enough money to live for five years.
While I was living off my severance income, my wife worked until she negotiated her own severance at the end of 2014. She is three years younger than me. Having her work and provide healthcare was very comforting and allowed me to reinvest 100% of our after-tax retirement portfolio income.
Then once both of us weren’t working full-time jobs in 2015, Financial Samurai started generating a livable income stream as well. This positive sequence of events is why planning is so important. It’s frankly why quitting your job to retire early is a suboptimal move.
Ideally, we want to live on between $15,000 – $18,000 a month in after-tax income to live our best lives while raising one or two children in expensive San Francisco or Honolulu. Using a 28% effective tax rate, we’re talking a target $250,000 – $300,000 a year in annual gross retirement income.
Here’s our latest source of income streams to fund our second retirement.
As you can see from the chart, we generate about $16,300 a month in after-tax retirement income if we use a 20% effective tax rate. The effective tax rate for investment income is lower than W2 wage income. Something to think about when forecasting your own retirement income needs from investments.
$16,300 a month or $195,600 a year in after-tax retirement income should be more than enough to provide for our current family of three as our all-in housing cost is less than $6,000 a month. Once all our housing cost is covered, our costs for food, transportation, and everything else aren’t too bad.
$16,300 a month will also allow us to continue saving at least 30% a month for a rainy day (~$5,000). Because we’ve been in the habit of saving at least 50% of our after-tax income since graduating from college in 1999 and 2001, respectively, it would feel foreign to not continue saving in retirement.
The main anticipated increase in cost is preschool tuition starting this Fall at $1,800 a month. The other potential increase in cost is if we are blessed with another child.
If we stay in San Francisco long term, our goal is to send our boy to public school after preschool if he can win the SF public school lottery system. If our son does not get into a reputable public school near by, then we’ll be forced to spend about $3,000 a month for elementary school and likely $5,000 a month for high school when the time comes.
These potential grade school tuition costs are the main reason why I’m striving towards $18,000 a month in after-tax retirement income, or ~$2,000 a month higher than current levels. I’ve got three years left to make this goal a reality.
Below is an analysis of the major retirement income categories.
Risk-Free Savings: $1,045/month (5% of total)
I love risk-free savings, especially after the Federal Reserve hiked interest rates multiple times since the end of 2015.
To be able to earn ~2.3% risk-free after making massive gains in the stock market and real estate market since 2009 sparks joy! Gone are the days of pitiful 0.1% savings interest rates.
My target is to always have between 5% – 10% of my retirement income and net worth in risk-free investments. You just never know what might happen in the future.
Stocks & Bonds: $7,560/month (37%)
After a tremendous rebound in the stock market in 2019, I decided to asset allocate more towards 3-month treasuries in my main House Fund portfolio.
As of now, my House Fund portfolio is roughly 20%/80% stocks/bonds because my plan is to buy another property within the next 6-12 months.
The House Fund portfolio had a nauseating $400,000 swing (-13%, then +23% so far) and I want to ensure that I protect the principal going forward. My other main public investment portfolio is closer to 60% stocks / 40% bonds. I plan to gradually shift the weighting closer to 50%/50%.
Below is my public stock and bond portfolio performance +9.2% vs. the S&P 500 +15.9% year-to-date according to Personal Capital’s performance tracker. With the income from my existing bond holdings, I should have relatively no problem closing out a 10-11% total return for the year.
As I edge closer towards retirement, my main goal is to minimize volatility and try and achieve a 5% – 7% total return equal to 2-3X the 10-year bond yield. 2018 was a positive year, +2% vs. -6.4% for the S&P 500. But I was up closer to 11%. Such volatility is unwelcome.
Real Estate: $6,550/month (32%)
Real estate used to dominate my retirement portfolio income (~60%) until I sold a significant SF rental house in 2017 for 30X annual gross rent.
I ended up reinvesting $600,000 of the proceeds in mostly dividend-paying stocks, $600,000 of the proceeds in mostly municipal bonds, and then $550,000 of the proceeds in real estate crowdfunding ($810,000 total) in order to not lose too much real estate exposure.
I did get a surprise $45,598.04 distribution on 4/16/2019 from the RS DME fund where I have a total of $800,000 invested. The fund has 17 investments, across 12 states, and 6 property types. My Class A Austin Multifamily property was sold for a 24.6% return over two years.
So far the fund is returning a 10% cash-on-cash return net of fees. I’m hoping the end IRR is much higher after the equity investments are sold within the next 2-3 years.
For retirement portfolio calculation purposes, although I received $45,598.04 in distribution, I’m only inputting the profits as passive income to stay conservative. Perhaps there will be another significant distribution later in the year.
Once about half my RS DME fund distributions are returned, I will look to reinvest about $300,000 in a couple Fundrise eREITs to further diversify my holdings and platform exposure.
So far I like the simplicity of investing in a real estate fund versus spending time trying to pick the best deals. But if I’m going to retire again, I’ll have more free time to do research on individual investments.
My goal is to always have at least 30% of my net worth exposed to real estate as it is my favorite asset class to build long term wealth.
I haven’t raised the rent on my SF 2/2 condo in almost three years. At $4,200 a month, the property is now under market value by $400 – $500 a month. But I plan to just keep the rent the same because they’ve been good tenants. I’ll wait until one or both decides to move out before raising the rent.
Our Lake Tahoe property is coming back to life! We’ve had a fantastic winter in 2018/2019, which has resulted in a roughly doubling of net rental income over last year.
As the storms have subsided, we plan to finally take our boy up to the mountains. Spending time with my own family has been a dream of mine since I first bought the property in 2007.
Alternative Income: $5,220/month (25.6%)
Online books sales for How To Engineer Your Layoff has steadily increased each year since the first edition was published in 2012. I wrote a new foreword for 2019 and updated some data.
My wife has spent the past four months updating the book for another edition launch in 2H2019. The new edition will have even more case studies and strategies to guide people to better negotiate a severance. We will likely raise the book’s price by 15% as well.
The amount of positive feedback we get from readers who’ve successfully negotiated their severance has been tremendous. If you plan to retire early, it behooves you to try and negotiate a severance. You have nothing to lose.
To generate $50,400 a year in almost passive online income from a book would require amassing a $1,008,000 portfolio generating a 5% return. Not needing to have startup capital is one reason why I’m so bullish on building online real estate. There is almost no risk except for putting your education and creativity to use.
As for my venture debt investments, I’m still waiting to get paid in full for my first venture debt fund from five years ago. The second venture debt fund just did a 25% capital call for a total of 92% of the capital committed. Depending on the final investment in the first fund, the IRR is going to be anywhere from 5% – 16%.
Finally, I invested in my first venture capital fund. This is a 10-year, $600 million fund by Kleiner Perkins where I don’t expect to see any income until perhaps year five. The main partner has a good track record and is a friend of a friend.
Enough To FIRE
Based on this deep-dive analysis, my wife and I should have enough to live a comfortable retirement lifestyle in San Francisco or Honolulu.
Keeping lifestyle inflation at bay while steadily growing our various incomes streams has been key to building our retirement portfolio. I’ve been wanting to buy a fancier house for the past couple of years now and have chosen not to so far.
What I find most interesting is that even though mathematically I shouldn’t have a problem retiring, I still have trepidation about selling Financial Samurai and retiring again.
Change is always hard, especially after you’ve spent a decade doing one thing. Giving up a steady income stream is also scary when you’ve been through the 2000 dotcom bubble and the 2009 financial crisis and now have a family to support.
Eventually, we’ll need to start spending our retirement portfolio income. But as of now, we plan to continue reinvesting 100% of our investment income and saving 80% of our active income until a retirement decision is made.
Readers, any of you planning to retire soon? If so, what type of deep dive retirement portfolio analysis have you done to ensure that financially everything will be OK once you retire? Do you see any holes in our retirement portfolio we need to work on shoring up? Featured art by Colleen Kong-Savage.