One of the key conclusions from doing a deep-dive analysis of my IRA with Empower, is that I should be able to live it up more in retirement. In fact, we should all be able to live it up more in retirement based on a higher recommended safe withdrawal rate by Bill Bengen, one of America's best retirement researchers.
Since 1999, I’ve always viewed all my tax-advantaged accounts as bonus money. My philosophy was simple: by not counting on these accounts to fund retirement, I’d be forced to build my taxable investment portfolio large enough to support an early retirement lifestyle. At the same time, by automatically maxing out my 401(k) every year, I’d ensure that life after 60 would be even more comfortable than if I hadn’t.
Yes, it can be hard to max out your 401(k) every year and expect nothing in return for decades. But early in my career, I realized there was no way I could last 40 years in banking with those hours and that level of stress. So I chose the easier of two hard paths: save aggressively and buy my freedom sooner.
Being Conservative And Living It Down In Early Retirement
Of course, when you retire at age 34, “freedom” still comes with limitations. Any withdrawal from a 401(k) or IRA before 59½ faces a 10% penalty plus taxes, so I wasn’t about to waste what I’d worked for. Instead, I devised five strategies for early retirement:
- Negotiated a severance package to cover living expenses for the first several years.
- Built multiple passive income streams to at least cover my basic living expenses.
- Earned supplemental income through Financial Samurai and occasional consulting.
- Encouraged my wife to work three more years before retiring herself at at 35.
- Cut expenses – most notably by downsizing homes in 2014 and renting out the old one for more semi-passive income.
At 34, I’d just eliminated a major source of income and was worried I’d made a huge mistake. Therefore, it was only logical to be conservative in early retirement just in case.
In hindsight, I probably should’ve worked five more years. But fear of failure kept me disciplined, and by 2015, our finances had stabilized enough for my wife to also negotiate a six-figure severance and retire too. We took the leap of faith because we no longer wanted to spend time doing something we disliked with our short lives.
Now It’s Finally Time To Live It Up
After another conversation with Bill Bengen, the father of the 4% Rule, I’ve decided it’s finally time to YOLO in retirement. I’m tired of always being so frugal and never allowing myself to spend on a few luxuries.
In his latest book A Richer Retirement, Bengen raises his SAFEMAX withdrawal rate from 4.15% to 4.7%, rounding up to 5%. His model assumes a 55% equities / 45% bonds portfolio – fairly conservative compared to my 99.8% equity-heavy IRA. A 5% SAFEMAX is considered the maximum annual withdrawal rate where a retiree won't run out of money after 30 years.
A shift from a 4% to 5% withdrawal rate increases spending power by 25%. That’s like going from spending $60,000 a year to $75,000 on a $1.5 million portfolio, without running out of money. And that doesn’t even factor in Social Security or side hustle income, both of which improve your odds dramatically.
Since 2012, I haven’t touched my retirement principal. In fact, I’ve saved and invested roughly 30% of my supplemental income each year. For example, over the past decade, I've contributed an average of $16,000 annually into my Solo 401(k). The freelance income comes from the occasional consulting and book advance income.
You can listen to my conversation with Bill on Apple or Spotify, or click the button below. Your positive reviews are appreciated as each episode takes hours to record, edit, and produce. Let me know if you're convinced that we should live it up more in retirement after listening.
Old Enough To Not Be So Frugal Anymore
What’s ironic about life is that the “old person” we used to imagine 20-30 years ago is now us. When that realization hits, it’s worth asking: did life turn out the way we hoped? If not, what are we waiting for?
At nearly 50, I don't want to regret not living it up to the max. I’ve had 13 years to experience the ups and downs of life without a paycheck or benefits. From paying $2,500/month for unsubsidized health insurance to finding creative ways to keep contributing to tax-advantaged accounts, early retirement hasn’t always been easy, especially when we became Dual Unemployed Parents to two kids. But it looks like we’re going to make it without having to return to work.
With fewer years left to fund, being a near-50-year-old retiree is far easier than being a mid-30s retiree. You’re more experienced, more grounded, and less anxious about all the unknowns. That said, I still have 18 years until my youngest graduates from college. Then there are my parents—and everyone’s health—to think about.
Your Retirement Portfolio Will Likely Keep Growing
After 13+ years of leaving my principal untouched, my retirement accounts have grown meaningfully alongside the markets. If I’d put my entire $3 million net worth in the S&P 500 in 2012 and withdrawn a steady-state $120,000 a year, the portfolio would be worth about $13.5 million today. That’s how powerful compounding can be. Meanwhile, Bill's SAFEMAX research assumes the withdrawal rate increases each year with inflation.
| Year | Start Balance | Withdrawal | S&P 500 Return % | End Balance |
|---|---|---|---|---|
| 2012 | $3,000,000 | $120,000 | 16.0% | $3,340,800 |
| 2013 | 3,340,800 | 120,000 | 32.4% | 4,257,939 |
| 2014 | 4,257,939 | 120,000 | 13.7% | 4,710,691 |
| 2015 | 4,710,691 | 120,000 | 1.4% | 4,648,859 |
| 2016 | 4,648,859 | 120,000 | 12.0% | 5,090,784 |
| 2017 | 5,090,784 | 120,000 | 21.8% | 6,051,854 |
| 2018 | 6,051,854 | 120,000 | -4.4% | 5,665,569 |
| 2019 | 5,665,569 | 120,000 | 31.5% | 7,279,067 |
| 2020 | 7,279,067 | 120,000 | 18.4% | 8,445,000 |
| 2021 | 8,445,000 | 120,000 | 28.7% | 10,685,715 |
| 2022 | 10,685,715 | 120,000 | -18.1% | 8,670,573 |
| 2023 | 8,670,573 | 120,000 | 26.3% | 10,783,444 |
| 2024 | 10,783,444 | 120,000 | 15.0% | 12,285,460 |
| 2025 | $12,285,460 | $120,000 | 10.0% | $13,550,006 |
- 5% withdrawal rate: ~$10 million today from $3 million in 2012
- 7% withdrawal rate (average of 400 retirees Bengen originally studied): ~$4 million today
Return Profile Of A More Traditional Retirement Portfolio Structure
Of course, I didn’t have the guts to go 100% equities when I left my job. We had recently gone through the global financial crisis and I was still highly uncertain about the future. So here’s what the results look like using a more realistic 60/40 retirement portfolio with real 2012–2024 60/40 returns (~8.2% average) and a projected +6% in 2025:
| Withdrawal Rate | 2025 Ending Balance |
|---|---|
| 4% | $5,959,300 |
| 5% | $5,146,696 |
| 6% | $4,438,007 |
| 7% | $3,820,844 |
Even with a balanced portfolio and regular withdrawals, the principal still doubled from $3 million to $6 million at 4% after just 13 years. So a 5% withdrawal rate doesn't seem unreasonable, as I'd still end up with a ~70% higher net worth 13 years later!
And if I live for 50 years after retiring in 2012 and withdrawing at 4%, my net worth grows to a whopping $38 million nominal using a 8.2% annual return (historical 60/40 annual return), or $12-$13 million inflation-adjusted real value. Therefore, clearly, if historical return assumptions of a 60/40 portfolio hold true, then a 4% SWR is too conservative.
Please send the running your own retirement analysis with a free financial tool or with a financial professional. The results are eye-opening.
Retirees Have The Ability To Adapt To Hardship
It’s been an incredible run since 2012, fueled by one of the most powerful bull markets in history. Sure, we had dips in 2018, early 2020, and 2022, but overall, investors have been richly rewarded.
Could we face another “lost decade” ahead? Possibly, with the S&P 500 trading at roughly 23X forward earnings. Ironically, it’s far better to retire during a bear market than during a bull market. If you retire in a bear market, it shows your finances are strong enough to withstand existing volatility. But if you retire in a bull market, you face a greater risk of drawdowns just when you start withdrawing.
The good thing is, most of us can adapt. Instead of withdrawing a steady 5% each year, we can pull back during tough times. We can also find ways to generate supplemental income – like teaching tennis in my case – if necessary.
One thing I didn’t fully grasp when I interviewed Bill Bengen was why the success rate of a 7% withdrawal rate was only about 50% in his book, even though only one household out of the 400 he studied actually ran out of money in his original research.
The key difference lies in his model’s assumptions: every household lives exactly 30 years after retirement and never deviates from a fixed, inflation-adjusted 7% withdrawal rate. In reality, not everyone lives that long, and most people naturally adjust spending based on market conditions. As a result, the real-life success rate of 399 out of 400 dying with enough money is much higher.
Today, with AI-driven productivity gains, the future might once again surprise us. I’m even willing to invest in AI companies for my children, to help save them from a life of disappointment.
It’s Time To Enjoy What We’ve Built
If you’ve invested diligently since 2012, chances are you’re sitting on far more wealth than you expected. We’ve worked hard, saved consistently, and benefited from one of the greatest bull markets in history.
So maybe now’s the time to ease up on the frugality, enjoy the fruits of your discipline, and live it up a little more.
Because if we’ve already done the hard part – saving, investing, and staying disciplined – then the next challenge is learning how to enjoy our wealth without guilt.
Fellow retirees, how have your investment portfolios and net worths done since you retired? Have any of you actually seen a meaningful decline in your portfolio or overall net worth? If not, why aren’t more people retiring earlier or spending more freely in retirement? The math clearly shows that if you stay invested, there’s a good chance you’ll end up even wealthier the longer you live.
Free Financial Analysis Offer From Empower
You can sign up for Empower’s free financial tools to help track and manage your net worth. I’ve been using their dashboard since leaving my day job in 2012, and it’s still part of my regular financial routine. My favorite feature is the portfolio fee analyzer, which revealed I was paying about $1,200 a year in hidden investment fees I didn’t even realize existed.
If you have over $100,000 in investable assets—whether in savings, taxable accounts, 401(k)s, or IRAs—you can also get a free financial check-up from an Empower advisor by signing up here. It’s a no-obligation way to have a seasoned professional, someone who reviews portfolios every day, take a closer look at your finances.
A fresh set of eyes can uncover hidden fees, inefficient allocations, or opportunities to improve your plan. I’m confident you’ll walk away with new insights about your retirement readiness, just as I did. It’s a great feeling to know you’re on track or that you’ll likely be fine no matter what happens next.
The statement is provided to you by Financial Samurai (“Promoter”) who has entered into a written referral agreement with Empower Advisory Group, LLC (“EAG”). Click here to learn more.
Subscribe To Financial Samurai
Listen and subscribe to The Financial Samurai podcast on Apple or Spotify. I interview experts in their respective fields and discuss some of the most interesting topics on this site. Your shares, ratings, and reviews are appreciated.
To expedite your journey to financial freedom, join over 60,000 others and subscribe to the free Financial Samurai newsletter. You can also get my posts in your e-mail inbox as soon as they come out by signing up here.
Financial Samurai is among the largest independently-owned personal finance websites, established in 2009. Everything is written based on firsthand experience and expertise.
After 31 years of working, I just retired this May. After, decades of maxing out my 457, my wife’s 457 and 403B, and adding to our taxable brokerage accounts, we amassed a 5.3M portfolio. However, lately instead of enjoying the fruits our labor, I feel overwhelmed. I have 750k in a tax deferred 457 and my wife has 1.1M in a tax deferred. These are ticking tax time bombs! I am 55 and blessed with annual $101,000 annual pension with 3% colas and wife will have a pension as well. She’ll probably take a reduced pension at 55, in four years, and hers will be about $65,000 with 3 % annual colas when she turns 61. With the pensions, no debt, 100% and paid medical in retirement paid by my city, for my entire family (kids till they turn 26), I feel is being overshadowed by the enormous tax I need to pay Uncle Sam. I finally decided to start converting my 457 every year till it’s all Roth. Listening to Ed Slott, he makes a lot of sense why we should opt for the Roth options, considering taxes aren’t that high and will probably go higher. Once they eliminated the stretch IRA with the SECURE ACT, he believes they really lost their luster.
I know I could easily pull out 5% and be fine since my pension covers everything and then some, but instead of spending it, I’m gonna pay the IRS to get them off my account as a silent partner.
Since you’re only 55 years old, doing the Roth IRA conversions once you no longer have day job income makes a lot of sense. But of course, it would’ve been better if you did them in your 20s and 30s, but no turning back time now!
At the end of the day, your tax advantage accounts are simply deferred income since he didn’t pay taxes on it in the first place. So I would just treated as such. No big deal as the Money was able to compound over the years and reduce your taxes while working.
Enjoy the good life!
So your (generous) pension and medical are being paid for by the government but you don’t want to pay your share (taxes) to keep that system running?
Oh so he should just decline the pension and medical (that the government offered over 30 years ago!) just so he can play by your financial “rules”? Stop being jealous of what you don’t have, especially on this site!
So wait, he’s suppose to decline the pension and medical (offered by the government over 30 years ago!) just so he can play by your financial “rules”? Stop hating on what others have, especially on this site!
I think there is some confusion around the SWR. It needs to be qualified. It only applies if your withdrawal in retirement includes selling assets. In Bengen’s books and podcasts he is always asked or addressed if the 4% or 5% or whatever includes principal and interest derived from your investments. And the answer he gives is “Yes”. But really it is only yes if part of your overall 4 or 5% withdrawal includes selling assets too.
SWR only applies if you have a certain amount of assets and you want to figure out if they will last X number of years. “How much can I withdrawal and not run out of money”. It is based on the idea that say you have 3 million dollars in principal, that January 1st of each year you liquidate 4% of that and shift it to say a checking account. You do that every January 1st for life.
Now lets say you have 10 million dollars and put it all in 30-year notes, yielding 4% and your annual expenses are say 200k. You are receiving 400k a year in interest so even with inflation in 30 years you will likely never will touch the principal. In that scenario the SWR idea does not apply, even though you are technically “withdrawing” 4% from your total portfolio each year. In this scenario your checking account (living expenses) are being refreshed as the dividend comes out – quarterly, semiannual. There is no need to sell anything Jan 1st or other times. (Sam, like me, loves these interest rates for this reason)
SWR, with the operative word being “withdrawal”, but of at least some assets/principal. SWR does not apply if your portfolio passively generates through low-risk assets dividends and interest enough to cover your expenses. Your actual SWR is then 0.
So SWR it is a helpful guide if you know going into retirement you will need to not just rely on interest and dividends. Otherwise don’t get caught up in it.
Hearing the wealth from many on this board, we will never need to dip deeply into our assets. In any case, even if I had to, it makes no sense to sell 200k (my 4%) of assets Jan 1 and move it to another account for that particular year. Not sure people really do that anymore?
“ Now let’s say you have 10 million dollars and put it all in 30-year notes, yielding 4% and your annual expenses are say 200k. You are receiving 400k a year in interest so even with inflation in 30 years you will likely never will touch the principal. In that scenario the SWR idea does not apply, even though you are technically “withdrawing” 4% from your total portfolio each year. In this scenario your checking account (living expenses) are being refreshed as the dividend comes out – quarterly, semiannual. There is no need to sell anything Jan 1st or other times. (Sam, like me, loves these interest rates for this reason)”
In my interview with Bill on my podcast, Bill clarifies this point. Yes theoretically, if you have, let’s say a 4.5% yielding risk free investment, you could withdraw 4% gross forever and never run out of money. But Bill goes on to say that inflation takes a bite out of your purchasing power and you may well indeed run out of money.
He goes on to say the stock component RETURNs of your portfolio accounts for about 60% I’m the reason why he recommended a percent and now 5% safe withdrawal rate. And that maybe he would have to demand 8% yielding bonds to be OK with being 100% bonds and that 8% yielding in bonds is equivalent to the historical return of a 60/40 portfolio.
Check out my podcast and let me know what you think!
Thx
So what’s your plan to spend more?
Bit by bit. How about you?
What’s your perspective (if one have the guts) to keep your portfolio 100% stocks since day one?
That would go against the heart of this post. I’d rather just find some part-time work that is meaningful that I enjoy. Everybody needs some purpose and something meaningful to do in retirement.
If you’re thinking about doing what you’re doing, I don’t think you have enough money or you having run the figures yet to retire comfortably. Better to keep on working instead.
It would have been a tumultuous, but lucrative ride. The problem is, the more money you have invested, the greater the gut punch when there’s a bear market.
Losing $30,000 out of a $100,000 portfolio is one thing. Losing $3 million out of a $10 million portfolio is another, especially when it can earn $400,000/year risk-free nowadays.
Personally, I would NOT have been able to sleep as well if all I had was my 99% in equities IRA portfolio after I retired. But it’s 99% equities because I have other assets, including real estate that generate enough livable passive income.
What’s your opinion about living the first years of retirement using loans colateralized by your stocks (at a ~6% interest rate), but no taxes on gains?
I’m a fearful person by upbringing and nature, so the unknowns of the future tend to worry me in general. My finances are in good shape, but that undercurrent of fear is hard to shake and surfaces every so often. That said, using projections based on realistic numbers has helped me feel more safe about my financial future. So I try and use data to shake off my fears of running out of money. I think it’s also my parents financial failures that also tend to trigger my money fears.
Anyway, since you’re very methodical and calculated with your finances, having a certain number you are comfortable splurging on each year sounds like a good way to help you feel good about spending. I’ve also found that spending on experiences versus material things tends to bring me more and longer lasting satisfaction and better memories as well. So that’s something you can consider as well. Overall you’re doing great and continue to be an inspiration to us readers, so thank you!
This constant fear in our head about running out of money or facing disaster. Scenarios is precisely the reason why we need to do the math and model out our retirement future.
After 13 years of being unemployed, I can unequivocally say that the fear in our heads is often way worse than reality.
The problem is, after a lifetime of living in fear of the unknown, we might just get to the end of our lives and regret having been so meek and uncourageous with the things that we wanted to do.
Hi Sam,
Thanks for this insightful article! I recently finished reading Bill Bengen’s A Richer Retirement, and it’s interesting to see these ideas reflected and explored here. The shift from the classic 4 % rule to a potential ~5 % withdrawal rate really resonated with me—especially the notion that after decades of diligent saving and investing, we deserve to enjoy our wealth, not just preserve it.
I appreciate the way you framed it: we’ve worked hard, stayed disciplined, and maybe now the hardest part is learning to relax into our retirement mindset and live it up a bit. Your point that we can adapt (adjusting spending in weak markets, supplementing income if needed) adds a realistic and empowering touch via setting guardrails vs hard withdrawl rules set in stone.
Thanks for challenging the “classic safe withdrawal” mindset and providing permission to live a fuller life in retirement—very timely for me and for many others on the journey.
Best regards,
Dan