Contribute To My 401k Or Invest In An After-Tax Brokerage Account?

The decision to contribute to a 401k or invest in an after-tax brokerage account is a dilemma that will be solved in this post. As someone who has built up millions in both accounts, there are some key considerations you need to make.

The great thing about a 401k is that you are contributing witth pre-tax money. The higher the tax bracket you are in, the more tax savings you will have. If you can start withdrawing from your 401k when you’re in a lower income tax bracket, then you’ve successfully conducted some tax engineering to boost your wealth.

The problem with the 401k is the 10% early withdrawal penalty before age 59.5. If the government gets desperate, they can raise the early withdrawal penalty percentage or increase the age limit. I ascribe a 75% chance one of these two things will occur over the next 30 years.

It’s easy to understand why saving for retirement is difficult. The value proposition is that you put your money away in an institution like Fidelity, which operates under the confines of the omnipotent government, who punishes you if you err from their rules, all for the chance that your money will grow decades down the road.

With no assurances from your money manager or the government that your money will be there in retirement, spending money now on instant gratification makes perfect sense. Give me the latest iPhone vs. the potential to have $25,000 more in retirement!

Therein lies the dilemma of the 401k contributor who can’t max out his or her account every year, and who therefore doesn’t have excessive after tax savings for liquidity and other purchases.

How Much To Contribute To A 401k

An easy way to determine how much to contribute to a 401k or after-tax investment account is to ask yourself what is the purpose for your savings. Having a more comfortable retirement isn’t a good enough answer. It’s important we be more specific with our wants, such as buying a car, a house, paying for graduate school, or taking care of a sick family member.

When I first started contributing to my 401k in 1999, I told myself the purpose for saving was to GET THE HELL OUT of Wall Street within 10 years! I knew I wasn’t going to last for decades like my parent’s lasted in their respective careers.

The hours and stress was just too punishing. I was fortunate to earn enough to max out the then $10,500 a year limit and contribute another 20% or so of my after tax income to my E*Trade account.

My target was to accumulate $100,000 in my 401k by age 27 along with $100,000 in my after-tax brokerage account so I could have the option of taking a two-year business school vacation, or moving back to Hawaii and just chill for a couple years. During either of these journeys, I’d really search deep to figure out what I really wanted to do with my life.

I aimed for $100,000 because it was a large enough amount where if I returned 10%, I was almost matching the maximum contribution amount at the time. Given I wouldn’t be working or contributing for a couple years, I was hoping that I could continue my retirement savings momentum through performance.

Why Don’t People Contribute More To A 401K?

Over the years, I’ve observed the #1 reason why more people aren’t willing to max out their 401k is because of their desire to save up for a house. Everything else seems obtainable through regular cash flow e.g. car payments, furniture, food, clothes, vacations, etc. But a down payment is a whopper of an expense that continues to torment the diligent saver.

Although I desired to own a piece of Manhattan real estate in 2001, I was about $50,000 short for a downpayment on a 2/2 condo and I wasn’t so sure about my career after the internet bubble burst. What a bad financial move not buying then!

Fortunately, the 401k system allows you to borrow from your 401k to buy a house. It’s just not exactly good financial practice borrowing to be able to buy something with more debt.

Borrowing From 401k To Buy Property

When you borrow from your 401k, you pay interest to yourself. The rate is typically a couple percentage points above the prime rate, which means it’s generally higher than your typical 30-year fixed mortgage rate. You can usually borrow up to half of your balance, or a maximum of $50,000.

It’s good such limits are created because you can imagine people pilfering their 401k for lots of superfluous things. Most loans must be repaid within five years, although some employers will give you up to 15 years if the money is used to buy a home.

Your 401k loan won’t count in your debt-to-income ratio when you apply for a mortgage because the loan is secured by the money in your 401k plan. 401k loans aren’t reported to the three major credit bureaus either, so the loan won’t hurt your credit score. In a way, I’m glad that nobody is punishing us for using our 401k money to buy a home. Applying for a mortgage is already a very cumbersome process.

But I would be very careful about going through the process of borrowing from your 401k to use as a downpayment to borrow more money to buy a home. Borrowing from your 401k isn’t as easy as snapping your fingers. Once you set the precedence that it’s OK to borrow from your retirement savings, then chances are higher that you will continue to borrow and borrow again.

Sure, you might get lucky and withdraw right before a crash. After all, buying something tangible is generally better than having your stocks disappear like vapor. But over the long run, you will likely lose out on some healthy returns. As they say, “Time in the market is more important than timing the market.“

The average 401k balance for all Americans is around $100,000 as of 4Q2017. The maximum contribution also rises to $18,500 for 2018.

Practice “Tax Location”

Contributing to a 401k or after-tax investment account is all about tax location.

Let’s say you have the luxury of maxing out your 401k and also growing a hefty after-tax investment account. It’s generally wise to buy growth stocks, or stocks that pay no dividends in your after-tax account. So long as you hold onto these stocks, they will hopefully grow at a faster compounded rate than non growth stocks and cause no tax liability.

On the flip side, you might want to buy more dividend income stocks in your pre-tax 401k or IRA. Your dividends and earnings gets compounded tax free until you finally have to withdraw from your account. If you are buying actively managed mutual funds with higher turnover, it makes sense to keep these funds in your pre-tax retirement accounts as well.

Whether you are maxing out your 401k or not, if you plan to hold income and growth stocks, you might as well allocate your positions accordingly. Nobody knows for sure the future of taxation rates, so it’s best to diversify and hedge. Here’s why I prefer investing in growth stocks over dividend stocks.

VIEW YOUR 401k LIKE A TAX WITH UPSIDE

It’s a good idea to treat our 401k like an extra tax we have to pay the government. Given taxes are mandatory, we should contribute the maximum amount to our 401k if we can afford it. After years of contribution, we hope our 401k will be free from penalties during withdrawal time. But if it isn’t, we’ll just chalk it up to another wasteful tax expenditure for the government to abuse.

Investing money in an after-tax online brokerage account provides good flexibility to grow your wealth while staying liquid. The key is to know yourself and not be tempted to liquidate your investments to buy things you don’t really need, or get emotional and sell during panic sessions or buy during euphoria periods.

I know myself and I have a temptation to do both! Having too much extra cash “made” me buy a $78,000 Mercedes Benz G500 when I was 25 years old. The only good thing is that I got rid of the SUV at a $15,000 loss the next year to buy a condo that has since appreciated in value.

Temptation and better long term performance is why I like to invest in private equity or venture debt with multi-year lockup agreements. Knowing my money can’t be touched without penalty tends to not only provide greater returns in the long-run, it also allows me to focus on doing more important things with my life.

To Summarize 401k Or Taxable Account:

1) Try to max out your 401k to save on taxes and get in a super-saver mentality. The maximum contribution amount for 2021 is $19,500 a year. The maximum contribution amount goes up $500 on average every two years o so.

2) Once you’ve been able to max out your 401k, aim to save at least 10% of your after-tax income after maxing out your 401k in a low-cost digital wealth advisor like Personal Capital, which automatically rebalances your money for you each month based off your risk tolerance.

3) The only major expense most people will really need to save up for is a downpayment on a property and college education, which is what the 529 plan is for. Hopefully everybody has disaster prevention health insurance. Calculate the downpayment amount based on 20% of the realistic purchase price for a home and divide it by your monthly after-tax, after-401k savings to figure out how long it will take for you to come up with your downpayment.

Adjust your 401k contribution and after-tax savings amounts to align with your desired time frame of owning a home. If your desired time frame for owning a home is ASAP, then contribute the maximum 401k amount that provides for a company match. To not do so would be to reject free money. Save all other proceeds in an after-tax investment account, savings, or CD account.

4) If you’re within a couple years of your goal to come up with the full downpayment, consider de-risking your investments from 100% equities to a 50/50 mixture of index equity funds and government bonds, and then to CDs or a simple money market account.

The worst thing that could happen is an obliteration of your downpayment right before you’re ready to buy. The proper asset allocation of stocks and bonds is important.

5) Once you’ve purchased a home, your number one mission should be to ramp up a comfortable after-tax liquidity amount so that you will never be a forced seller of your home.

I like a minimum of 6 months liquidity, but the number of months depends on you. Once you’ve developed your minimum liquidity amount, refocus on maxing out your 401k as much as possible. Your older self will thank you in 10 years!

6) Always do both. If you have aspirations of retiring before the age of 59.5, then you should consider aggressively building an after-tax investment account that generates passive income. Give yourself early retirement options! See the chart below as a basic guide.

Manage Your Finances In One Place

The best way to become financially independent and protect yourself is to get a handle on your finances by signing up with Personal Capital. They are a free online platform which aggregates all your financial accounts in one place so you can see where you can optimize.

Before Personal Capital, I had to log into eight different systems to track 25+ difference accounts (brokerage, multiple banks, 401K, etc) to manage my finances. Now, I can just log into Personal Capital to see how my stock accounts are doing and how my net worth is progressing. I can also see how much I’m spending every month.

The best tool is their Portfolio Fee Analyzer which runs your investment portfolio through its software to see what you are paying. I found out I was paying $1,700 a year in portfolio fees I had no idea I was paying! They also recently launched the best Retirement Planning Calculator around, using your real data to run thousands of algorithms to see what your probability is for retirement success. 

Once you register, simply click the Advisor Tolls and Investing tab on the top right and then click Retirement Planner. There’s no better free tool online to help you track your net worth, minimize investment expenses, and manage your wealth. Why gamble with your future?

Real Estate Diversification

In addition to investing in your 401k and taxable brokerage account, look to invest in real estate as well. The global pandemic has accelerated demographic shifts towards lower cost areas of the country due to the work from home trend.

Check out Fundrise and their eREITs. eREITs give investors a way to diversify their real estate exposure with lower volatility compared to stocks. Income is completely passive and there is much less concentration risk.

If you are bullish on the demographic shift towards lower-cost and less densely populated areas of the country, check out CrowdStreet. CrowdStreet focuses on individual commercial real estate opportunities in 18-hour cities.

Both platforms are free to sign up and explore. I’ve personally invested $810,000 in real estate crowdfunding across 18 properties to earn income 100% passively.

About the Author:

Sam began investing his own money ever since he opened an online brokerage account online in 1995. Sam loved investing so much that he decided to make a career out of investing by spending the next 13 years after college working at Goldman Sachs and Credit Suisse Group. During this time, Sam received his MBA from UC Berkeley with a focus on finance and real estate. He also became Series 7 and Series 63 registered.

In 2012, Sam was able to retire at the age of 34 largely due to his investments that now generate roughly $200,000 a year in passive income largely thanks to real estate crowdfunding investments in the heartland of America. He spends time playing tennis, hanging out with family, consulting for leading fintech companies, and writing online to help others achieve financial freedom.

For more nuanced personal finance content, join 100,000+ others and sign up for the free Financial Samurai newsletter. Financial Samurai is one of the largest independently-owned personal finance sites that started in 2009. Everything is written based off firsthand experience.