We’ve covered how crucial it is to plan for retirement if you’re hoping to retire in your fifties. Between the high cost of healthcare, the years of wealth building you’ll be missing out on, and the lack of Social Security benefits, retiring in your fifties certainly isn’t for the faint of heart. What’s more, there’s another hurdle you’ll need to overcome: taxes in retirement.
Taxes on Social Security benefits
It may not affect you early on in retirement when you’re not yet collecting benefits, but it’s important to know that your Social Security income could be subject to taxes. The tax rate is based on your combined income. You can determine your combined income by adding the following: Your adjusted gross income + nontaxable interest + ½ of your Social Security Benefits.
If your combined income is:
- $34,001 or more ($44,001 or more if you’re married filing jointly), up to 85% of your benefits will be subject to federal tax.
- Between $25,000 to $34,000 (or between $32,000 and $44,000 as a joint filer), you’ll be taxed on 50% of your benefits.
And if you live in any of the 10 states listed below, you could also be subject to state-level taxes on your benefits. The following states are considered not-so-friendly to retirees from a tax perspective: Colorado, Connecticut, Kansas, Minnesota, Montana, Nebraska, New Mexico, Rhode Island, Utah and Vermont.1
To learn more about the specifics of each state’s Social Security tax policy, take a look at our course called, “Your Government Benefits Are Subject To Taxes.”
Taxes Based on Location
Where you live may have a big factor on your taxes. Not only will cost-of-living and healthcare expenses differ by location, but federal and state taxes are based on where you live. This can have a big impact on your retirement savings everytime you withdraw. And if you plan to leave an estate, your estate could be taxed federally and at the state level. Alternatively, if you receive an inheritance, you’d also be subject to taxes based on your location.
Read our class, “How Location Impact Taxes,” to get a deeper look at how much each state taxes as well as any special tax deductions for older Americans. In addition, go to our Cost of Living calculator to compare different tax rates and see if moving impacts your Retirement Score.
Taxes on Investment Accounts
Retiring early could make it tempting for you to make early withdrawals from your retirement accounts. But doing so could result in heavy penalties (aside from the interest you’d be missing out on!). Early withdrawals from tax-deferred accounts will result in a 10% penalty. So, for example, if you withdraw $10,000, you’ll face $1,000 in penalties, which will reduce the amount of money you’ll be able to use. In addition, the entire amount of the withdrawal ($10,000 in this example), will be taxed as income at the end of the year.
If your money is sitting in a Roth account, it can continue to grow untouched. But if you withdraw your earnings before you reach the age of 59 ½, you will likely have to pay taxes along with a 10% penalty. The IRS does allow for some penalty-free withdrawals before age 59 ½, called “Qualified Distributions” if you meet specific requirements, such as:2
- If you become permanently disabled.
- For qualified college education expenses.
- For qualified birth or adoption expenses.
- To purchase your first home (there is a $10,000.00 lifetime cap).
If you meet one of these scenarios, you can avoid paying the early withdrawal penalty, but you may still have to pay taxes on your earnings.
If you have money in tax-deferred accounts like a traditional 401(k) or 403(b), SEP, or traditional IRA at the time you turn 73 (or 72 if you were born before December 31,1950), you will be required to take yearly distributions called Required Minimum Distributions, or RMDs. These amounts are then treated as taxable income. Learn more about RMDs in our class “Taxes in Retirement 101.”
If you’ve had a life-long dream of retiring in your fifties, you likely have money saved in several different types of accounts-- some of which are taxable and others which are not. What you may not realize is that you can drastically minimize the impact to your taxes by making strategic choices about the order in which you withdraw money from these accounts. This is referred to as withdrawal sequencing.
If you have multiple types of accounts, sequencing your withdrawal of funds can be a significant tax advantage, and there are a few different ways to go about it. Always be sure to keep RMDs in mind when deciding which withdrawal strategy to employ.
Below, we’ve included a section from our “Taxes in Retirement 101” course that outlines different withdrawal strategies you could employ. Choosing the right strategy will ensure you’re able to hang on to as much of your hard-earned money as possible during your long retirement.
Taxable First, then Tax-Deferred
- Withdraw from taxable accounts first (Social Security, annuities, other forms of earned income)
- Then start to draw down tax-deferred accounts (401(k)s, traditional IRAs, etc.)
- Leave the tax-exempt accounts (Roth IRA or Roth 401(k)) for last
Doing this allows you to potentially benefit from a lower tax rate and prioritizes leaving an inheritance for your heirs.
Taxable First, then Tax-Exempt:
- Drawdown taxable accounts first, like the first method,
- Then draw down the tax-exempt accounts (your Roth accounts),
- Finally, start drawing on the tax-deferred accounts (401(k)s, etc.).
This method reduces your tax bill throughout retirement and can make sense if you anticipate a lower tax rate in the future when you start drawing from the tax-deferred accounts.
Tax Bracket Management:
- Start with your tax-deferred accounts (e.g., IRA & 401K accounts) first, but only up to a specific tax bracket (e.g. 12%, 22%, etc),
- Then withdraw from taxable accounts,
- Leave tax-exempt funds for last.
This method prioritizes passing your estate to your heirs and may be a good strategy if you expect a higher tax rate in the future.
Taxes from Inherited Accounts
Are you planning an inheritance from a loved one will add cushion to your early retirement? Although that might be true, there’s still taxes and rules to consider if this inheritance might be “worth” it. Typically, any inherited accounts are still subject to RMDs and must be distributed within a 10-year period. These inherited accounts, outside of Roths, count as taxable income and can bump you into a new tax bracket. And depending on where you live, you may incur state taxes too. If you don’t want to deal with the taxes and maybe responsibilities of real property or creditors, you have an option to disclaim an inheritance. To understand all the tax liabilities and rules that come with an inheritance, go to our course “Taxes in Retirement 101” to get a deeper understanding.
SOURCES
- Washington, Katelyn, “States That Tax Social Security Benefits.” Kiplinger, 5 September 2023. https://www.kiplinger.com/retirement/social-security/603803/states-that-tax-social-security-benefits. Accessed 4 December 2023.
- “Retirement Topics - Exceptions to Tax on Early Distributions.” IRS, 29 August 2023, https://www.irs.gov/retirement-plans/plan-participant-employee/retirement-topics-tax-on-early-distributions. Accessed 30 November 2023.
