One of the most challenging aspects of saving for retirement is trying to figure out how much money to sock away. And since it's pretty much impossible to predict how long your retirement will last, it can be really tough to narrow down your ideal number.

Many people will say that $1 million should be enough to make for a comfortable retirement. When I first started building my nest egg, I, too, assumed that saving $1 million would put me in a pretty good spot.

A person at a laptop.

Image source: Getty Images.

But I've since realized that $1 million may not buy me the retirement I want. So now I'm setting my sights on a higher number.

The problem with only saving $1 million

Northwestern Mutual recently reported that Americans think they'll need an average of $1.46 million to retire without financial worry. But even that sum may not suffice for a lot of people.

If you follow the 4% rule for managing your nest egg, $1 million in savings gives you $40,000 of annual income your first year of retirement, and slightly more in future years as withdrawals are adjusted for inflation. $1.46 million gives you $58,400 your first year and a bit more thereafter.

If you're someone who's used to living on $60,000 a year, either figure may suit you just fine. But if you're used to earning a six-figure salary, you might struggle on $40,000 to $58,400 of income, especially if you have lofty goals for retirement that involve a lot of travel.

While there is Social Security to fall back on, the program may be looking at benefits cuts in about a decade's time. That won't make your monthly payments from the program disappear, but it's hard to get a handle on what they'll amount to. So your best bet is really to try to get the bulk of your retirement income from your personal savings and use your benefits from Social Security as a supplement.

One thing you can do to try to figure out your ideal retirement number is decide how much annual income you'd like to have as a senior and then multiply that sum by 25. So let's say you currently earn $120,000 a year and would like $90,000 a year in retirement. Let's also assume you can't count on Social Security for more than $20,000. That means you'd need $70,000 a year times 25, or $1.75 million.

I'm still figuring out my savings goal

At this point, I know that I need to look beyond the $1 million mark in the course of saving for retirement. But I'm admittedly not sure how far beyond I need to go.

Part of the problem is that I'm not yet sure exactly where I want to live in retirement and what my expenses will be. I also can't predict how inflation will trend over the next few decades and what general living costs will look like. So for now, I'm trying to just save as much as I reasonably can without making myself miserable in the process.

I'll make sacrifices like driving an older car and not upsizing or upgrading my home to carve out more money for savings. But I won't deny myself a yearly vacation or leisure spending on a monthly basis.

I figure that if I do this, all the while investing my savings in the stock market to hopefully generate strong returns, I'll ideally end up with enough of a nest egg to make my retirement enjoyable. While I'm not sure whether I'll aim for $1.46 million or $2 million or a completely different number, right now, I'm focused on just plain doing my best.