When you picture a successful investor, it's hard not to conjure up an image of Warren Buffett, who's a billionaire many times over.

Buffett has made some smart choices in his life to get to where he is today. For example, he spent modestly on a home and never upgraded despite easily being able to afford to. And he's talked up his generally frugal lifestyle on numerous occasions.

But one of the smartest things Buffett did was buying his first stock at age 11. Another wise choice Buffett made? Sticking with the stock market for many years.

A person raising one arm and fanning out a bunch of hundred-dollar bills.

Image source: Getty Images.

If you're trying to save for retirement, you may be frustrated by the fact that your nest egg doesn't seem to be growing all that quickly. But if you stick with the stock market like Buffett did, you may be surprised -- in a good way -- at the amount of money you're able to retire with.

Results should come eventually

Although Warren Buffett is a billionaire now, that wasn't always the case. In fact, you should know that 99% of Buffett's net worth was accumulated after he turned 65 years old.

When we think about how compounding works, that makes sense.

Compounding is the concept of earning interest on interest. In the context of credit cards, it's a terrible thing, because it can easily lead people deep into debt. But in the context of investing and building wealth, it's an incredibly helpful tool.

As you earn money in an investment portfolio, you're able to reinvest your gains as well as your principal. Over time, that could lead to a large balance -- even if your money doesn't seem to be gaining that much value year after year.

Imagine you're 30 years old with a 401(k) or IRA worth $100,000. Let's also assume you're no longer contributing money to that account, but rather, are just keeping it invested.

If your portfolio generates an average annual 8% return, which is a bit below the stock market's average, you'll end up with about $1,478,500 by age 65. But by age 31, you might only have $108,000. And by age 32, you may only be looking at about $116,600. By contrast, between the ages of 64 and 65, your portfolio might grow by almost $110,000 in a single year because you have a much larger balance to begin with.

The takeaway? Don't be discouraged if your IRA or 401(k) doesn't seem to be gaining a ton of value from one year to the next. Over time, you could end up with a serious amount of money.

Your investment strategy doesn't need to be complicated

While Warren Buffett had time on his side in the context of accumulating wealth, he also has a knack for picking stocks. If you don't, though, it's not a problem.

Buffett himself has long suggested that the average investor keep things simple by investing in S&P 500 index funds for retirement. These low-cost funds give you exposure to the broad market, and they don't require nearly the same amount of research as individual stocks.

The process of building a retirement egg nest isn't a quick one -- but that's not a bad thing. So don't be discouraged if as of your 30s, 40s, or even 50s, your IRA or 401(k) isn't where you want it to be. If you give things time and keep saving and investing, you may find that by the time you're ready to end your career, you're in a really good place financially.