I sold a '56 PSA 4 thinking I'd upgrade the card and almost immediately regretted it. Prices seemed to jump every time I looked. Finally paid up for a PSA 6 at a recent auction. I paid much more than I had planned to, but I'm glad to have this card back in my collection.
My guess is if you don't do it and wait, you'll regret it and end up paying more eventually.
Happy Hunting!