Sounds like my story with 1953 Topps Harvey Haddix. Some guy in Florida was buying singles and reselling as sets - and he considered Haddix a loss leader - so he was always bidding $30-$40 for low grade cards. After almost two years of always being the underbidder (and costing him a fortune), I wrote him one day and said that if he just let me win 1 card he could save a ton of money. He agreed. I won a card for about $20, and the price fell to about $5-10 for an ungraded, lower grade card within a few months.

Which shows what happens when two guys "need" a card.

I overpaid a lot on my 63 set. I got great deals on lots at the beginning but when you start to buy individual cards it gets pricey. If you buy on eBay the shipping kills you on a common or two that are in VGEX condition. The high numbers kill you. And buying at a show was no bargain either. I think I bought the Torre for 30% under "book price" and then saw another later on at the same show on a table that was 5 for $20. I see this a lot at shows - there are deals to be had and usually its at the disorganized tables where you can spend hours sorting through piles. Versus going to a guy that has them all in number order and you pluck your needs and overpay for convenience.

I have done great on some raw cards

I got a 58 Mantle for $90 that graded a 2.5
I got a Yaz RC for $25 that graded a 5

I have some other raw to graded success versus only a couple of duds (3 altered cards that didn't cost much to start with).

I over pay everytime I buy a baseball card. Why? Because back in the 1980s, those that wanted to relive the second childhood and those who thought they could become wealthy from baseball cards took and innocent hobby from the child and turned it into an industry for adults. There is no turning back to the days of the 1970s (or before) when values of cards were benign. So it is either pony up or don't collect. I notice too, that when I overpay for a card, then upgrade, no one has an interest in the card I just upgraded.