[ajquigs]

Interesting question.
For me, I have perhaps four holy grail cards, all of which I wanted since I was a teenager - Aaron rookie, 1933 Ruth #181, 1953 Paige, and 1952 Topps Jackie.
Because I'm not that condition sensitive I've been able to get examples that I think look great in my display and I'm very happy with ... grades 5, 2.5, 4.5, and 4. Thus, under the premise of your post perhaps I don't really have any holy grail cards at all.
Hypothetically, if I didn't have these cards and I had to trade all my others for them (assuming value was comparable) I think I'd consider it but likely wouldn't do it as I'd prefer to have a display of 100-200 cards rather than one or a small handful.