Once you've put away five years worth of your household's salaries and are contributing generously to your 401k and own a home. Then it may be okay to play with baseball cards... Damnit, forgot about college tuition
Easier said than done, I know. Try to focus on one niche that makes you happy. Hunt for bargains and resist overpaying for something that may reappear. Mine niche is htf Boston items now. Everything else in our shoebox is tradeable in order to fill holes. No appreciable value lost, or gained. But I enjoy it a lot more. I'll be on a fixed income shortly. So our collection is as valuable as it'll ever be in my financial lifetime.