Thanks, guys. Like Reacher I'm mainly thinking about this because it's cold here and my mind turns to summer boating when I'm not downhill skiing! My set up is more what Reacher described--there's about 3 inches of wood coming out each end of the mast, and that wood extends into the mast at least six inches, I believe.
I did not know what a slide hammer is--thanks!
Regarding the swelling--I was initially concerned about it being treated wood and thus leading to a mast failure (I know there's at least one story of that on this forum). But I don't believe it's treated wood. The swelling is on the part of the wood that protrudes out from the mast. I see it mainly on the bottom of the aft/mizzen mast, as it's a tight squeeze into the hole/step in the thwart and then into the holder. The other issue, much more minor, is that it is cracking just a bit and I worry about the strength of the hardware screwed into it (halyard block).
As I type this, I realize that the wood on my boat underwent a significant change in humidity, moving from dry Denver to the the variable moisture of the upper midwest. That doesn't matter much for the epoxied plywood, but exposed timber like the mast plug would change. That would explain why in the first year I had to file the plug down a bit to get it to fit, but it wasn't an issue this year.
Sorry for the rambling, but talking out loud is valuable, even on a forum. I think my plan will be to watch it more this coming season, and if cracks continue, I can gouge out as much as possible and insert the plastic plug like Reacher mentioned.