Never lost one, mine seem pretty tight where it would have to take a direct hit from a branch to push it out. I know I've read of others losing them and just can't imagine losing them just from shooting.
If I ever lost one and had to replaced it, I would use clear RTV on the damper to the bow, and also the weight to the damper to make it a permanent attachment to the bow. And clear RTV can be peeled off in the future if needed.