Take a close look at the ballistics for whatever caliber you are wanting to use and compare it to a 30 cal....especially if you plan on taking a longer shot!
Mtn goats have thick, dense bones and a lot of hair, and the thickest hide I've ever seen on a big game animal to penetrate with a bullet. They can take a pounding and keep moving. They also live in cliffs. You likely have a better chance of having to make multiple shots and a goat taking a death dive over cliffs and screwing up the gorgeous goat cape/breaking off horns if you go with a smaller caliber.
Obvious smaller than 30 caliber may work but is it really worth the risk? It's all up to you but a no brainer 30 caliber critter for me!