I currently own a 2014 Carbon Spyder Turbo with Z5 cams and a 2013 Spyder Turbo with RKT 2.1 cams. Both are shooters if I do my part. I plan on using the Cabron Spyder for my backcountry mulie hunt this year though based on a few factors (some of which have already been mentioned):
Weight - The carbon bow has allowed me to run a side bar and decrease my bow cant without adding a bunch of weight to the bow overall. 6 ounces may not sound like a lot, but when you're at full draw for a minute or two waiting for that bull or buck to turn broadside, 6 ounces can make a huge difference in how long you can hold your bow steady. If I was on a limited budget, then I'd probably get an aluminium bow, but if that's not the case, or if you don't mind buying a used carbon bow like I did, then that'd be the bow I'd choose.
Vibration - The difference in vibration is minimal, but it's there. Carbon bows just absorb more vibration and sound.
Durability - This is the only area where I think an aluminium Hoyt may edge out a carbon one. Until recently (2015) the finish on Hoyt's carbon bows have left a lot to be desired. I'm not trying to have a bow that wins a beauty contest, so this isn't a deciding factor for me personally, what finish issues I've had I just cover up with some Krylon and call it a day. But it's disappointing to see a high end carbon bow whose MSRP is $1200-1400 dollars have finish issues around the grip and riser. The aluminium bows, especially the black aluminium risers seems to be quite a bit more durable in that regard.
Hope this helps some!