bit of a different take.
Rather than pick bow/rifle based on anything else, I would first look at WHERE you are going to hunt, and use that to help guide your decision. The reason I say that is that on much of the east coast there is a huge discrepancy in how game-rich and difficult the hunting is between public versus private land. The private land is often ag land or actively managed and holds high concentrations of game, but tends to be extremely difficult or impossible for most people to access. Conversely, the public land tends to often be higher-elevation , much less fertile, with very little active forest management that would create game abundance or concentration. With spotting animals before putting a stalk on all but impossible given vegetation, archery is a lot easier to learn in a game-rich environment. While finding widely dispersed game in an eastern forest is often difficult without covering some ground, which makes getting into archery range extremely difficult, hence a lot of people gravitate to firearms of some sort. If you are going to end up really only having access to one sort of land to hunt, you might base your decision on what will work most easily there.
Also, look into season dates and duration for various implements. Often rifle seasons are very short 9a week or two or shorter), while archery seasons are quite long (several months). In many places only being a rifle hunter reduces your opportunity to hunt by a factor of ten or more. When trying to learn and simply fit hunting into a busy schedule that can be the difference between making it happen and not being able to make it happen, since it isnt already "built into" your lifestyle.
I'll also add that it's worth checking regs on crossbows. A ton of first-time hunters learn on a crossbow, becasue it it much easier to shoot accurately than any vertical bow, while not having the stigma of a firearm for many people. Doesnt sound like that's your issue and it wont change the eye-dominance, but it's an option.
The small game thing is also a great entry. If you have no access to a place to hunt deer, but you have greeat access to a marsh, well...you might be a lot better off aspiring to be a duck hunter.