If you want to rebore/rebarrel, go for it. However, you mentioned a “crappy, foggy scope” so I would start there. Those early post-64 70s weren’t pretty, but most could shoot. As Wapiti1 said, free floating may do it, but you won’t really know without a decent scope.
Caliber recommendations...