So some of this is conjecture on my part, and some was directly confirmed by the guy.
They knew approximately where I was hunting by where I parked. When I shot, he knew it was me and I confirmed it when he stopped me on my way out. I had been getting in around noon and hiking about an hour to reach my spot. The typically don't leave camp until 3ish, or at least that's when I see their trucks. I'm typically on location an hour by then. I told him I was going back. When I arrived the next day at 1, the guy (younger guy, same group) was flagging me with his orange from across the canyon, 800yds away. Not ideal, but I've had several people come much closer to me in that same area. It's public, so anything beyond 300yds is the most I can hope for.
We killed that evening so I didn't speak to anyone that night. He (the old man) flagged me down heading in the next morning to finish the pack out, and he was pissed that I didn't vacate the area since the guy was there before me. I.E. that was his intent. Find out where I was, confirm I was on elk, and beat me to the spot, then use "hunting etiquette" as a means to call dibs. I found it ironic that, that was his expectation. It would have been different if the range was closer. 800yds, he was well over 1,000 yds from where the elk was when I shot.
My conclusion: I was naive to give away information. Lesson learned. I will lie like a politician next year. I'm also looking into buying a suppressor. I'm half tempted to pop off some shots next time too, just to add confusion to their method. I will still stay on spot if someone is 800yds from me.