Set your brass back to the soft state before you form it. Then you've added several percent of strain forming it. That's a lot. That means that you went past the yield point, and plastically deformed your brass. So from a strain hardening standpoint, right after you formed it, it's stressed and hardened just like if you shot it many times. So after forming, for sure it would be wise to anneal.
I'm not an expert on sizing up and sizing down brass. But I do have some metallurgical training.
Maybe you didn't make a mistake. Maybe sizing it first was the good idea. But now certainly you should anneal it, and hit it one more time. The remaining strain that you introduce will be fairly small.
A lot of people will probably jump on me for this, but I believe that annealing temperature causes a coloration that's consistent with the temperature.
So I will take a new piece of LC , Lapua, etc, brass that I know has been annealed at the last step... And set up my annealing process to make my brass look just like theirs! I don't have the money, the machinery, or the metallurgical inspection equipment to do what they do. But since the color is directly related to the temperature I think I can get good enough to make world class ammunition that will readily reload many times.