If you head to the Big Bend (Alpine, TX) Nat'l Park, you might also consider:
1) Marfa (TX) "lights". Not a big deal, but pretty neat
2) Fort Davis
3) the McDonald

Observatory
(note) if you're interested in the Observatory, start calling now for reservations!
4) the "Sand Dunes" near Monahans, TX is....well....a bunch of shifting, blowing sand!
5) Balmorhea State Park near Toyahvale, TX is a unique place. You may not want to go swimming in Jan or Feb, but it's pretty, unique and has hiking trails.
P.S. the high 🏜 desert is gorgeous that time of year.
New Mexico - I'd try Ruidoso. It's kind of an "artsy-fartsy" kinda place, but interesting.
Carlsbad Caverns is probably the "diamond" of NM.
"City of Rocks" is neat if you stumble into it.
Red River, NM is nice if you're into ski areas.
The "Cumbres & Toltec Railway" in Chama, NM will be parked for the winter, but is still interesting.
NOTE:
PLEASE, PLEASE, PLEASE!
"IF" you happen to decide to visit Big Bend, go to the rangers station and register with them, give them an itinerary and STICK TO IT!*
Alpine, TX is considered the "Gateway to Big Bend Nat'l Park!"
The entry to Big Bend is
100 miles from Alpine!
If you get in trouble, it's a helluva long way to get help!
The Border Patrol "Air & Marine" unit (helicopters) is often called upon to remove injured hikers and locate lost ones!
Beautiful....but dangerous!
* - the manpower at the park is very slim.
IF you don't check-in at the appointed time, they WILL come looking for you. It's their job!
....and if you're not where you're "supposed" to be, Big Bend is HUGE. Lots of "nooks & crannies" to look in!
Last summer, a dad took his 2 sons hiking in Big Bend.
Unprepared, the heat got to them.
The dad tried to pack the youngest out.
The oldest son survived. The dad and youngest didn't make it.