Curious why Baker is a long shot? I know nothing about the town but I looked at some pictures and it looks like it has the necessary ingredients. Mountains, historic downtown, etc. I wonder what, exactly, the secret sauce really is???
I watched Flagstaff turn from sort of a seedy logging town to outdoor chic over a span of not really that many years, though it had a skiing / climbing / river running culture for a long time. And of course a university.
Blue Mtns. 20 min away, largely devoid of humans. Decent Mountain biking. Fantastic road bike riding. Two small ski areas within 1 hour drive. Wallowa Mtns. 2 hour drive, Columbia River 30 min drive, Spring Mtn. rock climbing 1.5 hour, other smaller chossier areas within one hour. Very nice climbing gym, 120 wineries, 3 breweries, 2 distilleries, 2 weed shops, great restaurants, thriving arts scene, symphony, chamber music festival, 3 colleges, a "real" town with old homes and streets lined with huge old trees. Fantastic local fruits and vegetables (and wine and beer), and friendly people. Not full of "body-Nazis" - people can converse on topics other than their latest training regimen and athletic accomplishments. Started to take-off before the last recession and bumper stickers started showing up with the slogan "Don't Bend Walla Walla".
If on average, if MT was 2k higher we'd be in fantastic shape for being a nice healthy zone to live in. Our fire season, lack of abundant alpine style climbing, choss heaps, knap weed and lack of diversity will keep us in a sleeping state of a state. Also, it can be hard to understand how a lack of diversity affects you if it it something you are used to or like having. I don't necessarily mean it by just your surroundings but mainly when you have casual convos with your friends and acquaintances who only have experience with a white world.