Like others have said, Red Rocks is a big part of it, but Colorado is also a bit of an island of civilization in the western part of the country. We're surrounded by Wyoming, Nebraska, Kansas, Oklahoma, New Mexico, and Utah - which probably aren't the states you think of first when you think of budding metropolises. So lots of events are held in Colorado that attract people from neighboring states.
Also, Denver International Airport is one of the busiest airports in the world since we're a huge connection hub to the rest of the country. So it's very easy to travel here.
Yeah, everyone is saying Red Rocks but they haven't even played there since 2009. They've played at Dicks 45 times since 2009 because they do 3-4 shows every Labor Day weekend.
I'm not sure why everyone is saying Red Rocks. Phish played three shows there in 2009 but they haven't played there since.
The real answer is that every Labor Day weekend they play 3 to 4 shows at Dick's Sporting Goods Park outside Denver (except in 2020 when the shows were cancelled).