You can enjoy it on the way from Arequipa to Colca, or once arrived to Chivay and better acclimatized at 3500m.
Ride starts at 4900mt height. Nearly 15,000 ft. Once at the highest point: Patapampa the downhill trail by the singletracks or down by the road begins, both ways takes you to Chivay.
The thing is that we prefer the tough way, so the road is not an option. The singletrack is quite easy, not too angled and is mostly fluid even with a hardtail. The Patapampa trail to Chivay is like 1 to 2 hours depending what trail you choose, so is always better to hire a local guide, find Pedro Samayani and he will help you with all stuff you need.
Once at the valley you may choose to continue or by the main road to Yanque village or by the gravel road by the other side of the Colca river, crossing Chivay to add a loop if you will stay at Chivay or end in some of the points on the road if you will not stay at Chivay village.