Ok, so it's true that you cannot go there with public transport (or at least, not easily), so we booked a tour with Sierra Verde Peru. It cost us S/.75 per person and includes the Lomas de Lachay and the Castillo de Chancay, with everything except the lunch. We were quite happy with it except that compared to the Lomas, the castle is not worth it. I would have preferred to spend more time in the Lomas (although we did spend 2 hours and a half there). I think now is one of the best time to go see them, since it's all green and foggy, and apparently everything blossoms in August.
All in all I really recommend going there on a week end, to get a bit of fresh air outside of Lima.