Upvote:4
The solution on short notice was to change Google search criteria to something along the lines of "private car hire Dunedin", which resulted in a chauffeur service which also rented their executive cars. With full insurance, it resulted in about 4 times the price of a cheap rental agency, but was a very safe and well handling car.
Upvote:4
As you noted the bus times aren't ideal, InterCity has a bus that leaves daily in the afternoon, as does Alpine Connexions.
It looks like Atomic Travel does have a morning, albeit not early morning, bus: 10:00 Dunedin - (via Cromwell) - Wanaka 14:30.