score:3
There is public transport available, as you mentioned. Bus route 7211 goes direct from central Chiayi and the TRA station to the HSR station in roughly 20-30 minutes - I really don't know why Google Maps is reporting this as a journey time of nearly 2 hours. There was also previously the 7212 route, but it looks like that's been withdrawn since the last time I was over.
For passengers travelling to the HSR station the journey is also free as far as I'm aware, but the last time I took the HSR was in 2014, so double-check - my subsequent journeys from Taipei have been via car. This Tripadvisor thread seems to suggest that the journey is still free for HSR ticket holders and NT$48 for those without a ticket.
Chiayi buses do accept EasyCard for short journeys. For longer journeys (i.e. to Alishan National Park) separate tickets need to be bought.
Source: My grandparents-in-law live in Chiayi.