score:3
Yes. As a dual citizen, you can choose which passport to present to the United States. If you use your UK passport (without a visa), you will be treated as a visa waiver program applicant, which means that you will need ESTA authorization to fly to the US.
If you present your Canadian passport, you will be treated as any other Canadian citizen, which in this case means that you will not need a visa. As a prospective "visitor for pleasure," you will be a B-2 applicant for admission. Entering in B-2 status has several advantages over the VWP, most notably that you will normally be admitted for six months rather than 90 days.
You will of course have to check in for the flight with your Canadian passport as well, because the airline will only allow you to board the plane with your UK passport if you have valid ESTA authorization.
For the return journey, I would actually recommend checking in with the Canadian passport so your departure is more likely to be correctly recorded in the US system. When you arrive in the UK, of course, it will be simplest to show your UK passport.
However, it is not strictly necessary to use the Canadian passport. Even if the US does not record your departure, you can set the record straight after the fact. You can check at https://i94.cbp.dhs.gov.
As noted in a comment, you also should not hesitate to show the UK passport when you check in if there is any question about your admissibility to the UK after you've shown the Canadian passport. You can explain that you checked in with the Canadian passport because you used it to enter the US and you wanted to make sure your exit record was matched to your entry record.