score:4
MileWise sounds similar to what you are looking for. It presents fares in cash/miles/points, and lets you specify how many points you have in various award accounts. The results are ranked via "the best ways to pay based on your rewards".
A major drawback is that the service cannot be used for American, Delta, or Southwest.