score:11
First up, dolphin meat (イルカ肉) is pretty bizarre even by Japanese standards, and second, Taiji is a hurting fishing town smack dab in the middle of nowhere that's gotten a lot of bad press thanks to The Cove. So yes, if you show up there solo and start asking about eating dolphin, you're unlikely to get a friendly reception. I'm not saying you'll be in physical danger -- this is Japan, after all -- but you will definitely be assumed to have ulterior motives, esp. if you don't have a Japanese buddy along to smooth things about.
Healthwise, mercury poisoning is a long-term concern as it accumulates in fish and people, but a single meal is not going to do any appreciable harm.
So if you'd like to sink your teeth into Flipper, here are three restaurants in and around Taiji that offer or at least have offered it: 東牟婁郡太地町のイルカに関連するお店を探す
(Note that the first of the four listings is a paid commercial for a restaurant in Ise-Katsuura, not Taiji.)
Here's 20 incl. the four above all around Wakayama Prefecture: 和歌山県のイルカに関連するお店を探す
And last but not least, there's even a place called Uomaru in Yurakucho, central Tokyo that has a single dolphin dish (イルカの甘露煮, ¥799) on their regular menu.
(courtesy of 海つばめ on Tabelog)
Yum yum? The cheaper joints offer dolphin sampler sets from around ¥1000, so it's not particularly expensive. And as far as I can gather, while the hunt is only in the winter, the meat is available frozen all year around and the three prefectures that hunt dolphin (Wakayama, Shizuoka and Iwate) cooperate to ensure supplies -- although the 2011 tsunami can't have done any good for the Iwate fisheries.