From my experience driving in Japan, the major problems you will run into are:
When you are turning, especially if you are the first car in line. You aren’t going to be driving down the road and just switch to the wrong lane, but when you are turning and not following a line of cars in front of you, habit kicks in and you can turn into the wrong lane.
Gear shift being on the opposite side and foot pedals never troubled me, but I can’t count the number of times I turned on the wipers instead of signaling or vice versa.
This isn’t strictly a driving problem, but crossing the street while walking is actually a bit of a problem at first. Instead of “look left->look right->look left->go” you have to switch it around and “look right->look left->look right->go”. Several times I saw people almost get hit because they looked the wrong way and stepped out in front of a car.