score:5
Yes, there is a direct underground tunnel connecting the two stations (three, actually: Keisei, JR and Tokyo Metro). You will need to exit and re-enter through the fate gates at both ends, but if you're using a Suica/Pasmo smart card the fare calculation will be automatic.
(courtesy Google)
The vast majority of nearby stations in Tokyo are similarly interconnected, but not quite "all". If you zoom in closer in Google Maps, it will usually show you exactly where the tunnels go in light pink, with exit numbers marked as well - very handy for getting your bearings in some of the larger stations!