During World War II, did the Luftwaffe try to bomb 10 Downing Street?

It was certainly possible to bomb individual buildings as the Luftwaffe did at RAF Kenley (this is near London) during the Hardest Day, using a combination of diving Ju-88s [not 87s] and low-altitude level bombing (on this occasion done with Do-17s). But this was pretty risky, resulted in high losses, and such low-level attacks were discontinued after the Hardest Day.

At their airfield at Cormeilles-en-Vexin, 9 Staffel (Squadron) KG 76 were briefed by their commander Hauptmann (Captain) Joachim Roth. The Staffel was to conduct a low-level attack against Kenley with Roth flying as a navigator in the lead aircraft. The unit had specialised in low-level attacks in France with great success. The nine Do 17s were to head across the Channel and make landfall at Beachy Head. From there they were to follow the Brighton–London rail line north-east to the target area. The crews were ordered to concentrate their attacks against buildings and hangars on the southern end of the airfield.

The Dorniers were to carry twenty 50 kg (110 lb) bombs each fitted with a fuse that would allow for function if released higher than 50 ft (15 m); the type of bomb previously used by the Staffel had to release from twice this height, making the unit's Do 17s correspondingly more vulnerable to ground fire.

The attack was to be part of a coordinated pincer movement against the airfields. Ju 88s from II./KG 76 were to dive-bomb buildings and hangars from high-altitude first. Five minutes later, 27 Do 17s from I. and II./KG 76 would level-bomb from high altitude to crater the runways and landing grounds while knocking out its defences. 9 Staffel KG 76, the specialist low-level strike unit, would go in and finish off any buildings still standing. It was a bold and imaginative plan. If it worked, it would wreck Kenley from end-to-end.

[...] Günter Unger lined up his Do 17 in order to attack a hangar and released his 20 110-lb bombs before his starboard engine was knocked out. Unteroffizier (Junior Officer or NCO) Schumacher watched as three hangars were destroyed by Unger's bombs.

At the end of the day just one hangar was left operational at Kenley. [...] In return 9 Staffel lost four Do 17s, three slightly damaged and two seriously damaged. Low-level attacks were abandoned after The Hardest Day.

(The Ju 88s were supposed to arrive first, but due to some German timing issues, they arrived after the low level attackers. And couldn't see the targets due to smoke caused by the low level attackers.)

Somewhat similarly, the British similarly used the [much faster] Mosquito in a low level bombing raid in the vicinity of specific buildings during some propaganda speeches (of Goring and Goebbels) in Jan 1943. According to the RAF memorial:

The bombing was inaccurate and did little damage but the effect was exactly what had been hoped for.

I.e. the commotion was heard over the German radio. But one out of six Mosquitos was shot down, highlighting the relatively high risk of such low-level raids, even with faster aircraft.

Possibly the final air raid of the war, on Berchtesgaden showed what could be achieved with high level bombing too:

Once the target was found, over 1,400 tons of bombs were dropped, including four 12,000 pound Tallboy bombs. The heavy payload was designed to destroy bunker networks that were believed to exist below the Obersalzberg complex. The SS barracks – the key target – were severely damaged. Houses belonging to Göring (who survived the raid in his bomb shelter) and Bormann were destroyed. The RAF official historian, Hilary Saunders, boasted that a thousand-pounder had made the deep end of Göring’s swimming pool a little bit deeper. The Berghof itself also sustained heavy damage. Days later, American and French troops arrived on the scene to rummage through the ruins for souvenirs.

The answers to a similar question on Quora point out that bombing accuracy just wasn't good enough in those times to target a specific building from the height of several thousand feet. Today this has become possible due to GPS-guided bombs and other similar equipment.

Nevertheless, there were some close hits, as described here (with a quote from Churchill's memoirs):

By October 1940, the intense bombing period known as the Blitz began. On 14 October, a huge bomb fell on Treasury Green near Downing Street, damaging the Number 10 kitchen and state rooms, and killing three Civil Servants doing Home Guard duty. Churchill was dining in the Garden Rooms when the air raid began. As he recalled in his memoir Their Finest Hour (1949):

We were dining in the garden-room of Number 10 when the usual night raid began. The steel shutters had been closed. Several loud explosions occurred around us at no great distance, and presently a bomb fell, perhaps a hundred yards away, on the Horse Guards Parade, making a great deal of noise.

Suddenly I had a providential impulse. The kitchen in Number 10 Downing Street is lofty and spacious, and looks out through a large plate-glass window about 25 feet high. The butler and parlour maid continued to serve the dinner with complete detachment, but I became acutely aware of this big window. I got up abruptly, went into the kitchen, told the butler to put the dinner on the hot plate in the dining-room, and ordered the cook and the other servants into the shelter, such as it was.

I had been seated again at the table only about 3 minutes when a really loud crash, close at hand, and a violent shock showed that the house had been struck. My detective came into the room and said much damage had been done. The kitchen, the pantry and the offices on the Treasury were shattered.

Keeping Downing Street safe became the priority of the Prime Minister and the War Cabinet. Steel reinforcement was added to the Garden Rooms, and heavy metal shutters were fixed over windows as protection from bombing raids. The Garden Rooms included a small dining room, bedroom and a meeting area which were used by Churchill throughout the war. In reality, though, the steel reinforcement would not have protected him against a direct hit.

In October 1939, the Cabinet had moved out of Number 10 and into secret underground war rooms in the bas*m*nt of the Office of Works opposite the Foreign Office, today's Churchill War Rooms.