Short version:
The David Gareja site straddles the border between Georgia and Azerbaijan. There was an episode in 2012 where the Azerbaijan authorities stopped tourists crossing to their side. The temporary resolution to that episode is still holding.
Your images demonstrate that you crossed the border, evidently without issue. The caves in your photographs are on the Azerbaijan side. I visited in 2016 without a problem despite meeting border guards from both sides.
Longer version:
The border between Georgia and Azerbaijan in this area follows the crest of a ridge; Georgia is on the northern (shallower) side, and Azerbaijan on the southern (steeper cliff-like) side. While these facts are not disputed, per this paper they do not define the border with enough precision to allow you to know when you cross the border. This has never been better defined because, as Gilles describes, the Georgians want the border moved such that the entire complex is in Georgian territory.
There is no man-made border demarcation at all at the site. There is a low fence that people assume marks the border, but it does not.
There are two monasteries at the site that people visit (ref: Lonely Planet for the area and Google Maps):
All visitors arrive from the Georgian side, there is no access from Azerbaijan.
In 2012 there was a brief episode where the Azerbaijanis stopped people crossing to see the caves on their side of the border. I don’t think anyone was harmed, they were just turned back. The issue was resolved a coupled of weeks later. This agreement is holding and if it breaks down it is likely that news will come out quite quickly. That said, if visiting Davit Gareja it’s worth being aware of the border situation and of the risk.
The UK Foreign Office says:
If you visit the Udabno caves at the Davit Gareja monastery site, take care not to cross the Georgia-Azerbaijan border, which is unmarked in this area.
I imagine other governments give similar advice. I’m not going to advise people to go against their government’s advice, but personally I would be more worried about venomous snakes than rampaging Azerbaijani border guards.
The David Gareja monastery area is very close to the border between Georgia and Azerbaijan, a few hundred meters away. If you went wandering in the hills, it’s quite possible that you might enter Azerbaijan.
The border is disputed. While the two countries agree on where the official border lays, Georgia contends that this location was artificially imposed by the Soviet Union and does not correspond to the historical border. Azerbaijan contends that while the area was indeed at some point in Georgia, it was in Azerbaijan before. Georgia has been trying to swap the Azerbaijani part of the David Gareja area for some other territory on the border since the two countries became independent in 1991, but in vain.
Azerbaijan and Georgia haven’t gone to war, but tension goes and ebbs. In May 2012, Azerbaijan deployed soldiers in the area and closed the Azerbaijani area to tourists. This only lasted a few weeks, but the issue remains unresolved.
I don’t know what the situation is right now, nor whether the particular cave that you visited was in Azerbaijan. The taxi driver may have voiced real concerns or may have exaggerated, but this is indeed a disputed area if not an open conflict zone.
Credit:stackoverflow.com‘