When the Pali suttas say that it is not the "same" thing that is born and dies what do they mean?

The Pali Suttas don't say that ("Pali suttas say that it is not the "same" person who is born and dies"), actually. So the question is not valid, corrupt.

If asked of what the Buddha said, the Suttas tell:

"And which is the carrier of the burden? 'The person,' it should be said. This venerable one with such a name, such a clan-name. This is called the carrier of the burden. ...The Burden

I think I agree with Samana Johann's answer, i.e. "conventionally" you give a name to a person, and say that, "the person named so-and-so was born in such a year, took their retirement 65 years later, and died in some other such a year."

Buddhism warns against having self-view, including e.g. "I am this body" or "this is my house". I think it gives at several reasons for that:

• SN 22.59 says something like, "the body is impermanent and you don't control it, so it's unworthy to be described as self"
• MN 22 says that sorrow etc arise from assuming any self-theory
• I think that later doctrines point out that things (everything), including anything like "body" etc. which you might be tempted to identify as "self", are "empty" instead of being (here I forget what term they use, but something like) "real" or "genuine" or "genuinely existent".

Nevertheless it's conventional to use "I" sometimes. "I" is the first word in this answer for example, for what that's worth, I often use it on this site to avoid speaking in a "this is the word of God" mode, i.e. to avoid e.g. making assertions such as, "Samana Johann's answer is correct", when I don't want to be that black-and-white about it. I also use words like "perhaps".

The parable of the chariot as explained to king Milinda says explicitly that such a thing as a named person is a "convention".

Buddhist doctrine describes or mentions the 5 skandhas and the 12 nidanas, and I think that's to explain what "really" exists instead of what people conventionally use to be self, e.g., "there isn't really 'me', instead what there really are are 5 skandhas, etc.", but I think those too are 'conventional' and empty.

SN 5.10 is maybe the earliest instance of the chariot allegory.

Then there's a doctrine of kamma ("I am the owner of my actions (kamma), heir to my actions, born of my actions, related through my actions") possibly to explain how past, present, and future are inter-related (the doctrine of the 12 nidanas is another description of that), but I think it does that without claiming that there's any identical or constant or eternal self.

When the Pali suttas say that it is not the "same" bundle of psycho-physical properties that is born and dies

Perhaps, if they did say such a thing, they might be saying something like that anything is dependently originated -- it exists because the conditions for its existence exist. And that the conditions at one time are different from the conditions at another time.

A baby exists depending on its parent, an old man doesn't, so baby and old man aren't the same.

That's true of the body but also true of perceptions, they're impermanent. Furthermore each perception is impermanent -- it doesn't exist, then it exists, then it doesn't exist, and then a different perception exists instead -- so it's not the same perception.

You might call it "a different bundle" in the same way that "grandfather's axe" or the "ship of theseus" isn't the same, though I don't know if that's Buddhist.

I read it as a reference to anicca/impermanence. Concepts like "baby" or "old man" deceives us to believe that skandhas that makes up for our existence are constant over time.

The five aggregates are form or matter, feelings or sensation, perception, mental formations and consciousness.

All of these are constantly changing, depending on conditions. None of it stays the same. Also, none of them are independent. All the five aggregates arise and fall dependent on conditions.

So, what stays the same? The self?

From the Vina Sutta, we find that when different parts of a musical instrument work together, music is created. Music exists for sure, but it does not exist standalone from different parts of the musical instrument. If you break down the musical instrument into its constituent parts, you cannot find music.

Similarly with the five aggregates and the self. When the five aggregates work together, you can find the self appearing. But if you break them down to pieces and examine them carefully, there is no self in the constituent parts.

Even the self that appears when the five aggregates work together is not the same and changes.

The self is basically a mental idea, a constant fundamental identity that we give ourselves.

In the sutta quote below from SN 12.61, we find that the Buddha says it is better for an uninstructed worldling to consider the body as his self, because he can see that it is impermanent - it arises, changes and passes away.

However, mind and consciousness are always changing from moment to moment. The uninstructed worldling may cling to the idea of a self being his consciousness or mind, that he cannot easily see arising and passing away, as he would, his body.

From SN 12.61:

“Bhikkhus, the uninstructed worldling might experience revulsion towards this body composed of the four great elements; he might become dispassionate towards it and be liberated from it. For what reason? Because growth and decline is seen in this body composed of the four great elements, it is seen being taken up and laid aside. Therefore the uninstructed worldling might experience revulsion towards this body composed of the four great elements; he might become dispassionate towards it and be liberated from it.

“But, bhikkhus, as to that which is called ‘mind’ and ‘mentality’ and ‘consciousness’ —the uninstructed worldling is unable to experience revulsion towards it, unable to become dispassionate towards it and be liberated from it. For what reason? Because for a long time this has been held to by him, appropriated, and grasped thus: ‘This is mine, this I am, this is my self.’ Therefore the uninstructed worldling is unable to experience revulsion towards it, unable to become dispassionate towards it and be liberated from it.

“It would be better, bhikkhus, for the uninstructed worldling to take as self this body composed of the four great elements rather than the mind. For what reason? Because this body composed of the four great elements is seen standing for one year, for two years, for three, four, five, or ten years, for twenty, thirty, forty, or fifty years, for a hundred years, or even longer. But that which is called ‘mind’ and ‘mentality’ and ‘consciousness’ arises as one thing and ceases as another by day and by night. Just as a monkey roaming through a forest grabs hold of one branch, lets that go and grabs another, then lets that go and grabs still another, so too that which is called ‘mind’ and ‘mentality’ and ‘consciousness’ arises as one thing and ceases as another by day and by night.

When the Pali suttas say that it is not the "same" bundle of psycho-physical properties that is born and dies

The words "birth" ("jati") and "death" ("marana') are defined in SN 12.2 as producing or imputing the idea of "a being" ("satta") upon the manifestations or changing appearances of the aggregates.

What is important to understand is not so much the aggregates but how ideas of 'self', 'persons' or 'beings' are imputed upon the aggregates.

For example, when you see "my mother", "my father", "my wife", "my friend", "my enemy", "my favourite pop star", etc; and Arahant, in the same situation, sees only aggregates, elements and sense objects.

Specifically: can we say, conventionally or otherwise, that the old man or the baby have aged, or that the baby grew up into an old man?

Convention is different than ultimate truth. In ultimate truth, only aggregates change & cease. This is why there are so many suttas that says an arahant is not born, does not age, does not die (example, MN 140; SN 22.85; etc).

When there is no idea or view of a "self", "person" or "being", the ideas of birth & death do not arise.

This topic is analyzed a hundred times over in all schools. For example, in The Questions of King Milinda:

The king said: ‘He who is born, Nāgasena, does he remain the same or become another?’

[Nagasena:] ‘Neither the same nor another.’

[King:] ‘Give me an illustration.’

[N] ‘Now what do you think, O king? You were once a baby, a tender thing, and small in size, lying flat on your back. Was that the same as you who are now grown up?’

[K] ‘No. That child was one, I am another.’

[N] ‘If you are not that child, it will follow that you have had neither mother nor father, no! nor teacher. You cannot have been taught either learning, or behaviour, or wisdom. What, great king! is the mother of the embryo in the first stage different from the mother of the embryo in the second stage, or the third, or the fourth ? Is the mother of the baby a different person from the mother of the grown-up man? Is the person who goes to school one, and the same when he has finished his schooling another? Is it one who commits a crime, another who is punished by having his hands or feet cut off ?’

[K] ‘Certainly not. But what would you, Sir, say to that? ’

The Elder replied: ‘Neither I am what is now the grown up, nor was I what was the tender tiny baby, flat on its back. But all these are tied in one by means of this body.’

[K] ‘Give me an illustration.’

[N] ‘Suppose a man, O king, were to light a lamp, would it burn the night through?’

[K] ‘Yes, it might do so.’

[N] ‘Now, is it the same flame that burns in the first watch of the night, and in the second?’

[K] ‘No.’

[N] ‘Or the same that burns in the second watch and in the third?’

[K] ‘No.’

[N] ‘Then is there one lamp in the first watch, and another in the second, and another in the third?’

[K] ‘No Sir. But thanks to that lamp the light shined all the night through.’

[Nagasena:] ‘Just so, O king, does the continuity of dharmas connect. One emerges, another dissolves, connecting as it were without [a clear boundary between] the previous and the next, thus the former-consciousness and the next-consciousness cannot be categorized as either the same nor as different.’

This is then analyzed fabulously in all conceivable details by Nagarjuna in his Mulamadhyamakakarika. Basically, the idea here is that reality is not as discrete as the mind would like to believe. Things morph and develop continuously, rather like drifting clouds which only temporarily take certain recognizable shapes and are strictly speaking never the same nor are not themselves at any given point in time.

Strictly speaking, things and people don't have an immutable "core" or an atomic identity that would move through time from one moment to another. Each consequtive state of things develops from previous state through morphing, moving, and interacting - but the idea that it's still the same entity is imputed by us, it's not there.

Another traditional simile, also used in Milindapanha, is that of the milk produce cycle. As milk goes sour, then becomes butter, then ghee - does it retain its identity or not? The Buddhist answer is that the identity is an imputation of the mind. To anyone interested in Western perspective on this, I recommend a short work called "Hierarchy Theory" by Valerie Ahl and T. F. H. Allen.