Why Buddha didn't want him to be called 'friend' by the monks?

Respect of the Buddha and the Dharma -- both of which are tied together -- must pre-exist for the successful study of the Buddha's Dharma and to attain the fruits.

As much as we cherish the Buddha's words, he is not our friend.

Normally, to be a friend implies that he/she will be there to hear your complaining, watch you fail/succeed, and join you in your house, entertainment, and other frivolous pursuit. Friend is a spoiled and vague word and to label one as such invites competition and abuse for the teacher and student alike. If the Buddha allowed himself to be considered a friend, his students would actually be able to grow beyond their cravings less and would remain reliant on his consultation.

Until one attains Arhatship or Buddhahood, the Buddha is not and should not be considered a friend but a Teacher. He has attained the universal status of an Enlightened One, a "Realised One", a "Worthy One" and many other special names fit for this Ultimate accomplishment.

When they first saw him they thought, "He is inferior to us (because he is given to luxury, forsaking the striving, etc.)."

When he approached them they no longer thought that, but still they called him "friend": which I understand to mean "my comrade, my equal".

The Buddha corrected them because he wanted them to learn the Dhamma from him, and it was important that they respect him as a teacher (because if they didn't respect him and didn't accept him as a teacher and listen to his teaching then that would be to their harm).

They were telling him that, as far as they knew, he had no attainment and no knowledge worth teaching:

For a third time, Prince, the group-of-five monks said this to me: “But you, friend Gotama, by that ascetic lifestyle, that practice, that difficult way of living, did not attain states beyond ordinary human beings, a distinction of what is truly noble knowledge and insight.

So how can you now, given to luxury, forsaking the striving, gone back to luxury, attain states beyond ordinary human beings, a distinction of what is truly noble knowledge and insight?”

He contradicted them:

When this was said, Prince, I said this to the group-of-five monks: “Are you aware, monks, of my having spoken to you like this before?”

“Certainly not, reverend Sir.”

“The Realised One, monks, is a Worthy One, a Perfect Sambuddha. Lend an ear, monks, I will instruct you about the attainment of the Deathless, I will teach the Dhamma, and following the path as it has been preached, after no long time in regard to that good for which young men of good family rightly go forth from the home to the homeless life, that unsurpassed conclusion to the spiritual life, you will dwell having known, experienced, and attained it yourselves in this very life,” and I was able, Prince, to persuade the group-of-five monks.

This version of the story includes the following explanation:

ON seeing their old teacher approach, the five bhikkus agreed among themselves not to salute him, nor to address him as a master, but by his name only. "For," so they said, "he has broken his vow and has abandoned holiness. He is no bhikkhu, but Gotama, and Gotama has become a man who lives in abundance and indulges in the pleasures of worldliness." But when the Blessed One approached in a dignified manner, they involuntarily rose from their seats and greeted him in spite of their resolution. Still they called him by his name and addressed him as "friend Gotama."

When they had thus received the Blessed One, he said: "Do not call the Tathagata by his name nor address him as 'friend,' for he is the Buddha, the Holy One. The Buddha looks with a kind heart equally on all living beings, and they therefore call him 'Father.' To disrespect a father is wrong; to despise him, is wicked. The Tathagata, the Buddha continued, does not seek salvation in austerities, but neither does he for that reason indulge in worldly pleasures, nor live in abundance. The Tathagata has found the middle path.

This is from an 1894 version titled The Gospel of Buddha by Paul Carus -- I don't know which original text this is a translation of (or whether it's even translation instead of invention).

One place where the story appears is the Ariyapariyesana Sutta (MN 26) which matches the story in the OP. This one is addressed to monks rather than to "Prince Bodhi" so I suppose there are several versions of this story.