Who said "If I want the opinion of the middle classes, I'll ask Her Majesty"?

This has been quoted a few times, in slight variation, like here, with attribution indeed to Disraeli:

— Harry Blamires: "The Victorian Age of Literature", Longman literature guides, Longman, 1988. p10 (gBooks)

And used here as well:

He [Prince Albert] never became really popular with the aristocracy or the working man, but it was otherwise with the middle classes, and they were the dominant factor in the national life in the nineteenth century. His solid qualities appealed to them, and they felt that he understood them. He could talk to them of schools and docks, of architecture and warehouses, and with them he slowly but surely became an undoubted success. He interpreted their point-of-view to the Queen, and with such accuracy that in due course a Prime Minister was to say:

“If ever I want to know what the middle-classes are thinking, I always ask Her Majesty’s opinion.”

— Look & Learn: "Queen Victoria was a symbol of Britain’s people and her Empire", Posted in Historical articles, History, Royalty on Thursday, 28 March 2013.

Caveat: while these two sources are not the worst one can think of in the usual terms of reliability, all they really do is confirm that it is not entire conflation in OP's mind. As both sources here lack a precise attribution, so that this is currently on the level of 'unconfirmed anecdote': "once said".

And to complement this caveat, we also see this version/variation of the theme, now attributed similarly vaguely, but to Lord Salisbury:

But although the queen shared some of the tastes and values of her most respectable subjects (Lord Salisbury later declared that if he knew what the queen thought about an issue, he knew what the middle classes would think), and although in later life her deliberate shunning of the more ostentatious trappings of royalty made it easy to think of her as a bourgeois widow at the head of the family firm, she was in fact sui generis, one of a kind. As Arthur Ponsonby put it, 'She bore no resemblance to an aristocratic English lady, she bore no resemblance to a wealthy middle-class Englishwoman, nor to any typical princess of a German court. … she was simply without prefix or suffix “The Queen”' (Ponsonby, 70).

— H. C. G. Matthew & K. D. Reynolds: "Victoria (1819–1901)", Oxford Dictionary of National Biography, Published in print: 23 September 2004Published online: 23 September 2004This version: 24 May 2012. doi

But at least this version can firmly be sourced for one later occasion, empasising that it was a repeated adage, in the House of Lords, after Victoria's death 1901:

I have said for years that I always thought that when I knew what the Queen thought I knew pretty certainly what view her subjects would take, and especially the middle classes of her subjects.

— Lords Chamber: Volume 89: debated on Friday 25 January 1901

Notably, 'being (like) middle class" was a conscious strategy of public relations for the royals — or rather: has been to a certain extent — fostered by various prime ministers since Victoria's reign. Cf.:
— David Tomkins: "Just Like Us: Victoria, Albert and the middle-class family (part 1 of 4)", Journal of Victorian Culture, 2019.

That 'lower class appeal' was of course counter balanced then with even more pomp for all the invented traditions in her time, like the coronation procedure…

Note that for the British monarchy this now rather long running strategy of — 'the queen being very much in touch with the people' with an emphasis on the middle class — as a concept was first 'tried' with Queen Caroline a few years earlier.

Rather, the pervasive perception of the queen's ill treatment was that it was pitting the nation as a whole against the higher orders. "One of the complaints now made against her MAJESTY," argued one editorial in Leigh Hunt's Examiner, was that she had appealed to the people; "but to whom else could she appeal? had not the higher, and, as they were called, the more illustrious classes of society, entirely deserted her?" The queen's cause came to be seen as the cause of the people; and the judgment of the people was seen to be embodied in the ubiquitously invoked voice of public opinion.

— Dror Wahrman: "'Middle-Class' Domesticity Goes Public: Gender, Class, and Politics from Queen Caroline to Queen Victoria", Journal of British Studies, Vol. 32, No. 4, Making the English Middle Class, ca. 1700-1850, 1993, pp. 396–432. (jstor)

The 'popularity and ceremonial appeal' of the monarchy fell off shortly after George III’s death in 1820, Colley writes, and it was not revived until late in the century. […] when Britain was predominantly a provincial society in which the London-based monarchy was alienated from the rational middle class in the provinces and remained controversial because of its residual political powers. Performing an ineptly managed ritual of public ceremonies that attracted little interest, this dowdy and unpopular monarchy, appears in Cannadine’s essay as an institution with so little public appeal that it was not even worth exploiting commercially. In a second essay Cannadine has taken his inquiry into the `myth’ of British royalty further. Victoria and Albert did not modernise the monarch as a ceremonial head of state, he writes; they aspired to govern as well as reign, and pageantry was of little interest to them. […]

— Alex Tyrrell & Yvonne Ward: "`God Bless Her Little Majesty.’ The Popularising of Monarchy in the 1840s, National Identities, Vol. 2, No. 2, 2000. (p109–125, doi)

This notion of the queen as a model was much played on in the years following the accession of Victoria, the 'Rosebud of England', when the wives and mothers of England could all claim to be queens in their own homes, however modest, and to follow the young queen in her celebration of marriage.26 Public opinion had decreed that the royal family must indeed be a family; kings and queens must be fathers and mothers in their own home if they were to be fathers and mothers to the people.

— Leonore Davidoff & Catherine Hall: "Family Fortunes. Men and Women of the English Middle Class 1780–1850", Routledge: London, New York, ³2019. p154

If this widely shared and cultivated sentiment of Victoria having close ties of understanding towards the middle classes could be tied really to Disraeli — with firm proof currently lacking — a possible reading of the line in question as something 'Disraeli looking down on the Queen for her low standing manners', then we might need the reminders that Disraeli was himself of a middle class background, was oriented indeed towards the upper classes, but moderated his negative views upon the middle classes in time, was primarily known for venerating Victoria and showering her with very flattering compliments constantly.
All in this configuration between classes perception between Victoria and Disraeli is exemplary condensed in this known anecdote:

Queen Victoria warned Disraeli that "it may not please the Navy in which so many of the highest rank serve … if a man of the Middle Classes is placed above them in that very high. [Talking about the knew First Lord of the Admiralty William Henry Smith]; italics in the original, LLC]
[…]
Victoria, who had named Disraeli Earl of Beaconsfield the year before, clearly did not consider her prime minister "a man of the Middle Classes," and whatever may be said of the social origins of the son of the Jewish literary critic who had become in turn a novelist, a Member of Parliament, and the squire of Hughendon Manor, he never saw himself nor was he seen by others as a tribune of the middle classes. The purpose of the Conservative party leadership, Disraeli wrote in 1848, was "to uphold the aristocratic settlement of this country." In the words of his biographer Robert Blake, "this assertion is the key to Disraeli's policy for the rest of his life."

— Walter L. Arnstein: "The Myth of the Triumphant Victorian Middle Class", The Historian, Vol. 37, No. 2 1975, pp. 205–221. (jstor)