score:1
This is covered under paragraphs 28 & 29 of the UK Immigration Rules https://www.gov.uk/guidance/immigration-rules/immigration-rules-part-1-leave-to-enter-or-stay-in-the-uk Applications for a visitor visa may be made to any British Diplomatic Mission, British Consular post or the office of any person outside the United Kingdom and Islands who has been authorised by the Secretary of State to accept applications for entry clearance. So you can legally apply from Switzerland.
What may be concerning your visa application consultant is that typically for a UK visitor visa it matters hugely to be able to demonstrate strong ties to your home country/place of residence that show you have a compelling reason to return there rather than overstay in the UK. Going for a job interview in the UK inherently shows immigrant intent. So you should look to provide strong proof of your intention to leave the UK at the end of your visit.