score:8
No system is so secure that there are no ways around it. Yes, the loophole you describe sort of exists, but it's not that straightforward in many airports. For one, at pretty much any airport, departing and arriving passengers are strictly separated, making it very difficult to join a crowd of arrivals when you are supposedly departing yourself.
Yes, in some cases, it would be possible to sneak out of a queue, or to not walk towards the airplane, but then you somehow have to get out of the airport, or on board another plane. Depending on the airport, this will not be easy and might result, when caught, in all sorts of safety measures being enacted.
Then, as @Gagravarr points out, cabin crew typically count the number of passengers on board and reconcile this with the passenger manifest.
Having said this, particularly in more out of the way airports, small airports in Africa, for example, safety measures are rather ineffective.
But, not only in out of the way airports: http://www.nydailynews.com/new-york/exclusive-150-flight-mexico-skip-customs-leave-jfk-article-1.2443629 and http://www.dailymail.co.uk/news/article-3414989/American-Airlines-passengers-arriving-international-flight-JFK-allowed-exit-without-going-Customs.html