score:6
As you said, the ninja or shinobi was first and foremost a master of disguise.
This was known as Hensojutsu, the art of disguise.
From Hatsumi, Masaaki. The way of the ninja: secret techniques. Kodansha International, 2004. p92
For most missions a sterotypical black ninja outfit would not have been the most prudent option. The ninja then attempted to look like whatever the situation called for. Some examples below.
Undercover operations are to be found throughout Japanese history, and one component of the modern ninja cult has been to exaggerate this fact by crediting certain historical figures with being ninja or βproto-ninja.β The legendary Prince Yamato Takeru resorts to subterfuge on at least one occasion including dressing up as a woman, making him a ninja in some eyes (Aston, 1972, p. 201)
And
He sent two Matsukura retainers as spies to Manila disguised as merchants, ostensibly to discuss reopening trade.
From Turnbull, Stephen. "The Ninja: An Invented Tradition?." Journal of Global Initiatives: Policy, Pedagogy, Perspective 9.1 (2015): 3. p12
For their commbat missions, however, a ninja fell back on armor that blended with his environment. The color needed to be flexible since black is not a good choice for blending in at all times. Colors like earth brown, watery blue and even snow white were utilised.
The author below descibes the ninja's armor as rainbow colored to fit in with the colors of nature.
One day, I was asked, "What color is Shinobi color?" I answered immediately, "Rainbow-color of course, the colors of a prism. They blend with nature's colors, don't they?
From Hatsumi, Masaaki. The way of the ninja: secret techniques. Kodansha International, 2004. p90
The author even refers to clan colors on page 92. You might notice that there is indeed a reference to black as the color of the Genji.
Certain old documents refer also to the "armor of the four clans," saying, "the head of the Genji wore black, the Heike violet, the Fujiwara light green, and the Tachibana yellow." This means using colored armor for Henso-jutsuβ not merely techniques for changing one's appearance, but methods for changing the whole flow of battle.
As conclusion: A ninja indeed used a specific outfit that blended in when infiltrating. The color black was used, but it was only a small part of the ninja apparel. Their clothes' color was adapted to their surroundings