Have you ever done geocaching or letterboxing? Both of these are sort of like modern-day treasure hunting games my family and I enjoy that involve using clues to locate a hidden box or object. The objects are usually obscured by a rock, or leaves, or some other camouflage, but if someone accidentally stumbles upon them there is nothing else protecting them and nothing stopping the someone from taking the object. When I was a kid my friends and I had a secret fort in the woods that operated on the same sort of security principle- nobody else knew it was there, so it must be safe.
The reality is that security by obscurity does not offer any real security. When it comes to computer system or application vulnerabilities, if one person can find it then so can others. One of those 'others' may have questionable morals or shady ethics and might use their knowledge of the obscure vulnerability and exploit it to compromise target computers.
I have addressed the issue in the article Security Through Obscurity: How Secure is Security by Obscurity?. At the very least, it is my opinion that the hole can be kept secret as an additional layer of security, but that there also must be mitigating controls or protective measures to safeguard the system in case someone else learns of the obscure hole.
Two security experts I respect a great deal have written a sort of point-counterpoint article on the subject. Roger Grimes and Jesper Johansson, authors of Windows Vista Security, address the value and the weaknesses in the security by obscurity philosophy and try to determine whether it has any merit as a security control. Read The Great Debate: Security by Obscurity.