This one has been one of my all-time pet peeve. Even seemingly intelligent people have uttered this to me.
One of the most frequent "come backs" when I discuss physics and especially electrons, etc. is that I always get the complaints that "we have not observed an electron", so it can't be real. Of course, one then ask what is meant by "observe", and immediately people who made such comments realized how silly it is to restrict this to simply things that we can see with our eyes. So then they retract by saying that we haven't "detected" an electron. All we have detected are just streams and streams of them in the form of currents, etc.
Automatically, I assume that these people do not buy into what have been observed in bubble chambers and other particle trackers in high-energy physics colliders. But even so, there are two different ways to tackle this:
1. The concept of an "electron" isn't just defined as an "object", certainly not in the classical sense. As with anything in physics, the entity is defined by a series of properties. These properties have consequences that are measurable. So if an electron has so-and-so properties, one should be able to measure all of them, especially if they involve the behavior of a collection of them. We do not need to directly observe or detect an electron to verify its existence. In fact, if one were to look closely, almost everything that we accept to "exist" is based on such inference. We accept something to exist when it fulfills all the criteria that corresponds to a set of property belonging to that object. The same is true with an electron. So to single out the electron as to not be a valid entity is rather puzzling.
2. Even with Point #1, scientists never rest on what we already have and continue to test our knowledge. Now there is a direct detection of single electrons via real-time counting of current measurement. A report of this work can be found here.
This is the most direct detection of single electron. Would this somehow silence all those doubters of the existence of the electron? One can only hope....
 J. Bylander et al., Nature v.434, p.361 (2005).