Here I take a central thesis of a direct realism theory of perception to be the idea that if we are directly aware of objects, and not a sense-datum or idea, then we have to say that things such as colour must be such that reference can be made to them without reference to any subjective or phenomenal experience of perceivers- we cannot reference colour except by way of referencing it as we experience it, ergo phenomenal concepts. However, how can colour be referenced in a way that avoids phenomenal concepts and still be about colour in any coherent way?
John McDowell explains further, referencing J.L. Mackie’s view of primary and secondary qualities (in which experiences of, say, red do not need to be understood in terms of the experiences the red object gives rise to):
‘According to Mackie, this conception of primary qualities that resemble colours as we see them is coherent; that nothing is characterized by such qualities is established by merely empirical argument. But is the idea coherent? This would require two things: first, that colours figure in perceptual experience experience neutrally, so to speak, rather than as essentially phenomenal qualities of objects, qualities that could not be adequately conceived except in terms of how their possessors would look; and, second, that we command a concept of resemblance that would enable us to construct notions of primary qualities out of the idea of resemblance to such neutral elements of experience. The first of these is quite dubious…But even if we try to let it pass, the second seems to be impossible. Starting with, say, redness as it (putatively neutrally) figures in our experience, we are asked to form the notion of a feature of objects which resembles that, but which is adequately conceivable otherwise than in terms of how its possessors would look (since if it were adequately conceivable only in those terms it would be secondary). But the second part of these instructions leaves it wholly mysterious what to make of the first: it precludes the required resemblance being in phenomenal respects, but it is quite unclear what other sense we could make of the notion of resemblance to redness as it figures in our experience.’ (‘Values and Secondary Qualities’, in ‘Essays on Moral Realism’, ed. Geoffrey Sayre-Mccord, p. 169)
I think the following argument can thus be extracted:
Direct realism holds that reference to colour (or any phenomenal quality) can be made apart from phenomenal concepts – or, there is a neutral figuring in experience for colour.
We cannot reference colour except by way of phenomenal concepts – or, there is no neutral figuring in experience for colour.
Therefore, a direct realism theory of perception is false.