How to hyperstimulate a sex-recognition algorithm. Women have higher luminous contrast between their facial skin and their lip/eye color. This contrast effect is influenced not only by degree of lightness but also by degree of redness (source).
Women are fairer in complexion because their skin has less melanin and less blood (Edwards and Duntley, 1939). Male castrates similarly look pale (Edwards et al., 1941). For this reason, relative lightness or darkness of skin seems to be key to sex recognition. People especially focus on the luminous contrast between facial skin and the color of the lips and eyes (Russell, 2009; Russell, 2010; see also Dupuis-Roy et al., 2009):
The luminance contrast between the eyes and the surrounding skin and the lips and the surrounding skin has been termed ‘facial contrast’. Female faces have greater facial contrast than male faces, and facial contrast plays an important role in sex classification and the perception of masculinity and femininity and also attractiveness (Porcheron et al., 2013)
This contrast effect has influenced the development of cosmetics. Different societies, sometimes independently of each other, have invented ways to increase facial contrast by darkening the lips and eye area and, conversely, by lightening facial skin (Russell, 2009; Russell, 2010).
Ancient Egypt was an early center for the development of cosmetics (Dayagi-Mendels, 1989). Indeed, the Egyptians “had most of the cosmetic aids which have ever been devised” (Corson, 1972, p.8), including rouge for the lips and cheeks, eyeliner (kohl), eyeshadows, and foundation […] In Mesopotamia, pots of colored paints for the eyes, and rouges for the lips have been found in Sumerian tombs near Ur from 5,000 years ago.
[…] Excavations at Harappa and Mohenjo-daro have found kohl pots and sticks for lining the eyes, as well as red iron oxides and white lead-based compounds that have been surmised to be rouge for the lips and cheeks and foundation for lightening the skin […] Evidence for ancient uses of received cosmetics in the East Asia is less clear, though there is a long history of the use of white face paint and rouge for the lips in China and Japan (Corson, 1972). Overall, there is evidence for the idea that the received style of cosmetics developed in multiple centers of early technology development, and spread outward to peripheral areas such as Europe and Southeast Asia, analogous to the spread of other technologies like agriculture and writing. (Russell, 2010)
This contrast effect may have likewise influenced evolution within our species. The most common human phenotype combines relatively light skin with dark hair and dark eyes. The opposite pattern is biologically possible but rare. In addition, the whitening of European skin might have provided a sufficient degree of luminous contrast for hair and eye color to diversify.
Not just luminous contrast
Facial contrast involves differences not only in luminosity, i.e., lightness/darkness, but also in hue, particularly the redness of the lips versus the lack of ruddiness of adjacent facial skin. Contrast in luminosity and hue declines with age and may provide a means for distinguishing younger from older women:
Several aspects of facial contrast – the luminance and color differences between the facial features and the skin surrounding those features – were found to decrease with age. These included the luminance contrast around the eyes and eyebrows, the red-green (a*) contrast around the mouth and eyes, and the yellow-blue (b*) contrast around the eyes. (Porcheron et al., 2013)
Hue contrast explains why redness—lipstick, red clothing, etc.—seems to enhance femininity even though women are the less ruddy sex. Red colors highlight this lack of ruddiness, thereby stimulating the mental algorithm that people use for sex recognition:
Stephen & McKeegan  found that people increase the redness (a*) of the lips to make a female face appear more feminine and attractive. Elliot & Niesta  found that pictures of women are perceived by men as more attractive and sexually desirable when they are associated with the color red–whether by the placement of a red border around the picture, or the presence of a red colored shirt on the woman. (Porcheron et al., 2013)
Some questions remain unanswered. Why, for instance, do many women redden their cheeks? Are the cheeks a secondary contrast zone? And what about women from racial groups with dark facial skin?
This last question seems to have given the study’s authors some trouble with the reviewers. Evidently, darker-skinned women still have some facial contrast. They just don’t have as much, probably because they don’t need as much. Dark skin tends to characterize highly polygynous societies; in fact, the correlation between darkness of skin and polygyny rate is greater than the one between lightness of skin and latitude (Manning et al., 2004). In such societies, women are almost sure to get mated and remain so as long as they are fertile. There is thus only weak selection for those who can better attract potential mates and more strongly stimulate sex-recognition algorithms.
On a final note, it’s no coincidence that the polygyny rate correlates inversely with latitude. In simple tropical societies, especially those with year-round farming, women can more easily provide for themselves and their children without male assistance. Since women cost so little to support, men will tend to seize all mating opportunities. A man will have as many wives as he can get, and not as many as he can support.
Dupuis-Roy, N., I. Fortin, D. Fiset, and F. Gosselin. (2009). Uncovering gender discrimination cues in a realistic setting, Journal of Vision, 9(2), 10, 1–8. http://journalofvision.org/9/2/10/
Edwards, E.A. and S.Q. Duntley. (1939). The pigments and color of living human skin, American Journal of Anatomy, 65, 1-33.http://onlinelibrary.wiley.com/doi/10.1002/aja.1000650102/abstract;jsessionid=43509EAC6110062BBF5676995DA16C6B.d01t01?deniedAccessCustomisedMessage=&userIsAuthenticated=false
Edwards, E.A., J.B. Hamilton, S.Q. Duntley, and G. Hubert. (1941). Cutaneous vascular and pigmentary changes in castrate and eunuchoid men, Endocrinology, 28, 119-128.http://endo.endojournals.org/content/28/1/119.short
Manning, J.T., Bundred, P.E., and Mather, F.M. (2004). Second to fourth digit ratio, sexual selection, and skin colour, Evolution and Human Behavior, 25, 38-50.http://www.ehbonline.org/article/S1090-5138(03)00082-5/abstract
Porcheron, A., E. Mauger, and R. Russell (2013). Aspects of facial contrast decrease with age and are cues for age perception, PLoS ONE 8(3): e57985http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0057985
Russell, R. (2010). Why cosmetics work. In Adams, R., Ambady, N., Nakayama, K., & Shimojo, S. (eds.) The Science of Social Vision, New York: Oxford.http://public.gettysburg.edu/~rrussell/Russell_SocialVision_cosmetics_chapter.pdf
Russell, R. (2009). A sex difference in facial contrast and its exaggeration by cosmetics, Perception, 38, 1211-1219http://public.gettysburg.edu/~rrussell/Russell_2009.pdf