The Gilded Portraits: Facial Skin Reflectance + A.I.
Subsurface scattering in computer graphics allows light to penetrate the surface of translucent objects (i.e., skin, milk, marble) and is then scattered by the interaction of these objects with other materials (Jensen, et al. 2001). This process gives computer-generated objects a glow like the effect seen in 17th century paintings such as Johannes Vermeer’s “Girl with a Pearl Earring” (or see above) that shows how light softens and glimmers on the subject’s facial features.
The problem, according to Theodore Kim (2021), “skin glow” from subsurface scattering actually mutes or dulls the features of darker skin. In the example above (“David”) you can see how subsurface scattering translates to rendering dark skin except for “c” that demonstrates a new term I learned about: specular reflection. It is specular reflection or “shine” that gives dark brown-skinned subjects like the ones in artist Kehinde Wiley’s large-scale figurative paintings their character. The amount of specularity of an object or subject depends on how shiny it is and how dark it is. So what if we could use technology to add specularity to an image? This is a question I’m addressing in my A.I. artwork: adding a shine (style) layer over the images.
To mitigate the subsurface scattering issue, we must adjust the algorithmic processes that apply specular reflection when creating or modeling subjects with darker skin tones. I recently started using neural image style transfer to create “gilded” portraits of prominent Black people who had passed away. In life, these people impacted or inspired me in a positive way. I think the Jessye Norman portrait was the first Deep Dream Generator experiment (see above). As time went on I adjusted the image style, adding the shine effect you see in Wiley’s (analog) paintings.
I often share these images on social media but not to sell them. I find enjoyment when looking at the results of using and collaborating with artificial intelligence (because that’s what it is). Also, I’m curious about how others see the work. Facial recognition A.I. has led to harmful technological designs, so incorporating A.I. in my creative work is one way to raise awareness and to mitigate against its harmful effects. I’m using a process known as image style transfer that renders new images using multiple styles (Miller 2019). The content (input images) is combined to recreate the original content in the style of other images.
The “gilded” series of A.I. portraits apply specular reflection (shine) as a layer on top of typical subsurface scattering (glow). The algorithm even creates a halo effect around the subjects. The halo was an unintended result of using the “deep style” algorithm. Also, what happens to the edges of things in an image is what led me to find and create style images that gave the portraits more shine. The styles come from antique tapestries and other shiny objects. I think this is just the beginning of a new chapter in my exploration of A.I.
References
Ghosh, Abhijeet, Tim Hawkins, Pieter Peers, Sune Frederiksen, and Paul Debevec. 2008. “Practical Modeling and Acquisition of Layered Facial Reflectance.” ACM SIGGRAPH Asia 2008.
Jensen, Henrik Wann, Stephen R. Marschner, Marc Levoy, and Pat Hanrahan. 2001. “A Practical Model for Subsurface Light Transport.” Proceedings of the 28th annual conference on Computer graphics and interactive techniques — SIGGRAPH ‘01.
Kim, Theodore. 2021. “Anti-Racist Graphics Research (SIGGRAPH 2021).” YouTube.
Kim, Theodore, H. J. Dorsey Rushmeier, Derek Nowrouzezahrai, Raqi Syed, Wojciech Jarosz and A. M. Darke. 2021. “Countering Racial Bias in Computer Graphics Research.” ArXiv abs/2103.15163.
Miller, Arthur I. 2019. The Artist in the Machine: Inside the New World of Machine-Created Art, Literature, and Music. Cambridge, MA: The MIT Press.