Neural Image Transfer to Prompt: Developing an Aesthetic
Damask refers to a broad group of vintage woven fabrics made on a jacquard loom. Before prompt-based AI generators entered the mainstream, I was experimenting with these fabric patterns as style reference images for AI-generated portraits. What attracted me to the fabrics was the intricate, woven details. I applied the patterns using neural style transfer, a class of algorithms or tools that manipulate digital images in order to adopt the appearance or visual style of another image. NST was first published in a 2015 paper titled “A Neural Algorithm of Artistic Style” by Leon Gatys et al. Some refer to NST as the “Gatys algorithm” and Deep Dream Generator refers to NST as “Deep Style.”
I spent several years exploring and experimenting with DDG’s Deep Style tool. In 2021 I applied fabric (lace) as an image style reference to a photo of Helen Keller for the 2021 FUTURES exhibition at the Smithsonian. In many of the photos Keller wore vintage lace. The curators obtained permission for me to use the Keller photo as a source image and from that I create the final image that was printed and displayed in the museum. Note: the lace was one of several reference images that I used.
A year later, the first prompt-based generative AI tools hit the mainstream, including OpenAI’s Dall-E and Midjourney. These tools eclipsed Deep Dream Generator’s ‘Deep Style’, so much so that DDG soon offered their own prompt-based tools. I wanted to see if I could incorporate what I was doing with vintage fabrics in DDG with the prompt-based tools.
What I was able to do is layer multiple images to create something similar but very different from the earlier DDG Deep Style images. The newer images have a lot more depth and color interaction. The color choices come from my art knowledge, esp. color theory. This was made possible through image editing and prompt engineering. Prompts are natural language texts describing tasks that an AI should perform. Midjourney also allows users to add images (as urls) to their prompts.
With prompt-based tools like Midjourney I can mix different fabric styles (see image above). For example, I was learning about sashiko (Japanese) embroidery and incorporated that into one of my prompts. Kuba cloth is another favorite and I was able to generate some of the patterns in recent AI-generated images. Kuba textiles are a type of raffia cloth unique to the Democratic Republic of the Congo, formerly Zaire, and noted for their elaboration and complexity of design and surface decoration.
My goal with these images is not to simply type “damask,” “kuba cloth,” or “sashiko” in a prompt, rather I’m challenging myself and the AI’s capacities for processing multiple visual stimuli in prompts. Researcher Abigail Housen refers to this process as aesthetic development and she identified different stages or patterns of behavior that cross both cultural and socioeconomic boundaries.
Note: Exposing oneself to art is the only way to develop one’s aesthetically receptive portions of their mind.
According to Housen most art viewers are accountive and constructive. Accountive viewers base their judgments on what they know and like. This includes people who post brief comments or use emoticons via social media. Constructive viewers use their own perceptions of, knowledge of, and values of their world to build frameworks for looking at art. This includes people who point out details in an image and tie in those details to something they know.
The other three stages of aesthetic development are classifying, interpretive, and re-creative. Classifying viewers have some art knowledge and can constructively critique images. Interpretive viewers seek to interact with art and go deeper to find underlying meaning. Re-creative viewers are “willing to suspend disbelief” and acknowledge that artworks have a life of their own. Art is not supposed to be stagnant or forced into boxes (ex. a canon). For me, the use of generative AI to create “art” is about re-creation: my art knowledge and memories are infused in the process of generating and prompting digital images.
References:
DeSantis, K., & Housen, A. (2005). A brief guide to developmental theory and aesthetic development. Visual Understanding in Education.
Art Viewing and Aesthetic Development: Designing for the Viewer