Leap-ing Through Time: From Gesture Technology to GenAI
Nearly a decade ago I was introduced to Leap Motion, a company that was attempting to revolutionize human-computer interaction with gesture technology. The Leap Motion Controller is/was an optical hand tracking module that captures the movements of your hands (ex. imagine drawing with your hands in mid-air). The co-founder was a guy named David Holz who would eventually release MidJourney. The Leap Motion Controller replaced the mouse and track pad on my laptop. I learned how to use the device to draw arrays using Processing programming language.
Processing is a free graphics library and integrated development environment (IDE) built for the electronic arts, new media art, and visual design communities with the purpose of teaching non-programmers the fundamentals of computer programming in a visual context. — wikipedia
In computer programming, an array holds any type of data, and each element can be individually assigned numbers or values. Using Processing, I created arrays to store data from the Leap Motion Controller. As I moved my hand in the Controller’s field, the software stored cursor coordinates as X Y variables (numbers). At every frame of animation, the variables were replaced with new numbers and the previous ones were discarded that, when displayed on a computer screen creates a trail.
I synced the movements of my hand within the Controller’s sensor field to music by Drexciya, an electronic music duo from Detroit (see above). The resulting images reminded me of artist Ellen Gallagher who also channeled Drexicya in her work. Gallagher created unique collages on scrimshaw by carving into the surface of thick sheets of watercolor paper and drawing with ink, watercolor, and pencil. My Leap/Processing-generated images disrupted the solid surface of the computer screen.
The emergence of generative AI or GenAI, more specifically neural style transfer or NST opened a door to new ideas. NST software algorithms manipulate digital images in order to adopt the visual style of another image. In 2016, I started using Deep Dream Generator or DDG, a platform that included NST tools. Meanwhile, Leap Motion founder David Holz was struggling to find a market for the Leap devices and he pivoted to GenAI. Before Holz released MidJourney and before text-to-image exploded on the world, I created works such as this one using DDG’s Deep Style tool:
I used the Leap/Processing output images as art styles in DDG/Deep Style. The content or source images included personal and stock photos and other images I found on the Web. Also, I used the Leap Motion Controller to “draw” on top of the Deep Style images. I worked mostly under the radar but then the world changed. With the Covid-19 pandemic came the release of MidJourney. Like ChatGPT, MidJourney is based on large language and diffusion models. This type of AI has been trained on huge volumes of data to acquire an understanding of natural language, and an ability to predict word successions forming coherent sentences.
Today, instead of coding, artists can to find ways to push the limits of digital imaging through GenAI. Image quality and detail depend on the ability to write precise, well-formulated prompts. Prompts can include descriptions of characters, objects, sets, and even indications of style, color, lighting or camera angle. Staying with the ‘Mami Wata’ subject matter and array/water theme I prompted MidJourney to generate new images. I’ve revisited this subject/theme across tools and platforms, using devices such as the Leap Motion Controller, Deep Style and text-to-image (ex. DDG, MidJourney).
David Holz recognized the potential to enhance AI-generated images and now MidJourney is used worldwide by graphic artists, designers, cartoonists, journalists, video game and animated film creators, and architects. Among the many other AI image generators MidJourney stands out for the aesthetic appeal and artistic quality of the content it produces. However, the tool does not act alone. The images that stand out are ones that come from artists’ ideas and prompts: visual language, subjects, themes, and other human-generated knowledge.
To exploit MidJourney’s full potential, it is necessary to understand the workings of GenAI, language models and machine learning. Expertise in prompt engineering is also important. Also, the idea of using a “seed” or source image came from practice using previous tools, guided by culturally relevant and creative frameworks that guide one’s approach to using specific GenAI tools such as MidJourney. Knowledge of folklore (water spirits), cosmology, literature, and even traditional artworks and styles are just as important as the AI algorithms and training models.