Designing Interactions: Urban Communication in the Age of GenAI

Nettrice Gaskins
4 min readJul 21, 2024

--

Cognitive-computer artifact diagram created by the author

For many years, the importance of cognitive artifacts — artificial devices designed to serve a representational function — and their use to enhance human abilities has been ignored within much of cognitive science despite earlier scientific inroads (Norman 1991). Scientists have paid little attention to how artifacts are invented, acquired, or transmitted across individuals or generations, especially in particular branches of education (and art) that are dedicated to outfitting learners with technologies for urban communication.

Accounts can be found in cultures that use technology to merge specialized forms of representation such as alphabets and drawings (ex. graffiti) and choreographic notations (dancing) based on symbolic, linguistic, mathematical and scientific formulations, programming languages, hardware (robotics, handheld devices), software (game platforms), etc. For example, Doze Green, a hip-hop pioneer and visual artist (above) adeptly connects Wild Style graffiti and street dancing. Green is part of a specific community of practice that invites various, regional styles. Here’s another example comparing West Coast krumping with West African dancing:

For Doze Green and the West Coast street dancers the technology is embodied and the user interface is the cypher or circle. When I first watched “Rize” in 2005 I was amazed by the similarities between the street and traditional African dancers. The film chronicles a dance movement that rose out of South Central Los Angeles with roots in clowning and street youth culture The clip above adds another layer of representation on the performances. Dance does not merely form a part of community life; it represents and reinforces the community itself. Its structures reproduce the organization and the values of the community. So what happens when you incorporate computers or even generative artificial intelligence?

Dancer Valencia James + Carnival AI app (Attracting Shapes)

Donald A. Norman (2009) puts technological innovations into two groups: conceptual breakthroughs that have a huge impact on society and continual improvements that merely improve on an existing technology. Because people are embedded in their environments, they are densely joined to the world outside, as well as the representations in their heads. For artists, GenAI adds a layer of representation and expression on our practices and performances. It is through interaction with multimodal interfaces that different communicative phenomena — gesture, gaze, posture, and object manipulations — can function collectively. In 2021, I was part of a three-person team that received a Mozilla Creative Media Award.

Using Carnival as a vehicle, the project creates a space where Caribbean and Black communities can engage with, learn about, and interrogate AI to benefit themselves. The project brings together AI, architecture, digital heritage, and art to create and feature AI-generated dancing sculptures, AI-generated Blue-Devils and Jab Jabs in the Carnival, and an app for visitors to create visual art through their own body’s movement and dance. — Mozilla

Tommy the Clown in Kendrick Lamar’s “Not Like Us” music video
“Not Like Us” video game

We developed the Carnival AI app to celebrate culture, using pose estimation that detects the position and orientation of a person or artifact. Users require camera-enabled devices to ‘play’ with the app. More recently, a game designer named Richie Branson released a free-to-play game titled Not Like Us: The Video Game that is inspired by the music video of the same name with Kendrick Lamar and what looks like the entire city of Compton, California. The music video features Tommy the Clown (above) who is credited for inventing clowning, which evolved into krumping (as seen in “Rize”). In the video Tommy presses a button on a Technics SL-1200 MK2, that was adopted by DJs such as Grandmaster Flash.

Technics SL-1200 MK2

The button press triggers the song “Not Like Us” with a performance by T Squad (Tommy’s crew) dancer Nvy who does a crip or blood-walk (c/b-walk) dance move in time to the music. Serena Williams, a Compton native, performed the walk at the 2012 Summer Olympic tennis tournament after winning a match at Wimbledon. Watching the “Not Like Us” music video reminded me of my earlier cognitive studies research, specifically interaction with multimodal interfaces and a constellation of cognitive-computer artifacts that are created or used by communities of practice (hip hop, street dance, etc.). I’m interested in how technological innovation is embodied through performance and enhanced using apps, games, etc.

--

--

Nettrice Gaskins
Nettrice Gaskins

Written by Nettrice Gaskins

Nettrice is a digital artist, academic, cultural critic and advocate of STEAM education.