You, Me, and Our Computers | Lisa Jamhoury

An exploration of social, embodied interaction over distance

In 2020, nearly all social interaction moved online due to the Covid-19 pandemic. Professional, academic, and leisure activities moved to Zoom, where people were mostly seated interacting with others over video and text chat. In contrast, You, Me, and Our Computers is a series of websites exploring social, embodied interaction over distance. Using computer vision, open-source machine learning algorithms, and peer-to-peer networking, YMOC places participants’ images and joint positioning on shared digital canvases and uses visual and auditory interactions to encourage explorations of virtual touch and co-presence over distance.

YMOC aims to make the new frontier of web-based social, embodied interaction more accessible. During the SloMoCo Spring Microresidency, YMOC will rapidly generate, test, and document the series of experimental web experiences in collaboration with the SloMoCo community. All code will be made available under open-source licenses and all documentation will be publicly available.

social

Spring and Summer microresidency
public presentation
Me, Me, and My Computer

Audio-video work created for Immediations, at ASU DesignSpace. Me, Me, and My Computer is a prologue to my SloMoCo project, Me, You, and Our Computers.

Sketch 1: Contours/Fills

Video of playtest for first sketch in YMOC series, Contours/fills.

Sketch 2: Overlap

Video of playtest for second sketch in YMOC series, Overlap. Tested with Renee Carmichael. She is in Argentina. I am in the US.

Sketch 3: Frame Difference

Video of playtest for third sketch in YMOC series, Frame Difference. Tested with Renee Carmichael. She is in Argentina. I am in the US.

Sketch 4: Facemosh

Video of playtest for fourth sketch in YMOC series, Facemosh. Tested with Pierre Drescher. We are on the same local network on separate computers.

Interlude: Building a Signaling Server for Simple-Peer

In the process of working on You, Me, and Our Computers, I have been developing my underlying server and client code to be released as an open-source tool. This week I released the first version of the shared server and client code and wrote a Medium post explaining the tool.

Prologue

In the first weeks of the residency, we created artworks for a drive-through show, Immediations, at ASU DesignSpace. The call requested ambient audio-video "immediations" to be projected at approximately 120 feet by 10 feet in an unused parking garage turned exhibition hall. I created Me, Me, and My Computer, a prologue to my SloMoCo project, Me, You, and Our Computers.

Sketch 1: Contours

Contours/fills is the first sketch in the YMOC series. It is a two-person sketch. Each person is represented with their webcam and their pose keypoints from PoseNet on an HTML5 canvas. Contour shapes are created for each person using the PoseNet keypoint data.

Sketch 2: Overlap

Overlap is the second sketch in the YMOC series. It is a two-person sketch. Each person is represented with their webcam and their pose keypoints from PoseNet on an HTML5 canvas.

Sketch 3: Frame Differencing

Frame Difference is the third sketch in the YMOC series. It is a two-person sketch. Each person is represented with their webcam and their pose keypoints from PoseNet on an HTML5 canvas. Frame differencing is used on each webcam to show movement from frame to frame. The differenced images are layered directly on top of each other.

Sketch 4: Facemosh

Facemosh is the fourth sketch in the YMOC series. It is a two-person sketch. Each person is represented with their face image as determined by the Mediapipe Facemesh algorithm on an HTML5 canvas. The face images are layered directly on top of each other with opacity so that they blend. Users control the blend by moving their head left to right.