You, Me, and Our Computers | Lisa Jamhoury
An exploration of social, embodied interaction over distance
In 2020, nearly all social interaction moved online due to the Covid-19 pandemic. Professional, academic, and leisure activities moved to Zoom, where people were mostly seated interacting with others over video and text chat. In contrast, You, Me, and Our Computers is a series of websites exploring social, embodied interaction over distance. Using computer vision, open-source machine learning algorithms, and peer-to-peer networking, YMOC places participants’ images and joint positioning on shared digital canvases and uses visual and auditory interactions to encourage explorations of virtual touch and co-presence over distance.
YMOC aims to make the new frontier of web-based social, embodied interaction more accessible. During the SloMoCo Spring Microresidency, YMOC will rapidly generate, test, and document the series of experimental web experiences in collaboration with the SloMoCo community. All code will be made available under open-source licenses and all documentation will be publicly available.
Interlude: Building a Signaling Server for Simple-Peer
In the process of working on You, Me, and Our Computers, I have been developing my underlying server and client code to be released as an open-source tool. This week I released the first version of the shared server and client code and wrote a Medium post explaining the tool.
In the first weeks of the residency, we created artworks for a drive-through show, Immediations, at ASU DesignSpace. The call requested ambient audio-video "immediations" to be projected at approximately 120 feet by 10 feet in an unused parking garage turned exhibition hall. I created Me, Me, and My Computer, a prologue to my SloMoCo project, Me, You, and Our Computers.
Sketch 3: Frame Differencing
Frame Difference is the third sketch in the YMOC series. It is a two-person sketch. Each person is represented with their webcam and their pose keypoints from PoseNet on an HTML5 canvas. Frame differencing is used on each webcam to show movement from frame to frame. The differenced images are layered directly on top of each other.
Sketch 4: Facemosh
Facemosh is the fourth sketch in the YMOC series. It is a two-person sketch. Each person is represented with their face image as determined by the Mediapipe Facemesh algorithm on an HTML5 canvas. The face images are layered directly on top of each other with opacity so that they blend. Users control the blend by moving their head left to right.