Image: Still image from playtesting YMOC Sketch 3: Frame Differencing
Frame Differencing is a two-person sketch. Each person is represented with their webcam and their pose keypoints from PoseNet. Frame differencing—a computational technique that only shows pixels that have changed from frame to frame—is used on each webcam. The technique in effect shows movement from frame to frame. The differenced images are layered directly on top of each other.
The posenet keypoints decipher how close or far apart the two people are. The opacity of the people’s images changes based on proximity. When the users are close to one another, the images are visible, as they move apart their images fade. Thus, in order to see your own image you must move and be close to the other person.
Images: Still images from playtesting YMOC Sketch 3: Frame Differencing. The opacity of the participants’ images change based on proximity. When the users are close to one another, the images are visible (top image), as they move apart their images fade (bottom image).
Image: Still image from playtesting with Renee. She is in Argentina. I am in the US.
I playtested this sketch with Renee Carmichael. We spent approximately five minutes on the site together. She’s located in Argentina, and I’m in Vermont, USA. We both experienced little to no visible latency. She was using a MacBook M1/Chrome. I was using an MSI gaming PC/Chrome. Here are some notes from her feedback:
Image: Still image from playtesting with Renee. In reviewing the documentation I noticed interesting moments of perceived intimacy. For example, in this still I appear to be holding Renee by the waist. My original thought about the sketch was that it would encourage intimacy, although I didn’t feel that it did during testing. Because of the quick fading of the pixels on screen, these “intimate” moments are lost to the users when interacting in real time.
This sketch was originally created as a sketch with only audio interaction. I added the images as an afterthought. Interestingly the sound didn’t work during testing and the images were much more successful than I had expected. The original idea behind the sketch was to use sound and imagery to encourage two people to overlap images. I thought that this would bring out a feeling of intimacy. However the quick fading of the images and the need to continue moving encouraged movement and less time for reflection.
The sketch was meant to mirror the image of the user. However, I reversed the canvases and instead the sketch was not mirrored. In effect this meant that when you would try to move one direction on screen, you would go the other. Interestingly for me this meant that I kept bumping into Renee when I would be trying to give her space. This added an interesting element to the interaction, but was not intentional. I’d like to test with the images properly mirrored.
