Purpose, Visibility, and Intersubjectivity in Video-Mediated Communication Technologies

Video-mediated communication may be able to benefit from a number of novel technologies, but designing for a good experience requires considering purpose, visibility, and intersubjectivity for both partners.

Skype, Google Hangouts, Facetime, and ShareTable are all examples of real-time video-mediated communication technologies. Designing, implementing, and deploying novel systems of this sort is a big research priority for me and every semester I get a few entrepreneuring students approaching me with ideas for cool new technology to try in this space: robots, virtual reality, augmented reality, projector-camera systems, and more. Frequently, I ask them to consider a few things first and if you’re new to thinking about computer-mediated communication, these may be helpful for you as well (many of these ideas come from my work with play over videochat).

In this case, let’s assume the “base case” of two people—Alice and Bob—using a potential new technology to communicate with each other (though the questions below can definitely be expanded to consider multi-user interfaces). Consider:

  1. (Purpose) Why is Alice using this technology? Why is Bob? The answer should be specific (e.g., not just “to communicate,” but “to plan a surprise party for Eve together”) and may be different for the two parties. It’s good to come up with at least three such use cases for the next questions.
  2. (Visibility) What does Alice see using this technology? What does Bob see? Consider how Alice is represented in Bob’s space, how Alice can control her view (and then flip it and consider the same things for Bob). Consider if this appropriate for their purposes. For example, maybe Alice is wearing VR goggles and controlling a robot moving through Bob’s room. It’s cool that she can see 360 degree views and control her gaze direction, but what does Bob see? Does he see a robot with a screen that shows Alice’s face encased in VR goggles? Does this achieve level of visibility that is appropriate for their purpose?
  3. (Intersubjectivity) How does Alice/Bob show something the other person? How does Alice/Bob understand what the other person is seeing? The first important case to consider is how Alice/Bob bring attention to themselves and how they know if their partner is actually paying attention to them. If Alice is being projected onto a wall but the camera for the system is on a robot, it will likely be difficult for her to know when Bob is looking at her (i.e., when he’s looking at the wall display it will seem that he’s looking away from the camera). It’s also useful to consider the ability to refer to other objects. Using current videochat this is actually quite hard! If Alice points towards her screen to a book on the shelf behind Bob, Bob would have no idea where she’s pointing (other than generally behind him). Solving this is hard—it’s definitely an open problem in the field—but the technology should at least address it well enough to support the scenarios posed in question 1.

Generally, I find that new idea pitches tend to propose inventions that provide a reasonable experience for Alice but a poor one for Bob. It is important to consider purpose, visibility, and intersubjectivity experience for both of them in order to conceive a system that is actually compelling.

Notes from the Field: Pokémon GO!

The obligatory evidence of being accosted by Pokémon during everyday activities :-)

The obligatory evidence of being accosted by Pokémon during everyday activities 🙂

Note: As promised, I’m revisiting this post (roughly 6 months later) to review my thoughts and predictions. Reflections are highlighted like this note.

Those of you who know me, know that I am obsessed with Pokémon (it was even the theme of my Intro to CS class last semester). I’ve blogged about Pokémon before (1 & 2), but now that everybody has bought into my insanity, I feel the need to do it again. Of course, I’m talking about the mobile augmented reality game Pokémon GO!, which has taken the U.S. by storm over the last two weeks.

The general Internet rhetoric regarding the game is a bit extreme (of course): it will kill your children (maybe in the process of arresting them), after robbing them, and stealing all of their data! Or, it will turn your children into monsters who mock the Holocaust or stare at their phones instead of the real world! Also, it may get YOU sued (as a property owner), even if you don’t play it. On the other hand, it could improve people’s health, get them to go outside, make friends, and help them discover and fall in love with their own cities!

I want to share my experience. I’ve been conducting “participatory observation” since the day Pokémon GO! came out: I’ve been playing myself and I’ve been stopping people on the street who are playing and asking them questions. I’ve done this in two cities (4 days in Minneapolis, 4 days in Chicago). Based on this, there are a three points I want to add to the above narrative:

      1. It is probably not doing anything to your children. There may be children playing this game, but I haven’t met any (only anecdotally: one man I met said he DID see one child playing it). All the players I’ve met have been roughly in the 19-35 range. I’m currently doing fieldwork at a middle school and NONE of the children I work with play. My main two main hypotheses for this: the game was aimed at the nostalgia of the older audience (e.g., it only has 1st generation Pokémon), and American children are not allowed to be as independent as Japanese children in exploring outside. Revisiting: Actually, this totally became more popular with the kids later — but it did take a solid 3 weeks post-release. Fail.
      2. It’s more about spatial computing, than the AR or stepcounts. The augmented reality (AR) is exciting at first and can always be turned on to catch a good photo op, but most experienced players turn it off to make catching Pokémon easier. The step count intervention is actually only a small part of the game and nothing new for Pokémon. What IS new and integral to the game is that everything happens in physical space, on a real map that you must move through to have meaningful encounters. This is positive because it creates opportunities for encounters with strangers, gets you moving, and (most importantly for me) leads to a playful exploration known as a dérive. It is negative when it leads to car accident, arrests, injuries, trespasses, disrespectful behavior, and all the other problems mentioned at the beginning of this post. Do the potential benefits exceed the costs? Each player must decide for themselves. Revisiting: I think this one was on the money — people play this as a spatial game not an AR game. Success.
      3. It is undeniably urban. My drive from Minneapolis to Chicago was a Pokémon desert, with very few spots along the way (though definitely enough quirky points of interest that could have qualified). This is because Pokémon GO! relies on user-generated geographic interest points (borrowed from the game Ingress). One big issue with that (pointed out by my colleagues in a recent publication) is that peer generated geographic data is significantly biased towards urban environments. So, while the download numbers may make it seem like the whole country is playing, it’s really just us city-dwellers. Revisiting: Others have picked up on this too, so I think I was right. Success.

And now, for a few predictions… (1) Most people will stop playing soon — it’s too repetitive and battery-draining and nostalgia can only get you so far (Revisiting: Yep, many sources confirm this. Success.); (2) if you buy a few of these devices when they become available, you will be able to turn a profit reselling them (Revisiting: Yep, it you bought it for $35 when it first came out, you can now resell it for over $50 since Amazon and Gamestop are both sold out. Success.); and (3) Pokémon Sun and Moon will be the best selling Pokémon generation ever. (Revisiting: Yep, Sun/Moon was the best selling not just Pokemon but Nintendo game ever. Success.) I’ll check in a few months to see how many of these came true! Revisiting: I’d say that went pretty well! Out of my 6 points/predictions, 5 were successes!

Have you been playing Pokémon GO!? What is your experience with it? Any stories that run counter to the points I’ve made above?

P.S. Team Mystic is the best team.

Pocket++: Changing Pants to Fit Phones

A very long time ago, I complained that women’s pant pockets are too small for a smartphone. Since then, smartphones have only gotten bigger! Here’s what it looks like when I try to fit my phone in my pocket (that is a regular iPhone 6, not a 6+!):

My phone won't fit in my pocket!

My phone won’t fit in my pocket!

Since then, I’ve created a number of rough prototypes to address this issue and finally came up with a solution that I see as the most efficient way of dealing with it. Namely, change your pockets! Here is what the solution looks like for me:

Now, my phone fits in my pocket!

Now, my phone fits in my pocket!

It fulfills my criteria of being comfortable while both sitting and standing, not being something I have to carry (i.e., a purse), and being easy to access. You can do this yourself if you have a sewing machine (hand-stitching won’t hold up to long-term wear, as I found out). However, the good news is that it’s also really easy to “outsource” the solution. Simple steps:

Measure how much of your phone sticks out and add one inch. That's how much more pocket you will need.

Measure how much of your phone sticks out and add one inch. That’s how much extra pocket you will need to add.

  1. Put the phone in your pocket and measure how much is sticking out with a ruler or tape measure. Add one inch to this amount (for comfort and future phone size increases). Do this for every style of jeans/pants that you want to alter (since pockets seem to vary drastically in size).
  2. Take to a local seamstress to alter. All they need to know if how many inches to extend each pocket (which you should have from step 1). You just need one pocket extended on each pair of pants. Image below shows what the pocket looks like from the inside. If you’re in the Twin Cities area, I recommend Blue Serge (the cost is $10 per pocket, but you could get a bulk discount to $7 if you bring more than two pairs of pants).
A well-sewn pocket is a thing of beauty!

Final outcome. A well-sewn pocket is a thing of beauty!

My favorite thing about this approach is that not only was I able to finally solve my smartphone problem, but that I was also able to support a wonderful seamstress in my local community. If you are the kind of person who doesn’t have a million types of pants and hates carrying a purse, see whether this solution works as well for you as it has for me!

Note: Mini blog post this month as I’m pretty caught up in teaching, an apartment move, and the upcoming wedding. But, I hope to be back in full force in June!