87% of People Got This Question about Their Door Lock Wrong!

You drive home and park. Your car is full of groceries and other shopping, which take many trips to bring into the house. Five minutes after you drove in, you are still making trips to the car. Is the door locked or unlocked?” What if I told you that 87% of people got this question wrong? Sensors and “smart” devices for your home may hold the promise of making life more convenient, but they may also make it harder to understand and predict things like the state of you “smart” door lock in common situations like the one above.

The main issue at hand is “feature interaction.” This is the idea that some of the features of your future smart home may want one thing (i.e., door locked for security), while others may want another (i.e., door unlocked for convenience). Software engineers who program future smart homes must come up with a clear set of rules for a device like a smart door lock so that it always behaves in clear and predictable ways. But, what is clear and predictable to a computer may not be clear and predictable to a person. My collaborator (Pamela Zave from AT&T Labs Research) and I found this out the hard way by running a study asking people to predict the state of their door lock in scenarios like the one above based on three rules applied to interaction between four features. None of the people in our study got all the questions right (and the one who got the closest was a lawyer!). See how you do by taking the 15-question quiz below:

How did it go? People in our study made some common mistakes that we describe in our paper. The bottom line is that “feature interaction” resolution rules that are simple for computers may require more effort for humans to understand. We think at the core of this may be a mismatch between logic and intuition. People intuit that an automated smart door lock should err on the side of keeping their door locked even in situations where it may be more convenient (and more similar to a regular non-smart door lock) to keep the door unlocked. It is important for researchers from multiple fields to work together to understand people’s intuitions and errors before programming future home systems, so that we won’t be left wondering whether our door is locked or unlocked!

Want more detail? Check out our CHI 2017 Publication.

Down with Downvotes: Why Do Subreddits Disable Downvotes?

This year, I took part in the comments workshop at CSCW 2017 (also known as the CCCCCCCR Workshop). The workshop applied a model I appreciate — we actually attempted to get some real work done during our time. As part of a great work group with Stuart Geiger and Tom Wilson, we dove into understanding how subreddit members reason about the decision to disable the downvoting feature.

First, a little background. By default, Reddit provides the readers with the option to give both positive (upvote) and negative (downvote) feedback on any post. However, the moderators of a particular subreddit may choose to disable this option through a simple hack of the interface (technically the option is still there, but it is invisible in the standard interface). Below is an screencap of two example subreddits, demonstrating the difference:

Two subreddits. On the left, the r/YAwriters does not make the downvoting option available. On the right, r/dailyprogrammer does.

Two subreddits. On the left, the r/YAwriters does not make the downvoting option available. On the right, r/dailyprogrammer does.

Subreddits that decide to disable or enable the downvoting feature generally post about this decision, allowing members of the community to weigh in. In this post, I’ll focus on our qualitative analysis of these of these discussions. To broadly sketch out our method:

  1. We began with a master Reddit dataset (helpfully provided for the workshop by Alex Leavitt and available publicly on Google BigQuery) and searched for four variations of the phrase “disabled downvoting.” Through this process, we identified 13 potential subreddits that have posted on this topic.
  2. From this set, we excluded subreddits where the option was mentioned but not discussed or enacted, ending up with 4 subreddits that made the decision to disable downvotes and were still disabled on the day of the workshop (Feb 2017) and 4 subreddits that made the decision to disable downvotes but reversed this decision sometime before the workshop.
  3. We open-coded all discussion in the post threads corresponding with the decision to disable and/or re-enable downvoting (generating over 150 open codes).
  4. We clustered these open-codes based on thematic affinity to identify major themes in response to the three guiding questions below.

Q1: Why do communities consider disabling the downvote? In our data, we observed four major reasons for this decision. First, if the community was small, they cited the ability and desire to self-moderate without the downvote. Second, subreddit members worried that downvotes introduced negativity, which was seen as particularly problematic in communities that encouraged creative contribution and growth (e.g., see r/YAwriters above). Third, communities may have perceived downvotes to be misused to mark differences of opinion (e.g., disliking a particular music genre) or personal disagreements rather than to signify irrelevant content. Finally, some cited that in very active communities certain reader “blanket downvote,” leading to new posts disappearing from the front page without ever being seen.

Q2: What are some objections to disabling the downvote? Despite the reasons given above to consider disabling downvoting, the decision was met with disapproving voices in some communities. The dissenters cited one or more of three main reasons. First, downvotes were seen as an important part of how a subreddit shapes itself and removing this option was seen as “not in the spirit of reddit.” Second, in certain communities, the move to disable downvotes was denounced as problematic pandering to the need for “safe spaces.” Finally, members worried that disabling downvoting removed a useful mechanism for the community to mark spam, irrelevant, and false posts.

Q3: Why did some communities choose to reverse their decision to disable downvotes? Four of the subreddits we examined chose to re-enable downvotes (one of these just 10 hours after the initial decision). We examined the three major reasons cited in those discussions. The first reason cited was that disabling was meant as a temporary change (e.g., after an influx of new members) or as a tentative experiment. Thus, the return to enabling downvotes was expected and unsurprising. The second reason cited was that the interface hack for disabling the downvote was insufficient. Since there were still ways to trigger a downvote without the graphical down arrows (e.g., keyboard shortcuts, mobile interface), the intervention may have disproportionately affected different groups of readers. The third reason cited was that a different solution to the consideration highlighted in Q1 was necessary due to the objections highlighted in Q2. One such solution was to set and make visually salient a set of rules for how a downvote is meant to be applied in that particular community (e.g., to mark irrelevant content but not to signify personal opinion).

There were also a few interesting implications for future research and design that emerged from this qualitative analysis. First, we found that users suggested a number of desired features and alternatives to downvoting that were simply not possible within the constraints of the current Reddit system. At the core of most of these suggestions, members of these communities desired a way to share their values with readers and encourage readers to apply the downvote or upvote with those values in mind. Second, we found that users had diverse hypotheses about predicted future effects of disabling downvoting with opposing theories regarding possible effects on increasing/decreasing the amount of discussion, increasing/decreasing posted content quality, and increasing/decreasing other participation (e.g., upvotes). Given the diversity of these folk theories, it seems that these questions are ripe for empirical investigation.

Technology in Recovery: Online Videochat Meetings

My blog post this month is co-authored with my student Sabirat Rubya and hosted on the ATTC (Addiction Technology Transfer Center) Messenger — an online publication aimed at service providers and clinicians who are focused on supporting recovery from substance use disorders. Check it out in the April 2017 ATTC Messenger issue!