Category Archives: Methods

Setting Up Automated Video Recording for Evaluating Your System in the Field

Posted on by
1

I designed a communication system for families as part of my thesis project (it’s called the ShareTable). During the deployment, I found that it was incredibly valuable to supplement weekly interviews and diaries with automated video recordings of system use. For example, through the diary and interview, I was able to gather that on a particular day the mom and daughter talked to the son using the ShareTable and that the topic of conversation was that he wasn’t feeling well. Interesting as a data point, but not particularly exciting. But, here’s what the same data point looked like when I transcribed the video of this session:

Mom: What’s going on, baby?
Son (age 11): Well, my throat is acting up…
Mom: Awww, well take care of yourself … what else is wrong, sweetheart? … You look like you’re really sad, honey!
Son: I just don’t feel good.
Mom: All right, well listen. I love you … Do you see my hand, holding on to your hand? [Strokes his projected hand]
Son: Yes, I do.
Mom: I love you, baby.
Son: I love you too, mom.
Daughter (age 7): Hey, Bubba. [Also puts her hand on the table]
Mom: There’s my hand. Keep your hand in there, we’re going to do a family handshake, okay? [All three move their hands on the table together]

Much more interesting! And, the video clued me in to the fact that the families were leveraging the camera-projector system provided for the table for creating a sense of metaphorical touch.

Obviously, there are a lot of trust issues in recording video of system use in private settings like the home. It comes down to developing a good protocol with your IRB, developing a trusting relationship with your participant, and answering any of their questions and concerns. In my study, I discussed the issue frankly with the families, stored recorded videos on the machine in their home (not remotely), and let them mark any video session for deletion before the researcher watched it. All in all, it is a bit of a hassle and some participants may never agree to this intrusion, but it was totally worth it for the richness and detail of data I got in the end.

So, if you’re interested in setting up automated video logging for your deployment, here is a simple arrangement that I found worked with minimal need for reinventing the wheel:

  1. Set up a standard webcam in the home, pointing to the area of interest that you would like to have recorded when your system is in use. I just used a Logitech USB camera, but you may find that you need a wireless solution like the Ai Ball.
  2. Download and install Flash Media Live Encoder on the machine that will store the video logs. You will need to know the location of FMLEcmd once it’s installed. If you have a modern operating system, you can just search for it.
  3. Run the FMLE GUI interface. Here you can select, where videos will save (or

    Don’t forget to change the CPU usage if you’re doing anything else with the machine that you’re using for recording.

    stream, if your participants are cool with that) and general features of the video and audio. Now, click on the wrench icon next to the format (see left). I found it really important to set the CPU usage to “Very Low” (it’s “Dedicated” by default), otherwise the recording was actually messing with the other things the system was doing. You may find that there are other settings here that you need to tweak.

  4. Now, in your program, you just need to do a system call to FMLE when you’re ready to start recording. Find and kill the FMLE process when you’re ready to stop. These are common instructions in any language, so you should be able to find out how to do it from your program. Here are quick code bits for doing it in C#:
To Start Recording:
this.fmleProcess = Process.Start(@”<wherever your FMLEcmd is>”);
int hWnd = this.fmleProcess.Handle.ToInt32();
ShowWindow(hWnd, SW_SHOWMINIMIZED);
To Stop Recording:
this.fmleProcess.CloseMainWindow();
this.fmleProcess.Close();

Okay, that’s it! All of the recordings should be saved according to the settings that you set up in the FMLE GUI. With my participants, I asked them to mark any sessions that they wanted deleted, I could identify those recordings by their creation date and time, and delete those without watching.

Hope this helps somebody out there. I’d love to hear about clever way that you record or log system use during field deployment (beyond text system logs).

Validated Measures for Family Communication Investigations

Posted on by
Reply

There are many advantages to using validated instruments while evaluating a family communication system. First, we can claim that the questions we ask actually measure what we think they measure. Second, if multiple studies use the same instrument, new forms of analysis become possible — allowing us to better build on each other’s work and conduct meta-analyses. Third, using validated instruments allows us to have better conversations outside of our immediate field. There are a number of validated instruments out there that may be useful and I want to highlight a few here. Other instruments that researchers in this domain have found relevant that have been listed on the Designing for Families wiki and that’s a good place to add to if you know of something good.

When evaluating communication technology, you may want to make claims about its effect on various aspects of the relationship between the people it is meant to connect. I suggest using the Network of Relationships Inventory (NRI) or Social Connectedness Questionnaire (specific version) in this case. If the relationship you are measuring is of a specific type (e.g. parent and child, married couple), it may also make sense to use a more targeted validated measure such as the Parent-Child Relationship Questionnaire. Sometimes, a technology may not be designed for a specific pair of people, but rather meant to increase a given users general sense of connectedness to others or emotional well-being. Two useful measures in this case are the Social Connectedness Questionnaire (overall version) and the Positive and Negative Affect Schedule (PANAS).

All of the measures I mentioned so far measure outcomes but do not ask the participant to reflect on the technology that used. If this seems useful to you, you may want to take a look at the Affective Benefits and Costs Questionnaire (ABC-Q). I’ve been designing and validating a more focused version of it called the Affective Benefits and Costs of Communication Technologies (ABCCT) questionnaire. I have preliminary validations of versions for use with both children and adults, so feel free to try them out.

However, all of the above measures are really only useful if the person has had an opportunity to try your technology over a period of time so that changes in relationships, affect, and communication practices can be detected. However, there are a few validated measures that I find to be valid after a short interaction (such as a lab study). The Networked Minds Social Presence Measure is appropriate to use with adults. I have not had success adapting it to children as I think they may have trouble reflecting on a short interaction in this abstract fashion. With children, I’ve had more success using validated observational metrics to code video of system use, for example the Howes levels of social play coding scheme when looking at remote play.

If there is no validated measure for the aspect you are investigating, it may make sense to design and validate something that others would be able to use. I’d love to hear about this kind of work if anybody is doing it and early designs of such surveys.

All Mentioned Measures:

  • Bel, D.T. van, Smolders, K.C.H.J., IJsselsteijn, W.A., and Kort, Y.A.W. de. Social connectedness : concept and measurement. International Conference on Intelligent Environments, IOS Press (2011), 67-74.
  • Furman, W. and Buhrmester, D. The Network of Relationships Inventory: Behavioral Systems Version. International journal of behavioral development 33, 5 (2009), 470-478.
  • Furman, W. Parent-Child Relationship Questionnaire. In J. Touliatos, B.F. Perlmutter, M.A. Straus and G.W. Holden, eds., Handbook of Family Measurement Techniques. SAGE, 2001, 285-289.
  • Harms, C. and Biocca, F. Internal consistency and reliability of the networked minds social presence measure. (2004), 246.
  • Howes, C., Unger, O., and Seidner, L.B. Social Pretend Play in Toddlers: Parallels with Social Play and with Solitary Pretend. Child Development Vol. 60, N. 1 (1989), 77-84.
  • IJsselsteijn, W., Baren, J. Van, Markopoulos, P., Romero, N., and Ruyter, B. de. Measuring Affective Benefits and Costs of Mediated Awareness: Development and Validation of the ABC-Questionnaire. In Awareness Systems. 2009, 473-488.
  • Thompson, E.R. Development and Validation of an Internationally Reliable Short-Form of the Positive and Negative Affect Schedule (PANAS). Journal of Cross-Cultural Psychology 38, 2 (2007), 227-242.
  • Yarosh, S. and Markopoulos, P. Design of an instrument for the evaluation of communication technologies with children. Proc. of IDC, ACM (2010), 266–269.

Experimenting at Home: The Role of Auto-Biographical Research in Designing for the Family

Posted on by
4

There has been a rich tradition in ethnography of gaining access to certain communities by using family connections (e.g., the famous Addler & Addler studies of schoolyard dynamics). Similarly, in the design of family communication technologies, it is not unusual for researchers to test new communication technologies with their own families (e.g., Hermes@Home by Saslis-Lagoudakis, Cheverst, Dix, Fitton, & Rouncefield).

Recently, at the “Technology for Today’s Family” workshop we discussed the possibilities of auto-biographical research in this domain. The appeal is clear: it solves the problem of finding willing families, it allows for longer deployments, it allows for ongoing debugging, and it allows access to the kind of data that would otherwise be impossible to get. However, how should such a study be reported? What is the possible role of investigations conducted in the researcher’s home?

In both of the examples from the first paragraph, the researchers revealed their relationship to the participants of the study. However, several recent studies in this domain (3 different that I am aware of) have left this information undisclosed. This never seemed malicious or purposefully misleading. Corresponding with these authors, they mentioned two major classes of reasons for not disclosing relationships to participants:

  • It is common in HCI to use participants you know (e.g., other students in your lab) and not reveal those relationships. It is up to the researcher to decide whether using known participants affected the study results. If the estimate is that it didn’t, there is no need to waste precious paper space disclosing relationships.
  • In general, participants in HCI studies are presented in an anonymous fashion. Articulating a relationship to the author would break this anonymity (since we don’t publish anonymously). Shouldn’t members of the researchers’ family receive the same protection as other participants?

I will not disclose the actual papers because I respect the anonymity of others’ families. But, I think this is an important question for us to decide as a community moving forward. Here are three proposed directions for moving forward:

  1. It’s fine as it is now. There are big advantages to the access that can be gained by doing research with our own families, and disclosing relationships to participants should be up to the researcher’s discretion.
  2. Familial relationships to participants should always be disclosed and implications discusses in the paper. The price of doing research with one’s own family is the loss of their anonymity. The researcher should make sure all family participants consent to their relationships to the author appearing in the paper.
  3. We shouldn’t do research with our own families at all. We run into the risk of designing more for families that are like our own, losing out on the chance to hear from other types of families.

I would really like to hear the opinions of others on this topic. What do you think would be the most reasonable approach to take?