Cable Deprivation Study Shows People Are Attached to Their TVs; Streaming Devices Have Issues to Overcome

 
 

 

At the 2011 TVNext event, we presented the results from an experiment that left five Boston-area families without cable TV for a week but equipped with a “connected TV” device.

 

The Set-Up:

Five families across demos, some with children, cable subscribers, 16hrs+/wk TV viewers. No cable for a week; instead have to use one of the devices we gave them: Roku, Boxee, Xbox360, AppleTV and GoogleTV.  Holiday week with heavy snow.  Left Flip cams for participants to record interactions. Compensated for participation.
 

Highlights:

  1. There are many modes of watching TV, and a lot of cultural nuances that define who is in control of the family remote and what is put on. We had one family that wouldn't turn the TV on during the entire day until their father arrives home. 
  2. People's TV preferences don't always fall into neat genres and groups defined by on-screen guides. One family liked to watch historic stuff and black and white movies.
  3. The constant flow of content is taken for granted and is unnoticeable until it's shut off. Then it's appreciated.
  4. Watching TV shouldn't be hard work or even require thinking. Devices should be able to arrange content into a continuous flow.
  5. Serendipity inherent in channel surfing is underrated.
  6. Lots of usability issues with the devices, which cut right through people's mental models of how TV should work. Inability to search for content across services hooked up to the device (like Netflix and Amazon on Demand with Roku) is debilitating and puzzling to the user.
  7. Unconventional navigation mechanisms are also a reason for frustrations. In an on-demand-only environment, which these devices are, a graceful search is a must.
  8. There were lots of technical glitches that might have had to do with people's home wireless networks, but they got blamed on the devices. Having to authorize services on devices was also annoying.
  9. People got excited by the small form factors of the devices and their overall hardware design.
  10. TV content selection on subscription services is generally poor outside of the most high-rated shows. Which is weird because that's where the long-tail stuff can really shine (i.e. I want an entire WWII channel)
  11. We thought DVR would kill live TV viewing, but in the current environment it is actually competing with all these boxes, at least as far as TV content is concerned.
  12. The paradox of choice is on full show – people refused to choose because there was so much to choose from.
  13. Devices and services are not really for any other viewers besides prime-time content consumers.  There’s nothing for stay-at-home moms with kids who watch daytime content.
  14. Every one of them said in one way or another “It’s a great device, but it’s not for me”. Some said it’s for active viewers, others said it’s for people who don’t care what they watch.

 

The Full Story

By the nature of our work, we are heavy consumers of all media, including television. Many of us are also owners and enthusiastic users of the “connected TV” devices, including the ones we tested. At the same time, we understand that we are in a minority when it comes to our own media consumption habits.

While we are confident that the evolution of the TV is set towards its “smartification”, we wanted to learn more about how people different from us watch television, how new technologies might change their routines, and what barriers lie on the way of the “connected TV” technology’s mainstream adoption.

None of the devices are advertised as cable replacements, and we didn’t set out to test their effectivenss as such. We decided to take the experiment to its extreme and disconnect cable boxes to bring out issues that would’ve remained under the surface had we let the devices co-exist with the families’ existing TV set-ups.

We had plenty of questions going in but no hypothesis to validate. Some of the findings were as surprising to us as they might be to you; others were less unexpected.

 

Methodology

We developed and sent out a screener survey to our in-house list. About 60 families volunteered. Of them, we picked five, one for each device.

The five families we selected were fairly diverse in their composition, media habits and levels of technical expertise (and we are extremely grateful to them for letting us disrupt their lives). Everyone had either Netflix or iTunes accounts (some had both), and most had devices other than their cable box already hooked up to their TVs. Every family watched at least 16 hours of television a week (self-reported), watched video online, had a modern TV set, a cable or satellite subscription, broadband access and a wireless set-up.

Each family was modestly compensated for their time upon the experiment’s completion; this is a standard practice. Each also received a small allowance to be spent on buying content via their devices. None of the participants was a Hill Holliday employee.

We visited each family twice, to install and then a week later to pick up their box. During each visit, we conducted a one-hour interview, first about their media habits in general and then about their particular experiences with the device over the previous week. Each interview was filmed.

We have also left each family with a Flip camera. Our instructions were simple: whenever you use the device, point the camera at the screen and record and comment on what you are doing.

We have ended up with some 15 hours of video footage, which was edited down to the six minutes of the final video. While we’ve got plenty of comments about each device, evaluating them individually wasn’t our goal. Instead, we looked for themes that were common to all five participating families.

 

What We Learned

While our sample was by no means representative, the results of our experiment point us toward some real issues that one should consider when thinking about the future of the “connected TV” technologies.

One finding that is probably obvious in retrospect is that TV is invisible until it’s shut off. It’s a bit like walking: you are aware of the direction in which you are headed but you don’t really focus on the individual steps until you come across an unusual terrain. The exclusively on-demand nature of the devices we tested is just such an unusual terrain that makes you think not only about “where” but also about the specifics of “how”.

The devices demand a lean-forward involvement with what has been traditionally considered a lean-back medium, and this requirement proved disconcerting to many when it lasted longer than the usual bursts of involvement with their DVRs or video-on-demand channels.

 

The Paradox Of Choice
Constantly having to pick what to watch next was daunting not only because it interrupted the usual flow of TV-time activities in the house or required interacting with unfamiliar interfaces but also because of the cognitive load involved in considering all of the numerous content alternatives. “I don’t want to have to think about it” was one of the strongest sentiments we’ve captured in our interviews. As with “the paradox of choice” phenomenon that describes how broadening the range of options leads to a decrease in overall consumption, we saw how families gave up on watching TV altogether when they couldn’t decide what it is that they wanted to watch. This problem is serious enough for Netflix to award a million-dollar prize for a better way to tell people what they should watch next; it didn’t seem the problem was sufficiently addressed by any of the devices.

 

Expectations
People have well-formed expectations about how a TV should work, and the devices didn’t seem to confirm well to these mental models. Surfing TV channels is seamless; “tasting” unfamiliar on-demand shows includes picking them from different menu categories and waiting for them to buffer first (and often paying for them up-front). This latency is tolerated in exchange for high-consideration longer-form content but it becomes too much of a friction when all one wants is the “in-n-out” material.

 

Usability
The lack of search spanning multiple video services on a single box was a usability flaw that stood out among other complaints that could be attributed to our users’ brief experience with unfamiliar technology. From the users’ perspective, there is no reason why they had to search the Netflix, Amazon On Demand and other services individually while looking for a particular piece of content on a single device.

Finally, the devices seemed perfectly suited for certain modes of TV consumption — viewing the high-consideration content I mentioned above, on occasions when it was available — but much less so for others. The two families with children found the devices didn’t fit how they watched TV at all, which seems like an interesting market opportunity to explore.

 

The experiment raised quote a storm in the tech media.

 

With Bill Letson, Chris Plating, Jenna Swan-Gross, DJ Capobianco, and Shakira MacLyons