Accessible social software for set-top boxes

I posted a comment on Tom Coates’s site warning that his charming and delightful proposals for social software for TV viewing would end up being inaccessible unless people tried very, very hard. Tom replied via snatchmail that I should write something that could then be linked and shown around.

So here it is.

The basic problem

Interactive-TV applications and set-top boxes are inaccessible to many people with disabilities. Deaf people can mostly use them just fine, until they interfere with captioning; low-vision people can’t handle the tiny fonts and terrible colours; blind people can’t use the onscreen menus at all; and people with a dexterity impairment can’t spend all day clicking through menus and paging through program listings for the simple reason that each click or keypress can be quite a production when using adaptive technology.

Also, while there have been a few tiny attempts to fix the problem – like WGBH’s failed and inadequate guidelines for STBs and the U.K. Vista project, whose researchers never, ever answer my E-mails – the reality is that nobody in the industry really gives a shit. Or gives enough of a shit to actually spend money to fix the problem.

Separately, I proved that even very-recent technology, like a Scientific Atlanta PVR, recapitulates all the usual mistakes and actually does some things much worse than before, like presenting you the low-vision user with a colour scheme of bubblegum pink on seafoam green. Read the many postings over at the now-shuttered Axxlog.

Social software is a good idea

Social software isn’t my thing, as I am not a Deep Thinker on Important Web-Related Topics the way Coates is (or Greenfield was). I don’t get invited to conferences to theorize because I’m not that theoretical; for ’05, I’m totally applied. (Besides, if I went to their conferences all I’d tell them is “Stop being so damned pie-in-the sky and guarantee that whatever you come up with is accessible.”)

Nonetheless, I write Weblogs, comment on other people’s Weblogs, have E-mail and chat going all day long every day of the year, and love Flickr to death. I don’t have to be a theoretician to be a user and a fan.

So. How do we make Tom’s concepts accessible?

Deaf/hard-of-hearing

This one is at least conceptually straightforward: Nothing displayed onscreen by the accessible social software for set-top boxes (ASSSTBs?) may interfere with captions, which, you must keep in mind, are always the last thing displayed and will sit on top of everything else.

So if the ASSSTB tells me that Tom is presently watching Buffy (which wouldn’t be news in the first place, really), then it must either appear where captions are unlikely to appear (screen top is often safe) or simply bump the captions up a couple of lines. The latter is done all the time through the use of data bridges in TV control rooms (example), which, in the Line 21 environment, attend to the preamble address code for each caption and simply rewrite it to cause the captions to appear on a different line. Perhaps someone who understands teletext and DVB and 708 digital captioning could write in and tell me how they handle similar functions in those environments.

For picture-in-picture display, you cannot clobber captions. You must block them from my television set and decode and display them yourself in the scrunched picture. (That implies I would have to set a preference to see captions in such circumstances. I can live with that. You must also use custom fonts.)

If you’re giving me multiple pictures-in-picture from multiple sources, the logic involved is somewhat more complicated but solvable, at least if you hire me to solve it for you.

When the full screen is used by ASSSTB or any other application, block captions completely. I know there’s still audio, but there is no video to go along with it anymore.

Low-vision

This one isn’t all that difficult at the design stage. You have to use bigger, better fonts – possibly from my project whenever we get them done – and give me my choice of foreground and background colours (and background opacity). I may be perfectly able to see your onscreen notifications under those conditions.

Hence, when you’re designing the system, do not assume any fixed font size. Assume, at the very least, two fixed font sizes and variable coloration. That last point may have implications for highlighting of selected objects, too.

Blind

Well, here’s the toughie. But technology may make it less tough.

First of all, no action on the part of the ASSSTB may interfere with the audio-description signal. If I’m watching described TV, nothing that the ASSSTB produces may change that. I may be interested to know if the friends on my network are also watching the described version.

Whenever there’s an onscreen display, there will have to be speech output, or at least commands that my STB can understand to produce speech. Here you may also employ auditory user interfaces, with several different pings, clicks, or sound effects to introduce the different ASSSTB states listed in Coates’s paper. So, in the simple case of a presence alert that Tom is watching Buffy, which, I reiterate, would not come to a surprise to me even if I couldn’t see a thing, the sequence of events could be as follows:

1. Buddy-alert sound effect (like a small ping)
   1. I ignore or accept it, sort of like a chat invitation
      1. If I ignore it, the system produces the same sound effect (re-pings) the number of times I have set up in my preferences (which could be zero more times), then leaves me alone
      2. If I accept it:
         1. My PVR automatically pauses the program I’m watching and keeps recording it
         2. Voice reads what’s on the screen
         3. I press dedicated keys, which may have been explained to me beforehand but could be re-explained if I wish, to switch channels or not switch channels
            1. If I switch to Buffy, my PVR continues to record my original show
            2. If I don’t switch, I pick up where I left off with my original show

For the watch-with-your-friends feature – a kind of iChat AV on your television – the system would have to tell me who is available and clearly announce whose avatar I’d selected as I press various keys. (By the way, if I were totally blind or simply had a preference, the ASSSTB could use audio and not video.)

Choosing channels and playing games could work quite well with this group (well, for certain games), but the system would have to be able to speak the name of the show on request. Yes, I could ask whoever’s in the room with me, but that method does not provide independence. (Besides, one of the U.K.’s few innovations in accessibility is the ability to watch audio description on headphones in the same room as people listening to main audio. Perhaps only you hear the speech.) On music-video programs, sufficient metadata would have to accompany the videos to identify them. I am aware that particular point is so unlikely as to be science fiction, but I am telling you what is required here.

The onscreen interfaces for the buddy list and sharing of a social library are relatively uncomplicated cases of speech output by comparison.

Mobility impairment

Essentially, comply with the User Agent Accessibility Guidelines. Further:

1. Make everything work with the smallest possible number of clicks.
2. Give me as much time as I select in my preferences in which to make decisions.
3. Default to harmlessness (e.g., if you have to dismiss a dialogue box because I ignored it, do not change my original state; if I miss a dialogue, let me recall it with a single keystroke).
4. Record my show on PVR whenever I start interacting with your interface.

Readily adaptable to computers

Functions like these could readily adapted to accessible applications on the disabled viewer’s computer rather than, or in addition to, the television set-top box. That might not be kosher from a rights standpoint, as it amounts to separate-but-equal, which may be the former but really cannot be the latter. But I can see some ways it might be OK. What all these concepts require, however, is user testing.

Any takers?

Tom’s got a good idea going. If anybody attempts to implement it, be forewarned that the public record now holds a range of strategies to make the entire experience accessible. You now have an explicit choice: Are you going to discriminate or are you not?