on subtitles and learning behavior

[joshua0]

I've always known that I learn better by reading than by listening, and for a while, I've suspected that that translates over to other aspects of my life. For instance, when I'm participating in a conversation, I often feel like I have an excessive amount of "dropped packets" -- times when I have to ask someone to repeat themselves, or in which an utterance that I wasn't expecting catches me completely off-guard. (I don't usually get comments on it, surprisingly, but I certainly feel like it's an issue; has anyone else noticed this in me?) But tonight, something happened that it all clicked.

I went over to a friend's house this evening to generally be unproductive, and after some hours, we settled on watching The Bourne Identity, which I had not seen before. After some time wrestling it onto a machine that could connect to the TV, we all sat down to watch it; when the first English dialogue started, however, we realized that VLC had subtitles turned on. By a unanimous vote of "can't be arsed", we left them on, and about 15 minutes into the movie, I realized something surprising: I actually had a god damn clue what was going on.

Now, this is unusual for me. I rarely watch TV or movies in part because when I do, I haven't a god damn clue what is going on. It feels like I have to pour intense amounts of resources into figuring out who's who, and what role they play; names mostly whizz past me. I mentioned this after I watched the movie, and zagarus noted, "Well, that's what the first 15 minutes are for"... but in my case, it often seems like I'm still left piecing together crucial bits of plot for the first hour and 15 minutes, and if I can figure it out by the time the movie ends, I feel like I'm doing pretty well.

When I figured this out, though, it all made a huge amount of sense to me. I remember thinking something like this when I left the subs on at some point in the past -- but this time, it just made watching the movie so much easier and so much more enjoyable. I felt like I was spending more time with the content of the film, and less time trying to put it all together. The interesting bit is that it didn't feel simply like a "missing words" thing so much as a "missing content" thing; rather than a "I can't hear" or "I can't understand", I always felt more like there was a cognitive disconnect. Even though I don't use the subtitles as a primary source of information, having them on screen in my peripheral vision, so I can scan them after the character has spoken (or while the character is speaking), just seemed to make it that much easier.

Looking back, this makes sense, too -- it gives some level of reason as to why I always just did so awfully badly in paying attention in lectures, for instance. I'd hear just fine, but the content just wasn't really meaningfully making any impact on me; and if I gave the presenter any less than my full attention, I'd get about 0% of it, just the same as if I do anything other than pay rapt attention to a movie.

I wonder if this is common. I'd immediately switch to using subtitles when watching The Wire, but apparently the authors seem to think that's a bad idea, so I guess I won't do that; but I think that from now on, when I watch movies, I'll turn the subtitles on! I also wonder what other things I can adapt in my life to work around this whatever-it-is; I bet there are probably some simple things I can change that make me substantially more functional.

Comments

[chrisamaphone]

I thought I was the only one that bad at following movies! I have the same issue when I hear people speak, too, though I know that's not uncommon among my friends (having complained about it on lj a few times and received surprising amounts of "omg I do that too"). People think I'm crazy when I say I prefer to communicate through text (email, SMS, IM) than voice because bla bla communication is some large % nonverbal bla bla intonation body language &c., and usually my best *rational* argument is that voice is a noisy/lossy channel and text is more precise, but as you say, it feels like there's something deeper going on -- like I really understand better through text.

[edanaher.livejournal.com]

I'm reminded of something which I think Rudich mentioned in Complexity Theory about teaching, though I can't think why it would have come up. And in any case, that's long enough ago that what he actually said has been completely replaced by my interpreation of it. That said...

Basically, the idea is roughly that you have some amount of "attention", and if you have to spend a significant amount of it parsing speech in real time, that leaves less for understanding and internalizing.

Reading, on the other hand, can go at your own pace, and if something seems off, you can jump back and re-read it, even within the brief span you have on closed captions.

Moreover, you typically read much faster than people talk, so you can read a phrase, understand it, and have several seconds for it to sink in before the next caption comes up. With speech, on the other hand, a long phrase requires continuous parsing, understanding, and internalizing, since you can't buffer the whole thing.

That said, it's particularly hard for me to judge if this is accurate, since I don't (AFAICT) suffer from this problem.

Actually, while I don't suffer from this precise problem, I do tend to leave loud parties when I stop wanting to spend the effort to understand people. That is, I can follow them, but it requires much more effort than normal (presumably I can't actually hear much of what they're saying, so I have to use English's redundancy and context to decode the bits I hear into a full sentence). I wonder if that's related?

[gwillen.livejournal.com]

I think this is not all that rare, in fact. I have a strong preference for subtitles-on when available. and I know a few other people who do that. To be clear, I also know a lot of other people who strongly oppose that -- I think it's definitely a feature that some but not all people have, which I might call a "mild speech processing disorder". (see e.g. http://en.wikipedia.org/wiki/Auditory_processing_disorder) I classify it as a processing rather than a hearing disorder because I've had my physical hearing checked plenty of times and it was always perfect. But I definitely have more than an average amount of trouble understanding speech, and it sounds like you're the same way.

[joshua0]

This is basically in line with how I feel. It's not like I didn't hear someone, it's just that somewhere in the tokenize / parse / validate phase, the packet got dropped. Often when I say "What?" to someone, I then go "Oh, ok", as whatever circuits in my brain they are get caught up, and it makes it the rest of the way through. *Thunk!* Oh!

Do you also feel like you have an excessive quantity of disfluencies in your day-to-day speech? Whenever I hear myself recorded, unless I'm trying very hard, I find that I often have a handful of intervals in which I hesitate, seeming to search for a word, and then continue on -- it's somewhat jarring for me to hear.

[gwillen.livejournal.com]

I don't, actually -- in fact I randomly saw myself speak on video recently (puzzlehunt runaround), and I was actually pretty shocked at how fluent I sounded.

[dreaminventor.livejournal.com]

Surprise surprise - I've found this happens to me too! (And the lectures bit is also very relevant - if I can't see what is being written or what a professor is writing, then I lose almost half the lecture).

Being able to visually dissect language being used is helpful for interpreting language. You have both the words being said clarified and you get the tone and intonation. There's less miscommunication to the viewer.

[valantrapitor.livejournal.com]

Yeah, speech processing takes away enough of my attention in movies (especially ones in which half the room is holding side conversations) that I'd much rather have subtitles instead.

Fun revelation I came to a couple of years back was that a part of why I enjoyed anime a lot more than TV was because of the subtitles - over sufficient exhaustion in speech processing the audio was roughly as intelligible for either English audio or Japanese audio (for which my brain had made rudimentary mappings of certain phrases to meanings due to having watched a lot of anime), but the subtitles (assuming they didn't suck) attached to the latter meant that I got a better idea of what's going on - because I could read much faster than I could hear -> parse, and because if you read quickly enough, you get second chances on reading. Now, there's something to be said for voice acting, sure, but I personally don't need to -have- to parse a person's speech to derive their emotional investment in a line they're saying. *shrug*

[valantrapitor.livejournal.com]

Also, I think the author of The Wire has misguided hatred against subtitles. At least, in the linked article, his justifications don't even make any sense. :P I think he just has a bad perspective of subtitles (perhaps from older days of media) that immediately maps them to parts of older movies/shows where they were only employed to let the audience know that people were speaking another language, not employed as an aid to consumption. You should feel free to subtitle up The Wire - it's a crying shame to not understand what half the dialog actually means, because it also means you miss out on significant characterization.