Videogames and correlation

I actually finished Chapter 4 and half of Chapter 5 of Box, Hunter and Hunter today. I'm in a little bit of hurry because I am going to be sitting in on the company statistics training class starting next week, so I want to be fresh on the material.

However, something that caught my eye today actually came from a medical device trade journal. The lead editorial in Medical Design referred to a study out of BYU that concluded that video games may lead to poor relationships." Shocking.

Naturally, this also set off my correlation/causation detector. I checked out the press release, and then moved on the actual article, which seems to be free access. At least, I just downloaded it from home without logging in to any of my journal subscriptions, so I think it is available to everyone. The press release was more circumspect, "As the amount of time playing video games went up, the quality of relationships with peers and parents went down." Again, shocking.

I thought the paper was interesting. The theoretical model the paper used was something called a media practice model. This is the claim that we select media based on who we are, but then a feedback effect occurs by which the medium shapes our personalities, so the medium functions as a kind of self-socialization. There seems to be a certain plausibility to this, but I haven't yet read the cited paper for this model. I await comment from the social scientists in the family.

However, there was a cited reference that annoyed me.

The research that has been conducted suggests a negative correlation between time spent playing video games and academic performance (Anand 2007; Anderson and Dill 2000), and a positive relationship between violent video game use and aggression (Anderson and Dill 2000).

There is an unfortunate equivocation here, in that "aggression" is being used in a technical sense. I know because I looked into this topic extensively a couple of years ago for a paper I wrote on the history of videogames. Aggression here refers to aggressive behaviors in the psychological sense, mostly meaning acting like a jerk, rather than shooting all your classmates and then turning the gun on yourself. However, pretty much every non-specialist, and probably many of the specialists, read the common sense of the word instead of the technical one.

Delving into the experimental design, the study was college students only. That is definite subset of all young people with characteristics that ought to be taken into account (IQ>110, somewhat limited range of SES). As long as we limit discussion to the study group, we won't get in trouble. This is a persistent problem in education research. Most pedagogical methods are tested by professors on college students, instead of on a general population of grammar school students. Also, the participants were self-selected, so we have that too. Ah well, we take what we can get.

To be perfectly honest, I have no idea how to interpret the results of this study. The authors launched right in and said that videogames are correlated with bad things, but I am mystified by the actual results, which mostly had the biggest effect based on what kind of university the students went to. What in the world does that mean? Who cares? Why use that as a factor? I don't know the answers to my own questions, I'm guessing the authors didn't either, since they didn't even bring this up. I don't have a good handle on effect sizes either. To be perfectly blunt, I don't give a crap about statistical significance unless there is a decent effect size. There is some discussion of slopes, but since we are using ordinal scales I don't know what the slopes mean.

Then we get to risk factor. This has become one of my pet peeves, since I have learned that "risk factor" is not even of much use in the field of epidemiology where it originated. Videogames are a "possible risk factor for emerging adult development." Risk factor for poor development I think they mean. The problem with risk factors is they need to be backed up with controlled experiments, otherwise we can't tell whether they are causative, random, or caused by an unknown third factor.

To the authors' credit, many of my complaints are actually featured in the conclusions section. However, when doing science (or attempting to) it is probably better to not discuss causes at all when you have no grounds to, rather than apologize for it later. Primarily because when some editor at a small trade journal writes about your research, he won't be so careful