- Share This Post
- Pin It
- 0
-
Sparkle (0)
After learning experimental design, and statistical analysis, oh, and actually doing ‘research’ and publishing, I can pick apart many a journal article, good as any other with my credentials and as well as many with less credential. These days experimental design is taught to school children. Ever judge a science fair? If experimental design is such common knowledge, why is there so much crap research published and promoted as science? Where the p really meets the bell curve is in behavioral and social science (an oxymoron?). Behavioral and social researchers really try to be like researchers in the hard sciences. They really try, but translating experimental design from a Petri-dish-controlled-environment to a group of humans is, well, unscientific. Great effort goes into an ‘experimental’ design where outcomes are behaviors under specified circumstances. Operational definitions of behavior are necessarily laborious. Efforts to create controlled circumstances and environments (medium) require pages of methodical description. Not to mention the incredible variability within each person. Groups of people are not homogeneous even if they are the same race, the same marital status, the same age, live in the same geographic location, same socioeconomic class, and the same family! Well, maybe if they are in the same family – but even there the variance is wide, and you can rarely get them to be the same age – except in twin studies, which reinforces the heavy effects of genetics. Genetics – now that’s scientific – studied in a lab and un-subjective and unchangeable data on each and every person. The human-as-Petri-dish is good for providing tissue samples and images of internal body parts.People are near-uncontrollable-Petri- dishes for behavioral research. Behavior, seated in the less-than-fully-understood-brain, is the result of multiple, complex and in-the-moment changes that are difficult to correlate, much less show cause and effect or prediction. How big of a sample does one need to characterize Americans on eating habits, smoking, drinking alcohol, substance use, sexual behavior or parenting? Nary has a research report included conclusive statements for resolving one the big questions in human existence. The cliché research report outcome – more research needs to be done on this topic. So let’s say you have some pretty good data – a big number of data-points of a discreetly defined behavior gathered by data-collectors who are blind to the point of the study (and therefore objective) and perfect in gathering the data in a reliable way. The data is appropriately crunched and the completely unbiased researchers interpret the data to say….. write and submit a report to a journal where reviewers consider the paper un-vacuously and after recommended edits, recommend the article for publication. Late in the game, some reporter, journalist or media-type picks-up the conclusion section on the research report, paraphrases portions considered interesting, and voila! Headline reads: Teens Having More Sex. Puleease! Good and bad science is bought and sold in the media market. Selling news is a for-profit business. Much-abused is survey research, perhaps effective in commerce, but even focus group responses are only meaningful if correlated with purchases (discreetly defined behavior). Survey, the bane of scientific research and a large portion of all social research, is subjective at its core and substantially weak for predicting behavior. Which scientists would give credence to asking a small sample of self-selected individuals to predict their own behavior in the future for actually predicting behavior?
The only survey that counts is in the Petri dish of the voting booth.
