Publication Bias and Diagoras

From the Social Science and Statistics Blog:

Diagoras was the original atheist and free thinker. ...In the context of publication bias, his contribution is shown in a story of his visit to a votive temple on the Aegean island of Samothrace. Those who escaped from shipwrecks or were saved from drowning at sea would display portraits of themselves here in thanks to the great sea god Neptune. "Surely", Diagoras was challenged by a believer, "these portraits are proof that the gods really do intervene in human affairs?" Diagoras' reply cements his claim to be the "father of publication bias": "yea, but . . . where are they painted that are drowned?"

Ben Franklin's Virtues (recycled)

Recycled blog from December 2006:

Benjamin Franklin at the age of 20 created a self-improvement project. He sought to cultivate his character by attempting to follow thirteen virtues. His autobiography lists his thirteen virtues:

1. "TEMPERANCE. Eat not to dullness; drink not to elevation."
This virtue reminds me of the Dalai Lama’s idea that the cause of human suffering is either in the excessive pursuit of pleasures or the excessive avoidance of pain.

2. "SILENCE. Speak not but what may benefit others or yourself; avoid trifling conversation."
Benjamin Franklin wrote and spoke volumes. He seemed to benefit others quite a bit. I like adding this rule: listen to others at least twice as much as you speak.

3. "ORDER. Let all your things have their places; let each part of your business have its time."

4. "RESOLUTION. Resolve to perform what you ought; perform without fail what you resolve."
Franklin was sent to France by congress to secure a desperately needed loan to support the American Revolutionary War. The French Prime minister, Count Vergennes ignored his request for a meeting. Franklin learned that Vergennes had a rare collection of books. He wrote to the Count asking to borrow one written by the French philosopher Voltaire. Vergennes lent him the volume, which Franklin returned a few weeks later with a letter of thanks. This correspondence not only broke the ice, but also appealed to Vergennes vanity. Before long, Franklin had secured the loan.

5. "FRUGALITY. Make no expense but to do good to others or yourself; i.e., waste nothing."

6. "INDUSTRY. Lose no time; be always employ'd in something useful; cut off all unnecessary actions."
Franklin was never idle. Aside from all his other accomplishments he raised money for Philadelphia's first libraries, police forces and fire companies.

7. "SINCERITY. Use no hurtful deceit; think innocently and justly, and, if you speak, speak accordingly."

8. "JUSTICE. Wrong none by doing injuries, or omitting the benefits that are your duty."

9. "MODERATION. Avoid extremes; forbear resenting injuries so much as you think they deserve."
Franklin was considered a master at consensus building. He advised Thomas Jefferson "never to engage in personalities". Jefferson later wrote that he never heard Franklin directly contradict anyone. Franklin wrote, "The conversations I engaged in went more pleasantly. The modest way in which I proposed my opinions procured them a readier reception and less contradiction."

10. "CLEANLINESS. Tolerate no uncleanliness in body, cloaths, or habitation."

11. "TRANQUILLITY. Be not disturbed at trifles, or at accidents common or unavoidable."
Franklin didn’t let his critics upset him. He urged others to make full use of other peoples envy and gossip for self-improvement. We often interpret negative remarks as malicious. Franklin wrote, "Love your enemies, for they shall tell you all your faults!" He said, "The sting of gossip is the truth of it".

12. "CHASTITY. Rarely use venery but for health or offspring, never to dullness, weakness, or the injury of your own or another's peace or reputation."

13. "HUMILITY. Imitate Jesus and Socrates."
When Franklin was a diplomat, he still introduced himself as Benjamin Franklin, a Printer. Adding contemporary people of which to imitate, I might include Ghandi, Martin Luther King Jr., the Dalai Lama, Mother Theresa, and Barry Sanders.

Possibly Harmful Psychological Treatments

From Scott Lilienfeld in the March 2007 Perspectives on Psychological Science:

The phrase primum non nocere ("first, do no harm") is a wellaccepted credo of the medical and mental health professions. Although emerging data indicate that several psychological treatments may produce harm in significant numbers of individuals, psychologists have until recently paid little attention to the problem of hazardous treatments. I critically evaluate and update earlier conclusions regarding deterioration effects in psychotherapy, outline methodological obstacles standing in the way of identifying potentially harmful therapies (PHTs), provide a provisional list of PHTs, discuss the implications of PHTs for clinical science and practice, and delineate fruitful areas for further research on PHTs. A heightened emphasis on PHTs should narrow the scientist–practitioner gap and safeguard mental health consumers against harm. Moreover, the literature on PHTs may provide insight into underlying mechanisms of change that cut across many domains of psychotherapy. The field of psychology should prioritize its efforts toward identifying PHTs and place greater emphasis on potentially dangerous than on empirically supported therapies.

Sex Roles

From the Freakonomics Blog:

A study by University of Toronto assistant professor of organizational behavior Jennifer Berdahl found that, contrary to the conventional belief that a woman’s acting “feminine” in the workplace leads to sexual harassment, just the opposite may be true. Berdahl’s paper concluded that women who “act like men” are more likely to experience harassment, possibly because of the conduct’s use as a tool to reinforce traditional gender roles.

Self-Reports of Happiness

A paper by Alan Krueger and David Schkade discusses the reliability of self-report measures of well-being.

ABSTRACT

This paper studies the test-retest reliability of a standard self-reported life satisfaction measure and of affect measures collected from a diary method. The sample consists of 229 women who were interviewed on Thursdays, two weeks apart, in Spring 2005. The correlation of net affect (i.e., duration-weighted positive feelings less negative feelings) measured two weeks apart is 0.64, which is slightly higher than the correlation of life satisfaction (r=0.59). Correlations between income, net affect and life satisfaction are presented, and adjusted for attenuation bias due to measurement error. Life satisfaction is found to correlate much more strongly with income than does net affect. Components of affect that are more person-specific are found to have a higher test-retest reliability than components of affect that are more specific to the particular situation. While reliability figures for subjective well-being measures are lower than those typically found for education, income and many other microeconomic variables, they are probably sufficiently high to support much of the research that is currently being undertaken on subjective well-being, particularly in studies where group means are compared (e.g., across activities or demographic groups).

The potential issue with a correlation of .64 is that is made up of many components besides reliability. That is, the test-retest correlation is thought to get at “repeatability” or “consistency.” However, in "state-like" variables, we would expect there to be a large change component over a two-week period. An analogue would be measuring weight, we want the scale to be reliable or consistent, but over a certain amount of time, my weight is expected to change. The more possibility for change there is in the variable, the more likely the reliability coefficient will be lower.

Self-Compassion

From Wired Science Blog:

When faced with hardship and failure, people fall roughly into two groups: those who "roll with life's punches, facing failures and problems with grace," and those who "dwell on calamities, criticize themselves and exaggerate problems."

What differentiates these people? And how to help dwellers turn into rollers? The answers, suggest a study published in this month's Journal of Personality and Social Psychology, don't involve self-esteem -- the target of many behavioral interventions -- but rather self-compassion: the ability to forgive yourself, to see your failings as universal rather than uniquely personal, and to see bitterness and anger in a detached manner.

See the May 2007 issue of the Journal of Personality and Social Psychology for the complete article.

Jeans and the Golden Ratio

From the Wired Blog:

A blue jeans company called The Proportion of Blu is using the "Divine Proportion" formula, also known as the "Golden Ratio," to make your butt look its best. Algebraically, it goes like this:

The formula has been used historically to align parts and proportions in building design, sculptures and paintings, and now -- your derrière. I kind of wonder, though, if this can really work since those of us who are not 6' models will need to hem the $220 jeans.

Set Point Theory of Happiness

From the APS Observer:

In the article “Adaptation and the Set-Point Model of Subjective Well-Being” in the April issue of Current Directions in Psychological Science, Richard E. Lucas (Michigan State University and German Institute for Economic Research, Berlin) reviews some recent studies suggesting that adaptation to changing life circumstances only goes so far. “Happiness levels do change, adaptation is not inevitable, and life events do matter,” Lucas asserts.

To study adaptation, Lucas and his colleagues used data from two large national prospective panel studies — one in Germany and the other in Great Britain. Unlike most previous studies of adaptation, these data were able to capture levels of life satisfaction both prior to and after major life events like marriage, divorce, unemployment, and illness or disability.

Lucas found that not all of life’s slings and arrows are created equal. On average, most people adapt quickly to marriage, for example — within just a couple of years, the peak in subjective well-being experienced around the time of getting married returns to its previous levels. People mostly adapt to the sorrows of losing a spouse too, but this takes longer — about 7 years. People who get divorced, however, do not, on average, return to the level of happiness they were at previously. The same is true of people who become unemployed: They do not fully bounce back, even after getting a new job.

Lucas also found that illness and injury have a strong negative effect on life satisfaction. That may not seem surprising, but it actually challenges a famous 1978 study that served as a pillar of set-point theory. That study, which examined the happiness levels of lottery winners and spinal-cord injury patients, found that lottery winners were not significantly happier than a control group and that spinal-cord injury patients were above neutral on the happiness scale. What got overlooked was that those with spinal-cord injuries were still significantly less happy than people in the control group (or lottery winners). Numerous more recent studies have confirmed that major illnesses and injury result in significant, lasting decreases in subjective well-being.

But Lucas also found that individual differences play an important role. There’s a lot of individual variation in the degree to which people adapt to what life throws at them. What’s more, individuals destined to experience certain life events actually differ in their subjective well-being from those not so fated — even well before the occurrence of those events. People who eventually marry and stay married, for example, tend to be happier even five years before their marriage than those who are destined to marry and get divorced.

Lucas stresses that his findings do not undercut the importance of adaptation processes. Some degree of adaptation necessarily protects us from prolonged emotional states that may be harmful and helps us attune to novel threats to our well-being rather than dwell on ones we are familiar with. Adaptation also helps us detach from goals that have proven unrealistic.

The full article can be found in the April 2007 issue of Current Directions in Psychological Science.

Subjects or Participants

A letter to the editor of the APS Observer:

Although the National Institute of Health takes no position on this issue, it is important for psychologists to know that federal regulations regarding the people in psychologists’ experiments refer to those people as subjects. 45 C.F.R. 46, a regulation adopted by 17 federal agencies, refers to “human subjects.” We are unaware of any effort to revise the 45 C.F.R. 46 to replace the term subjects with the term participants. Unless and until that change is made, it is imperative that psychologists refer to the people in their experiments as subjects when discussing their obligations under federal research regulations.

When publishing journal articles, psychologists are free to refer to the people in their experiments by any term they choose, and the American Psychological Association has the right to dictate language usage in APA journals. However, we do not think that the people in psychologists’ experiments should be referred to as participants even in journal articles. To avoid confusion, the same term should be used in journal articles as is used in federal regulations. Also, the term participants implies that subjects are equal partners with the experimenter in the research process, and that they are actively involved in the planning and implementation of the research. That may be a noble sentiment, but it is a false sentiment. Subjects are not equal partners with the experimenter, and terminology that implies that the two are equal is motivated by political correctness, not truth.

Whatever language a reader chooses to use in journal articles, s/he should not criticize peers for using the most accurate term for the people in their experiments. They are subjects.

Multiple Imputation

Multiple imputation is a statistical technique that imputes values for missing data. That is, it provides a best guess at what the missing data should be. Older techniques would substitute values for missing information. Mean substitution uses the mean of the variable for all missing values. The problem with mean substitution is that the standard error is lower than it should be (because all missing values don't deviate from the mean, but they do count as degrees of freedom). Multiple imputation techniques takes a distribution of possible values for the missing datum and uses one to replace it. Thus, standard errors are unbiased. The Social Science Statistics Blog has an excellent entry on the question of whether multiple imputation is making up data:

The fact is that the vast majority of our statistical techniques require rectangular data sets, and so data that look like swiss cheese make it really hard to do anything sensible with directly. Listwise deletion, where you excise horizontal slices out of the cheese wherever you see holes, discards a lot of cheese! What MI does instead is to fill in the holes in the data using all available information from the rest of the data set (thus moving some information around) and adding uncertainty to these imputations in the form of variation in the values across the different imputed data sets (thus taking back assertions of knowledge from the imputations when it is not predictable from the rest of the data and from duplication of the same information in different places in the data). If done properly, MI merely puts the data in a convenient rectangular format and enables the user (with some simple combining rules) to apply statsitical techniques to data acting as if it were fully observed. MI standard errors then are not too small, which would be the case if data were being made up.

Jon Haidt

Recently I have been reading Jon Haidt's great book, the Happiness Hypothesis (I am listening to the Audible version).

Here is a video of him speaking at the New Yorker conference.

Kitten Porn

From the Table of Malcontents Blog:

Facial Expressions in Japan and US

From livescience.com:

As a child growing up in Japan, Yuki was fascinated by pictures of American celebrities.

"Their smiles looked strange to me," Yuki told LiveScience. "They opened their mouths too widely, and raised the corners of their mouths in an exaggerated way."

Japanese people tend to shy away from overt displays of emotion, and rarely smile or frown with their mouths, Yuki explained, because the Japanese culture tends to emphasize conformity, humbleness and emotional suppression, traits that are thought to promote better relationships.

So when Yuki entered graduate school and began communicating with American scholars over e-mail, he was often confused by their use of emoticons such as smiley faces :) and sad faces, or :(.

"It took some time before I finally understood that they were faces," he wrote in an e-mail. In Japan, emoticons tend to emphasize the eyes, such as the happy face (^_^) and the sad face (;_;). "After seeing the difference between American and Japanese emoticons, it dawned on me that the faces looked exactly like typical American and Japanese smiles," he said.

Intrigued, Yuki decided to study this phenomenon. First, he and his colleagues asked groups of American and Japanese students to rate how happy or sad various computer-generated emoticons seemed to them. As Yuki predicted, the Japanese gave more weight to the emoticons' eyes when gauging emotions, whereas Americans gave more weight to the mouth. For example, the American subjects rated smiling emoticons with sad-looking eyes as happier than the Japanese subjects did.

Then he and his colleagues manipulated photographs of real faces to control the degree to which the eyes and the mouth were happy, sad or neutral. Again, the researchers found that Japanese subjects judged expressions based more on the eyes than the Americans, who looked to the mouth.

Interestingly, however, both the Americans and Japanese tended to rate faces with so-called "happy" eyes as neutral or sad. This could be because the muscles that are flexed around the eyes in genuine smiles are also quite active in sadness, said James Coan, a psychologist at the University of Virginia who was not involved in the research.

Research has shown that the expressive muscles around the eyes provide key clues about a person's genuine emotions, said Coan. Because Japanese people tend to focus on the eyes, they could be better, overall, than Americans at perceiving people's true feelings.

Statistics in Psychology Journals

Psychological Science has an article on how a 1999 article on statistical guidelines has changed the practice of presenting statistics in psychological journals:

Geoff Cumming, Fiona Fidler, and colleagues at La Trobe University in Melbourne, Australia, examined whether guidelines set forth in 1999 by the APA Task Force on Statistical Inference (TSFI) had been implemented in psychological research.

The authors analyzed articles published in 10 leading international psychology journals from 1998 to 2006, focusing on three practices central to the statistical reform debate: null hypothesis significance testing, confidence intervals, and figures with error bars. The results demonstrate that psychologists still rely on traditional null hypothesis significance testing but are also using considerably more graphs with error bars to report their research.

“For more than 50 years, statistical significance testing has been psychologists’ main statistical method” Fidler explains, “but there’s evidence it’s widely misunderstood and leads to dreadful research decisions.”

According to the authors, the shift toward using graphs with error bars signals a step forward in data interpretation. “Error bars,” says Fidler, “can give a clear impression of a study’s precision and lead to better conclusions.”

Many academic psychologists and journal editors agree. Results from a survey sent by the authors indicated that statistical reform is necessary in the field. But reform was not regarded as a priority, with a few editors noting resistance to change from some authors.

Cumming and Fidler insist that changes in statistical practices in research are needed if researchers and readers are to have a more accurate understanding of experimental results. They strongly recommend that scientific psychology “change its emphasis from the dichotomous decision making of null hypothesis significance testing to estimation of effect sizes.” Therefore, “researchers need further detailed guidance, examples of good practice, and editorial or institutional leadership.”

Achievement Index vs. GPA

From the Social Science Statistics Blog:

The Achievement Index is a way of calculating GPA that takes into account not only how well one performs in a class, but also how hard the class is relative to others in the institution...

The model, which is Bayesian, calculates "achievement index" scores for each student as latent variables that best explain the grade cutoffs for each class in the university. As a result, it captures several phenomena: (a) if a class is hard and full of very good students, then a high grade is more indicative of ability (and a low grade less indicative of lack of ability); (b) if a class is easy and full of poor students, then a high grade doesn't mean much; (c) if a certain instructor always gives As then the grade isn't that meaningful -- though it's more meaningful if the only people who take the class in the first place are the extremely bright, hard-working students. Your "achievement index" score thus reflects your actual grades as well as the difficulty level of the classes you have chosen.

Data from Articles

From the Social Science Statistics Blog:

Some journals require authors to make datasets and codes available for a while already... the American Economic Review requires authors to submit their data since 2004, and this information is now available on their website. The AER provides a basic readme document and files with the used variables for an increasing number of articles since late 2002; some authors also provide their program codes. There's a list of articles with available data here.

Voluntary Confessions

A recent study in the journal Psychological Science shows the way videos are produced can subtly trick viewers into thinking the confessions are voluntary depending on whether the video focused on the suspect or detective.

... Lassiter and colleagues from Northwestern University and the American Bar Foundation asked 21 judges and 24 law-enforcement officers to view a videotaped mock confession. The researchers presented participants with different versions of the confession in which the camera focused on only the suspect, only the detective, or both suspect and detective. Participants assessed how voluntarily the suspect confessed in each case.

The study found that judges and law enforcement officers considered the suspect-focus version of the confession to be more voluntary than the equal-focus and detective-focus versions.

“The phenomenon (camera-perspective bias) is rooted in a naturally occurring perceptual bias that affects everyone and which cannot be readily overcome regardless of people’s expertise or the amount of professional training they have received,” Lassiter said.

Street Signs

In the May 2007 Discover magazine there is a short article titled "Urban Unplanning". It describes how taking away street signs may actually make people drive more safely. The idea is based on the risk compensation effect. Here is a quote from the article:

"Basically, it means that animals tend to adjust their behavior to compensate for perceived risk. Applied to traffic, the idea is that people will drive more cautiously if they believe they are in a dangerous environment."

The conclusion is that because of all the traffic lights and signs everywhere, people "feel safe" on the road and thus don't pay as close of attention. If you remove lights and signs, people are more cautious. Therefore, there should be less accidents. Evidence in support of the risk compensation effect is that people riding bicycles who wear helmets tend to get into more accidents.

The article describes cities who are testing this theory by eliminating some signs and traffic lights.

"Since the program begin in 2004, accident rates at the town's main intersection have dropped to only one per year from a previous nine-year average of just over eight, congestion has fallen by 20 percent, and journey time has been reduced by 10 minutes."

Success and Creativity

An excerpt from the Mahalanobis Blog:

Success breeds its own failure in creative work because a successful writer or director gets less needed criticism, which creates self-indulgent, rambling works where the author doesn't tell us what's new, true and important (or interesting), but rather, books are movies that are are repetitive and full of off-point tangents. We all dislike criticism, in fact, the more correct the criticism, the more we dislike it, but it is far better to get criticism up front from a thoughtful individual, rather than the criticism of being ignored, if not now, then by posterity.

Racial Bias in Basketball

The New York Times has an article looking at the rate at which white referees call fouls on black players (and black referees call fouls on white players).

an excerpt from the Mahalonobis Blog:

The article by Joseph Price and Justin Wolfers, Racial Discrimination Among NBA Referees, shows that the coefficient on (%White Referees)*(Black dummy) in a regression predicting fouls per game is positive and significant. However, in table 2, they note that the mean fouls per 48 minutes played for Black players was 4.330 and for White players, 4.970 significant at the 1% level. The Black-White foul rate differential is 0.827 for all white referee crews, 0.574 for all black referee crews. In the multivariate regression, the coefficients on the Black dummy variable is always negative, meaning controlling for everything else they find most important and measurable, blacks receive fewer fouls than whites. So perhaps white refs aren't biased against blacks, but rather are merely less biased against white players? After all, if it were a simple story of calling more fouls on blacks, why should the coefficient on black players alone be negative and significant?

They explain this away by showing a highly selective regression in footnote 3, that is contradicted by the coefficient on the black player dummy in Table 4, highlighting that when a statistician finds what he wants, he stops. With tens of 'control' variables, any statistical presentation inherently draws from a large set of permutations known only to the researcher (choosing variables among 16 generates over 65,000 unique combinations). If the controls in footnote 3 were good enough for explaining why the black dummy is essentially zero, why were not those same controls used when proving that the coefficient on (black dummy)(%white refs) is positive in Table 4? After all, it's the tables, not the footnotes, where you put your best regressions.

There's a whole lot of multicollinearity going on that makes inference tenuous, and footnote 3 suggests the authors were very cavalier about the alternative hypothesis of there being an overall bias against whites that white refs mitigate but barely dent.

Bad Science

An excerpt from Newsweek:

it's hard to grasp how much of science is subjective, and especially how much leeway there is in choosing how to conduct a study. No one is alleging that scientists stack the deck on purpose. Let's just say that depending on how you design a study you can practically preordain the outcome. "There is an amazing array of things people do to botch a study," says Rebecca Maynard of the University of Pennsylvania.

For instance, 153 out of 167 government-funded studies of bisphenol-A, a chemical used to make plastic, find toxic effects in animals, such as low sperm counts. No industry-funded studies find any problem. It's not that the taxpayer-funded scientists are hallucinating, or that the industry scientists are blind. But here's a clue: many industry studies tested this estrogenlike chemical on a strain of rat that is insensitive to estrogen. That's like trying to measure how stress affects lactation ... using males.

Choosing the wrong methodology can lead science, and the public, astray. Early studies of hormone therapy compared women who chose to take estrogen pills and women who did not. The studies concluded that the pills prevent heart disease. Wrong. Women who chose to take hormones after menopause were healthier and more plugged into the medical system than women who did not. Differences in the women, not the effect of hormones, explained the difference in heart disease.