Nick Brown Smelled Bull

Nick Brown Smelled Bull

A plucky amateur dared to question a celebrated psychological finding. He wound up blowing the whole theory wide open.

Vinnie Rotondaro, Narratively, https://narratively.com/nick-brown-smelled-bull/

(Interjection by Don Chapin: An old saying goes, “statistics don’t lie, but
liars use statistics.” And this quite long article appears to shout a
corollary, “mathematics isn’t bullshit, but bullshit artists use
mathematics.” What this long article basically points out:

• Using a modicum of math in a ‘professional’ paper is a powerful
  “convincer,” because…
• Nobody, other than a dedicated mathematician, wants to wade
  through a mathematical proof… including myself, despite minoring
  in math through two engineering degrees. Whenever I had to review
  a mathematical development, I concentrated on the initial
  assumptions and typically found enough problems there to
  question the whole development. 🙂
• Typically, rather than apply the Galilean concept of modeling
  nature, someone makes an assumption and ‘develops’ the math to
  support the assumption.
• Amateurs often have more common sense than “professionals.”
• “Professionals” may well rely on reputation (“I have a PhD!”),
  rather than ability.
• Professional “peer review” may well be B.S. artists “influencing”
  other “professionally established” B.S. artists.
• “Professionals” may well be reluctant to question other
  “professionals.”
• With “professionals” ego and “position protection” often come
  before honest evaluation.
• The “established” technical community protects itself. )

It was autumn of 2011. Sitting in a dimly lit London classroom, taking notes
from a teacher’s slides, Nick Brown could not believe his eyes.

By training a computers man, the then-fifty-year-old Brit was looking to beef
up his people skills, and had enrolled in a part-time course in applied positive
psychology at the University of East London. “Evidence-based stuff” is how the
field of “positive human functioning” had been explained to him—scientific and
rigorous.

So then what was this? A butterfly graph, the calling card of chaos theory
mathematics, purporting to show the tipping point upon which individuals and
groups “flourish” or “languish.” Not a metaphor, no poetic allusion, but an
exact ratio: 2.9013 positive to 1 negative emotions. Cultivate a “positivity ratio” of greater than 2.9-to-1 and sail smoothly through life; fall below it, and sink like a stone.

The theory was well credentialed. Now cited in academic journals over 350
times, it was first put forth in a 2005 paper by Barbara Fredrickson, a
luminary of the positive psychology movement, and Marcial Losada, a Chilean
management consultant, and published in the American Psychologist, the
flagship peer-reviewed journal of the largest organization of psychologists in the
U.S.

But Brown smelled bullshit. A universal constant predicting success and
fulfillment, failure and discontent? “In what world could this be true?” he
wondered.

When class was over, he tapped the shoulder of a schoolmate he knew had a
background in natural sciences, but the man only shrugged.

“I just got a bee in my bonnet,” Brown says.

***
Before enrolling in the positive psychology program at the University of East
London, Brown had been in a self-described “rut.”

The married father of two had graduated from Cambridge University in 1981
with a degree in computer science, and spent most of his career as an IT
networks operator at an international organization in Strasbourg, France.

After nearly twenty years in the position, stretched thin between technical
duties and managerial headaches, he was looking for something new. So he
jumped at the chance to transfer into human resources when it presented
itself. The move didn’t deliver the change he was expecting, however. Still
operating in a large bureaucracy—the same organization, in fact—Brown was
now tasked with promoting staff welfare. But he had “little leeway to make
decisions,” and was constantly signing off on stuff he “thought was just plain
wrong.” Adding insult to injury, when charged with renewing his company’s
suppliers list for training and coaching materials, he wound up interacting with
“nuts” and “charlatans,” people who listed reiki and crystal healing among their
interests, or resorted to “hand-waving” when selling their wares.

He was fed up. Coming up on fifty, his mother ailing, “the general BS, the
constant, not particularly high, but nonstop level of moderate dishonesty,” was
beginning to wear on him.

Then one day in November 2010, Brown happened to find himself at a
Manchester conference attending a talk by popular British psychologist
Richard Wiseman, who had written a book called The Luck Factor.
“Basically, the way to be lucky is to just put yourself in situations where good
things can happen,” Brown remembers, “because more good things will happen
to you than bad on any given day, but nothing will happen to you if you just sit
indoors.”

After the talk, Wiseman signed books. The pile dwindled down and down, and
only four books remained when Brown made it near the front of the line. Four
people were standing between him and Wiseman.

“I thought, ‘Well okay I’m not going to get one,’” he says. “And then I was about
two feet from the front, and the woman in front of me, she was one step away
from him—had been queuing for twenty minutes—she just decided she didn’t
want a book anymore and walked off.”

“Well there you go,” Brown said, greeting Wiseman, “the science of luck.”

The two men laughed and got to talking. Brown explained his situation at
work. He asked Wiseman where he could find “evidenced-based stuff,” science-
backed skills he could use to motivate employees and gain the upper hand of
the hucksters and quacks hawking Tony Robbins-esque fluff.

The emerging field, Wiseman said, was called positive psychology.

***

As described by the Oxford University Press, positive psychology aims “to
study positive human nature, using only the most rigorous scientific tools and
theories.”

In a 1998 president’s address, then-president of the American Psychological
Association Martin Seligman announced the birth of positive psychology,
calling it, “a reoriented science that emphasizes the understanding and
building of the most positive qualities of an individual: optimism, courage,
work ethic, future-mindedness, interpersonal skill, the capacity for pleasure
and insight, and social responsibility.”

In large part, positive psychology can be defined by what it is not—the study of
mental illness (rather, it aims to preempt it)—and in contrast to what came
before it—a branch of the social sciences called humanistic psychology that
focuses on “growth-oriented” aspects of human nature, but which some in
positive psychology criticize as not being adequately scientific.

A typical positive psychology exercise, as described by Seligman, the
movement’s most visible figure, in his popular 2011 book Flourish, goes like so:
“Every night for the next week, set aside ten minutes before you go to sleep.
Write down three things that went well today and why they went well.”

The “What Went Well” or “Three Blessings” exercise, which is known in positive
psychology as a “positive intervention,” comes as part of a package of
treatments collectively called positive psychotherapy. And in a study of people
with severe depression, Seligman found that positive psychotherapy relieved,
“depressive symptoms on all outcome measures better than treatment as usual
and better than drugs.”

Similar findings have afforded positive psychology a level of public credibility
that few other psychological subfields enjoy. Centers and academic programs
have sprouted up across the world, the most influential of them being
Seligman’s own one-year, $45,000 Master of Applied Positive Psychology
program at the University of Pennsylvania.

In 2002, with a $2.8 million grant from the U.S. Department of Education, the
Penn Resiliency Program (part of the University of Pennsylvania’s Positive
Psychology Center) began a four-year study of positive psychology’s effects on
ninth-graders at a high school outside of Philadelphia. Six years later, in 2008,
Seligman entered into a far-reaching collaboration with the U.S. Army,
resulting in a $125 million government-funded “Army-wide” program known as
Comprehensive Soldier Fitness (CSF).

Guest-editing a January 2011 special edition of the American Psychologist that
was dedicated to the program, Seligman wrote, along with two military
personnel, that CSF’s goal is to, “increase the number of soldiers who derive
meaning and personal growth from their combat experience,” and “to decrease
the number of soldiers who develop stress pathologies.”

Recently, however, it has been reported that CSF has done little to reduce
PTSD. Nevertheless, the government is expanding the $50-million-per-year
program.

One of the authors who contributed to the American Psychologist’s special
edition on Comprehensive Soldier Fitness was Barbara Fredrickson. Co-
writing “Emotional Fitness and the Movement of Affective Science from Lab to
Field,” she cited her 2005 work on the “critical positivity ratio.”

Fredrickson is best known for her “broaden-and-build” theory, which posits
that the cultivation of positive emotions promotes greater and greater
wellbeing. She is considered a rock star of the positive psychology movement,
once having been praised by Seligman as its “laboratory genius.” Over the course of her career, according to her curriculum vitae, she has received $240,000 in awards and fellowships and over $9 million in grant funding.

Where the concept of a ratio of positive to negative emotions dates back to the
1950s, the specific origin of the critical positivity ratio took place in 2003,
writes Fredrickson in her general readership book Positivity: Top-Notch
Research Reveals the 3-to-1 Ratio That Will Change Your Life.

A Chilean business consultant named Marcial Losada, “who had begun to
dabble in what had become his passion: mathematical modeling of group
behavior,” sent her an out-of-the-blue email. Appealing to research
he performed in the 1990s that coded the language of sixty business teams for
positive and negative affect, Losada said he had “developed a mathematical
model—based on nonlinear dynamics—of (Fredrickson’s) broaden-and-build
theory.”

Two independent tests by Fredrickson studying the emotional ratios of
individuals seemed to confirm Losada’s findings. In 2005, the two unveiled the
critical positive ratio in an American Psychologist paper titled “Positive Affect
and the Complex Dynamics of Human Flourishing.”

The theory was bolder than bold: Mankind, whether working alone or in
groups, is governed by a mathematical tipping point, one specified by a ratio of
2.9013 positive to 1 negative emotions. When the tipping point is crested, a
kind of positive emotional chaos ensues—“that flapping of the butterfly’s wing,”
as Fredrickson puts it—resulting in human “flourishing.” When it is not met (or
if a limit of 11.6346 positive emotions is exceeded, as there is a limit to
positivity), everything comes grinding to a halt, or locks into stereotyped
patterns like water freezing into ice.

In Positivity, Fredrickson writes that it is possible to promote positive emotions
and decrease negative ones. The critical positivity ratio, then, represented a
stark dividing line—the difference between a bummer of a life and a blissful
one. Feel good about twelve things in your day and bad only about four, and by
the laws of the ratio you’ll be happy. Knock just one of the good emotions out,
however, and you won’t. So get that extra cookie and talk with someone you
love, and push yourself over the mathematical hump.

“Our discovery of the critical 2.9 positivity ratio,” Fredrickson and Losada
wrote, “may represent a breakthrough.”

***

Less than a month into his program at the University of East London, Nick
Brown was making breakthroughs of his own.

He had been poring over the original papers that informed Fredrickson and
Losada’s 2005 article—papers written or co-written by Marcial Losada. They
seemed “sketchy,” Brown says. In his research on business teams, for instance,
“the length of the business meetings weren’t even mentioned.”

“Normally you have a method and the method says we selected these people
and we picked these numbers and here’s the tables and here are the means
and here’s the standard deviation,” Brown says. “He just goes: ‘Satisfied that
the model fit my data, I then ran some simulations.’ The whole process was
indistinguishable from him having made the data up.”

In scrutinizing Fredrickson and Losada’s work, Brown happened upon a line in
their 2005 paper that caught his attention: “Losada (1999) established the
equivalence between his control parameter, c, and the Lorenzian control
parameter, r. Using the above equation, it is known that the positivity ratio
equivalent to r = 24.7368 is 2.9013.”

“So I started looking for where that formula came from,” he says

In 2000, writing in the American Psychologist, Martin Seligman and fellow
positive psychology pioneer Mihaly Csikszentmihalyi wrote that humanistic
psychology failed to “attract much of a cumulative empirical base,” that it,
“spawned myriad therapeutic self-help movements,” and that the legacy of the
movement is “prominently displayed in any large bookstore,” with the
“‘psychology’ section (containing) at least 10 shelves on crystal healing,
aromatherapy, and reaching the inner child for every shelf of books that tries to
uphold some scholarly standard.”

The dig “got a major groan out of me,” says Friedman, who prides himself on
the rigor of his work and the legacy of humanism.

Psychology has long been vexed by the question of the quantitative vs. the
qualitative, resulting in a kind of shame known as “physics envy,” whereby
researchers in so-called “softer” social sciences feel inadequate when compared
to “harder” physical sciences. Friedman says he tries to take a balanced view
on the subject. Seligman, on the other hand, he says, “claims that quantitative
work is more rigorous than qualitative, and that’s something I dispute. Each
can be rigorous in their own way, and each can be misused.”

And misused was exactly the word on his mind when he read Fredrickson and
Losada’s paper upon its publication in 2005. “I recollect just letting out a sigh,”
he says, “just more stuff of dubious worth.”

So he was intrigued when he received Brown’s email six years later, asking for
help to take it down.

“Dear Dr. Friedman,
Please excuse me writing to you spontaneously like this…”
“I had no idea what his background was,” Friedman says, “I didn’t know if it
was suitable, and I didn’t know how to tackle the math myself, so essentially I
responded, ‘I’m willing to play, tell me what’s on your mind.’”
Friedman suggested that Brown start drafting a critique. “He began to make
sense,” Friedman says. “Nick intuitively saw the flaws in the mathematics. He
realized how implausible this was.”

“But he couldn’t pin it down,” he says. The math was too much.

***

Enter Alan Sokal. A mathematician and physicist, Sokal is best known for a
trick he played over a decade ago, now known simply as the “Sokal Affair.”

In 1996, wanting to see if, “a leading North American journal of cultural
studies…would publish an article liberally salted with nonsense if (a) it
sounded good and (b) it flattered the editors’ ideological preconceptions,” Sokal
submitted a paper to a postmodern cultural studies journal called Social Text
that made little to no sense, and ended up getting it published. A great deal of
controversy ensued when Sokal announced his hoax, and a year afterward he
co-wrote a popular book titled Intellectual Impostures.

“I guess I’m seen as an all–purpose fighter against bullshit,” says Sokal, now
fifty-eight years old, with a voice reminiscent of Phil Hartman.

Brown had emailed the science gadfly in late November of 2011, but Sokal, who
was swamped with papers and class work, had filed it away. He had received
similar emails before, emails asking him for help debunking bad science,
“several times a year,” he says, “some less serious, some more serious.”
More often than not, the emails got lost in the mix, but somehow, combing
through his correspondence a few weeks later, he happened across Brown’s
and started reading.

Sokal “didn’t know what positive psychology was at the time,” he says, but he
saw that the issue was in his domain—“a misuse of math and physics”—and
that the mathematical offense was particularly obvious. “This one wasn’t even
hard.”

Brown related a list of inconsistencies he had found in the mathematical guts
of the theory. Finishing his email, he wrote:

“Here’s my problem. I am just this grad student with no qualifications or
credentials, starting out in the field. I don’t know how to express this kind of
idea especially coherently in academic written form, and I suspect that even if I
did, it would be unlikely to be published.”

“On the other hand,” he continued, “I don’t think that I’m a crank, and this is
starting to bug me…I would be very happy to be proved ignorant on this, by
someone who understands the science and mathematics better than I do.”
Sokal responded that he was “quite impressed with the cogency” of Brown’s
critique, and encouraged him to develop it into a full paper for publishing.
Later, the two met up for lunch in London, where Brown was studying and
Sokal was teaching. Soon after, another email reached Brown’s inbox.
Sokal, “came back with my draft one day,” Brown says, “and had written 3,000
words.”

***

By the late winter of 2012, Friedman, Sokal and Brown were all in touch via
email and working together towards a draft of what would become “The
Complex Dynamics of Wishful Thinking.”

The three men brought different skills to the plate. Brown was the outsider, the
instigator, who, knowing no better, dared to question the theory in the first
place. Friedman provided psychological expertise and played a diplomatic role,
helping guide the paper towards publication. Sokal was the finisher, the
infamous debunker with the know-how needed to dismantle the theory in hard,
mathematic language.

The article they wrote not only took to pieces Fredrickson and Losada’s 2005
paper, but also two earlier articles written or co-written by Losada. Taken
together, Brown, Sokal and Freidman tallied a litany of abuses, which they
related, one by one, in painstaking detail.

“We shall demonstrate that each one of the three articles is completely vitiated
by fundamental conceptual and mathematical errors,” they wrote.

But before getting to those fundamental errors, they agreed it would be best to
begin the piece with a rudimentary explanation of differential equations, the
mathematical building blocks that Losada and Fredrickson misused, and how
they work.

The pedagogical intro was written by Sokal and intended to have a two-pronged
effect. On the one hand, it enabled “anybody with one undergrad course of
calculus to understand the paper,” says Friedman. On the other, the intro
carried an implicit message, one directed towards psychologists that tacitly
scolded them for accepting the theory without understanding of the math
allegedly supporting it: “If you can’t understand this,” as Brown put it, “don’t
allow yourself to continue reading.”

Math aside, the paper has a biting, sardonic quality. “Most of the good lines are
Nick’s,” says Sokal.

In critiquing a section of Losada’s earlier research that characterized high
performance teams as “buoyant” and low performance teams as “stuck in a
viscous atmosphere,” to take an example, Brown wrote mockingly:
“One could describe a team’s interactions as ‘sparky’ and confidently predict
that their emotions would be subject to the same laws that govern the dielectric
breakdown of air under the influence of an electric field. Alternatively, the
interactions of a team of researchers whose journal articles are characterized
by ‘smoke and mirrors’ could be modeled using the physics of airborne
particulate combustion residues, combined in some way with classical optics.”

The line made it into the final; others were struck. Friedman worked to tone
down harsh language where he could, feeling protective of his profession and
suspecting “how much resistance there would be,” as Barbara Fredrickson is
an associate editor at the American Psychologist.

In the end, the paper that was submitted to the American Psychologist—the
same journal that had peer reviewed and published Fredrickson and Losada’s
theory in 2005—was relentless in its attack, tightly written and squarely
focused on bad math.

And through it all, the three met only once in person, when Friedman was in
London to give an academic address. It was a day spent scribbling equations in
chalk on blackboards and talking strategy – a debunkers’ bonhomie.
Afterwards, Friedman and Brown downed a few beers at a local pub. None of
the authors have seen each other since.

***

On July 2, 2012, Alan Sokal submitted the finished paper for peer review,
then titled “The Complex Dynamics of an Intellectual Imposture,” to the
American Psychologist.

On July 12th, they had a response. Citing the journal’s 85-percent rejection
rate, Managing Editor Gary VandenBos informed Sokal that the paper had
exceeded the two-month limit of which authors were allowed to respond to a
target article, and was “just not right for American Psychologist at the time.”
The following day at 2:32 p.m., Sokal replied. In an appeal that “took a bit of
bravado,” the team decided to go over VandenBos’ head, writing to American
Psychological Association CEO Norman Anderson that this was no ordinary
comment; that the paper would be published soon in some good journal; and
that if it were published somewhere else it would not look good for American
Psychologist. Sokal wrote that by not letting the paper go to peer review, the
American Psychological Association was running the risk of converting “a
minor scandal into a major one.”

Two hours later at 5:22 p.m., Anderson responded. The decision had been
reversed. The paper would go under review.

When asked to comment for this article, VandenBos wrote in an email that,
“the APA editorial peer review process for all of our scholarly journals is a
confidential process.” But as Friedman and Sokal explain, when reviewing a
paper, the managing editor first sends it to an associate editor “closest in field”
to the subject addressed in the paper. The only problem with this, Sokal explains, was “that the associate editor closest in field to our paper was Barbara Fredrickson.”

“But the editor in chief, he’s honorable,” says Sokal, and so the paper was
assigned to another associate editor who in turn sent it out to four reviewers.
In September, the results came in.

“The messages were mixed,” Sokal says. “One said it was ‘perfect.’ One said the
content was great, but that he wanted it to be toned down. Three and four had
some complaints about the tone and some rather more extensive criticisms of
the content.”

They went through the paper again, removing harsh criticisms and toning
down language. It was here that the title changed from “Complex Dynamics of
an Intellectual Imposture” to “Complex Dynamics of Wishful Thinking.”
Similarly, other references to “intellectual imposture” in the paper were
removed.

As for the third and fourth reviewers’ “more extensive” criticisms, Sokal says,
the team addressed “each and every” point, accepting “about 20 percent” of the
commenters’ criticisms and refuting “about 80 percent” of them. The cover
letter attached to the revised draft was 25-pages long. The paper was
recommended for publishing.

***

Nearly a year later, on July 15, 2013, “The Complex Dynamics of Wishful
Thinking” was published online by the American Psychologist. That same day, a
response from Fredrickson was also released, called “Updated Thinking on
Positivity Ratios.”

Barbara Fredrickson declined to comment for this article. In her published
response she wrote that she has “come to see sufficient reason to question the
particular mathematical framework,” employed in her 2005 paper.

Co-author Marcial Losada, Fredrickson wrote, “chose not to respond.” Emails
sent to Losada seeking comment for this story were not answered.

While ditching the math underlying her theory—the authors issued a formal
correction to their 2005 work on Sept. 16th—Fredrickson still believes that
research in positivity ratios holds “logic and importance.”

“My aim in this response article is not to defend Losada’s mathematical and
conceptual work,” she wrote. “Indeed, I have neither the expertise nor the insight to do so on my own. My aim, rather, is to update the empirical evidence
for the value and nonlinearity of positivity ratios.”

She echoed the point in a Sept. 16th letter to the editor about “The Magic Ratio
That Wasn’t,” an article written by Tom Bartlett of the Chronicle of Higher
Education. Frederickson stated: “It is important to recognize that considerable
theory and evidence point to the continued value of tracking and raising
positivity ratios.”

“Other elements of the original article remain valid and unaffected by this
change,” she wrote, “most notably the empirical finding—replicated across two
independent samples—that positivity ratios were significantly higher for
individuals identified as flourishing relative to those identified as
nonflourishing.”

In the same letter, Fredrickson announced that future printings of her book
Positivity would “feature a new note to readers” referencing the “current
scientific dialogue,” and that a link to her formal correction would be included
on the book’s website.

Brown, Sokal and Friedman were little satisfied with Fredrickson’s response, or
the authors’ correction. Nor were they happy with the treatment they say they
received at the hands of the American Psychologist.

The authors claim they agreed to let the American Psychologist show Barbara
Fredrickson an advance copy of their paper to inform her response, but only if
they were granted the “last word” in the form of a response to her response—an
agreement they say the American Psychologist subsequently broke. (After a
months-long appeals process, the authors have been granted permission to
answer Fredrickson in the American Psychologist.)

Fredrickson’s response, Brown says, “totally fails to address almost any of the
points we made, except to the extent that she says, ‘okay, well, if I throw my
collaborator under the bus maybe that will make you happy.’”

“She claims that out of the trinity of math theory and data, we maybe lost the
math, but we still have the familiar duo of theory and data you need to do
science,” Brown says, “but she doesn’t because the only theoretical support she
had were those equations.”

“She did the classic tar baby thing,” Friedman says. “Two out of three ain’t
bad.”

There are other inconsistencies, they note, odd facts confusing Fredrickson’s
claim that she studied the math but lacks the “expertise and insight” needed to
defend it.

Additionally, Fredrickson references a textbook called Differential Equations,
Dynamical Systems, and an Introduction to Chaos, a book Sokal says “is aimed
at upper-level undergraduates in mathematics, physics and related
disciplines.”

“If she had studied 1/10th of what she said she studied,” says Sokal, “she
would have seen right through Losada.”

Muddying the waters still, Martin Seligman, positive psychology’s most
influential figure, butchered facts relating to the critical positivity ratio in his
own book, misstating that “60 companies” rather than the factual sixty
business teams had been studied.

Emails sent to Seligman seeking comment on the discrepancy were not
answered.

“I contacted Seligman to clarify his use of the sixty corporations research vs.
sixty teams,” Friedman says, “and he responded that he wasn’t sure how he
came up with it. He distanced himself from the whole situation, saying that he
had never endorsed the ratio as an exact number.”

The two continued dialoguing on the topic. Eventually Seligman thanked
Friedman and his coauthors for helping to correct the scientific record, but “his
first response was brief and seemed rather annoyed,” Friedman says.

“Dismissive.”

“Very strange,” says Sokal. “It doesn’t give you confidence if two of the biggest
names in the subfield are at the very least so sloppy.”

***

To this day, Nick Brown still smells bullshit. He smells it despite the claim
that he and his coauthors make at the end of their paper that they do not,
“have an opinion about the degree to which excessive enthusiasm, sincere self–
deception, or other motivations may have influenced Losada and colleagues
when writing their articles.”

“I don’t like people who take the piss,” he says, and that’s exactly what he
thought Losada did by taking his theory to Fredrickson.