Adventures in Ethics and Science

Where the action is

2006-01-24T18:22:00.000-08:00

In case you're wondering why things are so quite here (post-wise), don't forget that I've moved the blog (as it extends into the future) to ScienceBlogs.

You might want to update your bookmark/blogroll/feed. If not, the other kids might accuse you of living in the past.

Welcome Koufax voters!

2006-01-22T17:11:00.000-08:00

I'm excited to be one of the many fine nominees for the "Best New Blog" Koufax Award for 2005. As noted in the previous post, "Adventures in Ethics and Science" has a cool new home over at ScienceBlogs. But, a lot of the good stuff is still here where it all started.

Because I know you want to make an informed decision about your vote (or, you know, put off doing actual work for a little while), here's a quick tour of my posts here. A few of these are big-traffic posts via search engine results, while others are posts that are dear to my heart (the "unsung heroes" of the archives). It's my hope that these will give you a taste of some of the issues in ethics and science that seize my hands and make me blog.

Scientific Misconduct (fabrication, falsification, plagiarism, and their pals)
When scientists get caught doing bad stuff -- especially when it's in the news -- it tends to set me off. I rant, but I also try to draw some lessons from it.

Aw Mom, scientific misconduct again?! (my initial thoughts on Luk van Parijs)
Lies that "don't matter"? (Van Parijs follow-up)
Completing the misconduct trifecta: plagiarism
Talking the talk vs. walking the walk (plagiarism update)

The Farm League
Some of the instances that might not rise (sink?) to the level of full-blown misconduct, but that are slimy enough that they ought to make responsible scientists glower.

Research with animals

Research with human subjects

Face transplants
(Written in March 2005, before the recent face transplant in France.)
Who'll protect kids from the EPA?
Students as a vulnerable population.

Playing well with other scientists
A round-up of day to day issues in the responsible conduct of scientific research.

Stem cell drama continues (and "magic hands" are raised) (about the tricky balance between having great bench technique and having an experiment that's reproducible)
All kinds of trouble: more on the Korean stem cell saga (wherein I examine the issue of authorship and responsibility)
Authorship matters. (about ghostwriting in the medical literature)
Science blogs for intra-scientific community communication.
Crackpottery, etiquette, and ethical duties. (When things get awkward at the professional conference ...)
Who's in the club, and why does it matter? (Should the scientific community worry about its gender make up?)

Teaching science, teaching ethics

Science for the rest of us
The public funds science; what are the public's interests where science is concerned? And what kind of duties do scientists have when it comes to getting the public to understand what science is up to?

Academic freedom, academic responsibility.
Policy decisions and scientific uncertainty.
Uncertainties, prudent planning, and duties.
Science and priorities.
Communicating science to the public.
I miss Sir Karl! (falsifiability and loopiness need not be mutually exclusive)

That should give you a feel for where I've been so far -- I'm looking forward to taking on a lot more, and I welcome your comments on all of it.

UPDATE: I think all the links are working properly now. Thanks to commenters and emailers who pointed out the broken ones.

Big news for this blog!

2006-01-11T06:31:00.000-08:00

It's moving!

The cool new digs are being hosted by Seed Media Group (which also puts out Seed Magazine). While the rants will still be mine, there are new tools that will be coming online over the course of the coming month(s) that are designed to better server both blogger (me) and reader (you) - and, we hope, will allow folks to navigate across blogs in the network in some novel ways

I'm excited to be making this move because the ScienceBlogs includes more than a dozen other great science bloggers (people you ought to be reading if you're not already) and is gunning to be "the largest and most engaged global conversation about science". This kind of conversation is exactly the kind of thing I think might improve science literacy, augment quality science journalism (while helping people tell good science journalism from dreck), and give non-scientists a better understanding of how the scientific marketplace of ideas works when it's working well. Also, it promises to be a rollicking good time!

This site will remain up (knock wood) as the archive of "Adventures in Ethics and Science" up to this point, but my new posts will go up on the new site. Two other tiny features you'll find on the new site:

My real name.
An unblurred picture of me (though it is about 20 years old ...)

What I do for my loyal readers!

Finally, since it's International De-Lurking Week, y'all can tell me what you think about the whole thing, either in this comment thread or in the comments on the new site.

Using unethical means to expose unethical conduct.

2006-01-10T09:20:00.000-08:00

An interesting piece of the Korean stem cell fiasco that escaped my notice the first time around is that the Korean investigative television program, "PD Notebook," that exposed the faking of photographs for the now-discredited Science article did so using techniques that violated journalistic ethics.

Take a moment to let that sink in.

Here's a lab that is reporting what looks to be great success with cutting edge scientific research. Then Choi Seung Ho, producer of "PD Notebook," gets an anonymous email from someone who claims to be a member of Hwang Woo-suk's laboratory, claiming that Hwang faked data in the Science paper. A good investigative journalist wants to get to the bottom of this to find out whether the stunningly successful research group really is stunningly successful or whether its fame rests on a pile of falsified data.

So, you have to talk to some of Hwang's co-workers, right? The question of journalistic ethics turns on how you talk to them. Here's what James Brooke writes (in the International Herald Tribune):

Choi is in the journalistic doghouse partly for tearing down a national icon, a charismatic, handsome scientist who was the modern, successful face that Koreans yearned to show to the world.

But he is also in the doghouse partly for allowing South Korea's ultra-competitive journalism world to spur him to use techniques that tarnished his work.

In one critical interview, a former co-worker of Hwang, now working at the University of Pittsburgh, is led to believe that his former boss is about to be arrested back home for fraud. When the worker, Kim Seon Jong, starts to talk about faking photographs for the Science article, he can be seen nervously asking if the interview was being filmed. No answer comes from the producer, who is holding a bag with a hidden camera. Instead, the producer hints that if he cooperates with MBC, they will protect him from arrest. To this date, no one has been arrested in the case.

For those keeping score at home, we have:

Conveying the (false) information the interviewee's boss is being arrested for fraud in the research of which the interviewee was a part.
Suggesting (falsely) that cooperation in the interview (which would presumably include ratting Hwang out) will protect the interviewee from (impending) arrest.
Filming an interviewee who did not want to be filmed without alerting him to the fact that he was being filmed or securing his permission to be filmed.

Unauthorized filming with a wee bit of coersion thrown in. Classy! As reported in the International Herald Tribune:

The Korean Broadcasting Commission reviewed the tapes, and a spokesman said the panel had "judged that it is highly likely that the program violated regulations on fairness, objectivity, human rights and statistics and public surveys under the Broadcast Law."

So, "PD Notebook" exposed Hwang's fabrications (which subsequent investigation by Seoul National University has determined really were fabrications), but used unethical means to secure the interview central to this exposure. You might wonder why this matters. Why should folks engaged in deceiving the scientific community, the business community, and the Korean public (who reportedly viewed Hwang as something of a national hero) have any right to expect other people to be honest with them? Isn't there poetic justice in lying to the liars to get the information with which to expose their lies?

Maybe there is. However, what kind of impact does this have on the next scientist with concerns about the boss's misdeeds? Is this scientist going to be brave enough to email a tip to a journalist, knowing that this might expose him or her to coercisive treatment and secret videotaping?

After all, it's not like the interactions between the many scientists on the research team (including "senior author" Gerry Schatten) were, by themselves, enough to head off or correct the fabrication and falsification. There is a legitimate question as to whether the depths of the deception would ever have been revealed if the press had not gotten involved. But rough handling of the journalists' sources of information this round will make it that much harder to find willing sources of information next round.

And, it's worth examining whether this departure from journalistic ethics is an aberation or part of a pattern. Here's what JoongAng Daily had to say:

The roles of producers and journalists are clearly separated in other countries, while producers play both those roles in "PD journalism" in Korea - they go out in the field, investigate, edit the content and write the scripts for the final program.

PD journalism was a byproduct of the authoritarian Korean regimes, media experts said. Academics in Korea do not reject the notion that PD journalism contributed greatly in promoting democracy here, where media have long been censored.
The distinctive form of PD journalism has both some good and bad points, media experts said. "PD journalism was referred to as a window to the truth and an excessive expression of subjective opinions at the same time," said Yoon Ho-jin, a senior researcher with the Korea Broadcasting Institute.

"Current affairs programs made by producers have played the role of critics of the government, and that role was valuable in some ways," said Kim Dong-yule, a senior researcher with the Korea Development Institute. "But the times have changed now, and producers must refrain from investigating with a conclusion already in mind."

"PD journalism has contributed to the maturing of our society with in-depth reporting," a senior executive at MBC said. "But, there have been many cases in which such programs were shaky in gate-keeping and verification."

(Emphasis added.)

In blog entries of yore I have noted that it is a serious problem when science writers make up their minds about how the story's going to go, then contact the scientists looking for quotes to support the story, sometimes even ignoring (or spinning) the quotes from scientists that don't support the stories they've already decided to write. In this respect good investigative journalism ought to be a lot like good science: you can start out with a hypothesis to guide your investigation, but in the end you must take pains to let your conclusions be guided by the evidence. "Taking pains" here means seriously considering the likelihood that your hunch is wrong, given the facts you've amassed. In this particular instance, it might mean considering whether coersion might make a frightened scientist offer testimony that isn't reliable (because he thought that this was the testimony that might save him from some threatened bad outcome). It would certainly mean, as well, seeking information that might explain behaviors that look suspicious. (This might mean asking Hwang or other co-workers to explain the Science photographs, for example.) Starting an investigation with your conclusion set in stone is just as intellectually dishonest as making up the data you report.

I suppose it's understandable why a reporter -- even one who hadn't made up his or her mind in advance -- might pressure an interviewee to in order to obtain information that it might be really hard to get otherwise. Indeed, given how good scientists can be at keeping secrets (some information is proprietary, after all, and other information you keep close to the vest until you're ready to publish it so as not to get scooped), it's possible that "PD Notebook" provided the crucial break that brought the house of cards tumbling down. But this just doesn't strike me as a sustainable model for keeping scientific researchers honest. Honest communication with the public (and a recognition that science is ultimately accountable to the public) is an important piece of fostering ethical conduct and rooting out misconduct. However, even more effective would be honest communications between scientists and a recognition that each scientist is accountable to the whole community of scientists. Tipping off a reporter is one way to try to head off misconduct in your lab, but there need to be mechanisms for tipping off folks in the community of science who have a vested interest in eliminating misconduct from science and its products from the scientific literature.

Knowing scientists as I do, it seems to me that pissed-off scientists could deliver a much better smackdown to a fabricator than any media outlet ever could.

Working to do human subjects research right.

2006-01-09T09:29:00.000-08:00

Today, some news that makes me smile (and not that bitter, cynical smile): UCSF has announced that it has received full accreditation for its program to protect research participants from the Association for the Accreditation of Human Research Protection Programs (AAHRPP).

This is a voluntary accreditation -- nothing the federal government requires, for example -- that undoubtedly required a great deal of work from UCSF investigators and administrators to obtain. (AAHRPP describes the process as including a preliminary self-assessment, followed by appropriate modifications of your institutions human subject protection program, preparation of a detailed written application, an on-site evaluation of your program by a team of experts, and review of these materials by the AAHRPP council on accreditation.) Here's what the UCSF news report has to say about the process:

While the US Department of Health and Human Services requires federally funded medical research centers to provide written assurance that all human research is in compliance with federal regulations and is guided by national ethical principles, the AAHRPP assessments are more rigorous and comprehensive.

AAHRPP reviews all programs involving research participants or their biological specimens– not only those programs that are federally funded -- and its assessment includes additional protections not required by the federal agency, such as community education and quality improvement activities.

The AAHRPP accreditation process took more than a year of preparation and several months of review, including evaluation of complex research protocols and related safety and privacy measures, as well as the caliber of training of investigators and the strength of the institutionwide commitment to human subjects' protection.

In the process, scrutiny was brought to bear on the effectiveness of interactions between various units needed to ensure the best protection, such as the institution's investigational drug pharmacy program, clinical research centers, committee examining potential conflicts of interest, office overseeing sponsored research, and overall medical center organization.

Notice that the focus here goes beyond whether an institution is following the letter of the law. Federal regulations on reserach with human subjects only extend to federally funded research. The AAHRPP is looking at all the research programs with human subjects at an institution -- whether funded by the feds, private donors, pharamaceutical companies, or any other entity -- to evaluate the human participant protections. And, as noted above, not only is AAHRPP asking for "additional protections not required by the federal agency, such as community education and quality improvement activities," but it is also attentitve to institutional features (e.g., "the effectiveness of interactions between various units needed to ensure the best protection") that are connected to how well programs to protect research participants function. The question is more than whether the institution is in compliance with standards right now, but whether the institution is set up in such a way that continued compliance is a robust part of the way things are done.

Why, you might ask, in the already busy crush of trying to get research done, would an institution take on the extra burden of applying for an accreditation that is not required of it? Part of the answer may be in the attention to how interaction between different units of an institution make compliance more natural. It's easier, in the long run, not to have to struggle to meet the necessary federal regulations -- having a system where the different units are all looking after subject protection, while maybe requiring more effort to set up, is a lower maintanence way to stay in compliance. Moreover, getting this sort of voluntary accreditation sends an unambiguous signal to the people in your organization that the institution is really committed to protecting human subjects, not just grudgingly meeting a bunch of onerous regulations imposed by the government. And, as AAHRPP points out,

Each time a new organization becomes accredited, the global benchmark for human research protection in science is raised.

In other words, working to make things good for human subjects at your institution is a way to make things better for human subjects everywhere.

The AAHRPP website makes for interesting reading, especially its discussion of five domains of standards for human research protection programs (Organization, Research Review Unit, including IRBs, Investigator, Sponsored Research, and Participant Outreach). Also, their advice for an institutional self-assessment looks like it would be valuable for any institution doing research with human subjects, regardless of whether that institution wanted to seek the AAHRPP accreditation.

Kudos to UCSF. Keep making us proud!

Is all animal research inhumane?

2006-01-08T11:38:00.000-08:00

I received an email from a reader in response to my last post on PETA's exposing of problems with the treatment of research animals at UNC. The reader pointed me to the website of an organization concerned with the treatment of lab animals in the Research Triangle, www.serat-nc.org. And, she wrote the following:

Some people may think that PETA is extreme. However, the true "extreme" is what happens to animals in labs. If the public knew, most would be outraged. But, of course our government hides such things very well. Those researchers who abuse animals in labs (which is ALL researchers, by my definition), cannot do an about turn and go home and not abuse animals or humans at their homes. Animal researchers are abusers, and there is enough research on people who abuse to know that abuse does not occur in isolation. The entire industry must change.

There are a bunch of claims here, some of which I'm going to pretty much leave alone because I don't have the expertise to evaluate them. Frankly, I don't know whether even the folks we would all agree are abusing animals in the lab are full-fledged abusers who cannot help but go forth and abuse spouses, children, family pets, neighbors, and such. (I'm not a psychologist or a sociologist, after all.) And, while I'd like to believe that the public would be outraged at unambigous cases of animal abuse, the public seems not to be outraged by quite a lot of things that I find outrageous.

I would, however, like to consider the claim that ALL researchers who do research with animals are abusing those animals.

First, I imagine there may be some research projects involving animal subjects where it's hard to locate an actual harm to the animals. (Consider, for example, positive reinforcement experiments that train pigeons to type. The pigeons are put in the unnatural position of having to interact with a typewriter, but they get food, are protected from predators, etc. Is this a worse life than scrounging through garbage and avoiding city buses? What if we throw in a daily hot stone massage?) For the sake of argument, let's set those aside.

I take it the animal research that is of real concern is that which brings about pain in the animals, or that which ends with the animals being "sacrificed" (i.e., killed). If we agree that animal pain and killing of animals are harms to be avoided (and not everyone will -- the U.S. is a meat eating nation, after all), does that mean that all research that causes animal pain or the killing of animals ought to be stopped?

We'd need to consider the sorts of harms that might come from ceasing animal research. It would, for example, have a marked effect on biomedical research -- including research with human subjects. The very first item in the Basic Principles in the World Medical Association Declaration of Helsinki reads:

Biomedical research involving human subjects must conform to generally accepted scientific principles and should be based on adequately performed laboratory and animal experimentation and on a thorough knowledge of the scientific literature.

(Emphasis added.)

In other words, ceasing research with animals heads off much new biomedical research with humans. You can't test a new drug on humans if it hasn't yet been tested in the appropriate animal system, no matter how promising that new drug may be. Unless WMA were to make significant revisions in the Declaration of Helsinki, an end to animal experimentation might mean an end to drug development and other lines of biomedical research. Ending animal suffering in the lab might mean there is more human suffering that we're unable to address with medical treatment.

Of course, there's the legitimate question of whether animal models are actually adequate models for the human conditions such biomedical science aims to address. (I've been told that we could cure most mouse cancers in fairly short order, but we're still quite a ways off on the human cancers the mouse cancers were intended to model.) Lately, researchers have developed an array of alternatives to animal research (in vitro studies, computer models, etc.), but these approaches have their limits, too. Surely the best system for studying human diseases and their treatments would be humans, but experimentation on humans is no less ethically problematic than research on animals.

Goodness of fit between a model and the system that is the target of the modeling is something scientists have to grapple with all the time. There are always ways that the model departs from the target. The practical question is how to work out models that get the important features of the target right. It may be the case that, imperfect as animal models are, they are still the best models we have for certain phenomena we are trying to figure out. But especially when our model systems come with ethical costs (not only animal research but epidemiological studies with humans), it seems like critically examining the model and keeping an eye out for alternative models that might work better is a good idea.

One could object that some of the research done with animals is simply unnecessary. For example, two of the research studies flagged by SERAT as especially problematic are a study of binge-drinking using rats and a study of gambling using primates. Even if binge-drinking and gambling are human behaviors that are problematic and need to be addressed, it's not obvious that the only ways to address them require a complete understanding of the underlying physiological mechanisms of these behaviors. Even if the physiology of binge-drinking or compulsive gambling were to remain something of a black box, there might be ways to change the environment to head off these behaviors.

Scientists might respond that knowing the physiological mechanism is of value even if we don't need that knowledge to solve the problem of heading off harmful behavior. Sometimes knowledge is a good in itself. However, if that knowledge comes at a cost, it's worth considering how that cost stacks up against the value of that knowledge. (Consider the costs of the Tuskegee syphilis experiment, whose aim was knowledge about the natural history of untreated syphilis. Certainly, such information would have value, but its value didn't justify the harms it brought to the subjects of the experiment.) So, the fact that scientists are curious and would like to get a piece of information does not, in itself, justify all the costs that getting to that knowledge might incur.

Careful readers that you are, you will have noticed that I've taken a consequentialist approach to this issue -- one of balancing competing harms and benefits. While this is how most of those who worry about ethical use of animals usually frame the problem, there are others who feel that a more Kantian approach is in order. (Maybe "Kantian" is not quite the right label, since Kant was concerned with respect for persons, and with not undermining the rational capacity in oneself or others. But stick with me here.) In research with human subjects, there are some lines you are not allowed to cross, regardless of the potential benefits of crossing them. For example, here are two items in the Helsinki Declaration's principles governing non-therapeutic research with human subjects:

1. In the purely scientific application of medical research carried out on a human being, it is the duty of the physician to remain the protector of the life and health of that person on whom biomedical research is being carried out.
...
4. In research on man, the interest of science and society should never take precedence over considerations related to the well-being of the subject.

(Emphasis added.)
The health and the life of a human research subject are always to be protected by the researcher. No matter what the payoff might be, whether in terms of solving practical problems for society or building scientific knowledge, you can't sacrifice the subject's health or life. This is a non-negotiable point -- like Kant's respect for persons -- around which your consequentialist calculations have to work.

Perhaps there are such lines we ought to recognize with animals in scientific research. I think when they are working as they should, IACUCs are trying to find and respect those lines. But it is also clear that we live in a society that has few qualms about doing fairly nasty things to animals for the sake of cheap food production, or entertainment (think cock-fights), or biker-wear. That society at large doesn't recognize a clear line separating appropriate and inappropriate ways to treat animals doesn't mean there isn't a line there we should recognize (as feminists, anti-racists, and the like will be happy to explain to you). My own hunch is that within a few generations, we may get to a point where certain ways of treating animals that are prevalent right now become unthinkable. But, not having gotten to that point makes it harder to argue for thoroughgoing changes in the rules for animal experimentation. As the situation at UNC illustrates, sometimes it's hard to get people to even follow the rules that are in place.

That said, let me suggest again that it is a strength of the community of scientists that scientists don't all march in lockstep on the matter of what humane treatment of laboratory animals require. Because different scientists have different views on this matter, they're more likely to actually talk to each other about it. In the course of these talks, scientists sometimes come up with clever strategies to get more scientific information with less -- or no -- animal harm. Given that scientists, as a group, have shown themselves to be quite good at answering hard questions using limited data, it might not take all that long for them to work out good ways to eliminate the need for animals in certain research projects, and to minimize the need for animals in others.

And scientists probably ought to care about animal-use worries of the public, not just of other scientists. At the same time, though, scientists should be ready to explain to the public how their animal use is essential to the research, and how that research benefits the public. Then, if members of the public disagree with the scientists (e.g., deciding to forego a bird flu vaccine if it involves animal research of which these members of the public do not approve), that's their choice. If no one used a medical treatment because of ethical qualms, the demand for that treatment would evaporate, and the researchers would turn their attentions elsewhere.

So, to my email correspondent: I'm not sure I think the problem of animal research is as black-and-white as you think it is. But, I'm inclined to think that science is moving toward higher standards for ethical use of animals, at least gradually. And, I think continued discussion on this issue is how that movement happens.

Just because they're out to get you doesn't mean they don't have a point.

2006-01-05T09:03:00.000-08:00

Since I'm in the blessed wee period between semesters, it's time to revisit some "old news" (i.e., stuff that I had to set aside in the end-of-semester crush). Today, a story from about a month ago, wherein the Rick Weiss of the Washington Post reports on the University of North Carolina's troubles obeying animal welfare regulations in its research labs.

You knew that the National Institutes of Health had all sorts of regulations governing the use of animals in research (and even an Office of Laboratory Animal Welfare, whose webpages have a bunch of helpful links for those involved in such research), right? You'd assume that the folks running a major research university (like UNC) would know that, too. Because you know who else knows it? PETA. And somehow, PETA had an inkling that researchers at UNC were maybe not taking the regulations on animal use all that seriously.

From the WaPo article linked above:

At the center of the storm is the University of North Carolina, which in the past four years has twice had the misfortune of hiring animal laboratory technicians who turned out to be undercover agents for People for the Ethical Treatment of Animals.

The first instance produced embarrassing video footage taken by the employee (one clip showed a lab worker using scissors to cut the heads off of baby rats while saying: "I don't put them to sleep. Maybe it's illegal, but it's easier."). It led to a damning report from the federal Office of Laboratory Animal Welfare. But no sanctions came down from that office, part of the National Institutes of Health, because by the end of that investigation OLAW had determined that the problems had been corrected.

By then, however, PETA had managed to have a new agent hired by UNC. After an 11-month tour of duty, that employee released a new batch of evidence, including more photos and videos, and OLAW opened a new investigation.

The recently released report of that second investigation is remarkable for its similarity to the first report, PETA activists note -- including its conclusion that no action needs to be taken because of reassurances that the university has again resolved the problems.

"We looked at the new report and thought, 'Did they just cut and paste the old one or what?' " said Kate Turlington, the PETA investigator who conducted the first undercover operation, during which she wore a hidden video camera while caring for sick lab animals and talking to co-workers.

Goodness gracious, where to start?

First, the NIH, whose regulations we're talking about. It seems like only yesterday I was blogging about problems that flow from having rules without meaningful enforcement. Maybe the NIH is applying major pressure to UNC behind the scenes to really address the problems with the treatment of its laboratory animals. (Maybe NIH can send in undercover agents as lab technicians!) Or maybe, being part of the federal government, NIH is not having such an easy time, what with resource issues and political pressures, functioning as we would like it to.

Not that I'm at all cynical about the government these days.

What about the UNC employees whose conduct PETA recorded and brought to light? It seems pretty clear that they were not only violating the regulations, but were also aware that they were violating the regulations. ("Maybe it's illegal, but it's easier.") Folks, this isn't lobbying or energy trading. This is laboratory science. It would also be easier to experiment on just two rats rather than hundreds. Or, for that matter, to just say you experimented on some rats and make up some persuasive data. Easy isn't what's driving the process here, and breaking the law is frowned upon.

So, UNC gets caught violating animal welfare regulations. They get hit with the "damning report" from OLAW. No sanctions from NIH yet, but the unpleasant publicity from PETA. You would think at that point that someone in charge at UNC would take serious action to make sure everyone doing animal research at UNC cleaned up his or her act. Otherwise, you'd be risking sanctions from an NIH angered that the "damning report" from OLAW had been ignored. And, you'd be risking putting your institution in a position where what PETA claims about it is true. Which, from a public relations point of view, seems like a mighty big risk.

And, which makes the subsequent PETA exposé pretty damn embarrassing (or should, if there is any self-awareness and shame still possible among those who oversee animal research at UNC). It almost looks like, institutionally, UNC doesn't care about animal welfare regulations. This is a problematic stance if, say, you'd like to take money from the federal government to support your research with animals. Moreover, paying lip service to the regulations without making sure they are followed is lax management at best; if you have a principled disagreement with the regulations, presenting reasoned arguments against them is much less slimy than winking at them and taking the money.

The most horrifying part of this all for UNC has got to be making PETA look (comparatively) reasonable. PETA doesn't want any animals used for research (or food, or clothing). PETA is not generally viewed as a voice of reason or moderation. PETA would have you believe that research with animals is usually inhumane, and that scientists and lab technicians can't be trusted to follow the animal welfare regulations.

Thanks to UNC, they have some hard evidence to back up that claim. This, coupled with the NIH's seeming unwillingness to actually enforce the regulations, has got to make things harder, at least on the PR front, for other scientists doing research with animals, even those who follow the animal welfare regulations scrupulously. When the public sees this kind of story, what's that going to do to the center of gravity of public opinion on animal research in particular and on the trustworthiness of science in general?

Nothing good.

But now, the very best part of the WaPo story:

Tony Waldrop, UNC's vice chancellor for research and development, said that many of the problems found in the second inspection were remnants of problems from earlier on, which were still in the process of being corrected. "It was not new information," he said, noting that a recent follow-up inspection resulted in "an absolute clean bill of health and full accreditation."

...

Perhaps most important, UNC says it has updated its screening and background checks for new hires.

In other words: It takes time to get the lab techs to actually treat animals humanely rather than breaking the law because it's easier; it's not like we can just tell our employees what to do! But in the meantime, we'll make damn sure we don't hire anyone who has worked with PETA or who shows other indications of a concern for animal welfare. That hasn't worked out so well for us.

The problem with cheaters.

2006-01-04T08:26:00.000-08:00

[Finally I'm actually healthy again, and not in a hotel charging $10 a day for internet access. So, on with the blog!]

It must be a law of nature that when past and current graduate students dine together at the end of December the conversation turns, sooner or later, to cheaters. First, of course, you discuss the head-slappingly stupid techniques cheating students employ. ("If they thought we wouldn't notice them doing that, they must think we're really stupid!") Then, you recount a sting operation or two (like planting someone next to a habitual cheater during an exam and having the plant spend the exam period writing utter nonsense -- all dutifully copied by the cheater onto her own exam). Finally, there is the wringing of hands over how the graduate students' efforts against cheaters are for nought given the policies at certain universities that, basically, don't let you do jack to the cheaters.

It's that last part that's been sticking in my craw since the cheating cheaters discussion of which I was a part on New Year's Eve.

Maybe I just lack the necessary perspective here. I am not now, nor have I ever been, a university administrator. I do not grant degrees, nor do I take in tuition money. I've just been in the trenches teaching. From my point of view, assignments and exams are tools for assessing how well my students have learned the material I have tried to teach them (and, therefore, of how effectively I've taught the material), and of how well they're reasoning about this material. Cheating, therefore, is a subversion of the communication that's supposed to tell us how well we've done with the process. What it ends up communicating, when detected (and detection is far more frequent than students seem to think it will be), is that the cheater doesn't actually care about learning the material on offer. And, I get that there are lots of things about which one may legitimately not care, but it seems like it's a good idea then not to take a course on them. Or, if one must take a course on them (as a requirement, say, for a major about which one does care), it seems like a better strategy to try to find something to care about in the material -- how is it connected to the thing I do care about, for example.

Indeed, part of what I find most offensive about cheating in my courses is that it is an attempt to appear as if one cares about the material that reveals the absence of actual effort to learn the material. Cheaters care about my course instrumentally, as a means to get a necessary requirement filled or to get a desired grade. And, they seem to think that I won't feel ill-used by their cheating.

But I'm not ranting about the students today. I'm down on systems that let cheating persist unchecked. On New Year's Eve, I heard tell of policies at three major research universities that make it next to impossible to do anything to a student you've caught cheating. One where a student isn't "caught" without multiple witnesses to the act -- one of whom has to be the professor of record for the course. (Teaching assistants, the prof always sticks around when the exam is being administered, right?). Another where professors and TAs are expressly forbidden from being in the room while students are taking exams (which leaves witnessing and reporting the cheating up to students ... who are not always so invested in taking up this responsibility). For all three of these institutions, there seems to be serious pressure from the administrative forces in the system not to impose sanctions (like suspension, or even failing grades) for even habitual cheaters. And the lack of institutional will to take a stand against cheating seems to have made some of the profs just ... give up trying to do anything about it in their own courses. (Who wants to take on the procedural nightmare involved even in administering a slap on the wrist?)

What the hell are these administrative forces thinking?

I'm sure much of their thinking is informed by legitimate concerns for the rights of the students to due process. If I were cynical, I might suggest that their thinking is also informed by the likelihood that the parents of the cheaters, the captains of industry paying upward of $40K a year for junior to get a name-brand diploma, may be inclined to call those administrative forces to lobby for junior to get a second (or third, or fourth, ...) chance. Certainly, it would be a problem if the system were set up in such a way that profs and TAs could merely allege cheating, without proving it, and thereby end a student's college career. But that's not what's happening. Rather, we seem to have a situation where habitual cheaters are not held to account at all, except for perhaps having to repeat a course.

This at universities where, occasionally, faculty members are booted for fabricating data.

My gut says the root problem here is the model of the university that the students and the administrative forces seem to have in mind. The operative assumption is that the student is a consumer and the university is providing a product. (I paid my money, whaddaya mean I don't get my degree?!) On this model, exams with the right answers are just the necessary paperwork you have to turn in to get the degree you came for. How much, really, should it matter how you got that paperwork filled out?

A better model, at least from where I sit, is one of community. While each of us has our individual interests, we have certain interests in common (like the honest exchange of information and ideas, or the creation of conditions that foster learning). This is why cheating is an abomination -- it strikes at our common interests, and makes it impossible for us to function well as a community. Administrative actions that don't recognize or address this aspect of cheating further undermine the community. When administrative forces don't get that cheating hurts the community, they reinforce the cheater's sense that the community doesn't matter.

Community may be the key to dealing with cheaters in the world of science, too. In many of the high-profile cases of fabrication, falsification, and plagiarism, it comes out that the cheater is a habitual cheater -- someone who has been cheating for some time, and who may even have been caught doing so but let go with a slap on the wrist. I've heard it said (and it seems reasonable to me) that the tendency to let scientists go with a slap on the wrist is reinforced by a lack of intermediate-level penalties for cheating; if all you have is the scientific equivalent of a death penalty, you may look for reasons to let people off. But, having been let off, sometimes repeatedly, the cheaters may start to get the message that cheating doesn't really matter all that much. Their "youthful" offenses are kept quiet, lest a promising young researcher's career be ruined.

Wouldn't it be better to bring the "youthful" offenses out into the light so the scientific community could make it clear how these kinds of behavior hurt the community and undermine the project science is trying to do? Shouldn't the community, in the process of training new scientists, take an active role in keeping these new scientists honest? Mercy comes from an understanding that people sometimes falter in their judgment; working together as a community to help members exercise good judgment seems like a better approach than leaving someone who has screwed up on his own with just the warning not to screw up again. The community ought to know about "prior bad acts", not so it can isolate the actors or consider them evil (because if that were the goal, you'd just boot them from the community on the first offense), but so the community can help the actors interact with the community in better ways and earn back the community's trust.

Truly evil actors will need to be booted, of course. But it seems reasonable that only a small proportion of cheaters are irredeemably evil. One of the strengths of community is that it can help bring you back after you've gotten off track. The trick, it seems, is understanding that you're part of a community in the first place.

Scenes from the jetway.

2005-12-27T12:03:00.000-08:00

Boarding a plane last night, a little scene played out. Prior to boarding, the gate agent had announced that the plane would not have room for wheeled suitcases in the overhead bins, and that anyone who had a wheeled suitcase s/he had intended to carry on should get a tag to check it at the jetway. In the gate area, one of the ticketed passengers, who had a big wheeled suitcase, approached the gate agent to argue about this instruction. For five minutes. After which, she refused to take a tag for her behemoth of a "carry-on".

Boarding. The same ticketed passenger handed over her boarding pass but refused, again, to take a tag for her bag. She walked down the jetway, causing the agent taking boarding passes to chase her down the tube (and, of course, delaying the boarding of everyone else in the line until the agent returned).

On the jetway, the folks set to load tagged bags into cargo asked her, "Didn't you get a tag? Did they run out?" Snottily, she asked, "Why do I have to check mine? There were lots of people ahead of me who brought theirs. I'm bringing my bag with me!"

Of course, the moment she tried to board the (really quite small) plane with this honkin' big suitcase, the flight attendant said, "No, that won't fit in the overhead bin. It needs a tag." The ticketed passenger from hell launched into a tirade about how she had a tight connection to her next flight at the destination and she couldn't lose the time it would require to retrieve the bag from cargo and bring it back to the jetway (seemingly oblivious to the delay she was causing in the boarding of the plane). The flight attendant showed her the overhead bins (each about 1/3 the size of the honkin' big suitcase.) The flight attendant then affixed a tag, handing the H.B.S. and the one other wheeled suitcase that someone else had tried to bring onto the plane, to the folks wrangling luggage at the jetway.

It may not surprise you that this scene made me think about ethics in general, and about rules imposed upon scientific researchers.

I'm not the most Kantian kid on the block, but one of Kant's insights I think is dead-on is that it's assy to make yourself an exception to rules you expect others to observe. If no one takes the jetway tag for his or her wheeled suitcase, there won't be room for anyone's carry-on luggage. Maybe if everyone else gets his or her wheeled suitcase checked, I'll have room to carry mine on (though not in this case) -- but I can only count on this payoff if everyone else plays by a different rule (follow the gate agent's instructions) than I play by (do what you want).

It's worth noting that the problem passenger here justified her behavior by saying:
(1) lots of other people are doing the same thing (false! just one other passenger tried to bring a wheeled suitcase on the plane), and (2) my interests (e.g., in not missing my connection) are such that I should be exempt from the rule. In fact, I don't think these justifications are prima facie crappy ones. If lots of other passengers were ignoring the instructions, there might be good reason to ask whether poor enforcement of the rule had undermined it altogether. Perhaps it is unjust to enforce the rule with me but not with all those other people ahead of me. And, in the event that it were possible to fit some small number of wheeled suitcases in the overhead bins (not true on this plane, but hypothetically on another), there may be interests such that we could identify which passengers could stow their wheeled suitcases overhead.

Here, of course, we had a rule that existed for a very good reason: the wheeled suitcases just didn't fit in the tiny overhead bins. The passenger trying to get around this rule was just being a jerk.

To a certain extent, what's true for the airline passenger is true for the research scientist, too. It's assy to count on other scientists to follow rules but decide that you don't have to follow them yourself. It would be assy to expect others to share reagents, while never sharing your own. It would be assy to expect others to review your manuscripts fairly if you yourself were a reliable source of venom in your reviews of the manuscripts of others. It would be assy to demand that others get their protocols approved by the IRB while making unauthorized changes in your own IRB-approved plans.

And yet ... the nightmare passenger was at least being open in her questioning of the rule. She wasn't trying to be sneaky. Granted, there were more respectful ways she could have expressed her doubt about the rule, and she could have considered the (physically) plausibility of the explanation offered for the rule. But breaking the rule while pretending to follow it would have been worse. Much of the really catastrophic stuff in science, ethically speaking, seems to happen when people act as though they are following the rules while breaking them big time.

The Doctor is In

2005-12-27T11:39:00.000-08:00

I haven't posted in a while because I've been wrestling with some nasty virus. Four days of fever and all the fun that goes with it! Personal favorite: delirium. (There is nothing like dreaming in HTML to rob "sleep" of all its restful properties.) Having finally gotten to the point where I'm feeling more healthy than unhealthy, my first inclination was to raise a cheer to modern medicine.

But ... why? I didn't schlepp to the doctor. I didn't take any medicines that would have killed the virus (and I know full well that antibiotics wouldn't do jack to a virus). Medical technology did allow me to check my temperature as frequently as I could manage (and, via ear rather than under my tongue, there was no increased urge to puke ... a definite plus). The few doses of ibuprofen I managed to take may have kept my fever from getting into the dangerously high range, but ibuprofen on an empty stomach ... urge to puke increasing.

In other words, modern medicine gave me a tool to monitor my fever, and another that let me reduce it, perhaps, if I could handle the puke-y feeling that came with it. Everything else was up to my immune system.

And given how happy I am that my immune system has gotten me through this little adventure, I'm now inclined to give the immune system (and other bits of "normal" functioning of the human body) at least a little more credit when I read about new and exciting medical treatments. This is not to say such treatments aren't doing something -- just that the immune system (and its ilk) may be responsible for a significant part of the outcome as well.

Dover ruling.

2005-12-20T08:40:00.000-08:00

It's here. The AP story, via Yahoo News:

Dover Area School Board members violated the Constitution when they ordered that its biology curriculum must include the notion that life on Earth was produced by an unidentified intelligent cause, U.S. District Judge John E. Jones III said. Several members repeatedly lied to cover their motives even while professing religious beliefs, he said.

[...]

"The citizens of the Dover area were poorly served by the members of the Board who voted for the ID Policy," Jones wrote.

The board's attorneys had said members were seeking to improve science education by exposing students to alternatives to Charles Darwin's theory that evolution develops through natural selection. Intelligent-design proponents argue that the theory cannot fully explain the existence of complex life forms.

The plaintiffs challenging the policy argued that intelligent design amounts to a secular repackaging of creationism, which the courts have already ruled cannot be taught in public schools. The judge agreed.

"We find that the secular purposes claimed by the Board amount to a pretext for the Board's real purpose, which was to promote religion in the public school classroom," he wrote in his 139-page opinion.

[...]

Said the judge: "It is ironic that several of these individuals, who so staunchly and proudly touted their religious convictions in public, would time and again lie to cover their tracks and disguise the real purpose behind the ID Policy."

One take home lesson: honest disagreement is one thing, but dishonesty will come back to bite you in the butt.

And, in case you were curious, here, from AP via the Seattle Post-Intelligencer, is the text of the Dover intelligent design statement in question in this case:

"The Pennsylvania Academic Standards require students to learn about Darwin's theory of evolution and eventually to take a standardized test of which evolution is a part.

"Because Darwin's theory is a theory, it continues to be tested as new evidence is discovered. The theory is not a fact. Gaps in the theory exist for which there is no evidence. A theory is defined as a well-tested explanation that unifies a broad range of observations.

"Intelligent design is an explanation of the origin of life that differs from Darwin's view. The reference book, 'Of Pandas and People,' is available in the library along with other resources for students who might be interested in gaining an understanding of what intelligent design actually involves.

"With respect to any theory, students are encouraged to keep an open mind. The school leaves the discussion of the origins of life to individual students and their families. As a standards-driven district, class instruction focuses upon preparing students to achieve proficiency on standards-based assessments."

(Bold emphasis added.)

So ... what exactly can we glean about the purpose of the public schools from this last paragraph? What if we substituted "linear equations" or "irregular verb conjugation" or "the branches of the federal government" for "the origins of life" here? Is there anyone who thinks that would be a good idea? (OK, I suppose it depends on how good your school system is -- still, there are some things you ought to be able to look to the schools to teach. Being, you know, schools.) And hey, I'm as disgruntled at the whole teaching-to-the-high-stakes-test movement as anyone (since I have to help survivors of that kind of secondary education understand how to learn stuff for real). But, it seems there are certain core competencies that are part of "learning biology", and that a responsible biology class will help its students attain those competencies. You can't call any old collection of ideas "biology" without doing violence to the meaning of the word.

For those who love to wallow in the legal language, here's the link to the page from which you can download the judge's opinion. (It takes a long time to download, but it's worth it.)

UPDATE: A good set of links to commentary on the ruling can be found at Science and Politics.

Getting linky with it.

2005-12-19T22:03:00.000-08:00

Some good stuff to read, especially if you're resurfacing from unpleasant tasks like writing or grading papers:

Sharing reagents/no good deed goes unpunished. YoungFemaleScientist has a great post about possible downsides to sharing published reagents. In particular, if the person who requests your reagent can't get it to work — and doesn't keep you in the loop so you can share your expertise — s/he might decide your reagent is crap. They might even go to your competitor for a similar reagent. Go read the post to see how this feeds into the eternal struggle to publish and have your findings recognized and used by your scientific peers. One important take-home message is that effective scientific communication involves asking questions, and it extends well past publishing findings or using published findings.

Sometimes "magic hands" are just hands that are better than yours. The Pluripotentate has a very nice response to my post about "magic hands" in the Korean stem cell scandal. The whole too-good-to-be-true nature of certain experimentalists' achievements can make you suspect ... well, that they really are too good to be true and these experimentalists are maybe makin' stuff up. But the Pluripotentate reminds us that this is not the only explanation:

I'd add another possibility -- stubbornness on the part of people trying to learn the technique. Right now I'm the magic hands in the lab on a particular technique. I remember being suspicious of the woman who taught me, until I finally got it to work. I finally gave in and made fresh buffers and followed the protocol exactly. So painful. So tedious. So rewarding.

The people I'm teaching now, who've been struggling for months, may look on me with some suspicion. But they're the ones hanging on to their precious contaminated Tris, the poor dears. One day, at the end of their ropes, they'll pour their buffers down the sink in frustration. And the week after that, the heavens will rain blessings down on their sparkling new magic hands.

Experiments are hard. That's why scientists get the big bucks (and the chicks, and the public's adulation, and a pony!).

Methodology matters. Check out Shakespeare's Sister's discussion of methodological flaws in a Norwegian study that purports to show that women who have abortions suffer “mental distress” longer than do women who have miscarriages. The dissection of the problems is detailed — go read it. Shake's Sis tells us:

... the mental health of the participating women who sought an abortion was almost statistically significantly poorer than the participating women who had a miscarriage, and the complexity of the abortion issue may account for discrepancies. That’s the problem with poor controls; you can end up with a study that has a completely meaningless conclusion. And yet here it goes—out into the world, reported as fact. Women who get abortions are more highly traumatized than women who have miscarriages. Even though it may be the women who got abortions and participated in the study were more inclined toward mental distress irrespective of their abortions, or that societal views of abortion—and specifically, women who get abortions—may facilitate feelings of shame and guilt.

It's not clear that the Norwegian researchers were trying to skew the results. Even before you start collecting data, good experimental design is hard, too.

It's nice that the blogosphere keeps chugging with such nice posts while I've been off in my cave grading.

Authorship matters.

2005-12-19T21:03:00.000-08:00

The whole lot of scientist involved with Hwang Woo-suk of Seoul National University and his stem cell research seem to be feeling the pull of the drain. Having admitted to fabricating some of the data in their Science paper, Hwang has agreed to withdraw it. Meanwhile, the "senior author" on the paper, Gerald Schatten, is being investigated by the University of Pittsburgh. As reported by the Washington Post:

At the very least, Schatten faces a formal reprimand once an internal school investigation is concluded.

"I will consider what disciplinary actions are appropriate in this case pending the findings," said Dr. Arthur Levine, dean of the medical school and Schatten's boss.

What's this, you say? Wasn't Schatten the whistleblower here? He was also the "senior author:

Scientists say that as "senior author" on the paper, it was his responsibility to catch the many errors Hwang has admitted.

However, Pittsburgh officials and the paper itself described Schatten's involvement in the cloning research as limited to consultation, helping the South Koreans prepare their manuscript and serving as their English-language translator.

Schatten did little, if any, actual research.

Levine said it's unclear why Schatten was given senior author status among the 24 South Korean scientists who also signed on to the paper.

"One should only be the senior author of a scientific paper when one has prepared and was responsible for all the data in that paper," Levine said. "It also implies the senior author is the chief of the lab where the experiment took place."

See? It wasn't just me saying that authorship brings certain responsibilities with it -- not just after the paper has been published, but before it's even sent off to the journal. While Schatten's level of (non-)involvement may not be unheard of for a senior author (ask around and you'll hear about lots of instances of people who are authors on a paper primarily because their high profile is hoped to increases the chances of publication), it seems to have blown up in his face rather spectacularly.

Meanwhile, the Wall Street Journal takes up the matter of scientific authorship as it plays out in ghostwritten articles for medical journals. Check out the discussion at Lawyers, Guns and Money. It's important to note that we're not just talking about ghostwriters who are employed to pretty up the prose of scientific communications -- we're talking about ghostwriters who work for drug companies. Quoting from WSJ as quoted by LG&M:

When articles are ghostwritten by someone paid by a company, the big question is whether the article gets slanted. That's what one former free-lance medical writer alleges she was told to do by a company hired by Johnson & Johnson.

Susanna Dodgson, who holds a doctorate in physiology, says she was hired in 2002 by Excerpta Medica, the El-sevier medical-communications firm, to write an article about J&J's anemia drug Eprex. A J&J unit had sponsored a study measuring whether Eprex patients could do well taking the drug only once a week. The company was facing competition from a rival drug sold by Amgen Inc. that could be given once a week or less.

Dr. Dodgson says she was given an instruction sheet directing her to emphasize the "main message of the study" -- that 79.3% of people with anemia had done well on a once-a-week Eprex dose. In fact, only 63.2% of patients re-sponded well as defined by the original study protocol, according to a report she was provided. That report said the study's goal "could not be reached." Both the instruction sheet and the report were viewed by The Wall Street Journal. The higher figure Dr. Dodgson was asked to highlight used a broader definition of success and excluded patients who dropped out of the trial or didn't adhere to all its rules.

The instructions noted that some patients on large doses didn't seem to do well with the once-weekly administration but warned that this point "has not been discussed with marketing and is not definitive!"

Let's be clear: There are articles in respected medical journals whose content has been spun by authors who didn't do any of the research but did get paid by drug companies with a financial stake in the research outcome reported. And, the names of these authors don't appear on the papers whose substance they have materially altered. Which means, of course, that it's really hard to hold these authors responsible for the content, since they are invisible to the other scientists relying on these published articles as sources of reliable information.

You think I'm getting carried away about this kind of ghostwriting? Here's another quote from the WSJ article:

Now questions about the practice are mounting as medical journals face unprecedented scrutiny of their role as gatekeeper for scientific information. Last week, the New England Journal of Medicine admitted that a 2000 article it published highlighting the advantages of Merck & Co.'s Vioxx painkiller omitted information about heart attacks among patients taking the drug. The journal has said the deletions were made by someone working from a Merck computer. Merck says the heart attacks happened after the study's cutoff date and it did nothing wrong.

(Bold emphasis added.)

Vioxx, baby. We're talking human lives (or, if you prefer, thousands of lawsuits that threaten the health of a major pharmaceutical company). This is not nitpicking. Scientific knowledge is only as good as the scientists who stand behind it. Ducking behind it is usually a bad sign.

If my philosophy of science students read this blog ...

2005-12-19T20:28:00.000-08:00

...they'd have some sense of my view of plagiarism and they'd maybe think twice. But if they don't read the assigned reading (or, you know, the syllabus), they probably aren't going to read this blog.

Especially lame venues for plagiarism I have discovered this term:

Online discussion threads. The idea is to use the discussions to grapple with difficult readings. As such, it is possible to get full marks for the discussion while essentially admitting that the reading made no sense to you at all -- provided you actually make an attempt to spell out what you're confused about, ask others in the discussion questions, etc. Why, oh why, would you cut and paste something smart-sounding about the general topic of the reading from some professor's web page and use that, without any attribution, as your "contribution" to the discussion?
Extra-credit assignments. Do they not realize what a big concession it is for me to offer extra credit in the first place? The course was designed with enough credit, as well as ways to cushion the effects of crises on one's grade. (For example, one can choose in which 10 of the 15 designated reading discussions to participate, the lowest of the short essays is dropped from the grade, etc.) Extra credit is mostly extra grading for me. So, given that I'm already grumpy, an obviously plagiarised response to the extra credit (of which I detected two today) makes my head burst into flames.
Keeping it in the family. Short essays are one of the venues in which you expect to find plagiarism, so mine are somewhat idiosyncratic. Most of the papers-for-sale don't really fit the questions the students need to answer in their short essays, so their best hope for dishonesty is to cheat off another student's essay, and so far, there haven't been a lot of students willing to accommodate the needs of the would-be plagiarist. But this term, a student emailed me late essays. Thing is, when I downloaded them and opened them up, they didn't have his name on them, but his girlfriend's. (She's in the class, too.) None of the other words in the essays had changed since she first handed them in, either.

I get that not all the people registered for my classes think they're all that important. But in that case, why not have a frickin' backbone and admit it? Don't try to put one over on me. Accept the grade you earn thinking your own thoughts and writing them in your own words. Be a grown-up!

That said, the grades are submitted!!

Please stand by. Our regular program will resume momentarily.

Update: Via Inside Higher Ed, this post about the paper mills. Given the typos in their ad, I'm not sure the papers would be worth the money. Also, despite claims that Turnitin.com is powerless to detect high-quality paper mill papers the way Google sniffs out papers pasted together from internet sources, it's worth noting that Turnitin also checks submitted papers against other student papers submitted to Turnitin. Just one other human being using the same paper (say, due to a clerical error at the paper mill) and your ass is grass.

Sloppy work on the part of the hired paper-writer, of course, can also put you on the hook for plagiarism.

What most dishonest students seem not to realize is that we're tipped off to plagiarism by writing styles that sound nothing like our students. Please don't underestimate our intelligence!

All kinds of trouble: more on the Korean stem cell saga

2005-12-14T09:18:00.000-08:00

Gerald Schatten and Seoul National University are in the news again today. (Here's what Joe Palca had to say about it on Morning Edition this morning.) The new development: apparently Schatten has asked Science to withdraw one of the papers Hwang and Schatten (and others) published describing how 11 different human stem cell lines had been created. The reason Schatten is asking that the paper be withdrawn, it is reported, is that he has learned from a credible source that some of the data and figures presented in that paper were fabricated.

Like nearly everything else in this case, there are complications. Schatten's credible source of information is another author of the paper ... which is fine, since another author would, presumably, be in a position to know something about the data actually collected, which photos correspond to which cell lines, etc. But, it's looking like this other author was working with Schatten in Pittsburgh when Schatten learned about the alleged fabrication. Which, possibly, means that the source of information here is golden-handed "P", late of Hwang's lab at Seoul National University. Remember that P not only has the vital "hand skills" to get the nuclear somatic transfer to work, but was also an egg donor for the research.

So, we have a junior researcher who may be the only one who can get a particular part of the experiment to succeed with any kind of regularity, who "donated" vital material that the boss needed for the research (under possibly coercive circumstances), who then came (with her magic hands) to work in the lab of a senior collaborator/competitor of her original boss, and is now handing the new boss information that could make possible a take-down of the old boss.

Do you think there might be any power dynamics at play in this situation?

Meanwhile, the obvious question comes up: If Schatten was a co-author of the paper, why on earth is he raising these concerns about it well after Science published it? How can you put your name on a manuscript if you're not confident that what's in it is accurate? Indeed, Joe Palca reported (in the Morning Edition story linked above) that Schatten "didn't do the research but came at the end and helped the Korean scientists write the paper and present the data." Is this enough involvement that Schatten knows enough to take responsibility for the claims made in the paper -- whether to bask in the glory of the scientific achievement it reports or to share in the blame for its mistakes?

Authorship is a slippery critter. Not only is there no clear standard for authorship that all scientists, or even all members of a particular scientific discipline, recognize, but the standards may vary lab to lab. (Exercise for the graduate student: Find out how your advisor determines who will be an author on a manuscript. Have a friend who works in a different lab get the same information from her advisor. Compare results.) The International Committee of Medical Journal Editors has taken a stab at establishing uniform authorship standards, at least for the journals that agree to use ICMJE standards. (Baby steps.) But these standards contain as clear a definition as you're likely to find of the author's involvement:

Authorship credit should be based on 1) substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published. Authors should meet conditions 1, 2, and 3.
When a large, multi-center group has conducted the work, the group should identify the individuals who accept direct responsibility for the manuscript. These individuals should fully meet the criteria for authorship defined above and editors will ask these individuals to complete journal-specific author and conflict of interest disclosure forms. When submitting a group author manuscript, the corresponding author should clearly indicate the preferred citation and should clearly identify all individual authors as well as the group name. Journals will generally list other members of the group in the acknowledgements. The National Library of Medicine indexes the group name and the names of individuals the group has identified as being directly responsible for the manuscript.
Acquisition of funding, collection of data, or general supervision of the research group, alone, does not justify authorship.
All persons designated as authors should qualify for authorship, and all those who qualify should be listed.
Each author should have participated sufficiently in the work to take public responsibility for appropriate portions of the content.

(Bold emphasis added.)

Now, following the first bullet above, Schatten probably qualified as an author in this framework: he made substantial contributions to drafting and/or critically revising the article for intellectual content (item 2), his help with presenting the data likely counts as a substantial contribution to analysis and interpretation of the data (item 1), and presumably he gave his approval to the final version of the manuscript that Science published. Yet, it is only now that Schatten is discovering that what he signed off on may not have been what it appeared to be.

Possibly this means that scientists need to be a bit more thorough before they give final approval to scientific manuscripts. Ask to see the data and the sources for the figures. Ask to be walked through the data analysis. Check up on your collaborators. Why should this kind of involvement be viewed as intrusive if it's really a collaboration?

Collaborations are central to this whole mess. You have to establish a level of trust with your collaborator. Given that scientific results are supposed to be scrutinized skeptically by other scientists, it strikes me that collaborators ought to bring a level of constructive skepticism to their interactions with each other. Show me how you got the result. Explain to me why this outcome doesn't actually mean X, Y, or Z instead of what we think it means. If a group of scientists can't handle challenges like these from each other, they probably shouldn't be collaborators.

Weird-ass power dynamics make this sort of challenge much more loaded and threatening than it should be among a group of honest scientists. Weird-ass power dynamics are probably a bad thing for a scientific collaboration.

More later? I wouldn't be surprised.

Technorati tags: stem cell scandal, authorship, collaboration

Passing thoughts from a brain addled by grading

2005-12-12T20:44:00.000-08:00

I love teaching, but I hate grading. Worse than the grading itself is the pathetic state in which it leaves my brain -- completely unfit for wrestling new blogable 'gators. But since you, gentle reader, may well have strolled by to avoid your own stack of grading, here are a few thought-like items:

On the matter of women in science and engineering: Check out the new post at Thus Spake Zuska, which includes some intriguing links with resources for women in science and engineering. The post also includes a "recipe for a complete feminist analysis of science and engineering: attention to equity, access, and climate; attention to how scientists and engineers could or should tranform their fields of endeavor; and attention to the considerable joy that thinking and doing in a technical mode evokes." I feel like after I've recovered from grading I may have more to say about this issue, but at the moment nothing I can say about it will be as coherent as what I wrote about the issue back in April.

Science, philosophy, and worries about relativism: At Majikthise, Lindsay Beyerstein comments on how scientists might mistake legitimate philosophical projects around the truth properties of statements for sloppy relativism. Her comments were sparked by a post at 3QuarksDaily calling out Richard Dawkins for making such a move, using a relativist straw horse to defend dismissing philosophy altogether from discussions of science. I think you know how I feel about whether philosophy has any place in discussions of science. (Hint: read the profile!) But, under the assumption that scientists are generally susceptible to reason, these posts do a nice job explaining the anatomy of the straw horse -- perhaps getting rid of the misunderstanding here will mean that the philosopher is not presumed an idiot. (Let the data decide that!)

End of semester cage-match: teaching chemistry vs. teaching philosophy!

Teaching philosophy seldom involves bad smells or risks of explosion. However, neither does it involve cool plastic model kits.
Teaching chemistry (it seems) sometimes involves having undergraduate boys question the solution set you have prepared, because you're a chick.
Teaching philosophy often involves having undergraduates (boys and girls) question your grading of their papers, because "there are no right answers".
In chemistry, one may enjoy the camraderie of grading exams with 11 other people.
To get that camraderie, you're dealing with 300+ students in the class taking the exam. (Even at my university, most philosophy classes are 40 or fewer.)
Reading all those essays scrawled in blue-books can make you cry.
Looking for the sign errors and such in order to assign partial credit on chem exams can destroy your eyes.

It may be a wash ... ask again at the end of the week!

Request-a-post: While I'm climbing up Grading Mountain, I'm happy to entertain requests for blogging topics. So if there's an issue you'd like me to talk about, just let me know.

Zoinks! Another publication ... sort of.

2005-12-08T09:16:00.000-08:00

Googling my name last night (shut up! you do it too!), I was shocked to discover that I have one more scientific publication than I thought I did. As second author, even! But this shock should be understandable given that I never saw the manuscript, not even in early drafts. Nor, for that matter, was I ever informed that it was published.

And yet, there's my name on it. If any problems with the content of the article came to light, I'd be on the hook for them.

What the heck is going on here?

I was, in fact, a student in the lab from which the manuscript was produced. I did, in fact, run a gazillion experiments and collect stacks of data on the experimental systems described in the article. So, my involvement in the research was actual rather than fabricated.

Most of the data was collected during my last summer in this laboratory, which was the summer right before I went off to graduate school. However, it may be worth noting that I worked in the lab as a "student intern" -- I wasn't (to the best of my recollection) funded at all by the PI's research grants, and instead had to scare up student research grants through my college so that I didn't have to sling hash during the hours I wasn't in the lab. This made me a bit of an outsider in the lab group -- I was there primarily to learn a variety of laboratory techniques that broadened my experience, giving me more research options in grad school. I think I left a forwarding address when I left the lab, but it's possible I didn't.

While I was in the lab group, my interactions with the PI were very limited. After my crash course in the experimental techniques I'd be using (from one of the senior lab techs, who was a very good teacher), the PI handed me a project to do, and I passed back the data as I collected it. That was pretty much it. I was never included in any discussion of why we were studying what we were studying, or of what the predicted outcomes might be for the particular experiments I was doing. I was a somewhat glorified piece of laboratory equipment. (Why glorified? I actually had to find some clever ways to deal with non-standard parts of the experiments. And, apparently, my data was very clean.)

In some ways, though it was pretty frustrating to feel like I was only a few steps up from a sonicator, the data may have benefitted from my being out of the conceptual loop. I was just reporting what I saw. I wouldn't have known how to fudge data if you had asked me to, simply because I had no bloody idea what "we" expected to see.

So, yes, I produced a bunch of data from experiments I did with my own two hands. And, some of my brain power was directed at tweaking the experimental design to make it work. And, it would seem, the fact that a discussion of this data resulted in a publication means that the data showed something of value.

But why am I an author (the second author, right after the PI) on this paper? If my contribution to the research was significant enough that I should be listed as an author, shouldn't I also have at least seen a manuscript (if not have participated in drafting it) before I was submitted to the journal? If I was listed as an author as a recognition of my hard work (and something that would potentially help my scientific career -- hey looky, a publication!), why wasn't I, say, told about it by the PI so I could realize the benefits of being published? (This was before Google, so there was no reasonable expectation that I'd quickly come across my achievement in the course of Googling myself.)

By the way, I don't really see how I can add this publication to my CV, given who I am and what I do now.

I take it that there are many more cases of "authorship" of this sort than most people outside the scientific bubble realize. I'm hopeful that there's less of it now than there was back then ... but I'm going to have to start asking around to see.

Stem cell drama continues (and "magic hands" are raised)

2005-12-07T09:13:00.000-08:00

I'm now thinking that whoever takes up the whole World Stem Cell Hub/Hwang and Schatten story is going to get not just a nice book, but also a major motion picture, action figures, and a spin-off reality show out of the deal. Here's the latest, via biotheics.net:

Now that Schatten and Hwang have split, observers here [Korea] say the three researchers should return to Korea to prevent the technology leaking. Schatten severed ties with his Hwang last month, citing ethical breaches in the procurement of human eggs.

"He secured Korean researchers who have key technology in embryonic stem cells. He could have everything for free after weeding out Hwang," said Sun Kyung, head of the Korean Artificial Organ Center.

"Schatten should send back the researchers to Korea without condition," he said.

However, one of three researchers, identified by her initial P, has apparently gone to ground with even her family members having been unable to contact her for over a week.

The woman researcher has exceptional "hand skills" in transplanting the nucleus of somatic cells to human eggs and contributed with Schatten to successfully clone the embryos of a monkey in October 2004. Whenever Hwang talked about his groundbreaking researchers, he mentioned the crucial role of P in his work.

P, you'll recall, is also one of the junior researchers in Hwang's team who was also an egg donor for the project.

Set aside for the time being the ethical questions about letting technology developed in Korea "leak" to a research team in the United States. Instead, let's talk about P's "hand skills".

And here, I feel perfectly comfortable saying that the real-life happenings have taken a novelistic turn, because I use that novel in my Ethics in Science classes. The novel in question is Cantor's Dilemma by Carl Djerassi, and one of the central issues in it is the relationship between the scientists who come up with theories to be tested and the scientists who execute the experimental tests. It's no surprise to anyone who's worked in science that some people are better in the lab than others. (Exercise for the graduate student: See if your P.I. can set up and successfully run your experiment from start to finish.) In the interests of getting good results fast (which is to say, faster than competing labs), many research groups figure out who has the "magic hands" -- who can coax the materials on hand into performing reliably -- and those skilled experimentalists are put on the top-priority projects. Others in the lab are supposed to try to learn from their technique, or at least stay out of their way.

All well and good except for an important feature of scientific practice - experimental results are supposed to be reproducible. This is not just a matter of a scientist with "hand skills" being able to replicate an experiment reliably. It is also important that other scientists, even scientists in different labs, should be able to repeat the experiments and get the same results.

So here's where things get tricky: Your research group wants to get to a discovery first. You deploy your experimentalists with golden hands. The experimentalists achieve success! You report your success (including the experimental details of how you proved your theory or achieved your technological breakthrough). And, if your achievement is sufficiently important, other research groups will try to replicate your work.

What if they can't?

One possibility is that the experimental challenges are significant and the experimentalists in these other research groups just need to practice the techniques for a while before they will be able to reproduce the work.

Another is that your description of the experimental procedure has omitted some crucial bit of information about the experiment. It could be something your golden handed experimentalists don't even realize is an important variable. (It could, on the other hand, be a detail you've been vague about in your report so as to discourage too much competition in the neighborhood of your experimental system.)

Yet another possibility is that you didn't observe what you think you did. The inability of other labs to replicate your results is supposed to clue you in to this possibility. It ought to send your team back to the lab, running the experiments again with a skeptical eye.

Or maybe, your golden handed experimentalists got great results by cheating somehow (either rigging the experiment or just making up results).

The notion that if you report full experimental details, other trained scientists in your field ought to be able (eventually) to run the experiments and get the same results you're reporting is central to science. Why experiments matter is they adjudicate between theories on the basis of empirical data that is (at least in theory) accessible to everyone in the scientific community. It is not enough, even with an experiment in hand, to convince oneself that one's theory is right; you need to be able to persuade a skeptical jury of your peers.

But that jury of peers is also your competition. If they can't replicate what you've done, is it because there's something wrong with it ... or because they're messing with you? Hey, if the research you're reporting is important enough, eventually someone should be able to replicate it. Assuming, of course, you've really included all the relevant details of the experimental protocol, and that they have someone on the team with enough "hand skills" to carry out the experiment.

Here, I'm assuming a bit of research that one wants to report. Things could get a lot more complicated if we assume the science in question is proprietary.

Bringing this all back to the saga of Hwang, Schatten, and P, what should we make of the possible defection of P from Hwang's lab to Schatten's? (I don't want to give away too much of the plot if you haven't read it, but Cantor's Dilemma features a similar defection ...) If P is the only one in the Hwang lab who is skilled enough to make the nuclear somatic transfer happen, then the Hwang lab now has some real problems on the experimental front. Meanwhile, if P has really defected to the Schatten lab, then that lab now has a skilled set of hands it didn't have before.

But if P is the only one who can do the nuclear somatic transfer successfully with human cells, this is problematic. It's not enough that P can make it work over and over. Scientists want to figure out what they're doing precisely enough that other scientists are able to do it, too. So unless P can successfully teach other scientists to make it work, something is fishy here.

I don't know what to say about whether having donated eggs to the project should increase or decrease P's credibility here. It is probably worth noting, however, that egg donation is a fairly difficult process for the donor. And, you don't even get nudie magazine to look at.

It will be interesting to see how the battle between competing interests of individual scientists and the interests of the scientific community as a whole turn out in this case.

Technorati tags: replicable experiments, magic hands, stem cell scandal

Professional Duties, Personal Convictions.

2005-12-05T08:40:00.000-08:00

The stereotype is that the scientist values knowledge above all else. Despite the impression people get that research scientists are emotionally detached when it comes to their lab animals -- viewing them as a means to obtain more information, and thus of merely instrumental value -- I don't know of a scientist who does or has worked with animals in research who hasn't had to do a gut-check.

Some scientists decide that the importance of the knowledge they produce in animal experiments really outweighs the harms to the animals. This is not a decision that it's morally acceptable to harm the animals at will, nor that the animals are "things" that can't suffer. Otherwise, committed scientists wouldn't go to such pains to figure out humane ways to euthanize experimental animals. (Check out, for example, PZ Myers's discussion of how to euthanize fish.) Some end up in situations where a choice is not forced, perhaps because they end up in fields where the research doesn't involve animals. (This was my situation. As an undergraduate, I worked in research labs that did animal research, but I did physical chemistry in graduate school. Not having to use animals to answer the questions we were trying to answer, there was no pressure to work out how we felt about animal research. I must confess, though, some people in my research group messed around with plants ...)

Still other scientists make a principled decision, based on their own convictions, not to participate in animal research (or on research with certain kinds of animals, like primates or vertebrates). Thinking For Food has a very nice post about making such a personal decision about one's scientific activities.

I fully appreciate the benefits of animal research, would not seek to ban it altogether, and understand that it is a necessary evil of modern society. On the other hand, many of the techniques are cruel and a fair percentage of research using animals isn't necessary. I personally have no desire to cause pain to animals (at least, not those with with backbones... I have fewer qualms when it comes to insects and other invertebrates, although there are certain animals within these groups I would be hesitant to experiment on) and having worked in animal research once, I'm well aware of the pain that can be inflicted even under the best animal welfare regulations.

The ethical stance that I've adopted has real world consequences for me as a biologist, and these are consequences I have accepted. Much of molecular biology, including the best paying jobs, involve research with mammals (mice in particular), and by refusing to work on mice, I have willingly and knowingly cut myself out from a large part of the job market.

Among the nice features of this musing is a recognition that research on animals is not a black-and-white issue. The choice is not between getting the information we want at the animal's expense and sparing the animals but not getting the information. Rather, there may be good ways to get some of the information animal research yields without using animals. Moral qualms can thus be a useful trigger to innovative thinking about how to answer various scientific questions. And certainly, you'd think that the information gained from studies without animals would be a useful complement to the results of animal studies.

The other thing I really like about this post is a recognition that choices have consequences (in this case, a narrowing of one's prospects of employement as a molecular biologist), but that accepting these consequences is a legitimate choice for a scientist to make. In other words, it is not the case that Science Central Command hands down the marching orders to all the scientists, who then execute them without question. Rather, the community of science consists of a bunch of agents who decide what kinds of personal sacrifices they're willing to make to have more job mobility, a higher scientific profile, or stacks of good data. The fact that different scientists weigh the factors in these choices differently makes the community as a whole stronger, rather than weaker.

The contrast, of course, would be to accept a job where you knew you had to work with mice and then refusing to do so on principle. This kind of move would get attention, but would also tend to communicate that you think the other people doing work with mice haven't given it any kind of thought. (It's perfectly possible that people who have given the issue a lot of thought, and who feel the pull of arguments against animal research, still decide that the animal research they're doing is justified; two people can deliberate on the same issue and come to different conclusions without one of them being a callous jerk.) To convince other scientists to pursue alternatives to research with mice, it would be more effective to actually do successful research without mice.

And this is where the discussion shifts from the scientist to the pharmacist:

I understand codes of ethics, and I understand how they can place limitations on what a person can do, but I also understand that if you have a limiting code of ethics, you should not pursue a career where you are going to come into ethical conflict with the requirements of your job. If you cannot bring yourself to dispense certain pharmaceuticals (and it need not just be contraception... anti-depressants and vaccines can be just as controversial among certain segments of the population), you shouldn't take a job as a pharmacist.

Doing research is rather more open-ended than filling a prescription, it is true. However, if your personal convictions are at odds with an essential requirement of your profession -- one where there really is no innovative way to get to the same goal by a different path that you can reconcile with your convictions -- it is time to look for a new profession.

(There are complicated issues here about the differences between the scientist and the health care provider, which I started thinking about back in April. Perhaps I'll untangle a bit more here soon.)

Quick take on the stem cell scandal at Seoul National University

2005-12-02T17:59:00.000-08:00

I haven't blogged yet about the saga of Prof. Hwang Woo-suk of Seoul National University. Hwang was doing cutting-edge research with stem cells from cloned embryos and now ... well, suffice it to say allegations of ethical improprieties in the research seem to have put the kibosh on Hwang's activities. He has stepped down as the chair of the World Stem Cell Hub after admitting there were ethical problems with how the (human) eggs used in the cloning effort were obtained. The international collaboration of which Hwang's cloning team was an important part was of great interest to American scientists, especially given federal funding restrictions on stem cell research and laws in some states against such research.

The heart of the ethical lapse? That the egg donors were paid (not in itself necessarily unethical) but that Hwang at first denied that they had been, and that two of the donors were in fact junior members of the research team (and thus might not have been making free donations but coerced ones).

Part of how this all came to light is that one of Hwang's American collaborators, Gerald Schatten of the University of Pittsburgh, pulled out of the collaboration rather publicly because (he said) having become aware of the ethical lapses, he couldn't stay a part of the collaboration and needed to sever all ties to Hwang's group. The scientific goodness of Hwang's results (which were published a couple of years ago) hasn't been called into question. Still, given the contentious nature of any research that involves cloning human embryos (at least in countries like the U.S.), it's easy to see how scientists involved in this sort of research would want to make sure everything is up to the very highest ethical standards.

So many issues here ... luckily, many have already been taken up at the bioethics web log. For instance:

What precisely is the ethical problem with how the eggs were obtained from the research? Read a post explaining why what happened is a big deal, and another post arguing that maybe it wasn't so much of a lapse.
How do different regulations, political climates, and ethical concerns in different countries make this sort of scenario a likely outcome of international collaborations? I was shocked to see this report that very few biotech researchers in Korea are familiar with, of even aware of, the international standards for research with human subjects embodied in the Declaration of Helsinki. Of course, I haven't seen the poll of U.S. biotech researchers for comparison ... Also, see this post that suggests that international collaborations born of American restrictions on research will lead to research where there's no way to tell if the ethics are on the level -- the solution being, of course, for the U.S. to fund (and oversee the ethics of) such research at home so its scientists aren't going offshore.
The role of the whistleblower, Gerald Schatten. My initial reaction to this case was that it is a good thing, when discovering your collaborator has made some very bad ethical choices, to confront him about it and call him out on his sh*t rather than keeping it quiet. But, possibly, the situation is more complicated. There are some reports that there might have been a scuffle over patent rights that precipitated Schatten's withdrawal. And, now it seems, one of the junior researchers who was an egg donor may have gone missing while working with Schatten in Pittsburgh. The conspiracy theory suggestion is that Schatten would be less interested in the contents of her ovaries than in the contents of her mind (including all the vital technical know-how that people outside of Hwang's lab have been trying to work out).

In other words, someone is going to get a really interesting book out of all this. Me, I'll be grading papers and reading job files. So keep following the story, and its many facets, at the bioethics web log.

From the mouths of ... college seniors.

2005-12-02T10:01:00.000-08:00

Last week, I was "in the field" working on my research. The professor whose laboratory I was observing mentioned to the students in the lab that I'm the one who teaches the ethics-for-science-majors class. One of the students, clearly a wiseass (and I can tell because I crack wise myself), asked, "Oh, do you teach how to fabricate data?"

"No," I replied. "In the class we talk about why fabricating data is a Very Bad Thing for scientists to do. Besides, you know that people get fired for doing that kind of thing."

"Sure," said the wiseass. "But what happens when your research just doesn't work? You need to produce results or you'll get fired. You keep your employer happy by generating results ... if things don't work out in the lab, what else are you supposed to do?" Smirking just a little, the wiseass rattled off a list of professions in which lying is de rigueur. Why not science, too?

Now, I recognize that these questions were tongue in cheek, asked by a student experiencing the colossal difficulty of getting original laboratory research to do anything one could descibe as "working". At the same time, students like the wiseass are keen observers of the world of science. Scientists in industry are part of an organization that needs to worry about the bottom line. No results may mean no marketable product, which eventually means the company goes under and the scientists are out of jobs. The folks at Merck know this is a real problem -- patent protection is about to end on one of their big money-makers, and the Vioxx results turned out not to be as promising as was initially thought (what with the life-threatening side effects). So, with fewer marketable inventions than was hoped, Merck looks like it will also be employing fewer scientists.

Obviously, you wouldn't want Merck scientists making stuff up (or even tweaking real results to make them look better) just to please the boss and have something to put on the market ... people could get hurt, and then come the lawsuits and the downsizing. But the close connection between success in the laboratory (and in the clinical trials) and keeping the company afloat has to do something to the psychology of the young researcher. It is a fact of the scientific life that most of the experiments people try do not work, and very few work on the first (or second, or third, ...) try. Kuhn may have been right that the good scientist, encountering difficulties in the lab, blames herself rather than her theory for the problem. (Also, there's a lot of blaming of equipment and reagents, often legitimate.) But when you're the new scientist in the lab, fresh out of school and with less experience than those around you, how can you help but worry that the boss will blame your lack of results on your scientific ineptitude? How long do you figure you have to turn it around before they thank you for your time and send you to human resources for your exit interview?

One of the characteristics of scientific research in the real world is you don't know how it will come out ahead of time. You may have some strong hunches, but hunches mean nothing without real results. And, if you're trying something that no one has ever done before, the very best plan is just a guess. It may fail spectacularly. There is no amount of hard work and technical skill that can guarantee your success. Maybe you'll find a way to solve the problem your research is aimed at solving, but maybe you won't.

Yet you're also trying to solve the problem of staying employed as a scientist in a world where solving problems (or not) can be the deciding factor in who stays employed. You can start to understand where the college senior contemplating a career in science gets stressed out.

Experiments that fail still tell us something, even if it's just information about approaches to solving the problem that don't work. If there were journals devoted to experimental approaches that did not work, I guarantee you they would be read regularly and thoroughly by working scientists. But short of recognizing the scholarly contributions of scientists who identify unsuccessful protocols, is there a way to let scientists just do good science without having to keep their eye on the bottom line?

Technorati tags: scientific integrity, pressure to produce

Blog against racism day

2005-12-01T11:19:00.000-08:00

Today is blog against racism day. Which is why I finally got around to writing the last post -- the issues people bother with, like education, that have an impact on other people may prompt different levels of emotional investment when the people impacted are of a different race or class than the people bothering with the issue.

Anyway, follow the link and see who else is posting about the various facets of racism today.

What's the big deal about high school biology class?

2005-12-01T09:17:00.000-08:00

It is no secret to my regular readers that I get a bit exercised about the science wars playing out in various school boards and court actions. This is no doubt unavoidable, given that I think about science for a living -- when you've got a horse in the race, you end up spending a lot of time at the track.

But lately, some thoughtful folks have been wondering whether people ought to be devoting quite so much energy (and emotional involvement) to the question of what a community decides to include in, or omit from, its high school biology curriculum. This dispatch from Dover, PA by Eyal Press (available online only to Nation subscribers, but maybe your library has it -- if not, email me) is an example of this kind of worry.

Press describes the Dover area as it unfolded for him in a drive-along with former Dover school board member Casey Brown:

We drove out past some cornfields, a sheep farm, a meadow and a couple of barns, along the back roads of York County, a region where between 1970 and 2000, 11 percent of the manufacturing jobs disappeared, and where in the more rural areas one in five children grows up in a low-income family (in the city of York the figure is one in three). Dover isn't dirt poor, but neither is it wealthy. It's the kind of place where people work hard and save what they can. Looking out at the soy, wheat and dairy farms while Brown explained that lots of older people in the area can't afford to keep up with their mortgages and end up walking away from their homes, I was struck by the thought that this was a part of the country where, a century ago, the populist movement might have made inroads by organizing small farmers against the monopolies and trusts. These days, of course, a different sort of populism prevails, infused by religion and defining itself against "outside" forces like the ACLU.

Press also went to see what the students in Dover thought of the controversy:

What do the intended beneficiaries of the Dover school board's actions make of the intelligent design debate? A few days before meeting Casey Brown, I drove out to Dover high school to find out. It was late in the afternoon and a couple of kids were milling about outside, waiting for rides. When I asked them what they thought of the controversy, they looked at me with blank stares that suggested I could not have posed a question of less relevance to their lives. "I think you should leave us alone," one of them said. "Everyone just sleeps through that class anyway," said another. I approached a third kid, who was standing alone. Nobody he knew ever talked about the issue, he told me; it was no big deal.

Press suggests that this is not just a matter of teen ennui. The schools in the area may not be up to the challenge of addressing the real needs of their students:

For the most part, though, kids in Dover seem perplexed that so much attention is being paid to what happens in a single class. It is a sentiment shared by Pat Jennings, an African-American woman who runs the Lighthouse Youth Center, an organization that offers after-school programs, recreational services and parenting and Bible study classes to kids throughout York County. The center, which is privately funded, is located in a brown-brick building in downtown York, next to a church. ... A deeply religious woman who describes her faith as "very important" to her, Jennings nonetheless confessed that she hasn't paid much attention to the evolution controversy, since she's too busy thinking about other problems the children she serves face--drugs, gangs, lack of access to opportunity, racism. "When we are in this building there are no Latinos, blacks, Caucasian children--just children," she explained after giving me a tour of the center. "But when I go out there"--she pointed to the street--"I'm reminded that I'm different."

"There's a lot of kids out there looking for something," Jennings continued. "They have questions that need answering. They're looking for someone to trust." I asked her if she thought schools were providing that thing. She shook her head. "I don't know if it's the schools or the parents or whatever, but something is wrong. The kids I see lack discipline. They lack reading skills." Listening to her, it was hard not to view the dust-up over intelligent design as a tragic illustration of how energy that could be poured into other problems is wasted on symbolic issues of comparatively minor significance.

Why those symbolic issues have assumed such importance in America has a lot to do with the fact that, in places like Dover, the only institutions around that seem willing to address the concerns of many people are fundamentalist churches.

I take it that Press is not primarily interested in taking scientists to task. Rather, his point seems to be that folks in Dover and places like it are much less concerned about "direction" of curriculum by fundamentalist churches because those churches are perceived as taking care of social needs that no one else -- including the government -- seems willing or able to address in these communities. It doesn't seem altogether irrational to bend a little to the folks keeping things together, especially if the bending involves changing the curriculum that the high school students are going to sleep through anyway, does it?

This is a variant of the ongoing debate I have at my university about what is supposed to be going on here:

Dr. F-R: A college education should help you understand different kinds of knowledge and reasoning. My class should help you understand what's distinctive about scientific knowledge.

Jaded Student: Dude, I really just want to sit in the chair and do the minimum I need to do to get the three units of upper division science general education credit. Don't bug me.

Dr. F-R: You're a college student! Learning this is good for you!

Jaded Student: I'm only in college so I can get a job that pays a decent wage. If I could do that any other way, I wouldn't be here.

Dr. F-R: How will you navigate the modern world without some understanding of science?

Jaded Student: Unless understanding science gets me a better salary it ain't gonna happen. Learning for its own sake is for suckers.

And here's where I want to say that Eyal Press is right that there are very bad things that are much larger than the details of the biology curriculum happening in communities like Dover, but that the fight over quality public education is central rather than merely symbolic. Education is not a dispensible luxury. Rather, it is an essential tool for people in making reasonable choices about their own lives. Education isn't just about teaching specific skills for the workforce; it also lays a foundation with which to learn new skills to keep up with a changing economy (or, dare I say it, with one's changing interests). Even more, education is supposed to open up a world quite apart from the world of work. The world may need ditch diggers (or repair technicians for the ditch-digging robots), but it would be a much better world if the ditch diggers (and repair technicians) not only earned a decent wage but also had enough left over to buy a few books and to think about things they wanted to think about. (Yes, I'm going on my "everyone deserves a life of the mind" rant. It happens.)

Making a better world may require choosing one's battles. Some would suggest that the battle over science education is a high-investment, low-payoff battle. But my own sense is that the minute we decide a certain population of students don't really need good science education, we've put up the white flag. Do we help students who are in difficult socio-economic circumstances by reducing their future prospects to succeed in further science classes or pursue a career in science? Do we help these students when we throw them out into the world as voters and consumers without a clear understanding of how scientific knowledge is produced and of how it is different from other kinds of knowledge? Might it not reinforce the feeling that the larger society really doesn't actually care much about you or your future if you find out that people with a voice didn't even whimper as you were subjected to an "education" these people wouldn't have allowed their own kids to suffer through?

One of the guiding ideals of science is that it is a project in which anyone can engage -- provided they have the necessary training. Scientists try to work out accounts of what's going on in the world that are tested against and built upon observation that human beings can make regardless of their home country, their socio-economic status, their race, their gender, their age. The scientific ideal of universality ought to make science a realm of work that is open to anyone willing to put in the work to become scientist. A career in science could be a real avenue for class mobility.

Unless, of course, we decide that public school students in less affluent communities (or more rural communities, or red states, or whatever) aren't really entitled to the best science education we can give them. If keeping them fed and out of gangs and passing the standardized tests in reading and writing is the extent of our obligation to these students, maybe a sound science education is a luxury. But if this is the case, we probably ought to cut out the whole "American dream" story and admit to ourselves that this place is not a meritocracy. Those who have the luxury of a quality education have an advantage over those who don't, and by golly they should own up to that. Especially when budgets are being hammered out, or when elections are coming up.

The kids in Dover, and Iowa, and Kansas, are part of your future and mine. Even if, as 15 year olds, they don't fully appreciate the stand being taken on their behalf, I'm not willing to back down from taking it, just the same way I'm not willing to let jaded students out of my classes without some learning taking place. Valuing other members of our society means valuing their future options to set their own course and to find meaning in their own lives. Good making good science education is not sufficient here, but my gut says it may be necessary.

Technorati tags: Dover, education, science wars

Just where is this handbasket headed?

2005-11-28T12:37:00.000-08:00

It's time for our regularly scheduled soul searching about what is happening to science (and its future) in the U.S. I was already going to blog about this, given recent news that Stanford has lost two scientists it was trying to recruit to Stanford's Institute for Cancer and Stem Cell Biology and Medicine to Singapore's Institute of Molecular and Cell Biology (which is viewed as a more favorable environment for this cutting edge research than even the deep-blue San Francisco Bay Area). But, not surprisingly, PZ Myers brought up similar worries in response to an article in Wired.

The big worries are that the U.S. is not supporting science in important ways -- U.S. universities aren't turning out enough science Ph.D.s (and science education at K-12 and undergraduate levels is not very good), there's not enough money being put into federally funded scientific research (or industry R&D), ideologically motivated restrictions on research are making it hard to do the research the really good scientists want to do, and scientists generally aren't getting the respect they deserve in the public square. The fear is that all these factors will coalesce in such a way that scientists start fleeing, first particular states in the U.S. (as suggested in this look at how different states fund or prohibit stem cell research), then the U.S. itself. At which point, presumably, everyone left in the U.S. will either be a frycook, a member of the punditry, or a participant in a reality show.

Are we totally screwed? Hard to tell from here, but it seems like this is a good moment for reflection about the precise nightmare scenario we're afraid of.

It is hard to deny that individual scientists will be influenced in their choices of where to go by the circumstances in the different locations under consideration. If you want to do research with stem cells, you probably don't want to take a university position in a state where stem cell research is illegal. If besides having a scientific career you also want to raise kids, you might want to do science in a place where the schools provide a good education (including in matters scientific). If you're a woman doing science, you might want your job to be in a country where it's legal for you to drive yourself to work. At the same time, scientists' choices are not always completely determined by their scientific agendas. You might forego a position at a ridiculously well-funded research institution, where you could get any state-of-the-art piece of equipment you want, in favor of a position at a less-well-funded institution where you can also contribute to excellent undergraduate education or live in a community you prefer; the trade-off would be finding creative ways to pursuit your research with fewer resources. But scientists make this kind of choice all the time. (My suspicion is that the scientists with tight resources tend to be more clever about experimental design because they have to be.) Moreover, there are more than a few scientists who tend to work, whether vocally or quietly, to change local circumstances (lobbying for more funding or more sensible regulations, supporting K-12 education, talking to their neighbors). These people manipulate experimental systems for a living -- are they going to accept their environment as immutable? Not so much (although some prefer moving to a different environment over tinkering with the environment they have). So, it would surprise me a lot if the sun rose one day on a rapture-like disappearance of all the scientists from the U.S. or even from Kansas.

At the same time, it is an important feature of science that you need a scientific community in which to do it. To keep research going, you need a critical mass of people in the lab -- not just to run the experiments, but to bounce ideas off each other and challenge each other's interpretations of the results. Because scientists have finite lifespans, this means you need to keep putting new scientists into the system even to maintain equilibrium. So, cutting back funding of science (including the training of new scientists in graduate school) could set up a situation where whole lines of research die out simply because there aren't enough scientists to maintain them. It's not clear, either, that "thinning the herd" of research lines this way would make for a stronger body of scientific research in the long run, given that it's often unclear in advance which lines of research will be the most important from the point of view of basic knowledge or practical implications. And, there's no guarantee that defunct research lines could be successfully resurrected later when more funding becomes available.

None of this would mean that there wouldn't still be people in the U.S. interested in pursuing science. However, to do so in the context of a scientific community, they would need to: (1) go somewhere that has a suitable scientific community already, (2) find a way to participate remotely with such a community, or (3) figure out how to constitute such a community ex nihilo where they are. The resources for exercising the second option have been steadily improving, but there are still many kinds of research one probably shouldn't do in jammies in one's garage. The third option is pretty hard without many like-minded friends and buckets of money. Assuming one is good at picking up languages, the first option may seem like the path of least resistance. But if the scientists all go to Singapore, that may have an impact not only on the science education available in the U.S., but also on who earns the royalties on patented inventions ... like pharamceuticals. So, as has been said before, this would not be a development without consequences for Joe Q. Public.

At the same time, futzing with one little piece of the current situation without addressing the others may lead to bad consequences, too. For example, increasing the number of science Ph.D.s produced in the U.S. only improves the scientific climate in the U.S. if there are ways for these Ph.D.s to stay engaged in science in the U.S. For starters, making sure there are enough jobs for all these Ph.D.s (non-exploitative jobs, not an endless cycle of post-docs until you achieve nirvana) would be a good thing. Large numbers of well-trained but under-employed and under-appreciated scientists enhances the feeling that the U.S. is not a good place to do science.

It does seem to me that the current situation presents an opportunity to consider cultivating a truly international community of science. Concerns have been voiced about how many foreign students come to the U.S., get science Ph.D.s, and then go home with them, thus shifting resources (in the form of education) from the U.S. to other countries. Setting aside the fact that foreign students pay lots of money to study in Ph.D. programs (whereas U.S. students are generally fully supported), as Ph.D.s in their home countries these scientists are well-situated to collaborate with U.S. scientists (and with U.S.-trained scientists in other countries). At a moment when it may be getting harder to do science locally, it seems like it should be getting easier to do it globally. Moreover, collaborations between scientists in different parts of the world who have different cultural and religious backgrounds but who share a commitment to the project of science just might give the rest of us a model for figuring out how to get along in a diverse community. It might even help us get in touch with our common humanity.

As for the point PZ was making in the Pharyngula discussion: Are we, as a nation, spending money on other things (like wars, or corporate tax breaks, or reality shows) that would be better spent supporting science and the benefits that flow from science to society as a whole? I'm no economist, but I think the answer is probably yes.

Technorati tags: politics, US science, international science

Teaching undergraduate science classes at research universities

2005-11-22T20:07:00.000-08:00

Inside Higher Ed reports a story that shouldn't surprise anyone: TA’s as the Key to Science Teaching. You see, according to Elaine Seymour, recently retired director of ethnography and evaluation research at the University of Colorado at Boulder, a lot of college students who start out as science majors leave the sciences, largely due to poor teaching in their science classes. And, one of the sources of poor teaching in science classes is graduate teaching assistants with little to no guidance about how to teach.

... she [Seymour] said the sad fact is that most science TA’s don’t receive much help in doing their jobs well. Many get absolutely no preparation and those who do tend to receive a brief session with no ongoing mechanism to learn about their teaching.

“Training,” Seymour says, is “a very unfortunate word,” and she prefers to talk about “educational professional development” for TA’s. One of the major problems, she found, is that when teaching assistants want to become better teachers, they often feel the need to keep quiet about it. “They consistently told us that if they want to teach and they are interested in that, they keep that to themselves. They are afraid that they will be taken as less serious students” by the professors supervising their work, Seymour says.

Word!

Is is rare, especially in the sciences, that the faculty supervising graduate students pay any attention to the need for "professional development" in the area of teaching. Nearly all the focus is put on learning to be a competent researcher. A charitable interpretation of this is that the faculty regard research as a relatively unknown activity to new graduate students, one that can only be learned by full immersion, while they regard good teaching as something that will come naturally to any intelligent graduate student who gives it a try.

A less charitable interpretation is that the faculty don't actually care about undergraduate teaching. Does undergraduate teaching bring in multimillion dollar grants? Not the way a kick-ass research program does. Does undergraduate teaching lead to high prestige publications or Nobel Prizes? Not so much. Will well-to-do parents keep ponying up $40K and more a year so junior gets the name of the Big Prestigious University on the diploma, setting him up to earn the big bucks, even if junior never gets within 20 feet of an actual professor in a science class? It would seem so.

Of course, even if faculty at such universities had some kind of commitment to undergraduate education (perhaps of the majors in their department, if not the swarms of pre-meds), it wouldn't necessarily mean that these faculty would also be committed to helping their graduate students learn how to teach. Most of these grad students are TAs in the pre-med courses, anyway. And indeed, large numbers of pre-meds who need science courses (and from whom faculty most want to distance themselves) are the reason that the large research universities take in as many science graduate students as they do. Given the reality that there are more science Ph.D.s being produced than there are Ph.D.-level jobs for them, it's pretty clear that a large number of graduate students are used primarily to assume the necessary but unpleasant teaching tasks with which the faculty would otherwise be bothered and to generate scads of data for their advisors' research programs.

Any time a graduate student spends developing pedagogy is time taken away from running experiments. For the graduate students who won't find jobs with their Ph.D.s (because there are too damn many of them), learning to be a good teacher is wasted effort, and it doesn't maximize the department's return on its investment (by moving the pre-meds through and getting more data at the bench). And, for those lucky graduate students who will find jobs with their Ph.D.s -- well of course, they'll want jobs at big research universities like the ones they trained at, and evryone knows that what really matters there is research, not teaching.

One wonders what will happen if the research money dries up, and if tuition payers start demanding quality teaching for their tuition dollar. It might be time to think about alternate strategies for training graduate students.

Technorati tags: science TA, graduate training, undergraduate science classes

Ethics and alternative medicine

2005-11-21T09:58:00.000-08:00

As requested by a fellow blogger, I'm weighing in with some thoughts about "complementary and alternative medical therapies" and the ethical implications thereof. There are plenty of other good posts to read on this general subject, most recently at Pharyngula, retired doc's thoughts, Notes from Dr. RW, and In the Pipeline.

Where I start on this is that doctors are in the odd position of trying to work from scientific knowledge and at the same time trying to provide care to human beings. A patient is not a petri dish. Not only is a human being wildly more complex than most of the experimental systems scientists try to tackle in the lab (where there are, after all, lots of controls in place), but the patient generally has strong opinions about his or her own experience. Sure, these opinions are subjective, but for the person who has them, they're mighty real. (Even if the excruciating pain I feel is "all in my head", the fact that it's in my head makes it real enough to me!)

Scientists are in the business of getting data on the observable features of the systems they're studying (and also working out clever ways to observe more of the features of those systems). They use that data to work out the structure of the system, the cause-and-effect connection, ways to predict what's coming in the system given certain conditions, and maybe even ways to alter conditions to bring about different outcomes. This can be challenging even in vitro. It is a much more impressive achievement in vivo. But it's worth noting that most in vivo research projects are vastly simplified compared to what you might think of as "the real world" of organisms (e.g., studying hundreds of mice that are very similar genetically, are housed in identical conditions, fed the same Mouse Chow at the same times every day, etc.).

In other words, to the extent that physicians are applying the fruits of scientific research to the treatment of their patients, they are often trying to extend results obtained in very controlled conditions to autonomous humans "in the wild". Given that most patients probably wouldn't want to adopt the correspondingly controlled environment, that's fine. But, it means that our best prediction of the likely outcome of intervention X is less certain than it would be under the conditions of the experiments that produced the best current knowledge about intervention X. Complicating things, even under experimental conditions, there is usually a range of outcomes observed in response to a given intervention. So, even while there may be very good evidence to support the prediction that intervention X will produce an outcome somewhere in a particular range, there may not be grounds for predicting that a particular individual will have an outcome in one part of the range rather than another.

"What does all this have to do with alternative medicine?" I hear you ask.

One of the big worries about complementary and alternative therapies is that they just don't have the science to back their efficacy. Yet, the public seems to be clamoring for them. In working out just what they should do about this state of affairs, physicians probably ought to examine why the alternatives are so attractive to so many patients. At least part of this, I think, can be explained in terms of what a tough-minded scientific approach to medicine brings with it, and how this might complicate the delivery of care (not just interventions) to the patient.

A patient shows up at the doctor's office looking for care that addresses a medical issue (an injury, a disease, prevention of some future ill). The doctor gets information about the patient's condition (via examination, lab work, and talking with the patient). Ideally, a medical intervention will effectively address the issue (repair the injury, cure disease or at least manage its symptoms, change the state of affairs in such a way as to lower the likelihood of the future ill). If the intervention can address the issue without making the patient feel like crap, that's a plus. The set of effective interventions the physician has to offer are generally those that have been proven in clinical trials. Ideally, you want trials in which administering intervention X is significantly more effective than administering an appropriately similar placebo (easier when we're talking about pills than non-drug interventions) in a double-blind set up, where neither the clinician nor the patient knows who's getting the intervention and who's getting the placebo. Such trials generate the "evidence" in evidence based medicine.

At present, very few of the alternative therapies people seek out (homeopathy, herbal medicines, chiropractic, accupuncture, etc.) have been through the double-blind clinical trial. By the evidence-based physician's lights, there is just no good reason to think they will be effective.

But for the patient who is given the best interventions evidence-based medicine has to offer, with no resolution to the issue that made him seek care in the first place, an alternative therapy may feel like something to try that might help the problem. In the same way that a 95% success rate of intervention X may be of no comfort at all if you're in the 5% not helped by that intervention, you may not care a fig if the alternative you try hasn't been demonstrated to be broadly effective -- you only care if it helps you. And even if your science-minded doctor tells you that it probably won't help you, if you feel better, you feel better. Even if it's just the placebo effect, it's still an effect, and feeling better is at least part of what you're looking for.

This does not mean, however, that physicians (and the medical schools that train physicians) ought to jump headlong into the uncritical acceptance of the whole plethora of alternative therapies. While patients need care from their physicians, they also expect that there is scientific knowledge guiding that care. Instead, it seems perfectly reasonable for the medical profession to subject the alternatives to the same sorts of clinical trials undergone by mainstream medical therapies. (Many such studies have already been done.) Such testing may turn up interventions that are as effective as mainstream interventions (remember that willow bark is an ancestor of aspirin). More importantly, it may turn up interventions that are harmful, about which patients should be informed. What to do with the interventions that prove themselves neither to be demonstrably effective nor harmful? The physician ought to be clear that the best scientfic evidence doesn't give any reason to believe that the intervention will take care of the medical issue -- but that there is little reason to expect it to be harmful, either. If the patient chose to explore the alternative, probably better that this happen with the physician's knowledge than on the sly. (It's better for the doctor to know how many parameters are being tweaked at a time, especially if the patient's condition changes.)

There is lots much that could be said here -- about who should conduct the clinical trials of alternative therapies (the medical establishment or the people selling the alternative therapies), about the psychological effects of paying a lot for a therapy on perceived improvement, on that feeling of alienation one gets from getting exactly three minutes of face-time from the treating physician and how that plays into perceptions of improvement), about the suspicions one might form that physicians are biased toward the pharmaceutical companies by something beyond scientific evidence (as I type this, my gaze falls on "Floxie", a bright blue plush drop of Fluoxetine -- regifted by a physician relative, who got it from a drug company rep). But on the question of how, ethically, a physician ought to handle the issue of alternative medical interventions, I would make the following suggestions:

Be clear about what is known from clinical trials and what it means to the individual patient. This goes for both mainstream and alternative therapies. Being clear about what is known (desired effects and side-effects) and what is uncertain (including just how a particular patient will respond to a particular intervention) is crucial. Saying, "This will work for you" and being wrong about it undermines your credibility.
Call the patient's attention to interventions clearly demonstrated to be dangerous and/or ineffective. This is worth doing even if the patient has shown no visible interest in these interventions; patients may choose not to discuss these with their physician (in the same way they may not be completely forthcoming about how much booze they drink, or how few vegetables they eat). To the extent that risks are known, especially for interventions readily accessable in the marketplace, patients need to know, too.
Let patients know which interventions are still untested, and what their untested status means to the patient. While "We don't know what X does to people in your condition" leaves open the possibility of good outcomes, it also leaves open the possibility of bad outcomes. Patients may have different risk-taking strategies than physicians when faced with uncertainties. This doesn't necessarily mean patients are muddle-headed; they're just concerned with a different payoff (getting better vs. having firm support from evidence). The physician ought not to get paternalistic here, but instead to give the patient a clear enough explanation of what is known and what is not known that the patient can make well-informed use of his or her autonomy.

There are quacks aplenty looking to make a buck on snake oil. But, the physician shouldn't let anger at the quacks spill over into hostility towards or impatience with the patients who are trying to figure out their options. Being upfront about what is known, and what is uncertain, about alternative and mainstream therapies is a good way for physicians to set themselves apart from the quacks.

Technorati tags: complementary and alternative therapies, medical ethics, evidence based medicine

Tangled Bank #41

2005-11-15T20:55:00.000-08:00

Tangled Bank #41. Hosted by Flags and Lollipops. It's up already! People can read their science stories early!!

Yes, I'm a little excited.

Scare-mongering, or avoiding miscommunication?

2005-11-15T14:35:00.000-08:00

Over at Crooked Timber, there's an interesting post about "quote doctoring" in the context of the battle over the reality of global climate change. In an interesting twist, John Quiggin's post looks at the doctoring of a quote about communicating scientific information to the public to miscommunicate what the original speaker was trying to communicate.

The quote in question, from Stanford University climatologist Stephen Schneider, is an interesting one:

On the one hand, as scientists we are ethically bound to the scientific method, in effect promising to tell the truth, the whole truth, and nothing but – which means that we must include all the doubts, the caveats, the ifs, ands, and buts. On the other hand, we are not just scientists but human beings as well. And like most people we’d like to see the world a better place, which in this context translates into our working to reduce the risk of potentially disastrous climatic change. To do that we need to get some broadbased support, to capture the public’s imagination. That, of course, entails getting loads of media coverage. So we have to offer up scary scenarios, make simplified, dramatic statements, and make little mention of any doubts we might have. This ‘double ethical bind’ we frequently find ourselves in cannot be solved by any formula. Each of us has to decide what the right balance is between being effective and being honest. I hope that means being both.

[fn1. Schell, J. (1989). ‘Our fragile earth’, in “Discover” 10(10):44-50, October.]

The quote-doctoring discussed at CT seemed intended to make it look like Schneider was saying that it's OK to make scary, inflated proclamations to the public (while in your lab coat, naturally) if it serves the greater good of protecting the environment. But, interestingly, a number of the commenters to the post asked: isn't that what the full Schneider quote is saying?

I don't think Schneider is advocating punking the public in defense of Mother Earth. Rather, I think he's trying to capture some of the difficulty of communicating science to a lay audience. Let's go in for the close reading:

"On the one hand, as scientists we are ethically bound to the scientific method, in effect promising to tell the truth, the whole truth, and nothing but – which means that we must include all the doubts, the caveats, the ifs, ands, and buts."
Scientists know that their conclusions are tentative. They are drawing the best conclusions they can with limited data. They know that what they know rests on an intricate inferential edifice, and that new data or wonky instruments or a change in reasonable background assumptions could result in changes -- sometimes big ones -- in what it's reasonable to conclude. When talking to other scientists, there is no problem with including doubts, caveats, ifs, ands, buts. Other scientists understand that these do not mean the conclusions are worthless, nor do they mean that the scientists reporting the conclusions don't know anything. Indeed, the scientists may know quite a lot. But, they are sensitive to the relations between their data, their assumptions, and their conclusions.

To put it more simply: the attitude in science is generally that you ought not to conclude anything that isn't fully supported by the data, and that you never know what the next datum will be. This is why good scientists actually bother to collect data rather than just making it all up. Scientist-to-scientist the question is not so much what do you think is going on here? as how does the data support this conclusion. Rather than come to a conclusion that turns out to be wrong, a scientist anticipates ways it might go wrong and data that might undermine it.

"On the other hand, we are not just scientists but human beings as well. And like most people we’d like to see the world a better place, which in this context translates into our working to reduce the risk of potentially disastrous climatic change."
In other words, scientists don't just pursue knowledge for knowledge's sake. Having learned something about the world, they sometime feel that they should use that knowledge to avoid Very Bad Outcomes.

"To do that we need to get some broadbased support, to capture the public’s imagination. That, of course, entails getting loads of media coverage."
Lay people don't generally read peer-reviewed scientific journals, attend group meetings in the science labs at nearby universities, or talk to their scientist neighbors about issues much beyond lawn care, mass transit, and the occasional school board election. Also, people hardly read the newspaper, maybe watch TV news while they're doing other things, and might miss a science-related news item on the car radio because they're screaming at the guy who just cut them off. If lay people don't seek out scientific information, then scientists who want to share the information (and enlist the help) necessary to avoid Very Bad Outcomes need to seek out the audience. And, unless you want them calling you at home (during dinner time!), this means they need to get help from the mass-media.

"So we have to offer up scary scenarios, make simplified, dramatic statements, and make little mention of any doubts we might have."
Is the mass-media going to give you enough air time (or column-inches above the fold) to explain your conclusions, plus the evidence that supports them, plus the uncertainties and how big these are relative to the strength of your conclusion, in the loving detail a fellow scientist would want to see? Here's a hint: will it help sell advertising? If not, you have to make your point clearly and efficiently. You have to make it in such a way that an audience accustomed to tuning out science will pay attention. You have to make it clear how the information you are presenting matters, in this case as a means to avoid the Very Bad Outcomes. Oh, and you have 60 seconds. Go!

Having tuned out science so effectively, members of the public generally have no understanding of error-bars, which means they can't tell big ones from little ones. Rather than recognizing that some doubt is a concommitant of the scientific method, they think that any expression of doubt means that you're only guessing. (Of course, if the point of science were merely to produce wild guesses, there are far cheaper ways to achieve that goal.) But how much time do you have to give your lay audience the full review of the basics of the scientific method, including how certain our inferences from empirical data can be? Will they check out of this review before you even get to the (suitably qualified) conclusions to which you want them to pay attention so we have time to avert the Very Bad Outcomes?

Maybe we start out with the "big picture" of what we've observed, what we think it means, and why we think it matters. Then, given a reasonable amount of time, we can answer questions. Which, I think, is what Schneider is saying here: "This ‘double ethical bind’ we frequently find ourselves in cannot be solved by any formula. Each of us has to decide what the right balance is between being effective and being honest. I hope that means being both."

Given the lay person's state of understanding about scientific methodology, trying to communicate scientific conclusions with all the scientific caveats may itself be a kind of mis-communication, since the lay person may end up with mistaken impressions about the goodness of scientific knowledge and the actually methodology (and its limitations) that goes into producing it. It doesn't help, of course, that scientist from industry and think tanks go to pains to make relatively small uncertainties look big -- big enough to drive a Hummer through.

It would help the communication immensely if the public actually understood enough about how science works to be able to comprehend the truth, the whole truth, and nothing but the truth from scientists who would like their knowledge to benefit even those who aren't scientists. But, it would seem, the public doesn't, in general, have that level of understanding. Until we figure out how to do something about that, it really does the least violence to the scientists' conclusions, and to the public, if the scientists translate their findings -- and their degree of confidence in them -- to laymen's terms.

Technorati tags: scientific communication, public interest

Talking the talk vs. walking the walk (plagiarism update)

2005-11-14T08:50:00.000-08:00

Earlier, I wrote about plagiarism in the engineering school at Ohio University. A masters graduate, Thomas Matrka, raised concerns about widespread instances of plagiarism in masters theses, while the administration ... well, didn't seem to view it as such a big problem as Mr. Matrka did.

Well, Mr. Matrka commented on my earlier post. I'm reproducing his comment, in its entirety, here:

Dear Dr. Free-Ride,

You pose five simple questions regarding plagiarism. Regardless of the context, the answer to all five questions is an indisputable, YES! An explanation follows.

1) “Do the practices Matrka identified constitute plagiarism?” Everyone knows that copying text word for word from a textbook without quotations is plagiarism. Some cases can be inadvertent, but the many of the cases I have discovered at Ohio University are extended and obviously intentional. There is no honest explanation for verbatim copies that include the same errors as the original, or misleading citations for works other than the actual work from which the text is stolen.

2) “Do the faculty have a duty to deal with past acts of plagiarism… and if so, how?” Plagiarism is a violation of university policy. Failure to distinguish theses containing plagiarism from those that are done honestly perpetuates the deception and ambiguity. How do administrators explain to the student who received a failing grade for plagiarizing a history paper that an engineering student who plagiarized received a master’s degree? How does a researcher properly cite work that is plagiarized? Some universities revoke degrees when plagiarism is discovered; ignoring past cases of plagiarism is inexcusable.

3) “Does pervasive plagiarism in a graduate program undermine the value of a degree granted by that program?” Most alumni, employers, and students expect a university to uphold its own policies. They do not ask, “do you allow plagiarism?” Cataloging theses and dissertations known to contain plagiarism unfairly creates suspicion around the work of all students and faculty.

4) “Do scientists and engineers have a common understanding of what counts as plagiarism?” Graduate students know they are plagiarizing when they open a book, copy it, and submit it for a thesis or dissertation.

5) "Do scientists and engineers agree that plagiarism is a species of scientific misconduct?” I am certain that any scientist or engineer would not appreciate their work being stolen and passed off as original work by another.

Most people agree that acts of plagiarism are very serious and intolerable. The problem at Ohio University’s Russ College of Engineering and Technology is the lengthy history of faculty approved theses and dissertations containing plagiarism. Acting on the evidence will undoubtedly raise questions of fraud and corruption. I will gladly share details with all that are interested.

First, let me thank Mr. Matrka for commenting on the post. It's nice to have the view of someone "on the ground" in a scenario to which I respond.

And (you knew this was coming), let me respond to these comments.

My questions in the original posts were not meant to suggest that engineering students (or engineering profs) were playing hooky when the other kids in school got The Talk about plagiarism. I'm sure almost all could provide at least a rough and ready definition of plagiarism if pressed to do so. But, as someone who spends a lot of time thinking about ethics, I'm drawn to questions about the differences between the values people say they are committed to and the values that seem to guide their behavior. In other words, if you talk the talk, but don't walk the walk (meaning, presumably, that you're walking a different walk), are your values the ones you're talking or the ones you're walking? And, does how we talk (and how we walk) change when we move from group to group (say, from an engineering school to the larger university community, or from a graduate classroom to a professional meeting of engineers)?

Mr. Matrka writes: "Everyone knows that copying text word for word from a textbook without quotations is plagiarism." This seems true. But it might be less clear what to make of use-without-citation of ideas, equations, or even common graphs from a widely used textbook. Often scientific training includes absorbing a more or less canonical body of knowledge that will serve as a common framework for discussions. It's conceivable that, after absorbing this knowledge, one might forget exactly which piece of knowledge came from which textbook (especially if that piece of knowledge is included in multiple textbooks, not to mention lectures). And, in some sense, the "textbook knowledge" is assumed to be so much the common grounding of a field that no one cites his source for it -- it's something one could find "in any textbook" on the subject.

I wonder whether the standard view that you don't have to cite ideas from the textbooks (because they appear pretty uniformly in all the textbooks) has made students and professors lazy about the citation of word for word quotations from textbooks. If you were putting it in your own words, you probably wouldn't cite it. Why put it in the textbook authors' words? Maybe because it's so clear in the textbook. Maybe because that way you know you haven't misstated the fact. I can only imagine that the faculty reading theses with uncited word-for-word quotations from textbooks either (1) don't realize that these are word-for-word quotations, since the faculty haven't spent quality time with the textbook in so long, or (2) do realize that they're word-for-word quotations but view this as a minor instance of plagiarism since it's not the kind of thing that would require citation if the thesis writer had expressed it in his own words.

Myself, I agree with Mr. Matrka that plagiarism is plagiarism. But, I understand the thinking behind saying this sort of plagiarism, while undoubtedly a sign of intellectual slothfulness, is not as evil as stealing someone else's description of a new experiment or someone else's bold new interpretation of experimental results.

Mr. Matrka writes: "Plagiarism is a violation of university policy. ... How do administrators explain to the student who received a failing grade for plagiarizing a history paper that an engineering student who plagiarized received a master’s degree?"

The talk: "As part of the university, we're committed to upholding university policy."
The walk: "Well, maybe we're not as committed to upholding it as are other departments."

It does seem here like university policies lose their force if they're only enforced some of the time, or only by some of the departments. Working at a university myself, I know that there are some policies I would defend to the death and others that I think are really wrong-headed ... but, there are plenty of ways for faculty to lobby to change the policies with which they don't agree. Just ignoring those policies, rather than at least voicing your objections to them, doesn't seem like much of a principled stand.

Mr. Matrka writes: "Most alumni, employers, and students expect a university to uphold its own policies. ... Cataloging theses and dissertations known to contain plagiarism unfairly creates suspicion around the work of all students and faculty."

In other words, if an institution (like a department, or a university) is known not to uphold its own policies, then folks dealing with that institution have no good reason for thinking that institution feels any commitment to the values contained in that policy. If you don't do anything about plagiarism, we don't actually get much out of your saying that p[lagiarism is bad; if you really thought it was bad, wouldn't you do something about it? And, if you have a policy against plagiarism which it is clear you have not enforced, is there good reason for us to believe that you have conscientiously enforced your other policies?

This might be a place where, if the institution in question really does have certain core values embedded in its policies, it ought to just drop the policies that aren't reflective of the values of the institution. It's a choice between being clear about what you stand for and looking like maybe you don't stand for anything at all (save the intake of tuition dollars). Yes, there may be difficulties if, say, the engineering department rejects certain values the rest of the university embraces, but it would probably be better to be clear about this than to have it revealed in a scandal.

Mr. Matrka writes: "I am certain that any scientist or engineer would not appreciate their work being stolen and passed off as original work by another." Again, I'm in total agreement. But, can we say "No scientist or engineer would appreciate X" implies "X is a species of scientific misconduct"? (Let X = having sugar put in your gas tank by a competitor.)

The official government definition(s) of scientific misconduct include plagiarism alongside fabrication and falsification. Most scientists seem to agree that having someone steal your words and/or ideas and present them as her own is a Very Bad Thing. But if the community of science as a whole (or even a subdiscipline within this community -- say, the community of engineers) agreed that this was not a very bad thing in certain circumstances -- say, they agreed that the contents of textbooks were community property for everyone to make use of as desired -- then it might be reasonable to formulate a more precise definition of plagiarism as recognized by the scientific community.

Please note that I'm not claiming that the scientific community or any of its disciplinary sub-communities actually hold this view about texbooks. But if they did, it seems to me, it would be better to be clear about it than to quietly be guided by a different set of values than those recognized in official policies.

If your values are good enough to walk, aren't they good enough to talk? And, if you wouldn't want to be caught talking them, why the heck would you walk them?

Technorati tags: plagiarism, scientific misconduct, community values

"Intelligent Design": not even interesting as philosophy

2005-11-09T20:22:00.000-08:00

I'm probably too tired to blog, but the I've been goaded into it by, of all things, commentary on a local school board election. In Minnesota.

At Pharyngula, here's the commentary on the returns:

All of the candidates have disavowed ID as a fit subject for science courses. It's clearly perceived as a toxic issue and all have tried to distance themselves from it. That's a good sign; unfortunately, it also makes it difficult to tell who is on what side. ... Adams, Langseth, and Maes, have said they don't support ID creationism, but they also waffle with vague suggestions that it's an "interesting idea", and maybe it could be taught in a philosophy course. Sorry, ladies, it is not interesting, and I really wish people would stop treating philosophy as a safe dumping ground for any crap idea that comes along.

Of course, I love it when a non-philosopher speaks up for philosophy as not a dumping ground for crap, and said as much in the comments. But in that comments came the, "Oh yeah? Where the hell are the philosophers publicly decrying 'Intelligent Design' as a crock? If the History of Science Society is on record as against the ID movement, why has the Philosophy of Science Association been silent on the matter?"

These are good questions. This post offers my best guesses (which, given the above-mentioned fatigue, are not exhaustively researched -- I'm shooting from the hip tonight).

Philosophers are speaking out against ID. I'm a philosopher, and I've gone off on ID repeatedly, both here and in my 3-dimensional existence. Philosophy of Biology (where Michael Ruse blogs) does too. So does Brian Leiter. So does Sahotra Sarkar. Anyone with a good search engine could turn up lots more in the blogosphere. Anyone who gains entry to a building with philosophy department offices could find a bunch just randomly knocking doors. We talk to people. We write letters to the editor. Critical thinking is our bag, baby -- how the hell do you think we feel about ID and the whole ID movement?

By the way, this open letter to the Dover school board from University of Pennsylvania faculty is signed by philosophers.

But maybe people have gotten so accustomed to tuning out philosophers (the most loathed of all those annoying "intellectuals" the American people cannot abide) that no one notices us shaking our tiny fists.
Philosophers have been speaking out against ID and creationism for a long time, and think the whole thing is played out. There really are some issues that philosophers have achieved "closure" on, even if folks outside the philosophical bubble are still working on them. Once the good philosophers have worked out all there is to say about an issue like creationism/ID, it's time to move onto a live problem.

For example, Philip Kitcher wrote Abusing Science: The Case Against Creationism (whose cover you see above) for a general audience. It was published in 1982 and is astoudingly clear. Many of his critiques of creationism carry right over to ID. Does he need to write this book again every decade just to earn his anti-ID cred? Or, would it be fair to ask members of the public to crack a book?

More recently, Elliott Sober has delivered scathing critiques of the "detection" of design claimed by Dembski. If I could be bothered tonight (what with the fatigue), I could generate a list of philosophical take-downs of both ID claims and the flaky anti-evolution strategies that accompany them. The philosophers have been on this a long time. Anyone remember David Hume's Dialogues Concerning Natural Religion? I think the 18th century counts as old skool, yo!

Again, this is one of those cases where being tuned out by the public may play an explanatory role. Here, given that the public has not attended to what the philosophers have said about the issue, the philosophers decide to tune out the public and do some more fulfilling philosophical work.
Intelligent Design is pretty sissy even for a philosophical theory. I say this as someone who has the utmost respect for purveyors of the interesting-but-clearly-wrong philosophical theories that one encounters in reading the history of philosophy. (Descartes is my favorite guy in this crowd. He's convinced me I can believe in my own existence, but his attempts to get us back reliable empirical data with which to do science don't work nearly as well as he wanted them to. Still, an interesting problem, and Meditations is a good read!)

Really, what is ID offering? Every time we can't imagine how something could come about through natural processes we holler Intelligent Designer?

Now, there may be interesting philosophical questions to be asked in the general vicinity. For instance, what conclusions are warranted when our imaginations fail us? Is "design" something of which humans are or could be reliable detectors? Is there a structural "floor" beyond which reduction does not succeed? But, for all of these questions, a serious philosopher would want to subject them to serious examination, maybe even considering real world cases of various sorts. The ID proponents don't seem to be doing any of that.

ID isn't a scientific theory. It's not even a convincing imitation of a scientific theory. Not much interest there for philosophers who want to understand scientific theory building and testing.

Maybe ID is interesting to folks who do philosophy of religion. Undoubtedly ID carries with it an interesting bundle of assumptions about the nature of the Designer. I confess that I am fairly ignorant of philosophy of religion, so I can't say whether ID is or should be a hot topic in that field.

In short: I can't think of why one would include ID in a philosophy class except as an example of how not to do philosophy (or science).

So why is the Philosophy of Science Association not on record against ID? I don't know, but I'll try to find out. Stay tuned!

Technorati tags: ID, philosophy

Doctor Free-Ride's Film Corner

2005-11-06T17:09:00.000-08:00

A member of the Adventures in Ethics and Science Field Team brought me a DVD to review, "Ethics in Biomedical Research". This is a DVD produced by the Howard Hughes Medical Institute. According to the HHMI website, the online catalogue offers "a variety of award-winning publications, videos and other materials—all free." That means this DVD is free for the asking, too.

As the title suggests, the focus of the DVD is the ethical issues around biomedical research. There are four parts: Overview (28 minutes), Animal subjects (19 minutes), Genetic alteration (17 minutes) and Scientific integrity (15 minutes). I was a bit surprised that human subjects didn't get their own dedicated section, but they are discussed in the Overview and the Genetic alteration parts.

The overarching message of the DVD is that ethical issues come up especially where scientists doing biomedical research and the public have overlapping interests (what can be cured, what kind of research is necessary to develop the cure, what will it cost, etc.). However, attention is also paid to ethical questions that come up within scientific communities, quite apart from the public's interests and concerns. The filmakers make it clear that ethical issues are complicated, requiring serious efforts to balance risks and benefits (including future outcomes which are uncertain). But, the DVD encourages scientists to face the ethical questions rather than setting them aside for someone else to handle. Indeed, the message is that taking concerns from different quarters seriously, and discussing them ahead of time (rather than after something bad has happened) ought to be part of the everyday activity of doing science.

The DVD has the kind of lovely footage you'd expect of laboratory apparatus, imaging of microbiological systems, and well-maintained laboratory animals. (I swear, they even made the fruit-flies cute.) There is also the standard footage of principal investigators sitting in their office chairs holding forth about the responsible conduct of research, members of Congress (and the President) speaking about stem-cell research, recipients of treatments that resulted from biomedical advances, protestors of various sorts, and a few professional ethicists. More surprising: we also get to hear the opinions of scientists who are not principal investigators -- actual students and lab technicians. And, there are at least two separate research groups having laboratory meetings devoted to discussing ethical issues in scientific research. (The coolness of watching such a group meeting is undercut a bit by the shaky-cam.)

As far as content goes, there are some important historical mileposts (the Nazi "medical" experiments and the Tuskegee syphilis experiment, the 1975 Asilomar meeting to evaluate the risks of recombinant DNA research). There is also mention of institutional, federal, and international standards that apply to particular kinds of biomedical research (especially research with animal and human subjects). The DVD does include a brief discussion of the three guiding ethical principles in the Belmont Report, and while it can't, for obvious reasons, present all the salient information from institutional guidelines and policy manuals, mentioning that such guideline and manuals exist conveys useful information to the scientist and the scientist-in-training.

But, as the introduction to the DVD makes clear, the sections of the DVD "pose questions but few answers." And in this regard, the DVD is extremely impressive. The interviewees present a wide range of opinions about various ethical issues, from germline alteration to authorship, from financial conflicts of interest to the pressures inherent in the competitive world of cutting-edge research. All of the views in the DVD are presented as worth taking seriously, and the film makers seem to have made a real effort to find some that might challenge more comfortable assumptions within the world of science. (For example, one of the interviewees in the Animal subjects section is Tom Regan. The aim of the DVD is clearly not to cram "all the answers" into 79 minutes of footage, but rather to raise questions and to open discussions -- not only between scientists and non-scientists, but also among scientists. The introduction claims that the DVD content is "presented to stimulate more in-depth discussion, such as in a research group meeting or a classroom setting."

Would I use this DVD in a classroom setting? While it doesn't add any content to my course, it might be useful to my students to see scientists talking seriously about scientific issues. Too, seeing the diversity of views the scientists express in the DVD, and their apparent willingness to work with others to figure out the most responsible course of action in different situations would probably be good for the handful of students I have who start out inclined to reject the whole enterprise of ethics because there are "no right answers" and it's all "just made up". But, I could see this DVD coming in handy in a course designed to prepare students to conduct independent laboratory research, especially in the biomedical sciences. (The Scientific integrity section would work well for students or scientific trainees in pretty much any scientific field.)

A notable absence in this DVD is any explicit role for philosophical tools like ethical theories. Possibly the filmmakers thought ethical theories would be of no use to scientists in the trenches ... but then, I have to ask, how should we reconcile this with the tendency to push off students' ethical training onto philosophy departments? Indeed, I have this nagging worry that DVDs like this (and, I really think this is an excellent DVD) will be substituted for discussions in classroom settings or in research group meetings. "Why yes, we take research ethics seriously. See? We have the DVD!"

Lest you think I'm being overly pessimistic, the member of the Adventures in Ethics and Science Field Team found this DVD tucked away on a bookshelf in a research laboratory. Apparently, it had been provided to the research group by the funding agency. It was still in the shrink wrap.

Technorati tags: biomedical ethics, DVD, RCR instruction

You want to communicate? Then let's communicate!

2005-11-04T11:36:00.000-08:00

Over at The Panda's Thumb, Wesley R. Elsberry, who had been attending the Dover trial, reports the following:

Robert Gentry was in the courtroom in the morning, and noticed me sitting with the plaintiffs. At a break, he told me that he was retracting his permission for me to provide his papers on my website. Along the way, he made a rather insulting insinuation that I would alter his materials in some way. Now, back at that press conference, Gentry complained that scientists did not want people to see his papers. I made a good faith offer to host them. I hosted “scientific creationism” files on my BBS back in the old days of direct dial-up, and I certainly did not alter those. I’m a scientist, and I definitely want to rebut the notion that I’m somehow engaged in keeping people from seeing the arguments made by antievolutionists. Far from it. I think antievolution materials make the case for keeping non-science out of science classrooms quite well.

(Bold emphasis added.)

What I was just saying about intellectually honest debate? I think you can see it in this case as well. Scientists cling to the belief that we're all better off if we consider all the arguments -- even (especially!) the ones opposed to our preferred theory or interpretation of the data. And the reason we're better off seeing them is that then we can subject all the arguments -- even our own -- to serious scrutiny, after which we might not have a final resolution but we will at least have better arguments to work with.

The problem is that certain parties seem not to want a serious scientific back-and-forth here. They don't want to subject their arguments to scientific scrutiny. They don't want to respond to any scientific critique, whether on the basis of evidence or logic or methodology. They don't want to be put in a position where they might have to abandon their hypotheses, so they avoid situations in which these hypotheses might be subjected to serious tests. In other words, these folks want no part of the intellectual engagement with the scientific community that is at the heart of the scientific method.

And, that's fine, unless they also want to claim that their hypotheses and arguments are perfectly good science and that those other naughty scientists are ignoring them.

If you want to engage in a scientific debate, engage in a scientific debate. If not, have a spine and be honest about it.

Technorati tags: scientific communications, scientific method, Dover

Scientific communication with scientists who might not get it

2005-11-04T09:24:00.000-08:00

PZ Myers is blogging on the scientific ethics beat ... so maybe I should blog about zebrafish? But honestly, there's plenty of material to go around on scientific ethics. No worries!

The question today is whether it's a good idea for scientists to grant permission to Creation Magazine to reproduce their figures and video clips. The scientists in question were studying the pollen-launching mechanisms of the bunchberry dogwood, among other plants. Their findings were published in Nature. Chad at Uncertain Principles (from whom PZ got the story) describes a colloquium with Dwight Whitaker, one of the scientists on the project:

During the question period, somebody raised the issue of "Intelligent Design," asking if this is the sort of thing that wing nuts are likely to pick up on, and how this sort of structure evolves. Dwight gave a very good explanation of the evolutionary origin, pointing out that the basic structural elements that make the little trebuchets are present in lots of other plants in the dogwood family, so the change from existing plants would be very small. He also explained how it would be evolutionarily advantageous for this particular dogwood plant to have an effective pollen-launching mechanism, as it's a shrubby little thing that can easily benefit from both wind-borne and insect-borne pollination.

In other words, this pollen-launching system fits nicely into evolutionary explanations. But, given that Creation Magazine has contacted the scientists asking for permission to reproduce figures and videos with which they demonstrated their findings, it seems pretty clear that someone thinks these materials can be used to bolster the case for creationism.

You're a scientist. On the one hand, you have a duty to share your findings with the community of scientists, because the other scientists in the community are supposed to be able to replicate it, chime in with their reasoned responses to it, use it as the starting point for further research, etc. On the other hand, you don't want your findings to be misrepresented -- to be identified as showing something they do not. You especially don't want your findings misrepresented by someone claiming the authority of Science who, you suspect, doesn't actually understand how scientific reasoning or scientific discourses work.

Do you grant the permission to reproduce the figures and video clips?

In this case, the scientists did. Quoth Chad:

In the end, they decided that they had an ethical obligation as scientists to make their data freely available, even to wing nuts (they did insist that the article include pointers to the original source, which isn't peddling nonsense). I tend to agree, but it is an interesting question: If you knew that your work was going to be used as "evidence" to support pseudo-science, would you give the whack jobs permission to use your figures?

Quoth PZ:

I wouldn't have to think twice. I'd give my approval. The data is there, and if I trust it to be an accurate reflection of the real world, but of course I would want it disseminated, even if the agent were as untrustworthy as a creationist.

Besides, I'd love to face off against a creationist who tried to use some of my zebrafish development movies, for instance, as an argument against me. It would be a perfect Annie Hall moment.

Although, if I had so much clout and influence that my denial would actually have an impact on their ability to spread their lies, I might have to rethink that. There is no real risk of that happening, though—I'm not the National Academy of Sciences.

Some things to note about these responses:

They opt for disseminating information rather than restricting it -- especially once the information has been published. This is part of the Mertonian norm of communism -- scientific knowledge belongs to everyone in the scientific community.
There is an assumption that, to a certain extent, the facts really do speak for themselves -- if you lay them out for people to see, there are certain interpretations that will seem better (scientifically speaking) and others that will show themselves as transparent efforts to twist the facts.
In insisting on pointers back to the original source of the figures and video clips, the authors are reinforcing their interest not only in providing accurate context for understanding the findings, but also their interest in staying involved in the dialogue about the findings and what they mean. In other words, it's within the rights of another scientist to disagree with how we interpret our findings -- but it's also within our rights to disagree with their position and try to explain why our interpretation was better.
While there is some concern that the requested use of the figures and clips might not be in the service of actual science, the authors seem willing to give the benefit of the doubt until presented with contrary evidence. Of course, there is the implicit threat that if the findings are used in ways that aren't scientifically legitimate that there will be a response from the scientists who granted permission calling out the other "scientists" on methodological shenanigans.
And, there is just a trace of worry that the authority of the scientists who presented the original findings might be hijacked to make a scientifically crummy claim look scientifically respectable to a lay audience that doesn't know any better.

Indeed, connected to this last point, PZ invokes the decision of the National Academies of Science not to grant copyright permission to the Kansas State Board of Education precisely because NAS wanted to head off an attempt to have its scientific authority hijacked. But I think, at least on the face of things, these two cases are different in some important ways.

First, NAS wasn't restricting use of scientific findings by other scientist (or, for that matter, by non-scientists). Rather, they were restricting use of painstakingly crafted educational standards in a way that would have fundamentally changed them, while at the same time claiming that the modified standards were still "based on research and on the work of over 18,000 scientists, science educators, teachers, school administrators and parents across the country that produced national standards as well as the school district teams and thousands of individuals who contributed to the benchmarks." In other words, the Kansas State Board of Education was trying to say, "These standards are an accurate reflection of what all these scientists and science educators say" -- when they were not.

How is that different from a creationist magazine taking pollen-launching mechanism findings and trying to claim that they are evidence for intelligent design? The scientific literature provides a way for the scientists who feel their work is being misrepresented, or misinterpreted, to respond. Publishing a paper doesn't end the discussion in science; the discussion keeps on going until scientists are done with it. On the other hand, educational standards have to be rather more "settled" in order to guide curricula. And, there's no obvious way for scientists to respond to the analogous misrepresentations and misinterpretions in state standards once they've been adopted (nor is there reason to assume that the folks adopting them are scientists who share the same assumptions about intellectually honest dialogue). NAS knew it wasn't dealing with a scientific question so much as a policy question, and it refused to provide cover for the policy by allowing misuse of its standards.

On the other hand, research scientists are working on knowledge that is growing, participating in discussions that are ongoing. There is a risk you take of including, in these discussions, people who don't really buy into the norms of good scientific dialogue ... but there is also an opportunity. Sometimes intellectual honesty and serious engagement with hard questions wins people over and really brings them into the community of scientists. (Sometimes it also impresses the lay people who are paying attention to the exchange.) Ethically impeccable scientific communication may have a bigger impact on those now suspicious of science than any piece of data.

Technorati tags: scientific communications, scientific norms

Closer to home

2005-11-02T14:19:00.000-08:00

In other news, today I got word from the IRB that my research protocol has been approved and I can start collecting data. Yay me!

With luck, this morning's attempt to fix our department photocopier was successful. I'll be needing that bad boy to duplicate surveys and consent forms.

Imagine how embarrassed I would have been (as a "local authority" on responsible conduct of research) if the protocol had been rejected ...

Completing the misconduct trifecta: plagiarism

2005-11-02T09:34:00.000-08:00

As I've just recently been discussing fabrication and falsification in the news, it seems inevitable that I should take up a news story about the "P" in FFP (fabrication, falsification, and plagiarism, the core of the government's definition of scientific misconduct; and no, I don't think the government is thinking of dropping the "P" because the "P" comes between it and its fans ...)

Inside Higher Ed delivers the relevant news item about concerns raised by a graduate of a Masters program in engineering at Ohio University. Briefly, Thomas Matkra, the engineer in question, feels that plagiarism in the masters theses at Ohio University — and, even more, complacency on the part of the faculty about acts of plagiarism in these theses — undermines the value on his degree. From the linked article:

Thomas Matrka did not set out to become a whistle blower.

In 2003, 10 years into his engineering career, he enrolled at Ohio to get a master’s degree. He got good grades, but as he worked on his thesis, he says, his adviser, M.K. Alam, the Moss Professor of Mechanical Engineering, repeatedly expressed dissatisfaction with his work. (Alam did not respond to requests for comment for this article.) Hoping for insight into projects that had previously won Alam’s approval, Matrka spent some time in the university’s library in the summer of 2004 thumbing through past theses.

He was struck by what he found. As he looked the papers over, Matrka says, he noted similarities — occasionally blatant, extended ones — between many of them. He discovered four theses, for example, in which the third chapters on “fluent and multiphase models” were virtually word for word. Two were from 1997 and two from 1998. Three others, from as many as six years apart, contained paragraphs and drawings that were almost identical. (Matrka provided pages from some of these theses to Inside Higher Ed for review.)

“Some of them were so blatantly obvious, where there was page after page copied from one another or from a textbook,” says Matrka. Some of the overlap is so obvious, he says, that it would be impossible for the professors who oversaw the theses not to have known about it. “It’s a faculty approval problem,” Matrka says. “It’s hard not to conclude that advisers condoned this.”

There is a cluster of connected question here:

Do the practices Matrka identified in these theses constitute plagiarism?
Do the faculty have a duty to deal with past acts of plagiarism (e.g., in theses of students already granted degrees) and, if so, how?
Does pervasive plagiarism in a graduate program undermine the value of a degree granted by that program?
Do scientists and engineers have a common understanding of what counts as plagiarism?
Do scientists and engineers agree that plagiarism is a species of scientific misconduct?

What is plagiarism? On Being a Scientist: Responsible Conduct in Research, Second Edition (1995) describes it as "using the ideas or words of another person without giving appropriate credit". So the central question is just what constitutes appropriate credit. And, it's not inconceivable that the standards of appropriateness are different in different contexts. The context in which a master's thesis is written may be quite different from the context in which a journal article, or a grant proposal, is written. Statements that may be regarded as part of the common pool of knowledge of credentialed professionals may be exactly the kind of thing a graduate student is expected to explicate in a thesis. In your manuscript for the journal, you probably don't footnote every textbook fact that you mention in your discussion. However, if you use the exact wording of the textbook author, or reproduce a figure exactly ... it seems like it would be better to err on the side of citing it, just to avoid confusion about whose words or whose figures they are.

So, is it clear to engineering professors and science professors (and working engineers and scientists) what "giving appropriate credit" amounts to? Is it clear to their students? It seems that, at the very least, what's going on at Ohio University indicates a mismatch in expectations between professors and graduate students. Matrka sees practices that look like plagiarism to him, and he expects the faculty to see this as a violation of the norms of the community and do something about it. The faculty ... well, possibly see some of these practices as plagiarism, but they don't seem to think they need to do very much to address past acts. And, it's not clear what they have in mind to head off future such acts ... if, in fact, they see these acts as problematic at all.

[Dennis] Irwin [engineering dean at Ohio University] says the college has begun “briefing graduate students on the nature of plagiarism, its consequences, and how to avoid plagiarizing others’ work,” and that it now requires electronic submission of theses and dissertations and a statement of originality signed by all students. Beginning this winter, he says, the college will “begin using comparison software to screen all of the theses submitted against all of those we have in electronic form.”

What the college will not do, he says, is ask his faculty to review what could be “tens of thousands of pages” of hard-copy theses and dissertations” in the library. That could take a huge amount of faculty time for an uncertain payoff, he says, “so you can probably see our problem in meeting any demand that all instances of plagiarism be removed from the library.”

Irwin adds: “I know Mr. Matrka is not satisfied with our actions to date, but all I’ve heard are accusations, and I haven’t been presented with any evidence that those accusations are true.”

Part of the problem, the dean says, may be a “difference in interpretation between what [Matrka] considers to be plagiarism” and the university’s own interpretation. With technical works like engineering theses, he says, “there are going to be similarities, particularly in equations and diagrams.” He adds: “If the same two people worked on the same experiment or apparatus, it is conceivable that they would jointly develop schematic drawing of that that might be used in both of their theses.”

Matrka admits that that possibility could explain some of the cases that looked fishy to him – which is why he has encouraged the university to turn the review over to faculty members more knowledgeable than he is.

But many of the other examples he has identified, he says, don’t require an expert’s eye. “Some of this stuff you wouldn’t get away with in high school.”

That the college of engineering has, apparently, just begun talking to graduate students about the nature of plagiarism suggests that in the past it was assumed that what counts as plagiarism in engineering was self-evident — or, perhaps, that plagiarism was not very important. The clear “ 'difference in interpretation between what [Matrka] considers to be plagiarism' and the university’s own interpretation" gives lie to the assumption that what counts as plagiarism is self-evident. If plagiarism is something that the college of engineering regards as important, there needs to be a more explicit discussion of what it is and why it matters.

And, if the message about why plagiarism matters is to be taken seriously, this may commit the faculty to doing something about past acts of plagiarism. Leaving theses with plagiarized pieces in the library sends a message about the acceptability of plagiarism. (Who looks at the old theses the most? People writing their own theses. Why? Because they're trying to discern what a good thesis looks like.) Also, tolerating plagiarism sends a message to the people who committed it that this is how we do it in this field. If you discover that someone who graduated 10 years ago made inappropriate use of sources, I'm not sure you yank their master's degree, but it seems like at the least, you ought to contact them and make it clear to them why they ought not to make similar inappropriate use of sources in the future.

If plagiarism doesn't matter to engineers, that's one thing. Maybe they should just go on record as saying, "We thing plagiarism is no big thing. Fabrication and falsification are right out, but ownership of words or ideas is a canard."
Then, there would be no confusion.

However, if plagiarism does matter to engineers, they have to do something about it. They have to be clear about what constitutes appropriate use and what does not, and they have to, as a community, bring the hammer down on inappropriate use. Otherwise, the community will be judged by its actions rather than its words, and the mismatch between the two won't earn the community much respect.

Scientific parents: Please take the opportunity to talk about plagiarism and proper attribution with your scientific offspring today!

Technorait tags: scientific misconduct, plagiarism

Tangled Bank #40, for your examination.

2005-11-02T09:30:00.000-08:00

Hop up on the table in The Examining Room of Dr. Charles and have a look at Tangled Bank #40.

You don't need to put on the paper gown unless you really want to ...

Lies that "don't matter"? (Van Parijs follow-up)

2005-10-31T08:55:00.000-08:00

The conduct of fired MIT biology professor Luk Van Parijs, as reconstructed in the investigation of his work of the last eight years or so, gets curiouser and curiouser. From the October 29th Boston Globe, a follow-up story by Bombardieri and Cook tells us that problems have surfaced not only with the research Van Parijs did at MIT, but also with papers he authored about research he did at Brigham and Women's Hospital while a graduate student. But the twist here is that it's not entirely clear how his fraud in these cases would have helped him. From the Globe article:

The new revelation deepens the mystery about a rising star who was popular with students and colleagues and appeared to be a gifted biologist. In both of the new cases, it appears that Van Parijs said he had done work that he had not done, work that would have been a small part of the overall experiment.

In one case, the data in question would not have affected the conclusion, said Dr. Abul Abbas, who directed the Brigham laboratory where Van Parijs worked and was the senior author on both papers. For the second paper, the questionable data may have affected the outcome, Abbas said.

So, it seems we have a guy fabricating or falsifying data that might not even change the conclusion of the papers for which these "data" were created.

Why?

I can think of a couple of plausible explanations here. One might be that he felt he needed more data to wave around to strengthen the impact of the actually good data he collected. (Replication is good, and more is better.) Another is that possibly some of the "good" data wasn't all that good either, but it was more convincingly faked. This might not be that crazy an idea. Suspicions about Van Parijs's work from his Brigham and Women's years are tied to some plots that look more similar than they should:

In the two papers, Van Parijs was investigating the function of T cells, which are part of the immune system. Van Parijs ran samples of the cells through a device known as a flow cytometer, which sorts the cells by the characteristics in which the scientists are interested. This produces plots, essentially diagrams with large numbers of dots, with each dot representing a cell.

In both papers, there are plots that appear to be almost identical, even though the paper says they are sets of cells from different mice. Using only one mouse would have saved time. The plots are not exact copies, though, which Abbas told the Globe has made him more concerned, because if the data are fraudulent, it implies they were done intentionally. Changing data or inventing it is considered a very serious offense, regardless of the effect the act has on the conclusions made in a research paper, scientists said.

(Bold emphasis added.)
This is the kind of fakery that was bound to be caught -- with a little reflection, Van Parijs could surely have turned out a better faked plot.

The other possibility here, which seems very weird, is that the fabrication and falsification were not done with the intention of producing "better" results, nor of affecting the reported results at all. But this would make Van Parijs ... a scientist who is lying to other scientists just because he can? Is this the scientific equivalent of torturing cats before moving on to your first murder of a human?

Perhaps. Again from the Globe article:

It is not unusual to see cases of fraud involving data that are tangential to the main point of a research paper, as is alleged in some of Van Parijs's work, according to C.K. Gunsalus, a special counsel at the University of Illinois at Urbana-Champaign, and specialist on research integrity.

''It is very common, and there is also a common defense, which is 'I have a PhD and I wouldn't have done something so stupid,' " said Gunsalus. Often, she said, this defense is successful. She also said that it was common to see a pattern of escalation, with small infractions building over time to larger ones.

(Again, the bold emphasis is mine.)

Do scientists who lie about insignificant things lose their taste for gathering real data, or do they get a taste for putting one over on other scientists? Either way, it seems clear that, in a field that is all about figuring out how things really work, telling lies is a Very Bad Thing. At this point, I'd imagine, citing a paper on which Van Parijs is an author would add about nothing, in terms of evidential support, to anyone else's serious scientific work -- this despite the fact that fact that Van Parijs's postdoctoral advisor, David Baltimore, told the Globe that "he knows from work that his lab has done following up on Van Parijs's research that a lot of what he did is, in fact, verifiable." (That "knowledge" rests on the assumption, of course, that members of the lab are doing legitimate experiments and analyses of these ... because surely Van Parijs was the only one who would ever dare to do otherwise.)

By the way, C.K. Gunsalus is the authority to consult on scientific integrity (and lack thereof) in university research settings. Despite some quite reasonable worries people have expressed (like YoungFemaleScientist in this excellent post) about folks accused of scientific misconduct being ruined forever even if the charges turn out to be baseless, Gunsalus has argued that more frequently the lack of real penalties allow the cheats to stay in the system and cheat again. There's a fairly high recidivism rate on cheating in science, according to Gunsalus; it's hardly ever the case that someone is caught for misconduct without having a history of similar deeds. (And that seems to be how the Van Parijs case is shaping up.) And what's the message to the rest of the scientific community if someone is caught fabricating and falsifying data, but is only given a slap on the wrist because it didn't effect the conclusions (or maybe it did, but other labs have "validated" the results)? The message is that lying isn't really a big deal.

Do you see now why some of us get worked up about dishonesty that seems insignificant in the grand scheme of things?

Gunsalus has a downloadable offprint that bundles two of her best articles: "How to Blow the Whistle and Still Have a Career Afterwards" and "Preventing the Need for Whistleblowing: Practical Advice for University Administrators." Both are beautifully written and full of practical advice. If you're a scientist or a science student (or a university administrator), you need to read them!

Technorati tags: scientific misconduct, scientific integrity, fabrication, falsification

Aw Mom, scientific misconduct again?!

2005-10-30T13:00:00.000-08:00

Can you believe there's another story in the news about a scientist caught fabricating and falsifying data? Also, the sky is blue.

This time, the miscreant scientist is Luk Van Parijs, who was an associate professor of biology at MIT until they fired his ass. His offenses include "fabricating data in a published scientific paper, in unpublished manuscripts, and in grant applications"; apparently, he also admitted to falsifying data. Gareth Cook and Marcella Bombardieri write about the incident for the Boston Globe.

Why does this kind of thing keep happening? Maybe if we knew, we could put a stop to it. In the meantime, here are my thoughts about the various players in this affair. (All quotes are from the above-linked Boston Globe article unless noted otherwise.)

The other researchers in Van Parijs's lab at MIT

The investigation began in August 2004, when a group of researchers in Van Parijs's laboratory brought their concerns to university administrators.

Well done, other researchers! The Globe article doesn't give details about who these researchers were. They may have included postdocs, graduate students, technicians, and visiting scholars. In any case, it is a good bet that they were lower on the foodchain than Van Parijs.

[Alice] Gast [associate provost and vice president for research at MIT] praised the scientists who made the initial allegations, saying that the university depends on all of its members to defend the integrity of research, even if it means the awkwardness of challenging friends and colleagues.

Keep in mind, if the other researchers really were lower on the foodchain than Van Parijs, that we're talking about something significantly more awkward than challenging a friend and colleague — we're talking about challenging the boss. You know, the guy whose letter of recommendation will make a big difference to your future as a scientist. The guy who needs to sign off on your dissertation. The guy who pays your salary.

Were there conversations in which these other researchers raised their concerns with Van Parijs before they brough their concerns to university administrators? The article doesn't say, but it's hard to imagine that there weren't. Maybe the questions were oblique. But you want to be reassured that the research projects of which you are a part are on the level. Really, there are so many messy consequences that flow from your boss being a cheat that any other theory with evidence to support it is preferable. Only when you're sure that he's really crossed to the other side do you get the admistrators involved.

Standing up for the integrity of one's own research, and of scientific research as a whole? That's a good thing. Mad props to the whistle-blowers.

MIT

MIT said Van Parijs quickly admitted fabricating data, as well as falsifying data, which means changing it in a misleading way. The confidential investigation was conducted by MIT scientists whom Gast declined to name. Van Parijs was placed on paid administrative leave in September 2004, and did not have access to his lab, she said...

Gast said that MIT is working with his coauthors on retracting published errors and that all of Van Parijs's colleagues have been very cooperative. The university is also preparing a report to the US Office of Research Integrity, an agency of the Department of Health and Human Services that investigates scientific misconduct involving federal funds, so that it can perform its own investigation. She said the university immediately stopped the spending on his grants when the problem was discovered, and would work with the government to determine what money needed to be returned.

MIT did well in taking the concerns about Van Parijs's work seriously. They started investigating the allegations quickly. They stopped spending the grant money, recognizing that the funders of the research thought they were funding the collection of actual data rather than fabrications. They are involved in correcting the problems in the published scientific literature stemming from Van Parijs's misconduct. MIT gets that science demands a high standard of integrity, and they don't want the MIT brand associated with behavior unbecoming a scientist.

At the beginning of the investigation, they put Van Parijs on paid administrative leave. After more than a year of getting the facts, they were comfortable enough in their assessment of his deeds (and his character) to fire him. Even if Van Parijs had been tenured (he was not), MIT would have been able to remove him "with cause". Scientific misconduct is a pretty clear cause for removal.

Kudos to MIT for finding the facts and taking appropriate action to remove a bad actor from its faculty. Beyond cleaning its own house, MIT acquits itself well by its "ongoing work to correct the scientific record". Not mentioned in the story is how MIT handled the dissolution of the Van Parijs lab; I hope efforts were made to find reasonable positions for the postdocs, grad students, and technicians displaced by this dissolution.

Luk Van Parijs
It goes without saying that a scientist ought not to fabricate or falsify data. Fabrication and falsification suck. These deeds are varieties of deception. Deceiving the people reading your papers in the journals, or the people reading your grant applications and deciding whether to fund your research, is crappy. And, fabrication and falsification suck even more when done in papers on which you have coauthors. You're dragging good scientists down with you. Even when you've been taken out of the game, they still have to worry about corrections, retractions, and the lasting impact on their reputations.

But wait, what's this?

In an e-mail sent to the Globe last night from his MIT account, Van Parijs said, ''I was shocked at the timing and manner in which MIT made the announcement since I had cooperated with the investigation to the fullest of my capabilities."

Excuse me? Dude, you were caught. Indeed, when confronted, you confessed your wrongdoing. MIT had to fire you. It's not like anyone was going to be able to trust any of your data again. You knew what you were doing was wrong ... and you still did it.

You cooperated? Great. But that doesn't mean MIT can, or should, keep your bad acts a secret. They can't protect your reputation without simultaneously hiding the fact that you put crap into the pool of scientific knowledge. They are not going to keep things hush-hush so you can, maybe, get a job in science someplace else and not have to work under a cloud of suspicion because of your prior bad acts.

Get a clue. You're not a bewildered first year grad student who doesn't know the rules of science. (Indeed, it's quite possible that your grad students were the ones with the moral compass and the courage to rat you out.) You're an associate professor in your mid-30s. If you don't know how to be an ethical scientist by now, is there any good reason to think you'll pick it up at some point in the future?

Van Parijs's mentors

He worked in the lab of Dr. Abul Abbas at Brigham and Women's Hospital for three or four years, until about 1997, according to Abbas, now chairman of the pathology department at the University of California, San Francisco. Abbas said he did not see any indication at the time that Van Parijs might falsify data. He said he is talking with people at Brigham and Women's Hospital to decide whether to investigate the work Van Parijs did while in Abbas's lab.

One imagines if Abbas did see an indication that Van Parijs's, or any other graduate student under his supervision, might falsify data, he would have grabbed him by the shoulders and given him a good shaking. A good research advisor wants you to learn how to do good research. Learning how to obtain clean, reliable data is a lot harder than learning how to make up data that suits your hypothesis.

[H]is career had been highly promising, including being hired as a postdoctoral fellow in the lab of famed Nobel laureate David Baltimore.

''He was a very personally attractive, excited, and thoughtful guy who cared about a wide range of science," Baltimore, now president of the California Institute of Technology, said in an interview yesterday. ''When I first heard there was a question about his work, it came as a very great surprise to me."

Dr. Baltimore, we meet again. Why is it that you are always surprised when someone raises a question about a collaborator's work? Is it that you have an optimistic view about your collaborators? Or do you maintain a certain ... distance from the day to day details of what they're up to? I'm not going to call it willful ignorance, but maybe you should consider collaborating more closely when you undertake collaborations.

The broader culture of science
By no means do I want to say that circumstances made Van Parijs cheat. He owns this mess. But, I have to wonder just a little if, given some changes in the culture in which he was working, we'd have fewer cases like this.

'He got a job and finished his postdoc training much faster than average people," said Xiao-Feng Qin, who worked on the same laboratory bench as Van Parijs at Caltech and is now an assistant professor in immunology at MD Anderson Cancer Center in Houston. ''I think people thought he was a golden boy, because he finished so fast."

You have to get results, and publish them, to stay in the game of science. Doing more, and doing it faster, gives you an edge over the competition for the scarce resources of jobs and grants. Getting really amazing results helps too.

But getting good results, in the real world, takes time. Doing things right sometimes means doing things over. Doing things right often means your hypotheses don't pan out. Writing up clear articles that present your findings accurately and explain what they do, and do not mean, takes time too.

Valorizing high output at great speed over careful research may be a problem. At the very least, we ought to not simply marvel out the golden boy's productivity; we ought to look at his output very carefully to make sure it holds up. Science isn't supposed to be about an impressive biography or being "personally attractive" and fun to talk with. We're supposed to be trading in real results!

Will the community of science learn the lessons from the good and bad actors in this case, take them to heart, and nip future misconduct in the bud? It would be nice to think so. But it wouldn't surprise me if we've got another case like this six months from now.

"Prove me wrong, kids. Prove me wrong!"

Technorati tags: scientific misconduct, fabrication, falsification

Protecting the meaning

2005-10-29T19:36:00.000-07:00

Science, by its nature, is an activity that has communication built into it. It's a big world, so scientists need to split up the job of figuring out what's going on with it, and they need to report their findings back to the team. Moreover, the sharing of information between scientists is a way for the individual scientist to be sure what she's observing is a real phenomenon rather than an equipment malfunction or a figment of her imagination. And, of course, scientists pass on information to non-scientists, whether practical information ("Hey, you might want to cook that at a higher temperature if you want to eat it and not get violently ill.") or more esoteric information ("Here are the cool things we could learn by accelerating particles and smashing them into each other; can we have some money?)

A perennial source of frustration for scientists who go to the trouble of telling lay people what they've found is that lay people manage to misunderstand the scientists so frequently. Certain science journalists have been known to make the problem worse by quoting scientists out of context or by playing "balanced reporting" games that don't accurately reflect the center of gravity of scientific opinion. In some ways, this seems like a risk inherent in any kind of communication: whatever you say (or write) can be misinterpreted by someone. Short of spending all your time trying to straighten out the folks who are confused or just not sharing any information in the first place, what are you going to do?

If you're the National Academy of Sciences or the National Science Teachers Association, you're going to do something that makes Dr. Free-Ride very, very proud.

You see, NAS and NSTA have published science education standards. They went to great lengths to make them good, and to put them into words that communicate clearly what students ought to understand about what science knows and how science knows it. In writing and publishing these standards, these organizations clearly hoped that they would be put to use in designing quality science classes.

They did not hope that their standards would be taken and modified in such a way as to remove or alter key parts of what NAS and NSTA were trying to communicate about scientific methodology and scientific knowledge. According to NAS and NSTA, that's precisely what happened when the Kansas State Board of Education took the two sets of standards and substantially altered them to create the state's science education standards. And, rather than let the Kansas State Board of Education misuse and misrepresent these carefully crafted standards, NAS and NSTA have decided to withhold copyright permission.

To really grasp the righteousness of this decision, it's worth looking at a chunk of the Kansas Science Education Standards, and the NAS response to it. I've taken both from the NAS review of the Kansas Science Education Standards, 14 pages of downloadable shock and awe.

First, from the statement on the deveopment of the Kansas standards:

The 1998-2001 science standards committee was able to build upon and benefited from a great deal of prior work done on a national level; the National Science Education Standards published by the National Research Council; Benchmarks for Science Literacy from Project 2061 of the American Association for the Advancement of Science (AAAS); and Pathways to the Science Standards, published by the National Science Teachers Association (NSTA). This allowed the foundation for the Kansas Science Education Standards (2001) to be based on research and on the work of over 18,000 scientists, science educators, teachers, school administrators and parents across the country that produced national standards as well as the school district teams and thousands of individuals who contributed to the benchmarks.

Now, the response of Barbara Schaal and Jay Labov, the NAS reviewers:

This statement suggests that the Kansas Standards are based in large part on these three documents from the NRC, AAAS, and NSTA. However, all three documents are clear about the central role of evolution to the life and physical sciences. Because of the changes that a minority of members of the Kansas State Board of Education made to those state standards in removing aspects of biological and physical evolution and related topics, all three organizations denied copyright permission to the Kansas Board in 1999 (see http://www4.nationalacademies.org/news.nsf/isbn/s09231999?OpenDocument). When the composition of the State Board of Education changed in 2000 and these areas of science were returned to the Kansas Science Standards, our three organizations jointly issued a statement praising this action of the Board (see http://www.nasonline.org/site/PageServer?pagename=NEWS_statement_president_02142001_BA_science_education).

In other words: If you're going to put together standards that fundamentally mislead about the state of scientific knowledge and the proper methodology of science, you'll have to do it without trying to anchor your standards in the authority of NAS and NSTA.

And indeed, it is clear, from the point-by-point analysis presented in the 14 page review of the standards, that this is exactly what the Kansas State Board of Education was trying to do. Download it, read it, and marvel.

No one is stopping the folks in Kansas from crafting their own science education standards. But to craft standards slipped carefully into the published standards of NAS and NSTA indicates that they thought the authority of NAS and NSTA on matters scientific was important. Indeed, in order to protect that very authority, NAS and NSTA essentially called shenanigans on the Kansas State Board of Education. And that's a use of copyrights I can really get behind.

(Hat tip to Jack Krebs at Panda's Thumb, whose post brought this matter to my attention.)

Technorati tags: science standards, scientific communications, copyrights

An appeal to my readers

2005-10-27T12:18:00.000-07:00

Yeah, I'm talkin' to you!

My department has become the Source of All Required Ethics Training for a number of departments, programs, and colleges at my university, and this seems to flow more in some cases from external pressures (e.g., what the accrediting agency or funding agency requires) than from a deep respect for the value of a philosophical grounding in ethics. Of course, I have blogged about this a little.

But now, in a setting where you all are not pressing my department for an ethics course that will achieve some necessary end for you, I would like to ask for your honest opinions:

What kind of ethics training does a scientist really need?

If you are a scientist (or scientist-in-training), what pieces of information are most useful in your day-to-day scientific activities from the point of view of being a responsible scientist? Is it an ethical theory? A piece of policy? A rule of thumb for deciding how to go forward in a tricky situation? Where or how did you learn it? Are there aspects of being a "responsible" scientist that you wish you had learned more about, and if so, what are they?

Do you think being a responsible scientist in your particular field puts special requirements on you, or creates particular challenges? (I'm interested in a broad swath of "science" here -- including experimental and theoretical sciences, natural sciences, physical sciences, social sciences, and computer science and mathematics.)

Another way to cast the question is: If you (or a student in your field) had to take a class in ethics, what would you put in that class to make it maximally relevant?

Or: What do you want all the scientists in your field to understand about what it means to be a responsible scientist?

If you are a non-scientist, I'm guessing you're here because you have some interest in how science interacts with the other stuff going on in our world (such as policy decisions, education, etc.). Let me put a related question to you all:

What do you want scientists to know about ethics/how to be a responsible scientist? You can, of course, answer based on your favorite dystopian vision of what happens to everyone else if scientist don't have or don't use this crucial ethical information.

You can respond in the comments, or by email to me (dr.freeride AT gmail DOT com). If you could identify yourself as a scientist (with your field) or a non-scientist, that would be helpful.

Thanks for your feedback!

How's the inventory in the (scientific) marketplace of ideas?

2005-10-25T14:49:00.000-07:00

Scientists, as much as any other people of reason, have an interest in creating a bustling marketplace of ideas. So, you might suppose that their attitude toward scientific theories, and even toward the wild ideas that aren't yet theories but suggest new directions, would be the more the better. And, you might be especially puzzled by the reluctance of scientists to support the "teach the controversy" position on the continuing Intelligent Design versus evolutionary biology contretemps.

Certainly, this is a view you might come to from the testimony of sociologist Steve Fuller in the Dover case. Here's how it was reported in the York Daily Record:

Fuller said intelligent design is a scientific theory that should be taught in school.

But during cross-examination, he said intelligent design — the idea that the complexity of life requires a designer — is "too young" to have developed rigorous testable formulas and sits on the fringe of science.

He suggested that perhaps scientists should have an "affirmative action" plan to help emerging ideas compete against the "dominant paradigms" of mainstream science.

The pool of peer reviewers is smaller than it has been because, as scientific research gets more and more specialized, there are fewer people in that specialty and even fewer of them are willing to peer review pieces, Fuller said. Consequently, grant money also goes to fewer researchers, he said.

"People don't want to judge the validity of a scientific theory based on who is talking about it and promoting it." ...

Fuller told the court that one of the problems of science is with the very definition of "scientific theory," which is the idea of well substantiated explanations that unify a broad range of observations. He said by requiring a theory to be "well substantiated," it makes it almost impossible for an idea to be accepted scientifically.

Let us grant that scientists can be creatures of habit, and that they can be suspicious of new ideas, just like anyone else. Let us also grant that there are ways in which actually-executed peer review departs from the ideal. And, let us grant that new ideas (especially the wacky-looking ones) may require some time before the scientific community can get a good read on whether they will amount to anything.

Does all this mean that there ought to be affirmative action in the Halls of Science for "emerging" and/or seemingly crackpot ideas?

First off, even if the answer to this question is "yes", it's not at all clear that a high school science classroom (most of whose denizens will not grow up to be scientists) would be the best place for this. The point of science pedagogy at this level is to convey something about our current understanding of various facets of the physical and natural world and, more importantly, about the methodology used to come to this understanding. Alternative paradigms probably pack more punch in later science classes (I'd say past the introductory level of college instruction or, even better, in one's first foray into research as an undergraduate). You need to really understand the dominant paradigm to be able to appreciate alternative to it. And, at least when I was a student, undergraduates and graduate students had enough of a taste for iconoclasm that they could be counted on to give most of the alternative paradigms on offer a good looking over. They might do this while the boss's eyes were averted — but this just reinforces my sense that the newish scientists are much less in the thrall of authority or tradition than Fuller seems to be suggesting.

Indeed, given the lore within different scientific disciplines about scientists who made great discoveries (and won prizes and stuff for doing so) by questioning the conventional wisdom of their field, you better believe there is a cachet associated with doing research at the paradigm's edge. It may not be the project you're working on that is the easiest to fund. It may not be a line of research that colleagues in your department think the most of (since they're busy doing research on other edges of the paradigm). But you bloody well keep looking to see if there's anything to it, because you would hate for some other young pup to be the one to convince the community of science that this alternative paradigm is the real deal.

And, while there may be instances where a more serious look at an alternative paradigm is called for, scientists will require at minimum that the alternative paradigm points the way to actual research that could be done to test it or solve live scientific problems with it. The requirement is not, despite what Fuller seems to suggest, that the new paradigm have all the answers. But, it has to fit with enough of the evidence and show real promise of solving outstanding problems that there's a sensible way to use it to guide scientific research. An idea that can't do that isn't ready for this particular marketplace of ideas.

Ed Brayton, also responding to Fuller's testimony, puts it rather well:

Fuller is of course correct to point out that there have been scientific revolutions in the past that have overturned much of what we thought we knew about the world. But those revolutions were the result of scientists actually doing science - building theories and models, testing them against the data, publishing the results for their peers to see and arguing over the results to reach a consensus - not by hiring PR firms and lobbying legislatures and school boards.

Finally, as for ID:

Not a new idea

Not especially helpful in guiding experimental research (else where are all the experimental reports of research guided by it?)

Both potentially iconoclastic and politically popular (at least among governmental folks and people with private think tanks that have lots of money to giveto researchers) -- thus, there's no reason to think mavericks wouldn't be pursuing it if they felt like it.

So, I'm not really feeling the need for ID affirmative action.

Of course you want scientists to be open minded. But, isn't that what all this scientist-on-scientist career competition is supposed to ensure? (If not, could we maybe ease up on the tenure-track-red-in-tooth-and-claw business?)

Technorati tags: intelligent design, alternative paradigms

Talking down, or keeping things real?

2005-10-24T14:26:00.000-07:00

At The Panda's Thumb, guest contributor Joe Meert has post about the Geological Society of America meeting and specifically, about what was said there about Intelligent Design.

I'm always happy to see what the scientists are talking about when they get together, and since I don't have much personal contact with geologists these days, it was useful to see what's on the geologists' minds. Given the importance of geology as a source of evidence for past biological goings on, they have, as you might imagine, rather strong reactions to efforts to impose ID in science classrooms.

But, there is a piece of the meeting (as reported by Meert) that has been gnawing at me a little. It's the strategy suggested by Don Wise. Of Wise's talk, Meert says:

He also noted that we should take our cues from politics. We live in an age of sound bites and using words like “incompetent design” can be more effective than trying to explain in scientific detail why it’s bad science. Wise encourages geologists to take lessons from politics; (1) don’t be defensive (2) keep your points simple and easy to remember (3) use humor to make your points (4) aim your points at the voters.

(Bold emphasis added.)

Now, I don't want to quibble with the claim that a more political strategy might be more effective than the standard scientific approach of looking at evidential support, testability, logical consistency and the like. Given the endumbening effects of years of TV and standardized tests, punchy catchphrases and slandarous mambos may be the most effective way to turn public opinion.

But ... part of what's groovy about science is that it isn't a popularity contest. It's about grabbing onto empirical data, building theories that hold up despite repeated attemt to knock them over, and coming up with an account of the world and its many phenomena that reflects something about how the world really is rather than just how we want it to be.

Now, you can participate in this kind of serious scientific debate and still have a sense of humor. (And if you can't, PZ and Orac can. So can most of the science teachers I've had since the tenth grade.) But my fear is not that bringing the funny will undercut the devastating logic of the scientific arguments. Rather, my fear is that bringing the funny may come to replace the devastating logic of the scientific arguments.

And that would be a problem.

The reason to talk to a scientist, rather than a witty political pundit, a funny pastor, or a stand-up comic, is to get the word on what we know from science and, more importantly, how we know it. Without the scientific explanations, all we have is a bunch of folks trying to win you over with their sparkling rhetoric. With the scientific explanations, there is something like a rational basis to believe the rhetoric or decide it's full of crap. I know it's terribly old fashioned, but I like to make up my mind about things based on reason rather than punchy delivery (or, for that matter, political power). Part of what wins people over to science is that power, while not entirely absent, is greatly modulated by the fact that every scientist in the community is supposed to be accountable to the same world, and to the community of other scientists trying to figure out that world. Losing that reality-based character of the scientific enterprise would be a big mistake. Indeed, leaving the reality-based character of the scientific enterprise (not just the claim to have the backing of reality, but the demonstration of how the story has to fit with reality in specific ways or be abandoned) out of interactions with lay people would be a mistake, too. Otherwise, it really does devolve into a "we say, they say."

(For the record, I think some of the strategies Don Wise outlines in the abstract of his talk really involve engaging in scientific arguments rather than eschewing them for surface flash. But I don't want political considerations in these frustrating times to end up sucking all the goodness out of scientific interactions with non-scientists. So let's be careful out there!)

Technorati tags: intelligent design, scientific reasoning, political strategy

Do you want them to learn it in the schoolyard?

2005-10-21T13:38:00.000-07:00

I just got back from talking with an outside evaluator about the federally funded training grant project at my university that tries to get more of our students to graduate school in science. The evaluator is here not at the behest of the funding agency, but rather at the request of the science professor here who oversees the program. Because, you know, he wants to know how good a job we're doing at what we think we're doing, so we can make improvements if needed, and he figured an outside guy who has evaluated other such programs could give us some good insight here.

Let me pause to note that the folks who are this serious about making their efforts successful are a big part of why I love working here.

Anyway, I was on the agenda because I teach the ethics course the federal funding agency requires of students supported by training grants like this one. (That is to say, they require some course on research ethics; I developed the particular course they're taking.) We had an interesting discussion, the evaluator and I, about the genesis of the course, the enrollment, the syllabus, and such. And, in the course of this discussion, we arrived at one of the nagging worries I have about courses like this:

It is possible that learning ethics (even ethics-for-scientists) from a class in a philosophy department will have less of an impact on science students than learning ethics from their science professors would have.

Part of this, I'm afraid, is the curse of the required class. Kids hate required classes, even if what they're required to take has potential value for their lives down the road. Anyone who has taught a class with prerequisites has probably had experience with students who took a class they were required to take and promptly forgot almost all of it ... because check the transcript, you did that class. That shouldn't mean you need to waste valuable brain real estate remembering anything you learned.

And seriously, a class from a philosopher? What the heck does that have to do with learning to be a good scientist? (Set aside the fact that a science professor approached me to develop this course, and that one of the science departments here decided, without consulting me about it, to make this class a requirement for their majors, with at least one other science department thinking about following suit.)

In my case, I actually have some ammunition by way of my misspent scientific youth. Y'all are looking at going to grad school in a science? Been there, done that, wrote the dissertation, got a Ph.D. But, there are other schools where the science majors take their ethics from philosophy departments and the philosophers can't necessarily throw down so effectively.

There's still the worry that, if you put all the discussions of ethics in a one semester course over in some other department, you convey the distinct impression that: (1) thinking about these issues for a semester is sufficient, and/or (2) no one in your home department can teach you what you need to know about ethics, and/or (3) grown-up scientists don't actually need to pay attention to ethics. Obviously, I think all of these are misapprehensions. Indeed, the science professors I've talked to here are really good at highlighting responsible conduct of research (RCR) issues in all kinds of contexts. These professors understand that RCR is still relevant in their work, and they even seem to talk to each other about the best ways to conduct their research rather than letting things blow up and calling in the ethicists in Haz-Mat suits.

It has also been pointed out to me that a bunch of science professors would actually enjoy coming to the ethics class but they don't for fear that it would stifle the class discussions. Given how discussions in this class tend to be (brutally honest, with lots of critical examination of how things are done in real labs -- some good, and some bad), it's probably true that the presence of an authority figure from a science department would change the dynamic. So I guess this is a real advantage of offering the course in the philosophy department: students from different scientific disciplines get to discuss their experiences among different branches of the tribe of science on neutral turf.

Still, I can't help but thinking it would be better if there was some kind of forum for discussions of ethics back in the students' home departments -- a lunch group, a once a month seminar or group-meeting type thing, something. This would help the students understand that their professors really care about this stuff, too. And, it would give the faculty a regular channel for talking together about RCR issues -- because people seem to do better with ethical decisions when they can chew them over with a group.

I imagine I'll keep thinking of ways to optimize this. If my history here is any guide, the science professors I'll be looking to for help implementing my harebrained schemes will be receptive.

They provide a nice contrast to the chair of another department here, who showed up, frantic, at our department the other day. Their department failed to get accreditation because the course (in our department) they had been using as an ethics course didn't satisfy the acrediting agency. So, they wanted us (of course) to whip out a specialized ethics course that would satisfy the acrediting agency. Immediately. In trying to impress upon our department chair just how badly they needed us to solve this problem for them, the chair of the other department exclaimed, "We don't know anything about ethics!"

Sugar-dumpling, that's what scares me ...

It does seem to reinforce the notion that ethics instruction is valuable wherever you can find it. But is it worth $13K? You be the judge!

My blog is worth $13,548.96.
How much is your blog worth?

Technorati tags: RCR, ethics instruction

Tangled Bank #39 has arrived

2005-10-19T09:24:00.000-07:00

The 39th edition of The Tangled Bank is being hosted by The Questionable Authority. Go read about science and be enriched.

National Chemistry Week: women in chemistry?

2005-10-18T15:50:00.000-07:00

On Day 3 of National Chemistry Week 2005, I thought I'd poke gingerly at that perennial blogosphere question: where are the women?

In 2005, more of them are in chemistry departments than there were a decade or two ago. While some (who I won't link directly, because puh-leez!) would have you believe that women can't hack hard sciences like chemistry (and thus can't write hard SF), there are loads of women who manifestly can because they do. Some even have quantum lectures that have cracked the top 100 podcasts. (A woman chemist who blogs is a "where are the women" twofer, right?)

But if we want to talk numbers, we're not looking at anything close to parity. From the data the American Chemical Society has compiled, in 1989 women earned 38.7% of the U.S. bachelors degrees granted in chemistry and 25.7% of the Ph.D.s. (My entering class of more than 50 in the fall of 1989 was roughly a quarter female, which seems to jibe with these stats.) In 1999, women were up to 45.5% of the bachelors and 29.7% of the Ph.D.s. In other words, even though it feels to me like there are significantly more chicks with Ph.D.s in chemistry than when I was in school, it's not actually that big an increase. The ACS Women Chemists 2000 report that about 43% of the women with chemistry Ph.D.s work in academia, with proportionately more teaching at schools granting AAs, BSs and MSs than at schools that grant Ph.D.s or at medical schools. More male chemistry professors with tenure than female, but a closer to even male-to-female ratio among the younger professors (assistant and associate level) than the older.

It looks like things are improving. On the down side, the academic job market is relatively tight, and there's more of an expectation that one will line up a string of postdocs and/or visiting assistant professorships between graduate school and the first "real" academic appointment. This is hard on everyone, male or female. But if you want to do something crazy like have kids, it can make it much harder not to exit the pipeline. In chemistry, this isn't just a matter of temperament. There are a great many experimental systems with which one ought not to interact if one is trying to gestate a healthy human. It's one thing if you're doing computational chemistry, but if you're doing organometallic syntheses ... let's just say, you're not working up till your due date.

People have gone round and round about whether, from the point of view of the science produced, it really makes much difference if the proportion of female chemists is high or low. I think it's an empirical question. But, in the meantime, the proportion of woman can matter quite a lot in terms of how many females get into chemistry in the first place. (I don't just mean get into it as a field of study or career, but get into it as a way of thinking they enjoy and are good at.) Dealing with male professors who let you know that they don't expect females to be any good at the science they are teaching can put you off that science right quick -- not because you think they're right, but because it's tiring dealing with that bullsh*t on a regular basis. Studying chemistry can be tons of fun, but it's hard sometimes to consider joining the professoriate if there's no one in their ranks who looks like you. Seeing more chemistry professors, male and female, who are working out ways to balance research and teaching with "outside" commitments like family would be a good thing for the profession as a whole. These female chemists say a lot of useful things about how we should rethink our comic book ideals about what it means to be a good chemist. I don't agree with some of the sweeping generalizations (women being more cooperative and such -- it's not an essential trait, dude), but it does seem like there are many styles and skills that would build a healthier community of chemistry.

We're not there yet, but I think there's movement in the right direction.

Edited to add: The Chemical Heritage Foundation had a traveling exhibition about women's contributions to chemistry. It traveled to science fairs and such, to get the word out to impressionable young minds that women have always been among those able to hack chemistry.

Technorati tags:
national chemistry week,
women chemists

Students as a vulnerable population.

2005-10-17T11:57:00.000-07:00

I just read Pat Shipman's article in the latest issue of American Scientist, in which Shipman cautions scientists not to be complacent about the Intelligent Design vs. Evolution brouhaha. Shipman makes a case that many scientists have made (especially in the blogosphere), but a few piece of this article really jumped out at me. The thing that connects them is the idea that children are being put at risk.

Speaking about the Dover case, Shipman lays out the standard reasons to take Intelligent Design as "scientifically unimportant," but then notes:

I might have settled back into complacency had I not learned that students in the public high school in my town—a town dominated by a major university—can "opt out" of learning about evolution if their parents send a letter to the school. Allowing students to "opt out" of learning the basic facts and theories of biology is about as wise as allowing them to "opt out" of algebra or English: It constitutes malfeasance.

Even if parents don't ask that Billy and Susie be excused from learning about evolution, they might not get to learn about it:

In at least 40 states, ID is being considered as an addition to the required science curriculum in public schools. This year a poll by the National Science Teachers Association showed that one-third of science teachers feel pressured to include ID, creationism or other "nonscientific alternatives" in their science classrooms. Some teachers are so intimidated by the threat of parental complaints that they skip material dealing with evolution in their classes.

In other words, what's on the menu in science class is being shaped in a significant way by public opinion rather than, say, by sound principles of science pedagogy. If there were a public backlash against the Law of Sines, would it be proper for Trigonometry teachers to drop it quietly from the curriculum? (Given current rates of "non-standard" punctuation among the general population, should grammar texts be consigned to the flames and graders wielding red pens be told to take a flying leap?)

Also in this article, Shipman brings up the case of Bryan Leonard, doctoral candidate in science education at Ohio State University. You'll probably recall that Leonard's Ph.D. defense was put on hold (by his advisor) because his committee wasn't properly constituted according to university policy, given that it lacked a faculty member with expertise in science education. This wouldn't have been news, except that Leonard testified as an expert witness at the evolution hearing held by the Kansas Board of Education. Leonard, a high school teacher, apparently wrote his dissertation on the research question:

When students are taught the scientific data both supporting and challenging macroevolution, do they maintain or change their beliefs over time? What empirical, cognitive and/or social factors influence students' beliefs?

One might quibble with the precise wording of the question (especially given that the consensus view among biologists is that there aren't any scientific data that challenge macroevolution). Setting that aside, there is an interesting question here about what makes a scientific theory believable to a high school student. Knowing the answer to a question like this might have all sorts of useful implications for efforts to improve science education. But, if how you get at the answer is by "teaching" kids something that isn't science as if it were science, then again we're talking about actions that constitute malfeasance. As put by three OSU professors in a letter to the dean of the graduate school (quoted by Shipman):

There are no valid scientific data challenging macroevolution. Mr. Leonard has been misinforming his students if he teaches them otherwise. His dissertation presents evidence that he has succeeded in persuading high school students to reject this fundamental principle of biology. As such, it involves deliberate miseducation of these students, a practice that we regard as unethical.

And, the professors asked the logical question: where was the IRB in this research involving human subjects?

Here's what the government has to say about research with children:

The issue of children as research subjects is a complex one since they are not considered able to make informed choices independently. Further, exposure of children, particularly healthy children, to more than minimal risks must be weighed carefully.

When including children in research, the role of the family should be considered in devising the protocol as well as in obtaining informed consent from the parents or guardians. If the research is based in schools, appropriate involvement and permission must be obtained from the school. Adequate measures must be developed to protect children's privacy and to ensure that their participation does not stigmatize them in the present or future.

The key question here is what the risks are to children who participate in the research, and how these compare to the potential benefits to the subject of participation.

One risk, it seems, it that the students could get an utterly misleading view of what science knows, and, more importantly, of how science works. Potentially, this could interfere with the students' ability to learn in subsequent science courses. (It might also discourage these students from pursuing further coursework in science.) Bad preparation in science could have a direct effect on the courses of study available to the college-bound students, and could impare the ability of the students to develop a sufficiently good understanding of scientific reasoning not to fall prey to all manner of quacks and charlatans (go ask Orac -- Orac knows). These outcomes are all too common given the current baseline of high school science education. It seems clear that the risks would be much greater if science education was actually designed to mislead in the service of answering Mr. Leonard's research question.

(Besides, in the rare cases when deception is part of an approved research protocol, afterwards the subjects are supposed to be told that they have been deceived, as well as why the deception was necessary to answer the research question. Did Mr. Leonard's research protocol do this?)

The overarching question here is what secondary education is supposed to do for (and to) the children who receive it, and whether certain ways of delivering it (including ID, leaving out evolution, misleading about the state of scientific knowledge or about the process of reasoning that produces that knowledge) undercut those aims. This is an important question to answer given that children are a vulnerable population. Their ability to make informed choices independently is not yet fully developed, and indeed education is supposed to help them develop this ability. Screwing around with this puts childrens' long-term well being at risk.

It might be objected that the parents should actually be the ones with the primary, or sole, responsibility for helping their children develop the ability to make informed choices independently. Unfortunately, some parents may not be able to do this, and others may actually view it as contrary to their own interests to make independent thinkers of their kids. (Certain teenagers make this view understandable.) Given that we have public schools (for a little while more, anyway), this seems to reflect a commitment to the idea that there are certain things children ought to learn, not only because it is good for them, but also because it is good for society as a whole. We owe the children a certain basic education, and we're better off as a society if they all get it. Similarly, we owe children certain basic kinds of protection regardless of what their parents might think about it. Is "excusing" your child from learning about evolutionary theory on the basis of religious objections ethically equivalent to denying your child a live-saving blood transfusion on the basis of religious objections? Perhaps not. But, I'm not sure it's as different as some science-semi-literate parents think it is.

Technorati tags: human subjects, ID

National Chemistry Week is here.

2005-10-16T21:30:00.000-07:00

Today is the first day of National Chemistry Week 2005. And, even though Bill Carroll, the current president of the American Chemical Society, is keeping a blog of his "Extreme Tour" in celebration of National Chemistry Week, I figured I should blog about it, too.

What's so great about chemistry? Of course, if you're a kid, chemistry has the allure of magic -- something might explode! (For those averse to permanent damage, there are plenty of cool chemistry activities that are much safer than whatever my brother did with his store-bought chemistry set to scorch the hell out of our parents' card table.) But I suspect it's real charm for students, at least when it's taught right, is that it's a science that looks for the "whys" pretty early in the game. In general, introductory chemistry doesn't involve much memorization (whether of equations, as in physics, or of Linnaean taxonomy, cell organelles, phases of mitosis, or any of the other important details one has to remember in a biology class). Rather, you learn how to use the Periodic Table almost like a decoder ring to figure out why various substances behave the way they do. From the very beginning, the chemistry student is thinking not just in terms of facts, but in terms of rationalizing those facts. For every weird exception you learn to a regular pattern, the challenge is to understand why it breaks the pattern.

In this chemical universe the student enters, things start to make sense in a way that everyday life hardly ever does. It can be downright seductive. But of course, the orderly chemical universe to which the student is exposed is the product of much labor in laboratories. What happens in the labs can seem chaotic rather than orderly, and sometimes it is only the determination of the chemists to find the underlying order that keeps the going back to the bench to tame the chaos. Needless to say, finding the order in chaos can be seductive, too.

While chemistry often gets props for being a practical subject to pursue (where "practical" usualy means leading to gainful employment, and the contrast class is something like philosophy), a lot of the people I know who went into chemistry were led by their hearts more than their heads. Chemistry just felt like the right way to engage with the world.

Primo Levi expressed this as well as anyone else has. Writing about his experiences as a chemistry student in Italy during the rise of Fascism on the eve of World War II, he said he felt

That the nobility of Man, acquired in a hundred centuries of trial and error, lay in making himself the conquerer of matter, and that I had enrolled in chemistry because I wanted to maintain faithful to that nobility. That conquering matter is to understand it, and understanding matter is necessary to understanding the universe and ourselves: and that therefore Mendeleev's Periodic Table, which just during those weeks we were laboriously learning to unravel, was poetry, loftier and more solemn than all the poetry we had swallowed doen in liceo; and come to think of it, it even rhymed! ...

[T]he chemistry and physics on which we fed, besides being in themselves nourishments vital in themselves, were the antidotes to Fascism ... because they were clear and distinct and verifiable at every step, and not a tissue of lies and emptiness like the radio and newspapers

(The Periodic Table, pp. 45-46.)

Why does it choke me up to see Levi want to conquer matter by understanding it, or to see that his motivation to understand matter is a desire to understand the universe and himself? Coming at a science like this, you can see why a couple centuries ago it was called natural philosophy. As nuts and bolts as the work of a chemist can be -- and Levi was for most of his career a chemist who took on problems in different industrial labs, including an IG-Farben lab while he was a prisoner at Auschwitz -- the drive here is to understand the substance of reality, to get at knowledge we can be sure of and can hold in common with others. Wanting something like this -- to understand of the universe we're in and how we fit into it, to share our experience with our fellow human beings -- feels like the most human of impulses. Science is not the show-offy acting out of the maladjusted braniac, but the labor of the human spirit.

Maybe if more of that got across to science students, and to the public at large, cultivating scientific literacy wouldn't seem so much like taking a dose of castor oil.

Technorati tags: national chemistry week, Primo Levi, scientific literacy

Science in crisis (?)

2005-10-14T10:31:00.000-07:00

Today, Inside Higher Ed has a story about a crisis in the training of new scientists, engineers, and mathematicians that I swear I've seen at least half a dozen times in the last dozen years. (And, given that for large periods of time within those dozen years I was paying more attention to research and thesis-writing than I was to the world, it's likely that I missed a few iterations.) The crisis is that, while in the last decade U.S. enrollments in science, technology, engineering, and mathematics (or STEM) have increased at the bachelors and masters level, doctoral programs in these fields have seen a decrease in enrollment. So, we have more students staying in the science pipeline longer than a decade ago (i.e., not leaking out in high school or college), but fewer are making it to the end of the pipeline and a Ph.D.

The first question to ask is whether this constitutes a real crisis (which means figuring out what would make it a crisis). Then, if it is a crisis, we'd need to figure out what to do about it.

Since I was an undergraduate a loooong time ago, I have heard doom-and-gloom stories about critical shortages of science Ph.D.s in the U.S. Of course, while in graduate school in chemistry I found out that U.S. universities were turning out something like 30% more chemistry Ph.D.s than the U.S. market could handle. So I confess to being a bit skeptical about the target numbers of science Ph.D.s certain folks think we ought to be reaching. Is this one of those situations where we're not actually striving to reach the optimal number of Ph.D. scientists to fully staff a healthy and active scientific community but rather pursuing growth in output for growth's sake? Does our economy depend on an every increasing production of Ph.D. scientists? (What's the futures market like on string theorists?) Is this just one more number we'd like to be able to hold up to compare ourselves favorably to Germany, or China, or India?

I hope not; it would be a silly reason to make more people suffer through the slings and arrows of a doctoral program. But there may well be good reasons. While people with bachelors and masters degrees in the sciences (and engineering and math, of course -- assume they're included; even economics) can find work in research labs, it's generally the folks with the Ph.D.s (or M.D.s) who are driving the original research. Maybe this is the cost of producing fewer Ph.D.s and more B.S.s and M.S.s: we move away from discovery, innovation, and deeper understanding and toward just being technicians. The market can probably absorb a lot more technicians than PIs, but leaving the original research to other countries that are better at producing large numbers of Ph.D. scientists might come back to bite us -- especially if there are skilled technicians who can be had for cheaper than those trained in the U.S.

And it's not clear that the costs will be primarily economic ones. Sure, with fewer Ph.D.s we may have a harder time supporting a thriving biotech industry, or attracting international students to our research universities (where they often inject cash into the system that their American classmates do not), or winning Nobel Prizes at the rate to which we've become accustomed. But who exactly will teach the increased number of students in the sciences at the bachelors and masters level when our Ph.D. output falls below the replacement rate?

(Given the large number of Ph.D. scientists looking for permanent positions, many with multiple post-docs under their belts, this does seem rather far fetched. I'm just reaching for a plausible explanation of the "crisis" here.)

The big distinction between Ph.D.s and scientists with lower levels of education -- that Ph.D.s are the ones who "really" do research -- might point to another reason to be concerned about our progress in increasing science enrollments at the lower levels. In many instances, centers of scientific research place all the emphasis on research, to the exclusion of attention to teaaching, public outreach, and other potentially useful functions a scientist could perform. (I'm not making this up: Sean Carroll talks about it here.) This already has an impact on all those undergraduate science majors. I would never let me children be undergraduate chemistry majors in my graduate chemistry department. The level of instruction from the professors (those guys with the Ph.D.s) was, with a few exceptions, pretty dismal. Occasionally, some good instruction could be had from a TA whose graduate advisor had not yet impressed on him or her that time spent on teaching was lost forever to what really mattered -- research. And, all to often undergraduate research experiences were supervised by graduate students rather than the professors. So, lots of those people getting B.S.s at the high powered research universities may know a lot less than they ought to about their scientific field and about research in it.

And, this is less than ideal if we want a population that actually understands something about science. Once you're out of school, your facility cranking out problem sets doesn't do much for you. It would be much more useful to have a grasp of how science tackles problems (problems whose solutions are not yet in the back of the book!), how science uses the tools it has and develops new tools, and how scientific patterns of thought set up conditions where we really can build a body of knowledge that we can count on (in part because many pairs of hands and eyes scrutinize it an continually update it). You may not need this kind of understanding of science to be an adequate technician (although it can be helpful when unexpected outcomes present themselves). On the other hand, if all you have is mad technical skillz, you may be replaced with a robot someday.

I think the crisis I'm feeling with science has less to do with the numbers game and more to do with recognizing that the value of the Ph.D. scientist is and ought to be more than a machine to produce original research. Yet, in many places, this is how scientists (especially scientists trying to build tenure cases) are regarded. When the rational choice is to shut out the rest of the world so you can do your research and get your high-impact publications, it makes it harder for people outside the system to get value out of the science Ph.D. Unless teaching is recognized as valuable too, the people who might be able to teach us the most about what science is up to right now will have other things they have to do. (And is it a surprise that advisors who haven't given much thought to classroom teaching sometimes have serious difficulties teaching their advisees in the lab?)

This isn't a trivial worry. If Ph.D. scientists are isolated from everything but their research, it becomes easier to marginalize them -- to assume that what they have to say when they make their rare appearances in broader discourses just doesn't matter. We let the public discourse write off science at our peril. So, it seems like maybe we need to start cultivating our scientists as communicators -- not just in journal articles, but in the public square. And we can't really cultivate it without making it count. Research universities ought to recognize that building public interest in and understanding of science is, in the long run, good for the health of the research university and good for the health of science education.

Having more science Ph.D.s might be a good thing, but doing more and better things with the science Ph.D.s we have might be even better.

Technorati tags:
science pipeline,
science+education,
tenure

Skeptics ahoy!

2005-10-13T12:10:00.000-07:00

The 19th Skeptics' Circle is up at Time to Lean. Check out the fairway, gawk at the oddities, and don't forget to have a funnelcake before you head home.

Impact factories.

2005-10-12T15:37:00.000-07:00

Via Crooked Timber, a story in the Chronicle of Higher Education about how the impact factor may be creating problems rather than solving them.

The impact factor is supposed to be a way to measure the importance of a journal, and of the articles published in that journal, in the great body of scientific research. To compute the impact factor for Journal X for a given year, you divide the number of citations in that year of articles published in Journal X in the two preceding years. Then, you divide that by the total number of articles published in Journal X in the two preceding years. What does this give you? A measure of how important other researchers in the field think the articles in Journal X are (since, the thinking goes, you cite articles that are important, and articles that are important get cited). Because we're looking at a ratio, we get a sense of the proportion of important articles in a journal rather than just the raw number of cited articles. So, if in 2005 there are 100 citations of articles published in 2003 and 2004 in PZ's Journal of Squid Canoodling, and 100 citations of articles published in 2003 and 2004 in Deep Thoughts from the Discovery Institute, but PZ J. Sq. Can. published 1000 articles in 2003-2004 while DTDI published only 300 in that time, the impact factor of PZ J. Sq. Can. is 0.1, while the impact factor of DTDI is 0.33.

As scientists at prestigious research universities know all too well, the impact factor is a way for tenure committees and granting agencies to judge how good your publication record really is. Instead of simply counting your publications, folks can also look at the impact factor of the journals in which you've published. This could be a helpful thing if one has a small number of publications but they're in widely cited journals. Similarly, it could bite you in the butt if your articles happen to be in journals that hardly get cited at all.

But, as seems to be the case any time you represent something like the importance of a journal (or of your scholarship) to a number, there are ways the impact factor doesn't tell you all you might want or need to know. For one thing, given that it's often useful to cite review articles, journals that publish lots of review articles get cited more, thus raising their impact factor. This doesn't tell you much about the impact of those journals' articles on original research. Also, remember that the impact number is calculated based on citations of articles from the preceding two years. It sometimes happens that really important results, for whatever reason, are not recognized (and cited) that quickly. So a three-year-old article that is cited like crazy will do nothing for the impact factor of the journal it's in. And of course, "little" journals that focus on fairly specialized scientific subfields have a much harder time getting high impact factors simply because there are fewer scientists who work in these subfields to cite the articles. (If these journals keep a really tight reign on the number of articles they publish, they could offset the low number of articles cited. But, this isn't necessarily what you want to do in a little subfield that is just taking off -- building the literature seems like a more natural impulse.)

Also (as the "journals" in my example above should suggest), sometimes articles are cited a lot to be made fun of.

The Crooked Timber discussion of the article talks about ways journal editors might try to game the system, raising some legitimate worries. Any place you are clear on the selection criterion, you can usually figure out multiple strategies to satisfy it. For example:

The gaming has grown so intense that some journal editors are violating ethical standards to draw more citations to their publications, say scientists. John M. Drake, a postdoctoral researcher at the National Center for Ecological Analysis and Synthesis, at the University of California at Santa Barbara, sent a manuscript to the Journal of Applied Ecology and received this e-mail response from an editor: "I should like you to look at some recent issues of the Journal of Applied Ecology and add citations to any relevant papers you might find. This helps our authors by drawing attention to their work, and also adds internal integrity to the Journal's themes."

Because the manuscript had not yet been accepted, the request borders on extortion, Mr. Drake says, even if it weren't meant that way. Authors may feel that they have to comply in order to get their papers published. "That's an abuse of editorial power," he says, "because of the apparent potential for extortion."

Robert P. Freckleton, a research fellow at the University of Oxford who is the journal editor who sent the message to Mr. Drake, says he never intended the request to be read as a requirement. "I'd be upset if people read it that way," he says. "That's kind of a generic line we use. We understand most authors don't actually do that." He changed the wording in the form letter last week to clear up misunderstandings, he said.

The benign reading of the editorial suggestion to add citations is, "Hey, there's other work out there, which you might not have noticed, that bears on yours in an interesting way -- have a look!" And, sure, who can argue against keeping up with the literature that relates to your own work? It might have been more persuasively above-board had the literature-to-look-at list included articles from other journals (in a Macy's sends you to Gimbel's kind of move), but you can't blame an editor for knowing his own journal best. The less benign reading is that the editors are taking active steps to artificially boost their journals' impact factors.

What gets lost in all this gamesmanship is the idea that scientific work ought to be evaluated on its own merits. Peer reviewers are supposed to be assessing the soundness of the science, not the sexiness of the finding. Sure, the sheer number of scientific journals in most fields makes it hard for any mortal to "keep up with the literature", which means that scientists look for quick ways to locate the papers most likely to be important. But the quick ways may not be the most reliable ways. And, depending on how sensitive peer review decisions are to impact factor gamesmanship, it is conceivable that things could reach a point where being published in a high impact journal has less to do with the soundness of your science than with the fashionability of your findings. At the extreme, scientists might have to spend a lot more time replicating published results, and might spend quite a bit of time ignorant of important findings that are published in out of the way places, or are waiting in the queue at the journals with high impact factors. And that would suck.

Technorati tags: impact factor, scientific literature

Art appreciation.

2005-10-11T14:47:00.000-07:00

This is an unsolicited piece of artwork from my offspring, apparently in support of teaching the controversy. I am unsure why the meatballs in this illustration are so large; perhaps it has some bearing on the progress in the Dover case.

Meanwhile, I suspect that the insects in the picture are subliminal advertizing for the Circus of the Spineless but am unable to confirm that money changed hands.

Truth in advertizing (university Nobel laureate edition).

2005-10-11T13:13:00.000-07:00

In yesterday's Los Angeles Times there's an interesting piece on how universities count "their" Nobel laureates. It goes without saying that this is not silent, internal, beaming with pride at the accomplishment of someone dear to us counting. Rather, we're talking about Nobel counts that get put out in university communications with the world.

First, what's going on with the counting?

There seem not to be uniform, agreed upon standards for identifying which institution gets to claim a Nobel laureate. Some, like UC Santa Barbara, are fairly strict, laying claim only to professors who won their Nobels while at UCSB and who are still active members of the UCSB faculty. Other schools, like the University of Chicago, claim as theirs Nobel laureates who were students at Chicago, or did research at Chicago, or are past or current members of the faculty. There's probably an argument of the "it takes a village" variety that could be made to support this kind of practice … but it seems like actually making an explicit argument about why you count Bobby Braniac as your Nobel laureate if he did two years of graduate school with you before transferring out and winning a Nobel a dozen years later might just serve to draw attention to the fact that your laureate count is inflated. (If you can legitimately claim a 15% contribution to Braniac's scientific trajectory, does that make him only a 0.15 laureate for your count? What does that do to the total number of Nobel laureates you can legitimately report?)

From the linked article:

There is good reason for ambiguity in the accounting.

"A university does many things," said David J. Gross, a UC Santa Barbara professor who won the Nobel Prize for physics in 2004. "It teaches, so it's proud of its students who went on to do good things. They're proud of their researchers who worked at the institution who have done good things. And of course they're proud of the people who are there now and their impact on current research and current teaching."

So one school might claim a Nobel laureate who was there as an undergraduate, another for graduate work, another for advanced research, and several for being on the faculty. Who can say which school is most worthy?

Caltech President David Baltimore, whose 1975 Nobel Prize for physiology or medicine is claimed by MIT, Rockefeller University and Caltech, sees no need to attempt an answer.

"It is sort of a game, and you might as well play it by whatever rules you want, like solitaire," he said.

(Bold emphasis added.)

Is it really OK for universities to count Nobel laureates by any rules they want?

To answer that, we need to understand why Nobel laureate counts are supposed to matter. Again, quoting from the article:

Nobel Prizes make schools attractive to prospective students, faculty and donors, conferring the aura of a winner. A university's roster of laureates is "probably more significant than [the college rankings in] U.S. News and World Report," said F. Sherwood Rowland, a UC Irvine professor who won the Nobel Prize for chemistry in 1995.

The fact that a school has one or more Nobel laureates is supposed to tell you something about that school. Just what it's supposed to tell you is a bit nebulous. If a university has a laureate who won the Nobel based on research done at that university, it tells you that high-powered research happens there. If a university has laureates who are still active in research, it tells you there may be opportunities for graduate students to learn from them. (Students might also learn from these luminaries in the classroom, but that's not clear from the mere presence of Nobel laureates on the faculty.) If a university was where a laureate completed his or her bachelor's degree, it tells you that an undergraduate education there is not sufficient to put people off research.

But closer inspection can tell you some other important things about a university. Some universities are the places laureates did the research recognized with Nobel prizes, while others tend to make senior hires of Nobel laureates. This tells you something not only about research conditions at the different kinds of schools but also about the philosophy for building excellent faculties (grow them yourself vs. buy them already famous). How many of a university's claimed laureates were, say, denied tenure while in the midst of their ground-breaking research? This says something about the university's receptiveness to cutting-edge ideas as well as shining a light on its standards for tenure.

Universities are reaching different audiences when they announce their Nobel body counts: students, faculty, administrators, alumni and other potential donors, funding agencies contemplating the feasibility of new research at a given institution, prospective students (and their tuition-payers), and propsective employees. For certain purposes (like securing donations for a new science building), just rolling off the list of your Nobel laureates may do the job of giving the donors warm fuzzies about the instituion. But from the point of view of presenting prospective students with an accurate picture of how their experience will be enhanced, certain ways of counting your Nobel laureates seem less than informative, if not downright deceptive. High school senior Carly Cranium might be better off knowing whether this university is one that has provided an excellent undergraduate education for Nobel laureates, or whether courses in her intended major will be taught by a Nobel laureate, or whether she'll have an opportunity to do research in a lab where prize winning research was done. Otherwise, the number of Nobel laureates ends up being a statistic with as much meaning to the prospective student as the number of acres the campus occupies and the percent of alumni who give money to the endowment.

For early-career scientists picking a suitable environment (for research, teaching, and maintaining a baseline level of sanity), it seems like the laureate count is meaningless without further information. It's nice to know a university will think fondly of you once you've won fame and fortune, but it's better still to know whether a place will nurture you before the world knows you're a star.

Technorati tags: Nobel laureates, honest accounting, university climate

When unfalsifiability is your business plan.

2005-10-10T08:52:00.000-07:00

This is a follow-up to my discussion last month of the Acu-Gen Baby Gender Mentor test, prompted by the report today on Morning Edition that authorities in Illinois are investigating the company marketing the test to determine whether the claims made to consumers in marketing the test rise to the level of consumer fraud.

So, for those just tuning in, the deal is that this test promises an accurate determination (99.9% accurate, if you want the numbers) of fetal gender, as early as 5 weeks into a pregnancy, from only a few drops of the mother's blood. And, they promise a 200% refund if the test is wrong.

Set aside, for the moment, concerns about whether there's good scientific evidence that this test could be so accurate. (That was the subject of the last entry.) Cast your gaze, for a moment, on what seems to be the standard operating procedure when a consumer tries to get a refund for an inaccurate test result.

Scenario 1: Baby Gender Mentor test says boy, but the sonogram says girl. The lab does a retest. If the retest still says "boy", the consumer is told that sonograms give inaccurate gender identification 20% of the time. Still a chance Baby Gender Mentor will be right, so no refund yet.

Scenario 2: Baby Gender Mentor test says boy, but amniocentesis says girl. The lab does a retest. If the retest still says "boy", the consumer is told that the pregnancy began with a set of fraternal twins, one boy and one girl, and that the boy was a "vanishing twin". No refund, even though the boy Baby Gender Mentor detected vanished.

Scenario 3: Sonogram indicates a single fetus, after which Baby Gender Mentor test says boy and amniocentesis says girl. The lab does a retest. If the retest still says "boy", the consumer is told that there was a vanishing boy twin, and that the sonogram that indicated that it was a single fetus is no proof of anything, since sonograms give inaccurate gender identification 20% of the time. No refund, because you can't prove Baby Gender Mentor was wrong!

Do you see the pattern here?

It is true, of course, that sonograms don't always give enough information to make an accurate determination, either of gender or of how many fetuses are present. (A woman of my acquaintance discovered, in her eighth month of pregnancy, that she was expecting twins — on the fifth sonogram.) But the great part here for the folks selling Baby Gender Mentor is they've got an excuse worked out for any mismatch between their test results and what other diagnostic tools (including visual inspection of the newborn) indicate. No, they aren't producing independent evidence that there was a vanishing twin, but since it could have happened, that's enough for them to say that their test result hasn't been falsified. And that, it seems, means that they'll never have to act on their double-your-money-back pledge.

With a business plan like that, maybe it's time to branch out from biotech to religion.

Technorati tags: biotech, unsupported claims, falsifiability

Getting excited about science: the 2005 Ig Nobel prizes.

2005-10-08T19:50:00.000-07:00

On Thursday, they awarded the 2005 Ig Nobel Prizes. They don't carry the same cachet as those other Nobel prizes, nor the same hefty cash awards, but the Ig Nobels are often awarded for scientific findings everyday folk can wrap their minds around.

For example, the Ig Nobel in economics went to the inventor of "clocky", an alarm clock that hides so you can't hit the snooze. Who can't appreciate an invention like that? The Ig Nobel in chemistry went to a pair of engineers who studied whether people swim faster or slower in syrup relative to water. (Turns out to be a wash because of the trade-off between drag and leverage.) And, a prize in fluid dynamics was awarded for calculation, from physical principles, of the pressure built up inside penguins before they defecate. It's really the untold story behind March of the Penguins.

Some of the recognized research, I think, is much easier for a scientist to appreciate than for the lay person. The winner of the prize in physics, one of whom was alive to receive the prize, had been conducting an experiment following drops of pitch as they dripped through a funnel -- since 1927. Since the pitch drops fell at the rate of about 1 every nine years, that means that in the 78 years they ran the experiment there were 8 or 9 drops. How's that for careful empiricism? And, if 8 or 9 data points seems too sparse for your tastes, there's the winner in nutrition who photographed and retrospectively analyzed every meal he has eaten for the last 34 years. You'd think the efforts of these scientists could at least make graduate students feel better ... but I'm not sure folks who haven't done the time to try to get an experiment to work, or to get enough data to get meaningful results, would be sufficiently impressed.

Also, a few of the Ig Nobel prizes usually go to projects that, uh, don't seem to yield much in the way of new knowledge. The stand-out this year is the prize in medicine, which was awarded to the inventor of neuticles. I'm not saying that a dog might not retain his self-esteem, post-neutering, better with neuticles than without ... but as far as I can tell there wasn't much canine self-esteem research. Yes, different sizes and levels of firmness offer the consumer/pet guardian lots of choices ... but this is a medical achievement? If you ask me, it just doesn't rise to the same level as watching pitch drops for 78 years. (This year may just be a fluke. The 2004 Ig Nobel Prize in medicine went to research on the effect of country music on suicide.)

I'm hopeful that schoolchildren hearing newsreports about some of these winners will be inspired to pursue science, engineering, and medicine in the hopes of snagging one of these prizes for themselves some day. In my heart of hearts, though, I fear they'll be more inspired by the winners of this year's Ig Nobel Prize for literature:

The Internet entrepreneurs of Nigeria, for creating and then using e-mail to distribute a bold series of short stories, thus introducing millions of readers to a cast of rich characters -- General Sani Abacha, Mrs. Mariam Sanni Abacha, Barrister Jon A Mbeki Esq., and others -- each of whom requires just a small amount of expense money so as to obtain access to the great wealth to which they are entitled and which they would like to share with the kind person who assists them.

Why do the poets get more respect than the scientists?

Technorati tag: Ig Nobel prizes

Getting philosophical, getting committed.

2005-10-06T09:17:00.000-07:00

There's something about the ongoing evolution versus intelligent fisticuffs that's been festering with me. It's one of the criticisms that's been leveled at evolutionary theory by folks like Phillip Johnson: the claim that evolution is a philosophical theory. Here's the claim, in context, as presented by a student newspaper at the University of New Mexico covering a talk Johnson gave there:

Johnson said the theory of evolution, or any theories like it, will not survive the 21st century because evolution is a philosophical theory.

He went on to say that one of the major flaws of the theory of evolution is that it excludes the possibility of divine intervention within the creation of living organisms.

“What we have is a theory that supports a moral view that nature is all there is and God is completely out of the picture,” Johnson said.

He said that one of the reasons God is left out of the theory is because scientists are either atheists or very liberal about religion.

Johnson’s speech concluded on the proposal that students should be taught a variety of theories regarding the way life is started — not just evolution.

Now, I know there's a long history of trash-talking various moments in the history of science by saying they look more like philosophy than science. Thomas Kuhn noted that a sure sign that your paradigm is in trouble is that the discussion gets philosophical. He wrote:

It is, I think, particularly in periods of acknowledged crisis that scientists have turned to philosophical analysis as a device for unlocking the riddles of their field. Scientists have not generally needed or wanted to be philosophers. Indeed, normal science usually holds creative philosophy at arm's length, and probably for good reasons. (Structure of Scientific Revolutions, p. 88)

Closer to home, I know what it's like to call one's mother and tell her her child is leaving a perfectly reputable scientific field to become a philosopher. I get that dropping the phi-bomb on a scientific theory is intended to damage reputations and hurt feelings.

But can we pull back for a moment to look at what the content of the slur is supposed to be?

PZ Myers responds (in part) to Johnson and his posse of trash-talkers this way:

You could also claim that Christianity, capitalism, and democracy are "philosophical theories"—that doesn't imply at all that they are going to expire. Evolution is not speculation and faith and guesswork, there is evidence…and what evolution tries to do is explain the evidence.

While I'm grateful for the assurance that philosophical theories aren't about to be yanked off the shelf like expired milk, PZ is gesturing towards a line one should draw that separates evolutionary theory from "philosophical theories" like Christianity, capitalism, and democracy. Johnson seems to be recognizing the same line, but disagreeing about what side evolutionary theory is on. PZ suggests that the "philosophical" side is where you'll find the theories based on speculation, faith, and guesswork. Johnson (as portrayed in the linked article -- even given his track record, I'm hesitant, given experiences with the school paper here, to assume the student paper at UNM necessarily got it right) seems to be saying "philosophical theories" are the ones that use their metaphysical commitments to support certain moral views and undermine others.

So, is the ideal supposed to be that scientific theories are utterly and completely free of philosophy? May I gently remind my scientific friends that, in the medieval university, we'd all be in the same department (or at least, on the same hall)?

Of course scientific theories bring some philosophy with them. You think the data we collect today can help us make good predictions about what will happen tomorrow? That reflects a metaphysical commitment you have about what kind of universe you're living in. And there's nothing wrong with having that commitment. Indeed, it's what helps some of us get out of bed in the morning. You want to show me the analysis that shows your results are statistically significant? Fine, but don't forget that the claim of statistical significance rests on metaphysical commitments about the normal distribution of data in the bit of the world you're studying. If you didn't start with some metaphysical hunches, there would be no way to do any science.

But, clearly, there is a difference between doing this and jumping into a "philosophical theory" of the sort Johnson and PZ seem to have in mind. And here, let me be the millionth person to point out that there is an important distinction between what one takes up as a methodological strategy and what one takes on as a metaphysical commitment. To Johnson, the fact that God is not mentioned anywhere in evolutionary theory is equivalent to biologists saying they're committed to the non-existence of God. To biologists, on the other hand, the non-mention of God reflects a methodological commitment to explain phenomena in the natural world by pointing to natural causes. Saying, "I'm only going to accept causes of types X, Y, and Z in explanations of this sort of phenomena" is not the same as saying, "There's nothing there but causes of types X, Y, and Z." If, as I pour a flask of water on a spoonful of table salt, I dance the tarantella, it would be silly to accuse the chemist, who explains why the salt dissolved by pointing to the structure of the salt and the structure of the water, of denying the existence of the tarantella. Clearly, the tarantella exists, but the chemist doesn't need it to explain why the salt dissolved.

(Occam's razor? Also philosophical. Don't let it freak you out.)

The deal with science — the thing that makes it different from some "philosophical theories" you might worry about — it that there's a serious attempt to do the job of describing, explaining, and manipulating the universe with a relatively lean set of metaphysical commitments, and to keep many of the commitments methodological. If you're in the business of using information from the observables, there are many junctures where the evidence is not going to tell you for certain whether P is true or not-P is true. There has to be a sensible way to deal with, or to bracket, the question of P so that science doesn't grind to a halt while you wait around for more evidence. Encounter a phenomenon that you're not sure is explainable in terms of any of the theories or data you have at the ready? You can respond by throwing your hands up and hypothesizing, "A wizard did it!" , or you can dig in and see whether further investigation of the phenomenon will yield an explanation. Sometimes it does, and sometimes it doesn't. In cases where it does not, science is still driven by a commitment to build an explanation in terms of stuff in the natural world, despite the fact that we may have to reframe our understanding of that natural world in fairly significant ways.

So really, philosophy is not the problem here. Rather, the problem is hanging certain metaphysical commitments on science that are extraneous to the job it's doing.

Which commitments are separable from which others, and which commitments are joined at the hip, can be a tricky business if you're not used to thinking carefully. (See, for example, the current debate over whether you can support disability rights and also support physician assisted suicide.) Even people who think for a living can let their assumptions go unquestioned if they've been humming along for a while. But, it seems to me, if you want to know what a scientific theory commits you to, you might want to talk to some scientists who use the theory. If you're really brave, you could even ask a philosopher of science.

Technorati tags: philosophical theories, metaphysical commitments, methodological commitments

A scientist's good name.

2005-10-03T21:26:00.000-07:00

Ms. PhD at YoungFemaleScientist recently blogged about the joys of dealing with coauthors on a collaborative project. After doing her experiment for the project, she waited patiently for a look at the manuscript that resulted from the collaboration. And when nothing arrived (after a more than decent interval) she asked how it was going. The response was essentially: We're sending the manuscript out next week. Oh, you want to look at it before it goes out? We'll get it to you the day before it goes out to the journal.

Uh, no.

And, since Ms. PhD hasn't taken leave of her senses (or forgotten her ethics training), that was pretty much her response. It is a shabby thing to do to a coauthor to cut her out of the manuscript preparation process, especially when she is willing to play an active role in making sure the manuscript (both in the details of her contribution, and overall) is in good shape before it goes out. And, there is something not quite right about a coauthor who wants to keep a collaborator out of this part of the loop. Maybe there are sensible reasons motivating it, but if none are offered, it's hard to be confident that that's what's going on.

Ms. PhD's assessment of the situation, especially from the point of view of a more junior collaborator, is so on the money that I need to quote it:

My gut instinct is that if it's really bad, I wouldn't even want my data in there, but I'm not even sure I can ask them to take my data out of it, or if I should just ask for an acknowledgment, or what. I'm only going to have one day- maybe less- to decide. And I don't imagine they would put in any edits I might want to suggest, since they're not going to have time.

I want to be especially careful because someone we know (distantly) is having a really hard time getting her grants, after having to retract some collaborative papers she published with a coauthor who later got caught for fabricating results. To me, that's really a nightmare scenario, especially in this day and age where you can't possibly know for sure, since you're almost always collaborating across (sometimes way across) disciplines.

At some point you really have to wonder if it's fair to hold the whole list of authors responsible for the sins of one greedy person, whom you're depending on to be the resident expert in their field.

And when the review process fails to detect it, you really have to wonder if the other authors should be expected to know. Since everything thinks peer review is so great and all that.

Collaboration is certainly a mixed blessing for the junior scientist. You can get some visibility by being involved in a high-impact paper with a big(ger)-name scientist, but you may not be taken seriously in the process of putting together the paper that's going to make the splash. As your collaborators see it, you're there to contribute one piece of the empirical scaffolding; after that, you're supposed to get out of the way. Leave the manuscript writing and editing to those of us with experience, missy!

But, as Ms. PhD points out, if your name is on the paper, you're taking responsibility for the goodness of what's in it. If you're ready to profit from it professionally, you're also putting yourself on the hook if there's something wrong with the paper. The "something wrong" could range from mistaken conclusions arising from sloppy data management to fabrication, and even if it was a collaborator's screw up, everyone on the paper gets to share in the shame — even the well established scientist. (I'm thinking of a city in Maryland ...) Putting your name on the paper is putting your reputation behind it. So there ain't no way in hell a collaborator should be keeping you out of the writing and revising loop. Indeed, it seems like coauthors should insist that everyone on the team read a full draft and contribute comments on it. Not only would this ensure that each author's contribution is represented accurately, but it would also be a good way to check whether the various pieces of the research are understandable to scientists who don't necessarily work with those precise experimental systems or methods. Think of it as the in-house peer review before the peer review.

Why wouldn't someone want to get a collaborator into the writing-and-proofing loop? I suppose researchers who are horrible procrastinators, or who get really blocked writing new stuff, or who get overly obsessive about comma placement, might slow down the process. Someone on the team ought to be able to hammer out the rough draft in a timely fashion, but it seems like all the writing and editing contributions could be done according to the group's deadlines. (People set deadlines for getting experiments run, so why not for drafting and editing?) Possibly a principal investigator might be hesitant to get feedback from a collaborator with who he had a long-standing feud about whether or not dangling participles are evil. Or, for that matter, from a collaborator who disagrees about the significance of a particular experimental result reported in the manuscript. But here again, peer review starts at home: if all the authors in the collaboration can't share their reasoned views and together come to some kind of agreement, why should they expect that their manuscript will be able to survive peer review by scientists not involved in the project?

A collaboration is a commitment that you try to make work for the benefit of all involved. But, if someone wants your data without your full involvement, it's time to ask whether this person is thinking about what's good for you, or for the collaboration as a whole. It is not wrong to insist that you be involved. In cases where someone is trying to prevent your full involvement, it is not wrong to explain that this is contrary to the spirit of collaboration, nor is it wrong to assert that you and your data cannot in good conscience remain in an arrangement where your proper participation is thwarted.

People who really need the data can either acknowledge your rights and responsibilities as the scientist who generated the data, or they can get their own damn data. When you're a junior scientist, pretty much all you have is your ability to generate good data and your good name. Anyone who would ask you to risk your good name by staying out of the process of generating the manuscript to which that name will be affixed forever, in front of everyone in the scientific community, is someone who isn't taking your best interests seriously.

Technorati tags: authorship, collaboration, peer review

I miss Sir Karl!

2005-10-01T20:12:00.000-07:00

Do you ever hit one of those points when you are just weary of some of the things with which it is your task to deal? I hit one of those; possibly the mountain of grading has something to do with it. But some of the common problems my students have run into are rather close to some of the problems I keep bumping into in discussions of science among the public at large.

Let me explain. My Philosophy of Science students have been reading Karl Popper and trying to get clear on how he drew the line between science and the other stuff. To help them do this, I’ve been having them do a group task where they take a suspicious hypothesis or cluster of hypotheses and see if there’s any reasonable way to test them. The Popperian line, of course, is that a test of an hypothesis is an attempt to falsify it. So the trick is to figure out some things that you ought to be able to observe under test conditions if the hypothesis is true, then set up the test conditions and see if you observe them.

But my students get distracted by the suspiciousness of the hypotheses. Give them a scenario with claims about alien abduction and whether or not there’s a plausible way to test the claims, they’ll go right to aliens → pseudo-science → untestable. In other words, if the hypothesis is one they’d reflexively reject, they tend to reject it immediately based on its implausibility rather than finding a way to get empirical evidence that would give them good grounds for rejecting it.

They forget Popper’s point that real science is often more implausible than pseudo-science. Seriously, put the implications of quantum mechanical theory up against the daily horoscope next to the comics. But QM makes testable predictions – there are particular outcomes that just wouldn’t fit with the theory. On the other hand, it’s pretty hard to construct a day that couldn’t somehow be reconciled with the “predictions” of the newspaper horoscope. If we get what’s distinctive about science, says Popper, we see that plausibility is not the essential thing; falsifiability is.

(Yeah, I know, there are shortcomings with Popper’s picture of science. We get to those in the course, soon. But it would please me if they understood Popper’s picture before we take it apart.)

This is where my students’ muddle with Popper shines some light on popular debates about science that are starting to get old (like the Intelligent Design vs. evolution slugfest). But the connection didn’t really click into place until I read this post at Generalizations, etc. There, Peter writes:

And why do they [the “intelligent design people”] get under my skin? Because I'm quaking in my boots that they will succeed in demolishing my precious covert religion of Scientific Materialism? Not likely. The problem is that I like science. It's an extraordinarily effective method for wringing useful information out of our vast and bewildering universe. Do you catch my drift? The problem here is that science is a method while "intelligent design" is a conclusion. If "intelligent design" were to enter the biology curriculum, the method would be circumvented to favor a conclusion that is dubious at best.

Part of the public discourse about evolution vs. ID has been whether it’s more plausible that the variety of life on this planet emerged from the operation of natural selection or from the action of an intelligent designer. But, see, plausibility is not what makes something science. Rather, the deal with science is that it uses a method of testing. You don’t get the conclusions — even the plausible ones — for free. You only get what you can support with the method. Popper said you get the very best support when you try to knock your hypothesis out of the water and the experimental results keep lining up as if the hypothesis were true rather than false. (I gotta think Popper was a fun bofriend.) And, even if it’s really well supported, you may have to set your conclusion aside in the light of new evidence that falsifies your theory. (Newtonian physics, anyone?) On the other hand, a theory that seems implausible and takes a hit when early attempts to test it are not helpful, like cold fusion, always have a shot at making a comeback if serious application of the method comes out in the theory’s favor.

So, if the public (or at least my students) could get clear on the distinction between what one believes and what would count as good, scientific reasons for believing it (grounded in the scientific method), maybe some apparent disagreements would be cleared up and I could feel less bone-tired than I do right now.

Edited to add: Ed Brayton gives an excellent analysis of an unfalsifiable hypothesis and why there is no conceivable way it could be falsified. Doesn't mean it couldn't be true, or that one couldn't believe it. But you can't use scientific evidence or methodology as your justification for believing it.

Technorati tags: scientific method, Popper.

Science, meet capitalism.

2005-09-29T12:10:00.000-07:00

There was an accident on the freeway this morning, which meant I listened to more NPR than my usual getting-to-work dose. Possibly, my peevishness at the boneheads who snarled the roads by colliding so inconsiderately is spilling over into peevishness at the scientists in the stories I heard. You be the judge.

Want to know the sex of your fetus really fast? You might be tempted to get the Baby Gender Mentor test (although you might not be tempted to say "Baby Gender Mentor test" five times fast). But, according to reporter Nell Boyce, you won't have much data to reassure you that this is money well spent.

Basically, Acu-Gen, the biotech firm offering the test, says that they can make an accurate fetal sex determination from a blood sample by 5 weeks after conception. They point to a bunch of publications that purportedly show that what they are offering is scientifically plausible and not a scam at all. The thing is, one of the scientists whose work is cited as supporting Acu-Gen's method, Farideh Bischoff, said in the NPR interview that she was skeptical that such high accuracy could be obtained so early in a pregnancy. (It's worth noting that Acu-Gen screws up the citation, giving it as "Farideh et al." rather than "Bischoff et al.")

Acu-Gen is the only company selling this test, so naturally, the details of the test are proprietary. But there seems not to be any body of, say, clinical trials that they can point to to reassure people who get the test that it's "99.9% accurate". In an interview on the Today Show, Sherry Bonelli, the CEO of PregnancyStore (the only retailer that sells Baby Gender Mentor test), said "They've actually followed more than 2000 women throughout their pregnancies and they've never been wrong." Of course, the emissary of Acu-Gen who responded to NPR's questions about the accuracy of the test, said, in effect, come back in a year when we've followed a bunch of pregnancies to term and can answer your concerns about accuracy.

Do you know that your test is accurate or don't you? (And by "know" I don't mean "know in your heart of hearts" so much as "know from the results of well-designed and well-conducted scientific studies with sufficiently large sample size that the results are reliable".) Pressed on whether tests of the accuracy had already been performed (as Bonelli claimed) or are currently being performed (as the Acu-Gen email to NPR suggested), the head of Acu-Gen, Chang Ming Wang, was evasive.

Diana Bianchi, a fetal DNA expert and another scientist cited by Acu-Gen in support of their method, said in the NPR interview that she was concerned about the claims of high test accuracy, given anecdotal evidence (from sonograms) that the test had given bad results in at least a handful of cases taken together with the lack of persuasive data to support the claims of high accuracy. Sherry Bonelli, Baby Gender Mentor test retailer (who was apparently much more willing to speak on the record than anyone who works for Acu-Gen), said that these scientists are skeptical because ... (wait for it) ... they're jealous of Acu-Gen! See, even though Acu-Gen hasn't provided any evidence that their claim (99.9% accuracy determining the fetus' sex at 5 weeks gestation from a drop of the mother's blood) is true, their critics haven't provided any evidence that Acu-Gen's claim is false. Given that the precise details of Acu-Gen's test are not available to these skeptical scientists (because they're proprietary), it's not obvious how they're supposed to produce such evidence. But that's not going to stop Bonelli from selling the test!

Of course, fetal sex determination via a drop of blood counts as a non-medical test, so the FDA doesn't regulate it (even though results from this test might well lead to a decision to pursue various obviously medical decisions for the expectant mother). Given that there are none of the clinical trials you'd expect for an FDA-regulated test, we're talking about something that, from the point of view of supporting data, is on par with "dietary supplements" advertised late at night on basic cable. Classy!

I wonder how other scientists in the biotech industry feel about this kind of thing. It seems like they should be concerned about a "science"-based product selling itself as science-based to the consumers but putting up very little science to back the claims that are separating consumers from their money. The more this kind of thing happens, the more opportunity there is for consumers to feel screwed over by shoddy science (and in case Acu-Gen's lawyers are reading this, I'm not claiming Baby Mentor Gender test is a scam -- I'm just pointing out that without any data to support it, there is no earthly reason a scientist or an educated consumer would accept its claims!). And, feeling screwed over by shoddy science would tend to feed into a low opinion of scientists. That would sure make it harder for serious scientists in biotech to connect with consumers. And, it would make things harder for scientists in general, even if they're not trying to sell anything but knowledge. Guilt by association sucks, but it's hard to avoid if you don't stand up and call bullsh*t on a member of your community who may be using the mantle of Science to make a fast buck.

One more quickie: this story of a biomedical firm whose listing on the New York Stock Exchange has been delayed. The apparent reason for this delay (NYSE hasn't given an official explanation)? Animal rights groups may have put pressure on the Exchange, because the firm in question, Life Sciences Research, Inc. does a lot of animal testing.

My regular readers (hi, Julie!) know that, while I like doggies and bunnies and duckies, I'm no animal liberationist. But, I'm not all indignant on behalf of Life Sciences Research, Inc. See, this is an example of market forces working, isn't it? There is certainly a demand for animal testing (which is why Life Sciences Research, Inc. has a thriving business), but there are also folks who are agin' it. In a free market (or whatever kind of market it is that we have), consumer opinions make a difference. Even the NYSE is influenced by public opinion.

Babe, that's just another cost of doing business.

Technorati tags: biotech, unsupported claims, science, capitalism

Research with human subjects -- mine.

2005-09-28T20:41:00.000-07:00

Yes, I'm getting ready to study some scientists (and scientists-in-training) at my university. Of course, there will be much Serious Philosophical Analysis, but what I'm planning to analyze are actual practices in science departments. ("Honey, look! There's a philosopher who's paying attention to the real world rather than starting from first principles! Turn on the sprinklers!")

The thing is, getting information on the practices means I'll be asking scientists and scientists-in-training questions, both in interviews and in questionnaires. And that means getting Institutional Review Board approval for my protocol (because respondents to questionnaires and interviews conducted as part of a research project are human subjects). And that means my protocol must ensure that "subjects are fully informed of their rights and of the potential risks and benefits of participation in the research."

Umm, potential risks of sharing information with a philosopher?

Participants might become reflective about their everyday professional activities, which can eat up a lot of time that could be spent on other necessary functions, like compiling assessment data or looking for a parking space.
Participants might become aware of gaps between the outcomes their professional activities aim at and the outcomes actually achieved. This could lead to feelings of disappointment. Alternatively, this might encourage adjustments of the professional activities to better attain the desired outcomes — another time sink (see above).
Student participants might become reflective about their role in the learning process, leading to alienation from their peers.
Faculty and student participants may be drawn into discussions with each other about the effectiveness of their various interactions. Discussions take time (see above).
Participants might become ensorcelled by the siren song of Philosophy (as the lead investigator of the proposed research did), putting them at risk for additional coursework and professional training, not to mention the stigma of having left a respectable field for ... Philosophy.
There is a small risk that participants may sustain paper-cuts from the questionnaire.

What am I forgetting here?

On a related note, there is a danger in giving someone like me a multipage policy for the protection of human research subjects, because it puts ideas in my head. For instance, I now want to find a way to incorporate into this research "taste and food quality evaluation" of "wholesome foods without additives". Possibly brownies.

Would a brownie with your questionnaire count as a potential benefit or a potential risk of participating in the study?

Technorati tag: human research subjects

Teaching scientific reasoning.

2005-09-27T14:53:00.000-07:00

Actually, make that "Trying to teach scientific reasoning to a group of students, the majority of whom are kind of freaked out about science."

Julie said I should blog about my online Philosophy of Science class. (She was a student on its maiden voyage; if she wants it blogged, it must be blogged!) So, given my thematic focus on this particular blog, I thought I'd discuss one particular type of activity I use in that class that aims to give students a feel for how scientists reason.

Of course, there are loads of activities that could fit under the broad umbrella of "scientific reasoning". There are computations (the kinds of things science majors do on problem sets). There is the activity of interpreting outcomes of experiments in the lab (and, unless things have changed a lot since I was an undergraduate, the task of working out a plausible explanation in the lab report of why things didn't work as planned). There is the challenge of framing a question and designing a research study or experiment that will lead to an answer. I could go on ... But, in my Philosophy of Science courses, as in most others, we don't have labs, and if I asked my students to do computation-heavy problem sets, they'd come after me.

So I go with what philosophy offers here: the thought experiment. For Philosophy of Science, though, these are thought experiments that are conducted in small groups. This is not just a strategy for making the online class feel more like a class (with other people! and discussion!); I give an analogous set of activities to the "live" version of this class.

The thought experiments I give are frequently drawn from incidents in the history of science, or from classic science fiction examples in the scientific literature. The groups of students are presented with a scenario and given a task along the lines of figuring out how to classify particular substances (by macroscopic properties? by microscopic properties?), or how to choose between two competing theories, or how to design tests for kooky-sounding hypotheses. In most of these tasks, the students start with some sort of interpretive framework cobbled together from the course readings, any prior scientific training they might have, and common sense. They need to use this to respond to certain bits of evidence or information. Before they can draw the conclusions they need to answer the questions I'm asking them to answer, they usually need to get more evidence, or adjust their interpretive framework, or both.

As a bonus, because this is a group exercise, the members of the group frequently have different interpretive frameworks and background assumptions. This means that part of the discussion is the gladiatorial battle between different interpretive frameworks and background assumptions. At the very least, the students become aware of their own assumptions and interpretive frameworks through this clash. (Usually they're so light you can hardly tell you're wearing them!) Often, the groups will succeed in coming to something like consensus by the end. Believe it or not, the consensus is usually built by the exchange of reasoned arguments.

This is lesson #1: Scientific exchanges often involve significant disagreements, but scientists work to come to agreement through rational engagement with each other. (This is why the groups are essential; most of my students don't disagree with themselves enough to learn this lesson from a solo project.)

In terms of dealing with the particular tasks I ask the students to complete, they run into some interesting problems. One is that they are fairly slavish in their loyalty to the facts they learned in high school science. Remember, these are students who, as a group, are scared of science. (There are a few notable exceptions -- but these exceptions tend not to be the ones who hold the high school textbook as the last word on the physical world.) Happy as I am that these students have retained something from their science education, it makes it harder to get them into the thought experiments sometimes, since the scenarios frequently ask how you would make a decision given certain sorts of experimental outcomes that might not jibe with reality as catalogued by the high school science class. To get some insight into what scientists do to get to knowledge, they have to imagine themselves into situations in which they might not know all the stuff that we know now and/or the phenomena in the scenario are slightly different from those in the universe they actually inhabit.

Indeed, this connects to another difficulty the students have with the scenarios drawn from the history of science: they think that the most important thing for them to do is "decide" the classification or theory choice in the way that Science actually decided the matter. It's almost like they see the actual judgment of science as the answer in the back of the book. So far, no one has actually overtly tried to reverse engineer their group response from the "right" historical answer, but they tend to use this answer as a selection criterion unless I intervene.

The thing I want them to experience here is that scientists don't have a "back of the book" to look to and check their answers. They're doing the best they can with partial information and an interpretive framework that proves itself in its usefulness. As such, it is completely possible, at certain junctures, that different groups of scientists could come to different conclusions about a particular problem. In the long run, the different groups can be expected to engage each other with reasoned arguments to come to consensus. But, the consensus doesn't come from having absolute proof that you have The One Right Answer. A lot of scientific decisions, after all, have to do with how to draw our categories, or what we want out of a theory or a model or a measuring device.

I mentioned before that I use activities like these in my live classes (where class meetings are 75 minutes long), but I've found them especially effective online. When the discussion happens on the online discussion board, students seem to work harder to express themselves clearly (because they're writing their contributions). They seem also to respond more seriously to the contributions of others in their groups (again, perhaps because they're written). The students have even been know to respond carefully and critically to their own contributions. And, rather than having to discuss and achieve consensus in 75 minutes, my online students typically have a week or two (depending on the complexity of the task) to really dig into the discussion and try to persuade each other. Not surprisingly, the greater length of time makes for a deeper discussion.

Bonus: Since I am privy to all the groups' discussions, not only can I play devil's advocate, dispense encouragement, and clarify any parameters that need clarifying, but I also know who any free-riders are, and I can award credit (or lack thereof) accordingly.

These are little steps toward building up a better understanding of what scientists are doing for the lay persons I teach. But I think they actually convey something that you don't usually get from canned lab experiments, either. (The handful of science majors who have taken this course strengthen my conviction about this.) A lot of the work ion real science is figuring out what to do next given what you've got. The scientist has a general plan of attack and a bucketful of strategies that seem promising and/or have worked in the past, but a lot of scientific reasoning starts out feeling like a shot in the dark.

Philosophers know a lot about taking shots in the dark!

Technorati tags: science education, teaching online, teaching-carnival

Blueprint to improve science journalism.

2005-09-26T13:23:00.000-07:00

Is there a way to get science journalism to work better? (What do I mean by “better”? The facts are reported accurately, and the non-scientist reader has a sense not only of why the science matters, but also of how the science was produced.) Could good science journalism go beyond helping people make rational decisions about what to eat, what to drive, and how to understand various bits of their world, to helping people have a better grasp of scientific reasoning — maybe even helping them see what is creative and beautiful and cool about science?

This post at Pharyngula has made me optimistic. PZ Myers, recognizing the good work of science journalist William Souder, reprints Souder’s take on why so much science journalism disappoints. It’s a beautiful analysis (where "it" = Souder, W. (2005) Of men and deformed frogs: a journalist's lament. In: Lannoo, M. 2005. Amphibian Declines: The Conservation Status of United States Species. U. Calif. Press, Berkeley, pp. 344-347.), and, to my mind, it points to some ways in which things could actually be improved if journalists and scientists put their minds to it. I’ll hit the key points Souder makes, and the optimistic places my mind went envisioning solutions to the problem.

[R]eporters—and more importantly their editors—tend not to see science as a developing story, but rather as a perplexing and boring process that produces "news" only intermittently, usually in the form of a readily digestible "discovery." This, for the most part, eliminates from science reporting what is elsewhere the gold standard in journalism—enterprise. A reporter trying to cover an ongoing story in science is likely to find room for only fragments of it in the paper or on the evening news. Very small fragments. Image that newspaper and TV journalists reported the results of elections, but said not a word about the campaigns leading up to voting day, and you begin to get an idea of the disparity.

What would it take for reporters to view science as a developing story? Why not a radical rethinking of the “science beat”? Reporters in the field in research laboratories and at scientific conferences. Reporters getting a feel not only for how the research is being done, but what’s motivating it, and what surprising twists and turns present themselves on the road between formulating a question and coming to something like an answer.

I just can’t imagine that embedded reporters in a lab would get in the way any more than they do in, say, a military operation. Plus, they might wash some glassware while talking with the researchers.

[J]ournalists are overly reliant on findings published in the scientific literature. In most forms of journalism getting "scooped" is a disaster. In science reporting, it's almost a requirement. The surest way to convince an editor to go with a science story is to show him or her that it has already been published in a scientific journal—or, preferably, that it will appear in one on the very same day you are proposing you run with your version. Here, I think, journalism and science must shoulder the blame equally. Journalists, in choosing only to cover periodic developments, give a false picture of the nature of scientific progress. A paper in a journal reporting a set of findings rarely represents a comprehensive view of the whole field of knowledge about a particular issue; rather it is a snapshot of one facet of our knowledge, incomplete and lacking context. No wonder the public often sees scientific discoveries as contradictory of one another. The public—that is to say, you and I—may feel a little like it's listening to a radio broadcast of a football game in which the plays aren't reported, but only a score is given every few minutes. In a seesaw game (and science is very much a seesaw game) you would never know when one reality might supplant another. At the same time, the scientific community makes better, more continuing coverage of science difficult when most journals require researchers to embargo their findings as a condition of publication. Why don't reporters do a better job of keeping track of what scientists are up to? Much of the time it is because scientists keep it a secret. Embargoing scientific findings that are in press enhances the status and confirms the supreme power of scientific journals—but it inhibits a full public understanding of what science does or does not know about many subjects of vital importance.

Souder is quite right that keeping results close to the vest until your paper in Nature or Science comes out fosters a popular picture of science as a collection of results rather than an ongoing process that involves corrections. The public sees a see-saw when the view from within science is of a jigsaw puzzle with a gazillion pieces. What to do?

Well, the scientific embeds would help by reporting on other parts of the scientific process besides the results. The fear, of course, is that the dispatches from Dr. Smartguy’s lab would undercut Dr. Smartguy’s ability to actually bring results to press in a top-flight, peer-reviewed scientific journal. (Won’t Dr. Smartguy’s competitors become avid consumers of science news?) Who in their right mind would agree to let journalists observe them if it cuts into their publication record. (And then there are grant proposals …)

But surely, just as there are certain sensitive details journalists embedded in military operations cannot report, it should be possible to specify sensitive details in the lab that are off limits until a publication has seen the light of day. Even holding those details back, there are many interesting and important stories to tell about how scientific knowledge is built. Indeed, science journalism might allow for more stories about beautifully designed experiments that didn’t work and promising leads that haven’t panned out (yet) than do the scholarly scientific journals. (Really, though, this might be something the scientific journals should rethink. I would have found it enormously helpful, when I was doing research in chemistry, if there had been a body of research on experiments that just didn’t work on my system. Would have saved me some time!)

Peer reviewing, of course, is what makes a result scientific knowledge, endorsed by the community of science. So, there is a danger in reporting (even obliquely) results that haven’t yet gotten through peer review. So another change that might help here would be to speed up the rate of peer review. To do this without sacrificing the quality of peer reviewing, you’d need more qualified peer reviewers — and they’d need to have the time to actually work through the manuscripts at a reasonable clip. To make that happen, it might be necessary for scientists (and the institutions that employ them) to recognize peer reviewing as an important scientific contribution. I’m not saying that reviewing a manuscript should “count” as much as producing one, but it could certainly be counted more than it is at present.

On a related note, an important part of the science beat should include an examination of peer review. Why is it so important to science? How does the process look, through the eyes of the reviewer and through the eyes of the reviewee? Once a paper has passed through peer review, why is that not the last word on the subject (and how do scientists deal with post-peer-review differences of opinion)? If the lay reader got even a bit of her head wrapped around this, it would be a big improvement over the status quo.

Finally, journalists, and (to a lesser but still substantial degree) scientists as well, place an inordinate significance on human health concerns with respect to ecological problems. Human health, of course, is a paramount consideration, but you should not have to have evidence of people keeling over or growing extra legs to sell an editor on a story about deformed frogs (or ozone depletion, global warming, endocrine disruption, water scarcity, shrinking biodiversity, etc., etc., unto oblivion).

Did I mention that Souder wrote about deformed frogs? But the point is generalizable. There are plenty of stories about science that the public would benefit from knowing even if they don’t have immediate implications for what we eat or drive or whatever. My hunch is that the public’s natural tendency is to be curious about what scientists are doing and what scientists think they’ll learn by doing it. Only the combine force of science education that makes science seem boring and impossibly hard, and scientists saying, “Look, don’t worry what it does, you wouldn’t understand” has deadened this natural curiosity. (C’mon, people are curious about Paris Hilton. Ain’t no way Paris Hilton is more interesting than a superconducting supercollider!)

Here again, I think scientists need to help journalists to make the situation better. Scientists really need to work out at least a good cocktail party explanation of what they’re studying, how they’re studying it, and why it matters. And, the “why it matters” part needn’t be closely linked to human health or gas efficiency or economic productivity. Scientists, and science journalists, can help the public understand that sometimes we are benefited by simply coming to better understanding of a piece of the world that intrigues us. But, if scientists can’t explain why this matters, no fair blaming the journalists for not explaining it.

So, now that we have an idea what needs to be done, let’s get to it!

Technorati tags: science journalism, science education, scientific knowledge production

Anti-science chickens coming home to roost.

2005-09-23T20:34:00.000-07:00

Remember when I was worrying about the government's relations with science? How I thought government interference with scientists and their results might undermine the ability for the government to actually produce science that anyone can trust?

We may be on our way, folks.

In a story this afternoon on The California Report (sorry, no permalink -- it's the first story in the September 23 archive), it was reported that the U.S. Department of Education is withholding a study on bilingual education. It only took three years and a couple million dollars to do the study, so no big deal. As the description of the story puts it, "Officials say the research failed to meet standards for quality. But skeptics question whether the decision is politically motivated."

See what happens? You get a reputation for trying to thwart the release of scientific results that go against your policy objectives (or, say, those of your big donors). Then, if you withhold a study whose results might have implications for your policy objectives, people will see this as business as usual. Even if there are actually valid scientific reasons for rejecting the study, no one is going to believe you didn't make them up. Which means, of course, that folks will be suspicious as well if a better version of the study comes out and happens to support the position you (or your donors) prefer.

Dr. Free-Ride's Better Half views this as a little victory for the cynical enemies of science. They've got things to the point that a piece of science produced under the government's auspices can be dismissed out of hand regardless of its actual scientific merit or shortcomings. And from there, it's not such a stretch to cutting science out of the public policy dialogue altogether.

I'm a little less negative about this. For one thing, science not done under the government's auspices can still hold its own under scrutiny. For another, it's not obvious to me that the public ends up agreeing that the science doesn't matter. If there were a serious politically motivated effort to withhold a scientific study, wouldn't that indicate that the pols were scared of the science? Wouldn't that be a clue that they knew, deep down, that the science should matter in the public policy dialogue?

Technorati tags: science and politics, government studies

Numbers don’t lie … unless they’re statistics.

2005-09-22T09:49:00.000-07:00

A colleague of mine was nice enough to point me to this news item from The Economist on the reliability of medical research papers. The article in question is “premium content”, but the article it discusses, ”Why Most Published Research Findings Are False”, is freely available. (Rock on, Public Library of Science!)

The paper is by John Ioannidis, an epidemiologist. It goes into loving detail, looking at factors like prior probability of your hypothesis being true, statistical power of the study, and the level of statistical significance, to show why certain common practices in medical research increase the probability that a “statistically significant” result is fairly meaningless. (If you’re not a math fan, it’s tough going. Even if you are a math fan but it’s been a while since you’ve mucked around with prob/stat, you may want to chew carefully.)

It’s an interesting read (and, I imagine, an important one for researchers who want to avoid some of the pitfalls Ioannidis indicates). Rather that recapping his argument here, which would necessarily involve either going into way more mathematical detail than you want me to, or dumbing it down, I’m just going to give you his corollaries:

Corollary 1: The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.

Corollary 2: The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.

Corollary 3: The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true.

Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.

Corollary 5: The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.

Corollary 6: The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.

A number of these didn’t surprise me. (“Financial interest might bias my results? Really?”) But I wasn’t expecting Corollary 4 at all. It does seem reasonable that if people in a research area don’t agree on precisely what they are, or should be, measuring, it’s harder to find out what’s really going on in that area. As Ioannidis puts it,

Adherence to common standards is likely to increase the proportion of true findings. The same applies to outcomes. True findings may be more common when outcomes are unequivocal and universally agreed (e.g., death) rather than when multifarious outcomes are devised (e.g., scales for schizophrenia outcomes). Similarly, fields that use commonly agreed, stereotyped analytical methods (e.g., Kaplan-Meier plots and the log-rank test) may yield a larger proportion of true findings than fields where analytical methods are still under experimentation (e.g., artificial intelligence methods) and only “best” results are reported.

As Ioannidis notes, the stage of research at which you’re working out what pieces of the system are important, what kinds of experiments might tell you something useful, etc., is very important for hypothesis generation. But, it would seem, to test these hypotheses in reliable ways you have to move from flexibility to rigidity — in other words, you need to separate the process of generating hypotheses from the process of testing those hypotheses. (Sir Karl Popper, in his grave, does whatever one would do in situations where rolling is not appropriate.)

Similarly, Corollary 6 is not as obvious. It would seem that more people working on a question would lead to better results than fewer people working on it. But hot fields are ones that are often relatively new (with appropriate standards still being worked out — see Corollary 4 again) and, perhaps more importantly, they are fields in which research teams are in fierce competition:

With many teams working on the same field and with massive experimental data being produced, timing is of the essence in beating competition. Thus, each team may prioritize on pursuing and disseminating its most impressive “positive” results. “Negative” results may become attractive for dissemination only if some other team has found a “positive” association on the same question. In that case, it may be attractive to refute a claim made in some prestigious journal.

In other words, the stakes of the competition can influence the flow of information from the labs to the literature. Moreover, it wouldn’t be surprising if the desire to get good results into print first makes mildly promising experimental results look better. (“Holy cow, we found it! Write it up, stat!)

Of course, scientists are human. Scientists can’t help but be biased by what they expect to see and by what they want to see. But that’s where the community of science is supposed to come it. All the individual biases are supposed to somehow cancel out in the scientific knowledge the community produces and endorses. Even if you see something in the data, if other scientists can’t see it, the community won’t accept it. And, the way the story is usually told, competition between scientists for discoveries and recognition and use-through-citation and groupies is supposed to make each of them scrutinize their competitor’s research findings and try to find fault with them.

Ioannidis seems to be saying, though, that it doesn’t always work out that way. Whether because scientists are trying to observe phenomena that are hard to pin down, or because negative findings only turn into publications when they show someone else’s positive findings were mistaken, or because scientists don’t always understand statistical methods well enough to set up good experiments of their own or critique the statistical methods used by papers in the literature, the community of science is ending up with less clarity about what it knows and with what kind of certainty. But, perhaps the insights of Ioannidis and others will help improve the situation.

Also worth noting, of course, is the fact that Ioannidis is concerned with the medical literature, rather than all scientific literature across all fields. There may be special difficulties that come from studying wicked-hard complex systems (like sick humans) that you don’t encounter, say, dealing with a chemical reaction in a beaker. Beyond dealing with different kinds of phenomena and agreed upon (or disputed) ways to characterize and probe them, scientists in different fields deal with different expectations as to what “scholarly productivity” looks like; I’m told the pressure to publish multiple papers a year is especially high in biomedical fields. Finally, while most of us couldn’t find a useful real-life application for string theory if we had spinach between our teeth, biomedical research seems pretty relevant to everyday concerns. Not only does the public eagerly anticipate scientific solutions to a panoply of problems with human health, but there’s a big old pharmaceutical industry that wants to be a part of it. But hey, no pressure!

My favorite section title in Ioannidis’ paper is “Claimed Research Findings May Often Be Simply Accurate Measures of the Prevailing Bias.” This is where the prescription lies: good science will depend on vigilant attention to what prevailing biases there are — in a research group, a department, a scientific field, the community of science, and in the larger societal structures in which scientists are embedded — and serious attempts to ensure that those biases are stripped out of the knowledge that scientists produce.

(And, for an extreme example of how research can give us more reliable information about researcher bias than about the system under study, check out this survey about motherhood and career choices given to freshman and senior women at Yale. If scientific and social-scientific journals had fashion spreads, this would be the “don’t” picture.)

ETA: For some reason, Blogger doesn't want you to see that last link. Here's the URL:
http://www.gelfmagazine.com/gelflog/archives/media.html#surveying_ivy_league_motherhood

Another URL or two:: There were Blogspot server problems the night this went up, so apparently the links are haunted. Here's where to find Dr. Ioannidis' article:

http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0020124

Here's the URL for the Public Library of Science - Medicine:

http://medicine.plosjournals.org/perlserv/?request=index-html

Technorati tags: research bias, scientific knowledge, statistics

What scientists know (or don't)

2005-09-20T21:45:00.000-07:00

I worry a lot about what lay people don't know about science. Sometimes the problem is that smart lay people have been scared away from thinking about science (by the people who tell them that you need a gigantic brain to do science). Other times, people seem to think they can hold forth on scientific debates despite the fact that they don't actually grasp the basics of the science they're talking about.

For an example of the "anyone can weigh in on science X" sort of person, consider the case of Timothy Birdnow, a property manager who has taken it upon himself to set out all manner of problems with the theory of evolution. Useful dissections of Mr. Birdnow's claims have been given by P.Z. Myers and The Questionable Authority. The diagnosis from The Questionable Authority is that Mr. Birdnow overlooks the fact that you can't coast on just your native intelligence here — you actually need to know some biology if you're going to weigh in on a biological debate without looking like a doofus:

In fact, scientists who dare to suggest that they might be more qualified to comment on scientific matters than non-scientists run the risk of being branded as "elitists". It's not elitism, folks, it's specialization. Modern society is far, far too complex for everyone to be good at everything. Most people select career paths that are specialized fields, and they are (normally) better at their own field than they are in other fields. This means that they are (normally) better able to develop informed opinions about matters within their fields than are people who lack the strong background in that area.

Not knowing Mr. Birdnow, it's hard for me to know whether this diagnosis is correct. My experience has been that lay people are much more likely to under-estimate their ability to "get" science than to over-estimate it. There are, of course, people who think they know everything, but they tend to be less likely to barge in to science than, say, philosophy (because how hard could philosophy be?). So my own completely unsubstantiated hunch is that perhaps Mr. Birdnow has a science coach on the side. And maybe that science coach has, um, a political agenda in whose service Mr. Birdnow is spokesmodeling. Or, maybe the science coach really does have a good grasp of biology (and the proper relation of DNA and RNA, etc.), and it's just that the cell phone keeps cutting out during the coaching.

But, it turns out, the specialization isn't just a matter of scientists doing science and property managers doing the property managing. Indeed, it would seem physicists might not know as much as they think about the state of evolutionary theory. Among other things, it seems that math-y scientists (like physicists) might not fully appreciate that theories that don't look like sets of equations can be perfectly good scientific theories. Also, physicists may not know how evolutionary biologists subject their theories to empirical test. It may all be science, but we have different scientific disciplines for a reason. While physicists may "get" the science in evolutionary theory better than, oh, property managers, they probably won't get it as well as someone who works on evolutionary theory for a living.

(Another discussion of the physicist who prompted these observations notes a certain irony in a string theorist policing the goodness of other scientific theories. I'm not going to cast aspersions. Some of my best friends ~~are~~ used to work on string theory.)

So: good to know some science to speak with authority about science. Good to know specific science X to speak with authority about specific science X.

But, economics? It seems a large number of economists (78% of 200 surveyed at the 2005 meeting of the American Economic Association) picked the wrong answer to what was supposed to be a straightforward economics problem. Could it be that the problem was not that straightforward? That not even economists are qualified to hold forth on economics? That economics, while dismal, is not a science?

Dude, I don't have all the answers; just some questions and some links.

Technorati tags: scientific specialization, scientific disciplines

Who'll protect kids from the EPA?

2005-09-20T10:00:00.000-07:00

Via my better half, an article from the Baltimore Sun reporting that new EPA rules allow testing of pesticides on children.

Now, the thing is, these are new rules that were prompted by criticisms of earlier problems -- including pesticide studies whose human subjects didn't know what they had been exposed to, nor even the purposes of the studies in which they were subjects! So, the idea was that the new rules would address these problems. Right?

Quoting from the article:

In unveiling the new rules last week, the EPA promised full protection for those most at risk of unethical testing.

"We regard as unethical and would never conduct, support, require or approve any study involving intentional exposure of pregnant women, infants or children to a pesticide," the rule states.

But within the 30 pages of rules are clear-cut exceptions that permit:

Testing of "abused or neglected" children without permission from parents or guardians.
"Ethically deficient" human research if it is considered crucial to "protect public health."
More than minimal health risk to a subject if there is a "direct benefit" to the child being tested, and the parents or guardians agree.
EPA acceptance of overseas industry studies, which are often performed in countries that have minimal or no ethical standards for testing, as long as the tests are not done directly for the EPA.

Shall we take these point by point?

1. It is not on the face of it outrageous to think that there might be certain instances in which participating in a scientific study could have a potential benefit for a human subject. If that human is a competent adult, the idea is to explain the potential benefit, as well as the potential harms, and let that competent adult make her own decision. If the human is a child, generally we look to the parent or guardian to make the decision that is in the best interests of the child. (And, there may be an argument here that the adult rendering consent for a child needs to be extra careful, not only about the immediate costs and benefits to the child, but also about how this decision may affect the child's later range of choices in certain, sometimes irreversible ways.)

An abused or neglected child, arguably, does not have access to parents or guardians who can make decisions in the best interests of the child. (Set aside, for the moment, concerns about who gets labeled as abused or neglected; I think there are really worries to raise here, but even if there weren't, we've still got stuff to worry about.) So, the abused or neglected child can't get proper consent from a parent or guardian to participate in a scientific study. But, they get to participate in a study without their parent's permission.

Why is the default position here getting to be a subject in a pesticide study? We're not talking about testing a new, promising drug (late in the drug-approval process, past the stage of figuring out how much of the compound a body can take without getting sick) when no other treatment is available for an illness that is doing you serious harm. We're talking about being exposed to pesticides. What is the potential benefit for the child?

2. I would love to see a succinct explanation of the "ethically deficient" research that is being allowed. Are we talking sloppy notebooks? Lying to the IRB? Being mean to the human subjects? Help me out here.

Also, what are the guidelines for what is crucial to protect public health? How precisely are we defining public health, and what are the boundaries on what is permissible in its protection? (Surely having nourishing food, safe water, and proper sanitation is essential to public health, and sometimes the government ... eh, just doesn't get around to it right away.)

Are unethical scientific experiments suddenly going to lead to big improvements in public health?

3. If we're going to allow kids to undertake more than a minimal health risk, someone needs to spell out what the "direct benefit" to the child being tested could be. Again, we're talking about exposing kids to pesticides. Is this expected to make them healthier? Smarter? What is the likely payoff that justifies the risk they are being asked to undertake? (And c'mon, if the parents are the ones rendering consent here, or if the kids are infants, they aren't even being asked, so it's even more important not to screw up.)

Or are we counting as the "direct benefit" something along the lines of warm meals and a ride to the clinic in an air-conditioned car? Because honestly, that hasn't worked out so well in the past. (Ask the Public Health Service.)

4. Oversees industry studies, performed in places where human subjects are not afforded the protections they are here ... because if you need to have the science to sell your products, you might as well be able to do the studies somewhere life is cheap. It's just good economic sense.

While I suppose it is possible to get scientifically valid data from studies where one treats human subjects unethically, it isn't something scientists like to do. Medical journals tend to have policies against publishing such results. Studies that treat human subjects badly could make it harder for other scientists, no matter how ethical, to find human subjects to participate in future studies -- especially if it makes the news. (In the aftermath of the Tuskegee syphilis experiment, how much harder was it to find willing participants in AIDS research than it might have been otherwise?)

Possibly there is some scientific knowledge that it would be hard to get except by asking human subjects to undertake significant risks. Depending on the nature of those risks, and the age of the subjects, it might even be that the clearest ways to get that knowledge would be, by accepted definitions, unethical.

Is this a good reason to relax the definition of what is ethical?

First, there may be other good ways to answer the scientific question that are not unethical, Sometimes ethical constraints make scientists more clever in how they approach problems. (On the other hand, it would seem that an overabundance of humans on whom to experiment made some of the medical researchers in Nazi Germany absolute morons, scientifically speaking.)

Second, it seems like we shouldn't even get to the let's-lighten-up-on-the-ethics stage unless the scientific knowledge in question is absolutely essential. The harm of not finding a workable answer to the scientific question has to be big, and it has to harm more than just the R&D team trying to bring a new product to market, or the shareholders, or the CEO.

If testing pesticides on children is so essential, and promises so much benefit to the children, then maybe we should go right to the children of the pesticide industry. No, not the kids of the parents working the line in the pesticide factory -- the kids of the CEOs, the stockholders, the lobbyists, etc.

The EPA will make sure it's OK.

Technorati tags: EPA, pesticide, ethics

Academic blogging survey/meme

2005-09-19T18:46:00.000-07:00

I picked it up at Pharrryngula, where every day can be Talk Like a Pirate Day with the mere click of a mouse.

Overview:
The following survey is for bloggers who are actual or aspiring academics (thus including students). It takes the form of a go-meme to provide bloggers a strong incentive to join in: the 'Link List' means that you will receive links from all those who pick up the survey 'downstream' from you. The aim is to create open-source data about academic blogs that is publicly available for further analysis. Analysts can find the data by searching for the tracking identifier-code: "acb109m3m3". Further details, and eventual updates with results, can be found on the original posting:
http://pixnaps.blogspot.com/2005/09/academic-blog-survey.html

Instructions:
Simply copy and paste this post to your own blog, replacing my survey answers with your own, as appropriate, and adding your blog to the Link List.

Important (1) Your post must include the four sections: Overview, Instructions, Link List, and Survey. (2) Remember to link to every blog in the Link List. (3) For tracking purposes, your post must include the following code: acb109m3m3

Link List (or 'extended hat-tip'):
1. Philosophy, et cetera
2. Pharyngula
3. Adventures in Ethics and Science
4. Add a link to your blog here.

Survey:

Demographics
Age - 37
Gender - Female
Location - San José, California, USA
Religion - Undecided
Began blogging - This one (the one that took) February 2005; earlier attempts started May 2003
Academic field - Philosophy of Science
Academic position [tenured?] - Assistant Professor [not yet]

Approximate blog stats
Rate of posting - almost daily (4-10 times per week, depending on what else is going on)
Average no. hits - 20/day
Average no. comments - 0/day
Blog content - all more or less on the theme of responsible science; some more about current events, some more personal.

Other Questions
1) Do you blog under your real name? Why / why not?
- No, but there are quite enough bread crumbs that anyone who wants to know my real identity can figure it out. The pseudonym is because it's not about me, and because I'm not interested in what the high and mighty at my university might have to say about my opinions.

2) Do colleagues or others in your department know that you blog? If so, has anyone reacted positively or negatively?
- Those who know what a blog is, know I have one. A subset of them read it. The comments have been positive.

3) Are you on the job market?
- No, and, if I get tenure, I hope never to be on the market again. (I like it here!)

4) Do you mention your blog on your CV or other job application material?
- No.

5) Has your blog been mentioned at all in interviews, tenure reviews, etc.? If so, provide details.
- Not that I'm aware of.

6) Why do you blog?
- Originally, the idea was to keep the students in my "Ethics in Science" class thinking beyond class time and assigned readings, to keep them up with relevant current events, etc. Then, the blog became a way for ME to think about issues I teach and things I'm working out in my research.

Also, I find it's a good way to write regularly (in smaller bites), and occasionally a good way to get feedback.

Plus, all the cool kids are doing it.

Pass it on!

Democratizing science.

2005-09-15T17:51:00.000-07:00

It's nice to know I'm not the only one who gets all exercised about some of the issues I talk about here. To wit: T.W. McKinney's comment on this post in which PZ Myers calls for knowledge to the people. McKinney points out that democratizing science might not be such a red-hot idea, depending on just what you have in mind when you say "democratizing":

I think this blog's ideological opponents might think the same of themselves, i.e. that they're "democratizing the process of scientific research." In a very broad sense, they're correct, too-- opening scientific methods to scrutiny by the court of public opinion is one way of "democratizing" science. They also take aim at the community of scientific experts-- who, it must be said, have (and commonsensically) wielded a lot of influence over public policy in various areas, etc. I believe that second aim of "democratization" renders the anti-science conservatives appealing to a lot of average people. If experts are appealed to in legal and political matters, their views (which, in most cases, resemble our best guess at the truth) appear to 'count more' than the average Joe's. So I, somewhat hesitatingly, might suggest that the anti-science folks are also aiming for democratization-- they are, as it were, populists challenging the hallowed ground of academia, and all of those scientists who think they're the one-and-only arbiters of 'the facts.' I'm not sympathetic to the leaders of this movement-- because they're attempting to destroy one of the most successful edifices of western civilization (probably *the* most, if I weren't so cautious in my phrasing)-- but I'm also aware that appeals to expert opinion in certain matters can leave people feeling like their views haven't been counted (or don't matter). Hence, the backlash.

It's good for people to have a say in what happens. It's even better if their say is backed up by reasons and, when possible, by evidence. Thus, we can not only hear what everyone has to say, but also try to find a sensible way to evaluate what has been said and figure out where to go from there.

This is how things are supposed to go in science's public square, too. But the crowd in science's public square has been trained to pay attentions to issues like relevance and credibility when someone makes a contribution. The rules of engagement are rather better defined here (if you saw it in your lab, you have to explain how we can see it in other labs, else we ain't buying) than they are in the broader public square where debates over politics and the fall TV line-up take place. Face it, I can insist that it just isn't relevant that Candidate X would be a good guy to have a beer with, but my insistence doesn't rule it out of the debate.

But, as McKinney notes, appealing to certain scientific standards to disqualify contributions to a scientific debate often looks to non-scientists like a power-play designed to maintain control of "expert" status and tell everyone else what to do. And, he notes (as I have in the past) that crappy science instruction may be a culprit here, leaving the public at large unable to distinguish legitimate moves within scientific discourse from unwarranted silencing of legitimate voices in the debate.

So again, let's fix science education.

But there's another relevant difference between the scientific public square and the broader public square: the goal that brings the scientific community together and brings their voices to the square is the shared goal of achieving a better understanding of how various bits of the world work. People coming to the broader public square nowadays don't seem to have a common goal that would, in its being attained, make everyone in the community better off. Rather, there's a struggle for goods -- for my guy to win so I get the stuff I want (which means your guy won't take my goodies and give them to you). Sad to say, it seems a situation designed to bring out the worst in people rather than, say, reasoned debate.

So maybe it's not that science needs more democracy (although I applaud Pharyngula and sites like it for bringing science to the people). Maybe what we really need is for the general public to engage each other more like scientists do.

Technorati tags: science, democracy