May 2008

Sun Mon Tue Wed Thu Fri Sat
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

« Global Warming – Part 2 | Main | Global Warming – Part 3 »

Comments

neopolitan

About your "even better link" - check who is behind the "Independent Institute" and whose voice is most heard (according to http://www.sourcewatch.org/index.php?title=Independent_Institute).

Fred Singer - whose name also appeared in http://www.durangobill.com/Swindle_Swindle.html and http://www.sourcewatch.org/index.php?title=Oregon_Institute_of_Science_and_Medicine (when he made positive comment on the questionable petition).

Seems this guy is a busy fellow.

cheers,

neopolitan

gr8hands

Scott,

Sorry, but it's difficult for me to accept the opinions of an economist on just about anything (particularly economics), because they aren't real scientists, and don't really use the scientific method -- so they wouldn't understand/recognize real science if it bit them on the butt! They don't even come up with accurate economic impact analyses -- at least I've never seen one that was accurate, or heard from a reliable source where one has been made.

So your "source" is rather poor, although he does have a particularly high opinion of himself and his qualifications. Those of us who have a real science background just snicker and shake our heads.

In real science, you're able to make extremely accurate predictions based on evidence (just look at how accurate we are about eclipses, sunrise/sunsets, tides, etc.). Yet no economist was able to predict the month, let alone the day, when the stock market crashed. Or the dot com bubble burst. Or when oil prices will reach any particular amount.

And for any of your readers who think that scientists would give you a favorable peer review in hopes that you would give them a favorable peer review -- that's just nuts. Scientists don't ever want to be known as having supported something that turned out to be wrong. They'd rather say "inconclusive results" instead of pointing to some particular conclusion that might be wrong.

Which is why something needs to be done correctly, fully, repeatedly before it gets published and then submitted for peer review. Otherwise it isn't science.

Too many people are confused about that.

Susan

I agree wholeheartedly with your assessment of people.

But I'll still take peer reviewed science over the opinions of whatever nutso/journalist/author/etc. who comes along ranting and raving at me. This is because;

a) Peer review means that multiple people unconnected by anything other than a profession have to look over an article and approve it. I'll take the opinion of multiple assholes over the opinion of one asshole, on the theory that it'll weed out the crazies.

b) I studied rhetorical theory in school, which basically says that as long as you say it really, really well, you can say anything and people will believe you. Personal experience has never contradicted this theory so far, which is why bullshit will get you far in life. But scientific papers *never* say anything well, so if I read it through and *still* believe them, their argument must have something going for it.

c) In general, peer review has nothing to do with opinion and everything to do with scientific method. You can hate the opinion expressed and still have no problem with the science. So "conformism" isn't really relevent, unless you're talking about those damn conformists who stubbornly insist that 1+1=2.

Peter Johnston

On posting here I came up against a number recognition panel "to stop automated robots from posting comments". As a moist robot without free will, this raises questions to me - is there a bias against non-moist robots, what is the difference between a robot and an automated robot and how many robots read your blog and want to post to it anyway?

Peter Johnston

Excellent article. My experience matches (perhaps that's why I think it's an excellent article - you agree with me). An extension of this is the expert you see on the news programmes. Why are these people "experts" - because they know the programme maker's mother, because they know a bit about the broad subject area (e.g. we had a computer expert on recently - how big a field is that and who can be an expert on all of it?), or because they don't have a real job so are available at short notice to come and participate on TV. Worst are the ones in pseudoscience who produce all sorts of research which actually comes from a survey - when did you last tell the truth to a survey - and when did you find one which wasn't steering you inexorably towards their point of view by not allowing you to give your true answer, only one of their boxes to tick.
A lot of this comes down to something you posted a while back - that once people hold a view they reject information which contradicts that view. Hence a peer review will welcome new research if it confirms their own views.

peter

Being so honest and yet humble, I would not presume to offer my opinion on your blog today.

David, Hungary

Scott,
Most Scientists would agree that Peer Review is not perfect - but like democracy, nobody seems to have come up with a better system. In non-peer-review journals, you get publications from some nutty guy promising the skin care product that will make women look younger, sold by Health Stores and Online 'clinics'. You find these articles by following apparent references at the bottom of ads in women's magazines (I do check these out from scientific interest – most are total bullshit).

Without peer review there is no quality control and a lot of crap to sift through before finding something useful. When there is the Quality Control of peer review, odd articles will slip through, sure, but not thousands of articles all saying the same thing.

To me as a scientist, a lot of what was said in the link you provided in your 'PS' seemed extreme to say the least – but it all became clear when I followed the link to the author.
"Robert Higgs is Senior Fellow in Political Economy". Sorry Scott, but that is not a science – this is definitely on the Arts side, not Science. So in this context, I agree with everything the guy says, as it relates to peer review in the areas where the articles are about IDEAS rather than things that can be measured. Economics is not a science in the context of issues like global warming, the amount of hot air coming from learned bodies rarely matches the proposals of one Scott Adams (future Nobel Prize Winner for Economics). Ergo – they are assholes.

In summary – peer reviewers in the Arts (including economics) will generally fit into the categories you listed. I am certain that a significant proportion of peer reviewers of articles on global issues will also fit in to them, but there will be a lot more who are straight and are trying to do their best as professional scientists.
If you want to stick with people who think science is crap, one of the comments last Saturday pointed to a site which fervently believes the Earth is flat, and that Scientists are all doing the work of the Devil. I don't know if you took the time to check it out – it scared the shit out of me that there are such anti-scientist people out there. http://www.truechristian.com/earthflat.html

SMEH

Saj lahko prdnemo, ni problema, ker smo na plaži. SPLOH!

Joe

I forgot to add. The idea that all the thousands of scientists who support the man made climate change theory wouldn't have checked the impacts of the sun is absurd. This effect is included in their models and doesn't account for the increases we are seeing in reality.

In reality - when you graph them CO2 is going up followed by global temperatures. There is some small effect from the sun but this doesn't account for the trends, whereas the CO2 concentrations in the atmosphere do.

When you look at the scale of industrialisation and land clearing (a huge contributor to global warming) around the world today it is hardly suprising that the atmosphere is responding to our actions.

Joe

It is possible that self interest can pervade the peer review process - a great example would be the volumes of research on how great Genetically Modified foods are. If the researcher and the peer reviewer are sponsored by say Monsanto to research how great GM foods are without looking at the other side then this can perpetuate. The scientists may all be objective, but limited by their research scope.

However, in the case of global warming this is not the case. In fact, taking this as a case in point, it is a miracle that the fossil fuel industry hasn't been able to flood the journals with anti-global warming science. That is because the science is very stong that human beings are causing climate change and it is bad.

The theory that climate change is a conspiracy of environmnentalists is ludicrous and amusing. I am a professional environmentalist, we hardly have enough money to pay ourselves and run basic services. To have the resources to fund the hundreds of thousands of papers that support the theory of man-made climate change - wow - we can only dream. We also have absolutely no motivation to make this up, as if there aren't enough environmental issues to campaign on already.

The fossil fuel industry on the other hand, they do have those resources and plenty of motivation to fight against a reality of climate change.

This is all the more evidence that all scientists aren't that bad after all, and climate change is something we should act on.

Your head in the sand, bad things don't usually happen, theory is based entirely on your perspective. People who lived in Europe or Hiroshima during the 1940's would have a very different perspective. For that matter so would any of the many previously comfortable civilisations that had their bum kicked during colonialism. Bad things happen to civilisations, just haven't happened in your life time yet.

Fixing climate change also has many other benefits in terms of making our lifestyles and environment generally healthier and consuming less resources through efficiency - thus helping other environmental causes.

Now that I have convinced you not to be a skeptic anymore, here is a useful link that will help you convince other skeptics...

http://gristmill.grist.org/skeptics


Akiyama

This is only tangentially related to the topic, but did you ever hear of a guy called Alan Sokal? He's a physicist who wrote a paper called "Transgressing the Boundaries: Toward a Transformative Hermeneutics of Quantum Gravity" and got it published in a philosophy journal, Social Text. It included lots of long words and scientific sounding concepts and lots of references to papers and books written by Social Text's editors. After it was published, he revealed he'd submitted it as an experiment, to see if a philosophy journal would publish something that was complete nonsense!

http://en.wikipedia.org/wiki/Sokal_Affair

Ajay Pal Singh Atwal

Ahh this is something with which I can relate to, unlike many of your cartoons.
Agreed peer review has its faults, but just for the sake of my 2 dimes, when open peer review is used it forces the reviewers to put more efforts and be more honest. See this: http://en.wikipedia.org/wiki/Peer_review#Dynamic_and_open_peer_review

Sometimes editors too need to be peer reviewed.

Some months back I saw the website of one of the ACM journals, where this open review was being carried out (cant recall the URL) and yes even other normal users could comment on the papers under review and also on the editor's comments.

Arturo

You still don't grok science, Scott, you wander into the realms of pseudoscience every time you try. Stop looking at the visible results, the end product and look at "method" instead. Only then can you judge the worth of a theory.

Anecdote: A pamphlet was published entitled "100 Authors Against Einstein", Einstein retorted "If I were wrong, one would be enough."

ie. Proving scientific theories is hard - Relativity is one of the most analyzed of all theories but it's still only a "theory".

Disproving things is much easier - you only need one decent anomaly for a theory to have to be abandoned.

This is why peer review works. Sure, 99 of 100 of the "peers" will be time wasters but usually there's somebody out there who's both interested in whatever it is you're proposing *and* competent.

And one person in a hundred (or even a million) is all it takes...

T.G.

Actually the US goverment is against climate change policies.
This seems like the USA war posture portrayed in an episode of South Park, where USA always plays safe doing whatever s**t they want but having quite a public opinion against so it looks fine as a whole.
I would add that in this case, it is more twisted, this blog is just as good as many to resume it, just reading what you americans think.
Mainly USA was against the climate change scientific conclussions, being accused of interested science, then they switched to appearing to endorse the climate change thing out of overwelming evidence that things are happening but without really doing nothing, but, then, knowing well the subject (like when you argue with an idiot on his terms, or the efective cop ... former thief) they accuse of the same thing they where obviously doing, of course, resources they have, undermining the whole thing.
It is amacing how you keep and potentiate to absurd limits the legacys your former metropoly left to you, call it, english units or twisted and dirt politics playing with public oppinion, demagogy, hypocrisy and so for (what is more funny is that your former metropoly is now your pet country).

stuart - velkairiwyth

Where did dogberts tail go in todays strip? You see it in the first slide, but come the third and the shot from behind... Tailless!!

*dun dun DUUUN!!!!*

Reckon someone stole it as memorabelia as they passed between previous slides?

neopolitan

About your "even better link" - check who is behind the "Independent Institute" and whose voice is most heard (according to http://www.sourcewatch.org/index.php?title=Independent_Institute).

Fred Singer - whose name also appeared in http://www.durangobill.com/Swindle_Swindle.html and http://www.sourcewatch.org/index.php?title=Oregon_Institute_of_Science_and_Medicine (when he made positive comment on the questionable petition).

Seems this guy is a busy fellow.

cheers,

neopolitan

Neil Wilson

Of course the modern world has done beyond peer review, where a tight cliche of individuals pass judgement on your life's work.

The modern equivalent is of course to publish the thing on the Internet, and then it can be reviewed, quoted and rubbished by as many people as can be bothered with it.

For Scientists to state that something that has been subject to worldwide review is somehow inferior to that judged by so-called experts is simply people trying to protect their livelihood.

Of course the biggest problem is journos after a story - scaring people half to death being the second best way to increase ratings after anything to do with sex.

NeilW

Steffen

I have done a few reviews so far and got reviewed a couple of times. And every time worked fine for me, although I heard some nasty stories at my institute. The main subject of a scientific article should always be how good it can explain what its point is. Naturally, the next issue is to find any possible flaws and then giving constructive suggestions on how to improve a manuscript.

The review process has some shortcomings. For instance, the decision is often based on the authors. If it is someone well known in the field the ms will be evaluated less restrictive because the reputation of the author suggests to the reviewer that the ms must be well written. This is quite often a strange criterion and in some cases to undeserved reputation.

Concerning the notion of there being a journal for any kind of result. This might be true, but my boss will be very reluctant to let me send a paper to a journal with low impact when the result could be published in a journal with high impact. Because the impact is a deciding factor for the money the institut gets granted...

Anyway, I close my statements with the link to an article about the quality of scientific articles. It is quite an amazing read.

http://improbable.com/2007/05/17/how-to-write-consistently-boring-scientific-literature/

Kathleen

Your comments are peer review are right on. I'm currently a graduate student in the hard sciences and there is a good amount of politics and bias involved in peer review. In fact, my adviser often goes on rampages about how some papers can leave out so much detail and get published, while he's having a difficult time getting a very detailed and supported paper published.

Although peer review has these issues, new discoveries in science are not discussed just once and immediately accepted. There's always someone following up on the work or trying to reproduce it and in the long run any major problems are found out and corrected. Bad data will eventually reveal itself, even if it takes years.

As far as getting a paper accepted, even if there is an asshole editor who won't accept it, there's a plethora of journals to submit papers to and one of them is bound to accept the paper. If not, then the paper probably sucks.

Kevin Kunreuther

Peer review in science historically has a terrible track record. Supposedly, it has made great strides, become more fair and less factional, in say, the last eighty years, but from talking to engineers and scientists friends, I am very inclined to say, things haven't changed much by far, there's only been a slight paradigm shift in alliances and prejudices. Still, if not for discussions in open forums, information could not be shared, dissected and discussed for the betterment of further enlightment. In Newton's day, information was shrouded, veiled, conveyed slowly and controlled by oligarchs of one kind or another, and jealous peers derided your work or claim you stole their ideas. And let's not forget the meddling church - if your work run up against church doctrine, and you published, at least worst your career was finished at worst worst you were dead man walking and so was your entire family. Nowadays, at worst worst, you're ridiculed by your peers because you're annoying and a crank and can't find a decent job at a university or in private sector in relation to your field of study until you've been "redeemed". At least worse, people disagree with you, but you still get the best jobs and respect because you're probably publicly not an annoying crank and have an unblemished and excellent resume.

Catherine

Why would you expect that two or three scientists would necessarily draw exactly the same conclusions about a manuscript or a grant (two of the main things that undergo peer review in the sciences)?

Acceptance or rejection of a manuscript or grant often depends almost entirely on significance. Perceived significance (like beauty) is inherently subjective. Science, like any other endeavor, is largely a normal distribution. The outstanding and the horrible are easy to identify and I suspect the correlation between reviews on these would be very homogeneous. It is in the middle, where subjective opinions will come into play. Is this work appropriate (high enough impact, interesting to diverse readers, etc.) for this specific journal? Is this work, compared with the other grants being reviewed at this time, in the top 7% or so?

Now add in the fact that almost all science today is "cross-disciplinary" meaning that reviewers of papers or grant proposals will have expertise in some, but not necessarily all, of the areas. Thus, you might have an engineer working with a biologist to review something. The biologist might find the proposal to be outstanding whereas the engineer identifies the fatal flaw.

As someone who has served on peer review panels for the last 7 years or so, I will say that in my experience, while there are some personality biases, most reviewers are fair, earnest and enthusiastic (if overworked and tired). We get to see the ideas and data long before anyone else. It is one of the greatest boons of being an academic.

Al

Well, see, the thing is, humans are programmed to be biased. There's no way around it. The closest you can likely get is "almost unbiased about this one topic"-ish, given that they know nothing on the topic (like that it even exists). But I have plenty of free time and I'm happy to see my peers succeed beyond me. But I'm often told that I'm really strange. And I can be "almost unbiased about this one topic"-ish about a great many things, since for most of my life I've been in the metaphorical cave on mars, with my hands over my ears, shouting at the top of my lungs in order to avoid knowing... well, anything that is commonly known, like politics, movies, and celebrities.

Ralph Becket

I'm a scientist and I can confirm that peer review is essentially about weeding out the howlers and obvious nonsense before it reaches an audience. Peer review is definitely not about reproducing experimental results; worse, it is quite possible for peer review to pass bad data or bad analysis. The number of scientists working in a given subfield is typically not that great (scientists are specialists) and it is possible to find an otherwise perfectly good paper being rejected because it treads on the toes of too many reviewers.

So: peer review is a sanity check, it is not a forensic audit. (This is one of the problems with the global warming debate: far too much weight is given to the value of peer review and concensus. I'm unaware of any other field where "the science is settled" which needs to use such arguments.)

-- Ralph

Jed Snole

The process does help to keep wishful thinking from affecting the outcome of experiments. You very rarely get any results these days where increasing X causes a nice linear increase or decrease in Y. Mostly you end up with a bunch of dots scattered all over a piece of graph paper. Then you have to figure out if they mean something. Humans are very good at pattern recognition - so good, in fact, that they can detect a pattern where none exists. When someone applies statistical analysis methods and smoothing to a bunch of random dots and then declares that they prove his hypothesis, it's a good sanity check to have someone say "It looks like just a bunch of dots to me."

Consider for a moment what would have happened if they went through a peer review process on the evidence before Bush went to war with Iraq. The people gathering and interpreting the evidence knew what the right answer had to be. A lot of the weapons of mass destruction that they saw in the satellite images were nothing more than just a couple of grayish pixels. They had no one to ask the necessary questions, so they saw what they wanted to see.

Roni

Jesus. Jesus would be the best peer reviewer. That's my answer for everything this week. No one argues against Jesus.

My favourite interview I saw this week was with a grand dragon of the KKK. When asked why they don't accept Jews into the KKK even though Jesus was a jew (because his mother was a jew) he said the blood line doesn't count for Jesus because whose to say he even has DNA. Brilliant!

My next favourite interview was that fundamentalist evolutionary scientist who says religion is a virus interviewing some priest, asking him how he knew the things in the bible are true and the priest says, because Jesus said "I am the truth". Fantastic!

The comments to this entry are closed.