Big Number Bogosity from a Christian College Kid

I know that I just posted a link to a stupid religious argument, but I was sent a link to another one, which I can’t resist mocking.

As I’ve written about quite often, we humans really stink at understanding big numbers, and how things scale. This is an example of that. We’ve got a jerk who’s about to graduate from a dinky christian college, who believes that there must be something special about the moral atmosphere at his college, because in his four years at the school, there hasn’t been a single murder.

Yeah, seriously. He really believes that his school is special, because it’s gone four whole years without a murder:

Considering that the USA Today calculated 857 college student deaths from 2000 to 2005, how does one school manage to escape unscathed? It’s certainly not chance or luck. For Patrick Henry College, it’s in our Christian culture.

Critics mock us for our strict rules – like no dancing or drinking on campus, no members of the opposite sex permitted in your dorm room, nightly curfew hours – and the lack of a social atmosphere it creates. We have been the subject of books (God’s Harvard), television shows, op-eds, and countless blogs who rant against our brand of overbearing right-wing Christianity that poisons society’s freedom.

Yet, what is the cost of students being able to “express” themselves? Is that freedom worth the cost of drunk driving deaths, drug related violence, and love affairs turned fatal?

There were 857 college student deaths in the five-year period from 2000 to 2005! Therefore, any college where there weren’t any murders in that period must be something really special. That christian culture must be making a really big difference, right?

Well, no.

According to Google Answers, the US Census Department reports that there are 2363 four year colleges in the US. So, assuming the widest possible distribution of student deaths, there were 1506 colleges with no student deaths in a five-year period. Or, put another way, more than 60% of colleges in the US went that five-year period without any violent student deaths.

Or, let’s try looking at it another way. According to the census, there are 15.9 million people currently enrolled in college. The school that, according to the author, is so remarkable for going without any murders in the last four years? It has 325 students. Not 325 per class – 325 total.

In other words, among a group making up less than 2/1000ths of one percent of the college population, there were no murders. Assuming that the distribution of violent deaths is perfectly uniform (which it obviously isn’t; but let’s just keep things simple), given that there were 857 violent deaths in the student population as a whole, how many violent deaths would you expect among the student body at his dinky christian college?

That would be a big, fat zero.

The fact that there were no violent deaths at his school isn’t remarkable, not at all. But to a twit who’s incapable of actually understanding what numbers mean, that’s not the conclusion to be drawn. It’s also not that the violent death among college students is actually remarkably rare. Nor is it that most college students will go through college without any violent deaths on campus. No – according to a twit, with 857 violent campus deaths over five years, the only reasonable conclusion is that there must be something special about the ridiculous religious rules at his college that prevented the great rampaging plague of violence from touching the students at his school.

I actually spent five years as an undergraduate at Rutgers University in NJ. During that time, there were no violent student deaths. (There was one death by alchohol poisoning; and there was one drunk driving accident that killed four students.) But zero violent deaths. Gosh, Rutgers must have been an absolutely amazingly moral university! And gosh, we had all of those horrible sinful things, like dancing, and co-ed dorms! How did we manage to go all that time with no violence?

It must have been the prayers of the very nice Rabbi at the Chabad house on campus. Yeah, that must be it! Couldn’t just be random chance, right?

Ok, now let me stop being quite so pettily snide for a moment.

What’s going on here is really simple. We hear a whole lot about violence on campus. And when you hear about eight-hundred and some-odd violent deaths on campus, it sounds like a lot. So, intuitively, it sure seems like there must be a whole lot of violence on campus, and it must be really common. So if you can go through your whole time in college without having any violence occur on campus, it seems like it must be unusual.

That’s because, as usual, we really suck at understanding big numbers and scale. 800 sounds like a lot. The idea that there are nearly sixteen million college students is just not something that we understand on an intuitive level. The idea that nearly a thousand deaths could be a tiny drop in the bucket – that it really amounts to just one death per 100,000 students per year – it just doesn’t make sense to us. A number like 800 is, just barely, intuitively meaningful to us. One million isn’t. Fifteen million isn’t. And a ratio with a number that we can’t really grasp intuitively on the bottom? That’s not going to be meaningful either.

Bozo-boy is making an extremely common mistake. He’s just simply failing to comprehend how numbers scale; he’s not understanding what big numbers really mean.

The Danger When You Don't Know What You Don't Know

A little bit of knowledge is a dangerous thing.

There’s no shortage of stupidity in the world. And, alas, it comes in many, many different kinds. Among the ones that bug me, pretty much the worst is the stupidity that comes from believing that you know something that you don’t.

This is particularly dangerous for people like me, who write blogs like this one where we try to explain math and science to non-mathemicians/non-scientists. Part of what we do, when we’re writing our blogs, is try to take complicated ideas, and explain them in ways that make them at least somewhat comprehensible to non-experts.

There are, arising from this, two dangers that face a math or science blogger.

  1. There is the danger of screwing up ourselves. I’ve demonstrated this plenty of times. I’m not an expert in all of the things that I’ve tried to write about, and I’ve made some pretty glaring errors. I do my best to acknowledge and correct those errors, but it’s all too easy to deceive myself into thinking that I understand something better than I actually do. I’m embarrassed every time that I do that.
  2. There is the danger of doing a good enough job that our readers believe that they really understand something on the basis of our incomplete explanation. When you’re writing for a popular audience, you don’t generally get into every detail of the subject. You do your best to just find a way of explaining it in a way that gives people some intuitive handle on the idea. It’s not perfect, but that’s life. I’ve read a couple of books on relativity, and I don’t pretend to really fully understand it. I can’t quite wrap my head around all of the math. That’s after reading several entire books aimed at a popular audience. Even at that length, you can’t explain all of the details if you’re writing for non-experts. And if you can’t do it in a three-hundred page book, then you certainly can’t do it in a single blog post! But sometimes, a reader will see a simplified popular explanation, and believe that because they understand that, that they’ve gotten the whole thing. In my experience, relativity is one of the most common examples of this phenomenon.

Todays post is an example of how terribly wrong you can go by taking an intuitive explanation of something, believing that you understand the whole thing from that intuitive explanation, and running with it, headfirst, right into a brick wall.

Continue reading

Iterative Hockey Stick Analysis? Gimme a break!

This past weekend, my friend Orac sent me a link to an interesting piece
of bad math. One of Orac’s big interest is vaccination and
anti-vaccinationists. The piece is a newsletter by a group calling itself the “Sound Choice
Pharmaceutical Institute” (SCPI), which purports to show a link
between vaccinations and autism. But instead of the usual anti-vac rubbish about
thimerosol, they claim that “residual human DNA contamintants from aborted human fetal cells”
causes autism.

Among others, Orac already covered the nonsense
of that from a biological/medical
perspective. What he didn’t do, and why he forwarded this newsletter to me, is because
the basis of their argument is that they discovered key change points in the
autism rate that correlate perfectly with the introduction of various vaccines.

In fact, they claim to have discovered three different inflection points:

  1. 1979, the year that the MMR 2 vaccine was approved in the US;
  2. 1988, the year that a 2nd dose of the MMR 2 was added to the recommended vaccination
    schedule; and
  3. 1995, the year that the chickenpox vaccine was approved in the US.

They claim to have discovered these inflection points using “iterative hockey stick analysis”.

Continue reading

Finger Trees Done Right (I hope)

A while ago, I wrote a couple of posts that claimed to talk about finger trees. Unfortunately, I really botched it. I’d read a bunch of data structure papers, and managed to get myself thoroughly scrambled. What I wrote about was distantly related to finger trees, and it was useful to help understand how fingertrees work – but it was not, in any way, shape, or form, actually a description of fingertrees. Since then, I’ve been meaning to write a proper post explaining finger trees – but with the work on my book, and with chaos at work, I just haven’t had the time. This time, in order to do my damnedest to make sure that I don’t screw it up again, I’m basically go to describe finger trees over a couple of posts by walking through the best finger-tree paper that I could find. The paper is “Finger Trees: a simple general-purpose data structure”, by Ralf Hinze and Ross Patterson. This might by the paper that introduced the structure, but I’m not sure.

The point of finger trees is pretty simple. It’s very similar to the point of zippers. Programming in functional languages is terrific. As I’ve described before, there are a lot of advantages to writing functional code. But there are also a lot of places where a naive implementation of an algorithm using a functional data structure is dreadfully inefficient. Functional code may be prettier, more maintainable, and more reusable – but imperative code is frequently much more efficient. When you’re doing an operation that, conceptually, modifies a piece of a complex data structure, then functional code can really suck. Finger trees give you a way around that – for many common updatabale data structures, you can build finger-tree versions that are very close to or fully as good as imperative, updating structures.

Continue reading

Friday Random Ten, 4/23/2010

  1. Stellardrive, Inlandsix: Reasonably good instrumental prog. They’re
    not particularly exceptional, but they’re decent.
  2. Gong, “The Octave Doctors and the Crystal Machine”: Gong is a
    perfect example of one of the differences between the great prog bands,
    and a lot of the neo-progressive stuff. I can’t quite describe exactly what it
    is – but you listen to a band like Gong, and you never get bored. You can listen
    to it over, and over – and it’s always interesting. Even though the individual
    features of the music are similar to what a lot of less brilliant bands do,
    they manage to put them together in a different way. I can listen to a neo-prog
    band like Jadis or Frost once or twice a month; if I listen to them more than
    that, they start to bug me. But I can listen to Gong twice a day, and never
    lose interest.
  3. Parallel or 90 Degrees, “Backup”: One of the really great neo-progressives.
    Po90 is Andy Tillison’s other band, and they are brilliant. Not as brilliant as
    groups like Gong, but pretty damned amazing.
  4. Jadis, “All You’ve Ever Known”: Here’s exactly what I’m talking about.
    The beginning of this Jadis track is actually sort-of like the Gong track above.
    But somehow, it’s dull when Jadis does it. Listening to them right after
    Gong and Po90, they frankly sound terrible. I really like Jadis, but they can’t
    hold a candle to the prog greats.
  5. And So I Watch You From Afar, “If it Ain’t Broke, Break It”: Really good
    post-rock. ASIWYFA is on the louder end of post-rock, and they’re really good
    at it. They’re one of my most recent post-rock discoveries, after being recommended
    to me by a reader of the blog, and I’m really enjoying them.
  6. Genesis, “Your Own Special Way”: And now, my favorite band of all time.
    I love Genesis. Even after Peter Gabriel left, they still wrote some of the
    best prog rock of all time. There’s a reason why so many neo-prog bands were
    inspired by them. Even when they’re doing a song like this, which is basically a silly sappy ballad,
    they make it into something really special.
  7. Jacob Hoffman with Kandel’s Orchestra, “Doina and Hora”: an incredibly old
    recording of traditional klezmer, led by probably the greatest Klezmer xylophone player
    ever. If you have any appreciation for Klezmer, this will absolutely knock your
    socks off.
  8. The Flower Kings, “Soul Vortex”: Ah, the Flower Kings. The only
    neo-progressive band that I’ve found that’s really as good as the original
    prog guys. Whatever that elusive “it” that the great bands had that made them
    endlessly listenable was, Roine Stolt and the Flower Kings have it.
  9. Transatlantic, “The Return of the Giant Hogweed”: On their latest album,
    Translatlantic added a disk of covers of their influences. Naturally, no
    group made up of members of the best neo-progressive bands could possibly
    not include a classic Genesis track. It’s a very faithful cover, and
    it works really well.
  10. Marillion, “Forgotten Sons”: An old favorite of mine: one of the
    lesser known tracks from Marillion’s very first full album. From the
    very start, Marillion was really something special.

Book Update

Very quick post here: the third beta of my AppEngine book “Code in the Cloud” was released this morning. If you’ve bought a copy of the beta, you can go to your pragmatic account, and download a fresh copy with all of the fixes and new material.

If you haven’t bought a copy… Well, if you’re interested in cloud programming, I’d like to think that this book is a pretty good overview of the subject. It’s about Google AppEngine, but I’ve done my best to write it so that it discusses the nature of cloud programming in general, using AppEngine as a specific example of a cloud platform. Buying a copy supports your friendly math blogger and makes me happy; and when I’m happy, I’m more likely to write more posts for the blog :-).

The main reason that I’m mentioning it is that a few people have asked me to provide a forum on the blog for questions. I’m happy to answer questions, and I’d love to hear feedback from anyone who’s read it – both positive and negative. (And, to be honest, the negative feedback is generally more useful, so I’m very serious when I say that constructive negative feedback is welcome. Anything that you can tell me now, before it’s printed, is something that I can fix!)

So if you’ve got any questions or comments about the book, please go ahead and put them in the comments here. On the other hand, if you find any errors, you’re welcome to put them here, but it would help me more if you could file an erratum at the book’s pragmatic press page, linked above.

Shocking Fraud from Financial Scum

Against my better judgement, I’ve ended up writing a lot about the
financial mess that we’re currently going through. If you’ve read that, you
know that my opinion is that the mess amounts to a giant pile of fraud.

But even having spent so much time reading and studying what was
going on, the latest news from the financial mess shocks me.
Even knowing how utterly sleazy and dishonest many people in the financial world
have been, even knowing about the stuff they’ve been doing, the kinds of
out and out fraud that they’ve perpetrated, the latest news makes them
look even more evil than I could have imagined.

Continue reading

I am a racist

(Unfortunately, this post has been linked to by a white supremacist site. Instead of providing a forum for their foulness, I’m shutting down comments on this post.)

Unfortunately, I lost the link that inspired this. But I recently saw a post by a conservative about “reclaiming” the word racist. It went on to list a collection of reasons why he was a racist. The gist of it was that all of us dirty liberals were the real racists – because there’s no possible reason for us to support things like affirmative action, welfare, etc., unless we really, deep down, believe that minorities – particularly blacks – are stupid animals incapable of taking care of themselves.

It’s typical bullshit. So I’m responding in my own way. Because, you see, I am a racist. I’m not proud of that fact – but growing up in a deeply racist and sexist culture, you can’t avoid absorbing racist and sexist messages and attitudes into your worldview. And the blogger who inspired this is, like me, a member of the privileged elite. The difference between us is that I at least try to notice the effects of my privilege. I don’t support social justice programs like affirmative action, welfare, and job training because I think that poor black people need help because they’re less smart than me: I think that people like me have unfair advantages that we rarely appreciate, and that everyone deserves the same advantages that I’ve been lucky enough to receive. But however idealistic I am, however commited I am to social justice, the fact remains: I am, to my shame, a racist.

  1. I am a racist – because I never noticed all of the unearned privileges that are given to me until someone pointed them out.
  2. I am a racist – because even after learning about the unearned privileges
    that I recieve, I still don’t notice them.
  3. I am a racist, because I have grown up in a culture that, at every turn, teaches
    me that to be white is to be better, and smarter, and I have absorbed that lesson.
  4. I am a racist, because I instinctively react to members of minorities with fear.
  5. I am a racist, because I live in a sunset town.
  6. I am a racist, because I believe that I deserve the success I
    have, even though I know people who are more smart, capable, and
    talented than I am never had the chances that I did to
    be successful, because of the color of their skin.
  7. I am a racist – because I am a white man who has directly benefited from
    the unfair preferences that have been directed towards me all of my life.
  8. I am a racist – because every day, I benefit from the denial of
    basic privileges to other people.
  9. I am a racist, because I do not notice the things that are denied to people
    who are different from me.
  10. I am a racist, because I do not notice the advantages that I have over
    others.
  11. I am a racist, because even when I do manage to notice what is denied
    to people of different races and backgrounds, I don’t speak up.

The point of this isn’t just to do a sort of “walk of shame”. The
point is that I am an incredibly lucky person, who has benefited from
all sorts of things – from where I was born, to the color of my skin,
to the background of my parents, to my gender. I have recieved, and
continue to receive benefits because of those, and many other factors
that have nothing to do with my own merit. And except for
very rare occasions, that goes unremarked, unnoticed.

People like me think of ourselves as the default – as “normal”
people. We consider the incredible advantages that we receive to
be normal, unremarkable. We don’t notice just how much we benefit
from that assumption of our own normality – the benefits we
receive fade into invisibility. We don’t even notice that they exist. And
then when someone who doesn’t get those benefits
has trouble, we naturally blame them for not being as successful as we
are.

The underlying theme of people like the jerk who inspired this
post is: “I made it by myself, without any help. So
they should be able to make it by themselves, without any
help either.”

But that’s bullshit, because none of us “made it by ourselves”. We’re
the beneficiaries of the system we live in.

I grew up in a wealthy town in NJ. We didn’t consider ourselves
wealthy – but by comparison to lots of other people, we really were.
I went to a very good school system. We complained about it a lot:
the textbooks were too old; the equipment in the science labs were too
beaten up; the classes were too easy, and so on.

When I was in college, I got to teach a summer program for top
students from schools in Newark, Camden, and Jersey City. And I
discovered that my students went to schools where they didn’t have to
worry about their books being too old – because they didn’t
have any books. I mean that literally: in their english
classes, they didn’t have books, because their schools had
never been able to buy new books since it opened – and the
books had long since fallen apart. They didn’t complain about the
lousy lab equipment – because their schools had never had
science labs at all. How could people coming from schools like that
possibly hope to compete with students from a school like
mine? I didn’t admitted to college over people from their schools because
I was smarter. I got admitted into college over people from their
schools because I was richer and whiter.

And when my students went to the campus bookstore to buy
basic supplies like paper and pencils, the people who worked there
followed them around the store – because what would a
bunch of poor black kids be doing in a bookstore if they weren’t
there to rob it?

I write this math blog for fun. How did I get the background to do
it? I come from a highly educated family. They taught me to read
before I even started preschool. I’d learned about statistics from my
father when I was in third grade. I learned about algebra in sixth
grade, even though my school didn’t teach it until 8th or 9th. I
learned calculus in my freshman year in high school – even though my
school didn’t teach it until a senior year AP class. I was learning this stuff
long before the school taught it to me; and my parents made sure that
they bought a house in a very expensive school district where there would
be things like AP classes. My parents paid for me to go to college – which gave
me the time to take courses not just because I needed them to graduate,
but because they covered things that I wanted to learn, just for fun.

How could a person from a family that just managed to scrape by,
who lived in a school system that couldn’t afford textbooks for the
basic classes, much less the AP classes, how could they compete with
me? It’s damned close to impossible. Not because they’re any less
smart, or any less talented. But because I’ve had an absolutely
uncountable number of advantages. Every day of my life, I’ve been
given benefits which helped make it possible for me to become who and
what I am. I’m here partially because I’ve worked damned hard
to get here. But that work, by itself, wouldn’t have gotten me to where I am,
without luck and privilege.

People like me need to remember that. We didn’t earn what we have
all by ourselves. We may have earned part of it – but only
part. An awful lot of what we have is built on privilege: on the advantages
that we’ve been given because of race, gender, wealth, and family.

Financial Shenanigans: the Repo 105

glenfarclas-105-aged-40-years-lr.jpeg

I’m glad to report that electricity has been restored to the Chu-Carroll
household. So now I’m trying to catch up.

During the outage, I got a bunch of questions about the latest news coming
out of the big financial disasters. A major report came out about the failure
of Lehman Brothers, and one thing that’s been mentioned frequently is
something called repo105.

The whole repo105 thing is interesting to me, not so much because of what
it actually means, but because of how it’s been reported. The term has been
mentioned everywhere – but trying to find any information about just what the
hell it means seems to be next to impossible. It’s absolutely amazing how many
places have reported on it without bothering to explain it.

Continue reading

Code in the Cloud: My Book Beta is Available!

As I’ve mentioned before, I’ve been spending a lot of time working on a book.
Initially, I was working on a book made up of a collection of material from blog posts;
along the way, I got diverted, and ended up writing a book about cloud computing using
Google’s AppEngine tools. The book isn’t finished, but my publisher, the Pragmatic Programmers,
have a program that they call beta books. Once a book is roughly 60% done, you
can buy it at a discount, and download drafts electronically immediately. As more sections
get done, you can download each new version. And when the book is finally finished, you
get a final copy.

We released the first beta version of the book today. You can look at
excerpts, or buy a copy, by going to
the books page
at Pragmatic’s website.

If you’re interested in what cloud computing is, and how to build cloud applications – or if
you just feel like doing something to support you friendly local math-blogger – please take
a look, and consider getting a copy. I’m not going to harp about the book a lot on the blog; you’re
not going to see a ton of posts that are thinly veiled advertisements, or updates tracking
sales, or anything like that. If there’s something that I would have written about anyway,
and it’s appropriate to mention the book, then I’ll feel free to mention it, but I won’t
waste your time hyping it.

In other news, here’s the main reason that things have been dead on this blog since
the weekend:

photo.jpeg

That’s the view from my driveway as of monday morning. Over the weekend,
we had one of the worst windstorms to hit New York in about thirty years. That
mess is two oak trees, each close to 2 meters in diameter, which came down on
our street on saturday. (If you look closely towards the right hand side, you
can see the remains of my neighbors car.) The telephone pole in the picture
was snapped not by getting hit by a tree, but simply by the wind. Since that
pole had our electrical transformer, and those trees took out the wiring that
fed that transformer, we are (obviously) without electricity, internet, or
(most importantly) heat.

Con-ed is promising to restore our electricity by friday. I’m not holding my
breath.

Anyway, back to the happy stuff. The book exists in electronic form! Buy
a copy for yourself, your friends, your neighbors, and your dog! We’ve got lots
of wonderful new expenses to deal with recovering from that storm! 🙂