Censorship == They Won't Work For Me For Free!

Sorry, but as a software guy, I just couldn’t resist mocking the sheer insane
hypocrisy of this.

There’s a right-wing political site out there, called RedState.com. RedState
is serious far-right – constantly bemoaning the nanny-state, the culture of entitlement,
the virtues of personal responsibility, and so on. According to RedState, Social Security
is bad – people should save for their own retirements, not rely on the government to take care of them. Socialized medicine must be avoided at all costs: people should pay for their
own medical insurance, not expect the state to do it for them. And so on.

So, RedState initially set up their state on Scoop. If you don’t know, Scoop is
some software brewed up for a geek-news and discussion site called Kuro5shin. Scoop
is a free software, implemented in Perl, and distributed under that manifesto of
right-wing entepreneurship, the GNU Public License. (For those not in the know, I’m
being sarcastic here. While it’s generally silly to try to attribute political attributes to software licenses, I think it would be reasonable to say that if you characterized open-source licenses in terms of the political spectrum, the GPL would be solidly in the socialist area.)

After running for a while, Scoop wasn’t up to the load. So they switched to
another free package, called Drupal. Drupal is, like Scoop, free, and distributed under the GPL. But Drupal didn’t have as many features as Scoop, which frustrated
the RedState guys. So what does a good, responsible, self-sufficient, entrepreneurial
organization like RedState do when the free software that they’re using isn’t up to the job
that they want it to do?

Naturally: They whine about how no one will fix it for them for free, and how
the unwillingness of people to do free work for them is just a totally unfair
attempt by those nasty rotten liberals to censor them:

The bad news: our liberal “friends” – you know, the ones who believe so strongly in free speech and open debate – have done what they can to prevent us from making these improvements, so that our influence will be minimized just as we head into the 2008 presidential primary season.

No, our Blue State buddies haven’t succeeded in stopping us from improving our website. But they’ve made it more difficult and more expensive – which is why I’m coming to you for help.

Let me explain …

You see, when we started RedState in May of 2004, we used a website program called Scoop — the same program a lot of similar sites on the left used. But, as the number of visitors to our site grew, Scoop kept crashing on us.

If we’d been a liberal website, we would have been able to fix the problem quickly and relatively cheaply. The online left loves Scoop. Unfortunately, there weren’t really any conservative Scoop developers out there to help us. We kept crashing and were out of money. We had to close down or take drastic action.

Well, we didn’t close down. We ditched Scoop and moved to the best alternative at the time, a program called Drupal. But, in accomplishing the switch, budget constraints forced us to sacrifice some popular site features in order to alleviate the strain on our overused servers.

Needless to say, we always regarded those “downgrades” as temporary, and we hoped to restore the eliminated features – and to add new and even better ones – as soon as we could afford to.

Unfortunately, we still can’t afford to. But we’re convinced that America can afford even less to have us operating at anything less than our absolute peak potential during the coming presidential election season.

So we’ve decided to move ahead with our upgrades without delay, and despite not having the cash on hand – hoping and praying that RedState.com readers like you will help us make up the shortfall with a generous donation.

Continue reading

Simple Lempel-Ziv Compression in Erlang

I decided to do a little bit of something useful with Erlang, both to have some code to show, and to get some sense of what it’s like writing something beyond a completely trivial example. Because the capabilities of Erlang shine when you get into low-level bit oriented things, I thought that writing a bit of data compression code would be make for a good example. I’m going to present it in two parts: first a simple but stupid version of the algorithm; and then the encoding part, which into bit twiddling, and potentially gets more interesting.

I’m going to use a simple version of Lempel-Ziv compression. Before I get into the detail
of how LZ compression works, I’ve got an interesting story about how I learned about it. My first summer in grad school, I was looking for a job. One of my college friends worked for a subsidiary of Travellers insurance, and he got me an internship with them.

Our group (3 of us) worked on programs that ran on salepeoples’ laptops. Since this
was 1991, laptops were still pretty primitive. We had to run in 128K of memory, with the best machines having 10 megabytes of hard disk, and the worst machines having two floppies. So memory use was always a huge issue.

Being an insurance company, naturally things were pretty bureaucratic. They hired me to write a prototype of a new program. I don’t remember the details of what I was supposed to write. But the way things worked, they wanted to build this new tool for the salespeople. But the company wouldn’t let them propose a new project without having staffing for it. So they hired me to work on it. But because they hadn’t proposed the project before they hired me,
I had nothing to do while they proposed it, and worked out their requirements. So I worked for them for a total of about 12 weeks; and it took them about 9 1/2 weeks to get to the point where they had an approved project for me to work on. So I had over two months with nothing
to do. So I kept pestering the other two guys about what I could do to help them.

One of the things they wanted was a way of doing a splash screen. They had a GIF image of the company logo, and they thought it would be nice to be able to splash it on the screen whenever the sales app loaded. But they didn’t have an image viewer that they could call from inside their program. So they asked me to write one. GIF images are encoded using LZ. So I coded that up the LZ decoder to get the bitmap out of a GIF in C, and they were happy. Then I decided, as long as I had an LZ decompressor, I should write an LZ compressor, and then we’d have some generic data compression code. So I went ahead and did that, and added a
nice, effective set of data compression routines to our libraries. But the manager was actually pissed at me: I’d added a feature to our library – data compression – without getting permission. The right way of doing things was to write a proposal, and pass it around the various levels of petty bureaucrats for two months while I sat and twiddled my thumbs on the payroll.

Anyway… Back to compression.

Continue reading

Bad God Proofs: the Islamic Version

I’ve been sent Yet Another Proof of God. This one goes to rather a lot of trouble
to appear to be mathematical. I thought that it would be fun to rip it apart. For a change, this one is from an Islamic moron, rather than the usual Christian moron.
Alas, it’s pretty much as stupid and shallowly wrong as the usual christian one.

Continue reading

Common Sense: Math is the Root of All Evil (including Evolution)

Sorry that the blog has been so quiet lately; I managed to catch a vicious flu for the first time since I started getting flu shots, so I’ve been feeling too ill to write. I’m still far from recovered, but I’m feeling well enough to share a bit of delightful foolishness with you.

After seeing my recent post about a relativity denier, a reader sent me a link to another extremely amusing anti-relativity site. (In fact, I’ve recieved a bunch of links to anti-relativity sites; I’m only posting the most amusing ones.) This one has
several particularly amusing properties, but from my point of view what makes it
such a great target is that it uses the mathematical precision of relativity as part of its argument against it. You see, math is ultimately the basis of a grand anti-religious conspiracy to replace god with randomness and evolution!

The site is called “Common Sense Science”. Alas, the full range of their insanity isn’t available for me to view: they publish textbooks and journals, and to get the full details about their “theories”,
you must buy the books and/or subscribe to the journals. But there’s more than
enough there on the site to see what they’re going on about.

Continue reading

The Evil, Imprecise, Confusing Metric System

Yet another alert reader just sent me a link to this extremely
humorous blog
. It’s not recent, but it’s silly enough that it’s worth pointing
out even now. I’m not one hundred percent sure that this isn’t a parody. Looking at
the blog as a whole, I think it’s serious. Pathetic, but serious.

According to scientists, The French / Eurotrash “metric” system is in a state of crisis today, as scientists discovered that the weight of the “kilogram” is flip-flopping faster than John McCain at an episcopalian rally.

The Kilo – a unit with an identity crisis: This is one of the approximately 30 standard kilograms stored in a French castle. The polished metallic pellet a the center of this contraption is made of an alloy of precious metals, provided at government expense. Only in Europe would government subsidize such folly!

Since Europe converted to the metric system a few years back, this could have a grave impact on that continent’s faltering economy. Imagine a situation where customers wishing to buy some meat, or fill their car with gas literally have no idea how much their goods will cost, simply because nobody knows the exact definition of a weight.

Jesus, the humble carpenter no doubt used a ruler to measure his wood every day, however the Bible clearly states that he had no need of the metric system.

How much simpler life would be if we only adopted the standard units of weights and measures mentioned in the King James Bible? Biblical measures are authorized by God, and like God are not tied down to fickle matter. This is yet another example of how Science only really makes sense when it is rooted in the Bible.

Continue reading

Building up more: from Groups to Rings

If you’re looking at groups, you’re looking at an abstraction of the idea of numbers, to try to reduce it to minimal properties. As I’ve already explained, a group is a set of values with one operation, and which satisfies several simple properties. From that simple structure comes the
basic mathematical concept of symmetry.

Once you understand some of the basics of groups and symmetry, you can move in two directions. You can ask “What happens if I add something?”; or you can ask “What happens if I remove something?”.

You can either add operations – which can lead you to a two-operation structure called a ring; or you can add properties – in which the simplest step leads you to something called an abelian group. When it comes to removing, you can remove properties, which leads you to a simpler structure called a groupoid. Eventually, I’m going to follow both the upward and the downward paths. For now, we’ll start with the upward path, since it’s easier.

Building up from groups, we can progress to rings. A group captures one simple property of
a set of number-like objects. A ring brings us closer to capturing the structure of the system
of numbers. The way that it does this is by adding a second operation. A group has one operation
with symmetric properties; a ring adds a second symmetric operation, with a well-defined relationship
between the two operations.

Continue reading

The Meaning of Division: Quotient Groups

After that nasty diversion into economics and politics, we now return to your
regularly scheduled math blogging. And what a relief! In celebration, today I’ll give
you something short, sweet, and beautiful: quotient groups. To me, this is a shining
example of the beauty of abstract algebra. We’ve abstracted away from numbers to these
crazy group things, and one reward is that we can see what division really means. It’s
more than just a simple bit of arithmetic: division is a way of describing a fundamental
structural relationship that pervades mathematics.

Continue reading

Insurance: Why it sucks

Look folks, I don’t want to become an economics blogger! Stop sending me economics questions. I hate to disappoint my readers and not answer their questions, but this economics stuff is almost terminally dull to me.

The mortgage posts have gotten an insane amount of traffic, which has in turn brought in a huge number of questions. Most of them are about details of the whole mortgage situation – and honestly, I can’t answer those. I don’t know the details, only the basics, and I can’t explain what I don’t know.

On the other hand, a lot of people have used my down-to-earth explanation of the mortgages as
a springboard to ask for a similar explanation of another issue that’s been getting a lot of attention in America – insurance. Why is health insurance such a big problem?

I’m going to focus specifically on the insurance part. There are plenty of things wrong with the
American medical care system. But I’m going to focus specifically on how insurance works, and
how simple min/max calculations led to the current situation.

Continue reading

Responding to a FAQ: What is Tranching?

I’ve received an amazing number of requests in the short period of time since my last post to
explain “Tranching”. I mentioned it off-handedly, but a lot of people have heard about its role in the
whole sub-prime mess, and wanted to know just what it means. I don’t particularly like writing
about economics; it’s just not my bag. But enough people are asking that I feel like I need to answer the question. But this is it folks – no more of this nonsense after today! There are plenty of other people writing about this, who know more about it, and who are more interested in it, than I am.

It’s also a relatively simple idea. You’ve got a bundle made up of, say, 100 loans. You want
to sell at least part of that bundle as a super-safe rated investment. But because it’s formed from
risky loans, you know that at least some of them are likely to default.

So what you do is divide it into tiers called tranches. Suppose you’ve got 5 tiers – levels 1
through 5, where 5 is the top tier, and five is the lowest. Each tier covers a part of the bundle. So you might sell 20% of each tier. When the loans are repaid, the way that it works is that tier 1 is repaid first. When tier 1 is fully repaid, then you start paying tier 2, and so on. You can think of it like an overflowing cascade: repayments go to tier one; when it’s full, the overflow goes to tier 2; when that’s full, the overflow goes to tier 3, and so on. So if anyone doesn’t repay the money,
the people holding the lowest tier, T5, lose everything before T4 loses anything, and so on up the stack.

Let’s look at an example. Suppose you sell $100 million worth of loans. You divide it into
5 tranches. So you sell $200,000 of the value of the loans as tier 1, $200,000 as tier 2, etc.

  1. During the first year of the loans, $100,000 is expected to be repaid, and $100,000 is
    repaid. Each tier gets $20,000.
  2. During the second year, $100,000 is expected to be repaid, but repayment falls short. Only
    $90,000 is repaid. Tiers one, two, three, and four each get their $20,000; tier 4 takes
    the loss, and only gets $10,000.
  3. During the third year, things get worse – only $50,000 gets repaid. Tier one and two each
    still get $20,000. Tier three loses half of its expected payment, collecting $10,000. Tiers
    three and four get nothing.

Tranching is a good idea. Once again, it’s a good way of dividing risk. Anyone who invests in risky
loans is taking a chance, but tranching let you divide the chances up, so that people who want safety
can buy the top tranches, get less of a profit, but know that they’re not going to get screwed unless
things really go seriously bad; people who are willing to take their chances in the lower tranches know
that they’re taking a significant risk, but they can potentially make a lot more money.

The key, though, is dividing into tranches properly. If, as in the example above, you had five tranches, each taking up 20% of the pie, then the top tranch would be pretty safe: you’d have to lose 80% of the value of the loans before that tranch lost anything – and in mortgage loans, even
shitty ones, losing 80% of the value would be quite extraordinary.

But that’s not how banks set things up. Remember that you’ve got investors practically begging to
get their grubby little paws onto these high-return, low-risk bonds. But it’s really only the upper tranch that people wanted. The lower tranches – especially the lowest – were hard to get rid of. So, what the banks did is, once again, screw around with things. Some bundles of shit loans were divided into tranches where 80% of the value of the loans were sold as
tip-top ultra-safe investments. That means that if a package of crappy high-risk loans loses 20% through
loan default and foreclosure that the principal of the ultra-safe investments are going to be
lost. That’s not what any sane person considers “ultra-safe”.

And then, they took the bottom tranches, and re-bundled them. Take the bottom tranch from a hundred different packages of shit loans, bundle it into a new set of bonds, and re-tranch it. Then sell the top 80% of that as the top tranch of a bundle. And so on. In many cases, the investors have no clue how many levels of re-bundling are going on to create their top-tranch low-risk bond. And you’ve got all sorts of people who thought they were buying conservative investments who are now stuck with their money invested in bundles of low-tranch shit loans.

The Total Stupidity of Crowds: Bad Mortgages and Circular Solutions

Reading the news lately, I’ve come across an amazing example of how ubiquitous bad math can be used. Most of you have probably heard about what’s been called “the sub-prime crisis”. Despite a lot of media hand-wringing about how complicated it all is, the sub-prime crisis is really a very simple phenomenon: basically, you’ve got a lot of banks that have loaned out money without worrying about whether or not it could get paid back, and now those loans aren’t getting paid back, which is causing all sorts of grief to people who invested in them.

Continue reading