In the post about Koch curves, I talked about how a grammar-rewrite system could be used to describe fractals. There’s a bit more to the grammar idea that I originally suggested. There’s something called an L-system (short for Lindenmayer system, after Aristid Lindenmayer, who invented it for describing the growth patterns of plants), which is a variant of the Thue grammar, which is extremely useful for generating a wide range of interesting fractals for describing plant growth, turbulence patterns, and lots of other things.
While reading Mandelbrot’s text on fractals, I found something that surprised me: a relationship between Shannon’s information theory and fractals. Thinking about it a bit, it’s not really that suprising; in fact, it’s more surprising that I’ve managed to read so much about information theory without encountering the fractal nature of noise in a more than cursory way. But noise in a communication channel is fractal – and relates to one of the earliest pathological fractal sets: Cantor’s set, which Mandelbrot elegantly terms “Cantor’s dust”. Since I find that a wonderfully description, almost poetic way of describing it, I’ll adopt Mandelbrot’s terminology.
Cantor’s dust is a pathological set. It’s caused no small amount of consternation among mathematicians and physicists who find it too strange, too bizarre to accept as anything more than an extreme artifact of logic in the realm of pure math. But it’s a very simple thing. To make it, you start a line segment. Cut it into three identical parts. Erase the middle one. Then repeat the cutting into thirds and erasing the middle on each of the two remaining segments – and then the segments remaining from that, and so on. The diagram below shows a few steps in the construction of the cantor dust.
Why should it be called a dust? Because geometrically, in the limit, it’s got to be a set of completely disconnected points – a scattering of dust across the original line-segment.
It’s so simple – what is pathological about it? Well, it’s clearly 0-dimensional in the pure sense – as I said above, it’s just a collection of points. Topologically, it’s a set of points with empty neighborhoods. And yet – look at it. It’s clearly not zero-dimensional. It’s got a 1-dimensional geometric structure. But naive topology insists that it doesn’t. But there’s worse to it. It’s a similar problem to what we saw in the shape-filling curves. Clearly, the dust disappears into nothingness. Every part of it has zero length – it’s seems like it must converge to something very close to the empty set. And yet it doesn’t: the set of points in the Cantor dust has the same cardinality as the cardinality of the set of points in the original line.
For those of us who came of age as math geeks in the late 20th century, this doesn’t really seem that strange at all. But you’ve got to remember: the 20th century was a time of great turmoil in math. At the beginning of the century, the great work was solving mathematics: turning all of math into a glorious, perfect, clean, rational, elegant edifice. The common belief of the time was that math was beautiful and perfect – that while the real world might be ugly, might have all sorts of imperfections and irrationalities, that those real-world flaws could never touch the realm of pure math: math was, in the words of one famous mathematician “the perfect mind of God”. And then came the crash: the ramifications of Cantor’s set theory, Gödel’s incompleteness, Church and Turing’s uncomputability, fractals, Chaitin’s strange numbers… The edifice collapsed; math was flawed, imperfect, incomplete as anything else in the world. It was hugely traumatic, and there was (and in some circles still is) a great deal of resistance to the idea that so much irregularity or ever irrationality was a part of the world of math – that the part of math that we can really grasp and use is just an infinitessimal part of the monstrous world of what really exists in our abstractions.
But getting back to the point at hand: what does Cantor’s dust have to do with information and noise?
Imagine that you’re listening to sound through a telephone wire with incredibly precise recording equipment. You’re sending a perfectly clear sine-wave over the line, in order to see how much noise there is.
You start pretty high – you only want to record noises that exceed, say, 20% of the amplitude of the basic sine-wave. You wind up with a pattern of bursts of noise. Those noises are scattered around, temporally. Now, mark off every time period of greater that 5 minutes where there is no noise. Those are gaps in the noise – the largest gaps that you’re going to look at. Now look in the bursts of noise – that is, the periods of time where there was no gap in the noise longer than 5 minutes. Look for periods of 1 minute where there wasn’t any noise. In between the 5 minute gaps, you’ll get a collection of smaller 1 minute gaps, separated by smaller bursts of noise. Then look into those 1 minute gaps, for 10 second periods with no noise – and you’ll break the bursts of noise up further, into bursts longer than 10 seconds, but shorter than a minute. Keep doing that, and eventually, you’ll run out of noise. But turn down your noise threshold so that you can hear noise of a smaller amplitude, and you can find more noise, and more gaps, breaking up the bursts.
If you look at the distribution of noise, one thing you’ll notice is that the levels are independent: the length of the longest gaps has no relation to the frequency of smaller gaps between them. And the other thing you’ll notice is that the frequency of gaps is self-similar: the distribution of long gaps relative to sections of the recording of long length are the same as the distribution of short gaps relative to shorter sections of the recording. The noise distribution is fractal! In fact, it’s pretty much a slightly randomized version of Cantor’s dust.
Understanding the structure of noise isn’t just interesting in the abstract: it provides a necessary piece of knowledge, which is used regularly by communication engineers to determine the necessary properties of a communication channel in order to ensure proper transmission and storage of information. Recognizing the fractal nature of noise makes it possible to better predict the properties of that channel, and determine how much information we can safely pump through it, and how much redundancy we need to add to the information to prevent data loss.
One of the strangest things in fractals, at least to me, is the idea of space filling curves. A space filling curve is a curve constructed using a Koch-like replacement method, but instead of being self-avoiding, it eventually contacts itself at every point.
What’s so strange about these things is that they start out as a non-self-contacting curve. Through further steps in the construction process, they get closer and closer to self-contacting, without touching. But in the limit, when the construction process is complete, you have a filled square.
Why is that so odd? Because you’ve basically taken a one-dimensional thing – a line – with no width at all – and by bending it enough times, you’ve wound up with a two-dimensional figure. This isn’t just odd to me – this was considered a crisis by many mathematicians – it seems to break some of the fundamental assumptions of geometry: how did we get width from something with no width? It’s nonsensical!
I just finally got my copy of Mandelbrot’s book on fractals. In his discussion of curve fractals (that is, fractals formed from an unbroken line, isomorphic to the interval (0,1)), he describes them in terms of shorelines rather than borders. I’ve got to admit
that his metaphor is better than mine, and I’ll adopt it for this post.
In my last post, I discussed the idea of how a border (or, better, a shoreline) has
a kind of fractal structure. It’s jagged, and the jags themselves have jagged edges, and *those* jags have jagged edges, and so on. Today, I’m going to show a bit of how to
generate curve fractals with that kind of structure.
Part of what makes fractals so fascinating is that in addition to being beautiful, they also describe real things – they’re genuinely useful and important for helping us to describe and understand the world around us. A great example of this is maps and measurement.
Suppose you want to measure the length of the border between Portugal and Spain. How long is it? You’d think that that’s a straightforward question, wouldn’t you?
It’s not. Spain and Portugal have a natural border, defined by geography. And in Portuguese books, the length of that border has been measured as more than 20% longer than it has in Spanish books. This difference has nothing to do with border conflicts or disagreements about where the border lies. The difference comes from the structure of the border, and way that it gets measured.
Natural structures don’t measure the way that we might like them to. Imagine that you walked the border between Portugal and Spain using a pair of chained flags like they use to mark the down in football – so you’d be measuring the border on 10 yard line segments. You’ll get one measure of the length of the border, we’ll call it Lyards
Now, imagine that you did the same thing, but instead of using 10 yard segments, you used 10 foot segments – that is, segments 1/3 the length. You won’t get the same length; you’ll get a different length, Lfeet.
Then do it again, but with a rope 10 inches long. You’ll get a *third* length, Linches.
Linches will be greater than Lfeet, which will be greater that Lyards.
The problem is that the border isn’t smooth, it isn’t a differentiable curve. As you move to progressively smaller scales, the border features progressively smaller features. At a 10 mile scale, you’ll be looking at features like valleys, rivers, cliffs, etc, and defining the precise border in terms of those. But when you go to the ten-yard scale, you’ll find that the valleys divide into foothills, and the border line should wind between hills. Get down to the ten-foot scale, and you’ll start noticing boulders, jags in the lines, twists in the river. Go down to the 10-inch scale, and you’ll start noticing rocks, jagged shapes. By this point, rivers will have ceased to appear as lines, but they’ll be wide bands, and if you want to find the middle, you’ll need to look at the shapes of the banks, which are irregular and jagged down to the millimeter scale. The diagram above shows a simple example of what I mean – it starts with a real clip taken from a map of the border, and then shows two possible zooms of that showing more detail at smaller scales.
The border is fractal. If you try to measure its dimension, topologically, it’s one-dimension – the line of the border. But if you look at its dimension metrically, and compute its Hausdorff dimension, you’ll find that it’s not 2, but it’s a lot more than 1.
Shapes like this really are fractal. To give you an idea – which of the two photos below is real, and which is generated using a fractal equation?
The most well-known of the fractals is the infamous Mandelbrot set. It’s one of the first things that was really studied as a fractal. It was discovered by Benoit Mandelbrot during his early study of fractals in the context of the complex dynamics of quadratic polynomials the 1980s, and studied in greater detail by Douady and Hubbard in the early to mid-80s.
It’s a beautiful example of what makes fractals so attractive to us: it’s got an extremely simple definition; an incredibly complex structure; and it’s a rich source of amazing, beautiful images. It’s also been glommed onto by an amazing number of woo-meisters, who babble on about how it represents “fractal energies” – “fractal” has become a woo-term almost as prevalent as “quantum”, and every woo-site that babbles about fractals invariably uses an image of the Mandelbrot set. It’s also become a magnet for artists – the beauty of its structure, coming from a simple bit of math captures the interest of quite a lot of folks. Two musical examples are Jonathon Coulton and the post-rock band “Mandelbrot Set”. (If you like post-rock, I definitely recommend checking out MS; and a player for brilliant Mandelbrot set song is embedded below.)
So what is the Mandelbrot set?
Take the set of functions where for each , is a complex constant. That gives an infinite set of simple functions over the complex numbers. For each possible complex number , you look at the recurrence relation generated by repeatedly applying , starting with :
If doesn’t diverge (escape) towards infinity as gets larger, then the complex number is a member of the Mandelbrot set. That’s it – that simple definition – repeatedly apply for complex numbers – produces the astonishing complexity of the Mandelbrot set.
If we use that definition of the Mandelbrot set, and draw the members of the set in black, we get an image like the one above. That’s nice, but it’s probably not what you expected. We’re all used to the beautiful colored bands and auras around that basic pointy black blob. Those colored regions are not really part of the set.
The way we get the colored bands is by considering *how long* it takes for the points to start to diverge. Each color band is an escape interval – that is, some measure of how many iterations it takes for the repeated application of to diverge. Images like the ones to the right and below are generated using various variants of escape-interval colorings.
My personal favorite rendering of the Mandelbrot set is an image called the Buddhabrot. In the Buddhabrot, what you do is look at values of which *aren’t* in the mandebrot set. For each point before it escapes, plot a point. That gives you the escape path for the value . If you take a large number of escape paths for randomly selected values of , and you plot them so that the brightness of a pixel is determined by the number of escape paths that cross that pixel, you get the Budddhabrot. It’s fascinating because it reveals the structure in a particularly amazing way. If you look at a simple unzoomed image of the madelbrot set, what you see is a spiky black blob; the actually complexity of the structure isn’t obvious until you spend some time looking at it. The Buddhabrot is more obvious – you can see the astonishing complexity much more easily.
I thought in addition to the graph theory (which I’m enjoying writing, but doesn’t seem to be all that popular), I’d also try doing some writing about fractals. I know pretty much nothing about fractals, but I’ve wanted to learn about them for a while, and one of the advantages of having this blog is that it gives me an excuse to learn about things that that interest me so that I can write about them.
Fractals are amazing things. They can be beautiful: everyone has seen beautiful fractal images – like the ones posted by my fellow SBer Karmen. And they’re also useful: there are a lot of phenomena in nature that seem to involve fractal structures.
But what is a fractal?
The word is a contraction of fractional dimension. The idea of that is that there are several different ways of measuring the dimensionality of a structure using topology. The structures that we call fractals are things that have a kind of fine structure that gives them a strange kind of dimensionality; their conventional topological dimension is smaller than their Hausdorff dimension. (You can look up details of what topological dimension and Hausdorff dimension mean in one of my topology articles.) The details aren’t all that important here: the key thing to understand is that there’s a fractal is a structure that breaks the usual concept of dimension: it’s shape has aspects that suggest higher dimensions. The Sierpinski carpet, for example, is topologically one-dimensional. But if you look at it, you have a clear sense of a two-dimensional figure.
That’s all frightfully abstract. Let’s take a look at one of the simplest fractals. This is called Sierpinski’s carpet. There’s a picture of a finite approximation of it over to the right. The way that you generate this fractal is to take a square. Divide the square into 9 sub-squares, and remove the center one. Then take each of the 8 squares around the edges, and do the same thing to them: break them into 9, remove the center, then repeat on the even smaller squares. Do that an infinite number of times.
When you look at the carpet, you probably think it looks two dimensional. But topologically, it is a one-dimensional space. The “edges” of the resulting figure are infinitely narrow – they have no width that needs a second dimension to describe. The whole thing is an infinitely complicated structure of lines: the total area covered by the carpet is 0! Since it’s just lines, topologically, it’s one-dimensional.
In fact, it is more than just a one dimensional shape; what it is is a kind of canonical one dimensional shape: any one-dimensional space is topologically equivalent (homeomorphic) to a subset of the carpet.
But when we look at it, we can see it has a clear structure in two dimensions. In fact, it’s a structure which really can’t be described as one-dimensional – we defined by cutting finite sized pieces from a square, which is a 2-dimensional figure. It isn’t really two dimensional; it isn’t really one dimensional. The best way of describing it is by its Hausdorff dimension, which is 1.89. So it’s almost, but not quite, two dimensional.
Sierpinski’s carpet is a very typical fractal; it’s got the traits that we use to identify fractals, which are the following:
Self-similarity: a fractal has a structure that repeats itself on ever smaller scales. In the case of the carpet, you can take any non-blank square, and it’s exactly the same as a smaller version of the entire carpet.
Fine structure: a fractal has a fine structure at arbitrarily small scales. In the case of the carpet, no matter how small you get, it’s always got even smaller subdivisions.
Fractional dimension: its Hausdorff dimension is not an integer. Its Hausdorff dimension is also usually larger than its topological dimension. Again looking at the carpet, it’s topological dimension is 1; it’s Hausdorff dimension is 1.89.
where for each 
, 
is a complex constant. That gives an infinite set of simple functions over the complex numbers. For each possible complex number 
, starting with 
:
doesn’t diverge (escape) towards infinity as 
gets larger, then the complex number 
for complex numbers – produces the astonishing complexity of the Mandelbrot set.
to diverge. Images like the ones to the right and below are generated using various variants of escape-interval colorings.
