The last major property of a chaotic system is topological mixing. You can
think of mixing as being, in some sense, the opposite of the dense periodic
orbits property. Intuitively, the dense orbits tell you that things that are
arbitrarily close together for arbitrarily long periods of time can have
vastly different behaviors. Mixing means that things that are arbitrarily far
apart will eventually wind up looking nearly the same – if only for a little
while.
Let’s start with a formal definition.
As you can guess from the name, topological mixing is a property defined
using topology. In topology, we generally define things in terms of open sets
and neighborhoods. I don’t want to go too deep into detail – but an
open set captures the notion of a collection of points with a well-defined boundary
that is not part of the set. So, for example, in a simple 2-dimensional
euclidean space, the contents of a circle are one kind of open set; the boundary is
the circle itself.
Now, imagine that you’ve got a dynamical system whose phase space is
defined as a topological space. The system is defined by a recurrence
relation: sn+1 = f(sn). Now, suppose that in this
dynamical system, we can expand the state function so that it works as a
continous map over sets. So if we have an open set of points A, then we can
talk about the set of points that that open set will be mapped to by f. Speaking
informally, we can say that if B=f(A), B is the space of points that could be mapped
to by points in A.
The phase space is topologically mixing if, for any two open spaces A
and B, there is some integer N such that fN(A) ∩ B &neq; 0. That is, no matter where you start,
no matter how far away you are from some other point, eventually,
you’ll wind up arbitrarily close to that other point. (Note: I originally left out the quantification of N.)
Now, let’s put that together with the other basic properties of
a chaotic system. In informal terms, what it means is:
Exactly where you start has a huge impact on where you’ll end up.
No matter how close together two points are, no matter how long their
trajectories are close together, at any time, they can
suddenly go in completely different directions.
No matter how far apart two points are, no matter how long
their trajectories stay far apart, eventually, they’ll
wind up in almost the same place.
All of this is a fancy and complicated way of saying that in a chaotic
system, you never know what the heck is going to happen. No matter how long
the system’s behavior appears to be perfectly stable and predictable, there’s
absolutely no guarantee that the behavior is actually in a periodic orbit. It
could, at any time, diverge into something totally unpredictable.
Anyway – I’ve spent more than enough time on the definition; I think I’ve
pretty well driven this into the ground. But I hope that in doing so, I’ve
gotten across the degree of unpredictability of a chaotic system. There’s a
reason that chaotic systems are considered to be a nightmare for numerical
analysis of dynamical systems. It means that the most miniscule errors
in any aspect of anything will produce drastic divergence.
So when you build a model of a chaotic system, you know that it’s going to
break down. No matter how careful you are, even if you had impossibly perfect measurements,
just the nature of numerical computation – the limited precision and roundoff
errors of numerical representations – mean that your model is going to break.
From here, I’m going to move from defining things to analyzing things. Chaotic
systems are a nightmare for modeling. But there are ways of recognizing when
a systems behavior is going to become chaotic. What I’m going to do next is look
at how we can describe and analyze systems in order to recognize and predict
when they’ll become chaotic.
Based on a recommendation from a commenter, I’ve gotten another book on Chaos theory, and it’s frankly vastly better than the two I was using before.
Anyway, I want to first return to dense periodic orbits in chaotic systems, which is what I discussed in the previous chaos theory post. There’s a glaring hole in that post. I didn’t so much get it wrong as I did miss the fundamental point.
If you recall, the basic definition of a chaotic system is a dynamic system with a specific set of properties:
Sensitivity to initial conditions,
Dense periodic orbits, and
topological mixing
The property that we want to focus on right now is the
dense periodic orbits.
In a dynamical system, an orbit isn’t what we typically think of as orbits. If you look at all of the paths through the phase space of a system, you can divide it into partitions. If the system enters a state in any partition, then every state that it ever goes through will be part of the same partition. Each of those partitions is called an orbit. What makes this so different from our intuitive notion of orbits is that the intuitive orbit repeats. In a dynamical system, an orbit is just a set of points, paths through the phase space of the system. It may never do anything remotely close to repeating – but it’s an orbit. For example, if I describe a system which is the state of an object floating down a river, the path that it takes is an orbit. But it obviously can’t repeat – the object isn’t going to go back up to the beginning of the river.
An orbit that repeats is called a periodic orbit. So our intuitive notion of orbits is really about periodic orbits.
Periodic orbits are tightly connected to chaotic systems. In a chaotic system, one of the basic properties is a particular kind of unpredictability. Sensitivity to initial conditions is what most people think of – but the orbital property is actually more interesting.
A chaotic system has dense periodic orbits. Now, what does that mean? I explained it once before, but I managed to miss one of the most interesting bits of it.
The points of a chaotic system are dense around the periodic orbits. In mathematical terms, that means that every point in the attractor for the chaotic system is arbitrarily close to some point on a periodic orbit. Pick a point in the chaotic attractor, and pick a distance greater than zero. No matter how small that distance is, there’s a periodic orbit within that distance of the point in the attractor.
The last property of the chaotic system – the one which makes the dense periodic orbits so interesting – is topological mixing. I’m not going to go into detail about it here – that’s for the next post. But what happens when you combine topological mixing with the density around the periodic orbits is that you get an amazing kind of unpredictability.
You can find stable states of the system, where everything just cycles through an orbit. And you can find an instance of the system that appears to be in that stable state. But in fact, virtually all of the time, you’ll be wrong. The most minuscule deviation, any unmeasurably small difference between the theoretical stable state and the actual state of the system – and at some point, your behavior will diverge. You could stay close to the stable state for a very long time – and then, whammo! the system will do something that appears to be completely insane.
What the density around periodic orbits means is that even though most of the points in the phase space aren’t part of periodic orbits, you can’t possibly distinguish them from the ones that are. A point that appears to be stable probably isn’t. And the difference between real stability and apparent stability is unmeasurably, indistinguishably small. It’s not just the initial conditions of the system that are sensitive. The entire system is sensitive. Even if you managed to get it into a stable state, the slightest perturbation, the tiniest change, could cause a drastic change at some unpredictable time in the future.
This is the real butterfly effect. A butterfly flaps its wings – and the tiny movement of air caused by that pushes the weather system that tiny bit off of a stable orbit, and winds up causing the diversion that leads to a hurricane. The tiniest change at any time can completely blow up.
It also gives us a handle on another property of chaotic systems as models of real phenomena: we can’t reverse them. Knowing the measured state of a chaotic system, we cannot tell how it got there. Even if it appears to be in a stable state, if it’s part of a chaotic system, it could have just “swung in” the chaotic state from something very different. Or it could have been in what appeared to be a stable state for a long time, and then suddenly diverge. Density effectively means that we can’t distinguish the stable case from either of the two chaotic cases.
Another one of the fundamental properties of a chaotic system is dense periodic orbits. It’s a bit of an odd one: a chaotic system doesn’t have to have periodic orbits at all. But if it does, then they have to be dense.
The dense periodic orbit rule is, in many ways, very similar to the sensitivity to initial conditions. But personally, I find it rather more interesting a way of describing key concept. The idea is, when you’ve got a dense periodic orbit, it’s an odd thing. It’s a repeating system, which will cycle through the same behavior, over and over again. But when you look at a state of the system, you can’t tell which fixed path it’s on. In fact, minuscule differences in the position, differences so small that you can’t measure them, can put you onto dramatically different paths. There’s the similarity with the initial conditions rule: you’ve got the same basic idea of tiny changes producing dramatic results.
In order to understand this, we need to step back, and look at the some basics: what’s an orbit? What’s a periodic orbit? And what are dense orbits?
To begin with, what’s an orbit?
If you’ve got a dynamical system, you can usually identify certain patterns in it. In fact, you can (at least in theory) take its phase space and partition it into a collection of sub-spaces which have the property that if at any point in time, the system is in a state in one partition, it will never enter a state in any other partition. Those partitions are called orbits.
Looking at that naively, with the background that most of us have associated with the word “orbit”, you’re probably thinking of orbits as being something very much like planetary orbits. And that’s not entirely a bad connection to make: planetary orbits are orbits in the dynamical system sense. But an orbit in a dynamical system is more like the real orbits that the planets follow than like the idealized ellipses that we usually think of. Planets don’t really travel around the sun in smooth elliptical paths – they wobble. They’re pulled a little bit this way, a little bit that way by their own moons, and by other bodies also orbiting the sun. In a complex gravitational system like the solar system, the orbits are complex paths. They might never repeat – but they’re still orbits: a state where where Jupiter was orbiting 25% closer to the sun that it is now would never be on an orbital path that intersects with the current state of the solar system. he intuitive notion of “orbit” is closer to what dynamical systems call a periodic orbit: that is, an orbit that repeats its path.
A periodic orbit is an orbit that repeats over time. That is, if the system is described as a function f(t), then a periodic orbit is a set of points Q where ∃Δt : ∀q∈Q: if f(t)=q, then f(t+Δt)=q.
Lots of non-chaotic things have periodic orbits. A really simple dynamical system with a periodic orbit is a pendulum. It’s got a period, and it loops round and round through a fixed cycle of states from its phase space. You can see it as something very much like a planetary orbit, as shown in the figure to the right.
On the other hand, in general, the real orbits of the planets in the solar system are not periodic. The solar system never passes through exactly the same state twice. There’s no point in time at which everything will be exactly the same.
But the solar system (and, I think, most chaotic systems) are, if not periodic, then nearly periodic. The exact same state will never occur twice – but it will come arbitrarily close. You have a system of orbits that look almost periodic.
But then you get to the density issues. A dynamical system with dense orbits is one where you have lots of different orbits which are all closely tangled up. Making even the tiniest change in the state of the system will shift the system into an entirely different orbit, one which may be dramatically different.
Again, think of a pendulum. In a typical pendulum, if you give the pendulum a little nudge, you’ve changed its swing: you either increased or decreased the amplitude of its swing. If it were an ideal pendulum, your tiny nudge will permanently change the orbit. Even the tiniest perturbation of it will create a permanently change. But it’s not a particularly dramatic change.
On the other hand, think of a system of planetary orbits. Give one of the planets a nudge. It might do almost nothing. Or it might result in a total breakdown of the stability of the system. There’s a very small difference between a path where a satellite is captured into gravitational orbit around a large body, and a path where the satellite is ejected in a slingshot.
Or for another example, think of a damped driven pendulum. That’s one of the classic examples of a chaotic system. It’s a pendulum that has some force that acts to reduce the swing when it gets too high; and it’s got another force that ensures that it keeps swinging. Under the right conditions, you can get very unpredictable behavior. The damped driven pendulum produces a set of orbits that really demonstrate this, as shown to the right. Tiny changes in the state of the pendulum put you in different parts of the phase space very quickly.
In terms of Chaos, you can think of the orbits in terms of an attractor. Remember, an attractor is a black hole in the phase space of a system, which is surrounded by a basin. Within the basin, you’re basically trapped in a system of periodic orbits. You’ll circle around the attractor forever, unable to escape, inevitably trapped in a system of periodic or nearly orbits. But even the tiniest change can push you into an entirely different orbit, because the orbits are densely tangled up around the attractor.
One thing that I wanted to do when writing about Chaos is take a bit of time to really home in on each of the basic properties of chaos, and take a more detailed look at what they mean.
To refresh your memory, for a dynamical system to be chaotic, it needs to have three basic properties:
Sensitivity to initial conditions,
Dense periodic orbits, and
topological mixing
The phrase “sensitivity to initial conditions” is actually a fairly poor description of what we really want to say about chaotic systems. Lots of things are sensitive to initial conditions, but are definitely not chaotic.
Before I get into it, I want to explain why I’m obsessing over this condition. It is, in many ways, the least important condition of chaos! But here I am obsessing over it.
As I said in the first post in the series, it’s the most widely known property of chaos. But I hate the way that it’s usually described. It’s just wrong. What chaos means by sensitivity to initial conditions is really quite different from the more general concept of sensitivity to initial conditions.
To illustrate, I need to get a bit formal, and really define “sensitivity to initial conditions”.
To start, we’ve got a dynamical system, which we’ll call f. To give us a way of talking about “differences”, we’ll establish a measure on f. Without going into full detail, a measure is a function which maps each point x in the phase space of f to a real number, and which has the property that points that are close together in f have measure values which are close together.
Given two points x and y in the phase space of f, the distance between those points is the absolute value of the difference of their measures, .
So, we’ve got our dynamical system, with a measure over it for defining distances. One more bit of notation, and we’ll be ready to get to the important part. When we start our system at an initial point , we’ll write it .
What sensitivity to initial conditions means is that no matter how close together two initial points and are, if you run the system for long enough starting at each point, the results will be separated by as large a value as you want. Phrased informally, that’s actually confusing; but when you formalize it, it actually gets simpler to understand:
Take the system with measure . Then f is sensitive to initial conditions if and only if:
Select any two points and such that:
Let diff(t) = . (Let diff(t) be the distance between and at time .)
(No matter what value you chose for G, at some point in time T, diff(T) will be larger than G.)
Now – reading that, a naive understanding would be that the diff(T) increases monotonically as T increases – that is, that for any two values and , if with measure . And for our non-chaotic system, we’ll use , with .
Think about arbitrarily small differences starting values. In the quadratic equation, even if you start off with a miniscule difference – starting at v0=1.00001 and v1=1.00002 – you’ll get a divergence. They’ll start off very close together – after 10 steps, they only differ by 0.1. But they rapidly start to diverge. After 15 steps, they differ by about 0.5. By 16 steps, they differ by about 1.8; by 20 steps, they differ by about 1.2×109! That’s clearly a huge sensitivity to initial conditions – an initial difference of 1×10-5, and in just 20 steps, their difference is measured in billions. Pick any arbitrarily large number that you want, and if you scan far enough out, you’ll get a difference bigger than it. But there’s nothing chaotic about it – it’s just an incredibly rapidly growing curve!
In contrast, they logistic curve is amazing. Look far enough out, and you can find a point in time where the difference in measure between starting at 0.00001 and 0.00002 is as large as you could possibly want; but also, look far enough out past that divergence point, and you’ll find a point in time where the difference is as small as you could possible want! The measure values of systems starting at x and y will sometimes be close together, and sometimes far apart. They’ll continually vary – sometimes getting closer together, sometimes getting farther apart. At some point in time, they’ll be arbitrarily far apart. At other times, they’ll be arbitrarily close together.
That’s a major hallmark of chaos. It’s not just that given arbitrarily close together starting points, they’ll eventually be far apart. That’s not chaotic. It’s that they’ll be far apart at some times, and close together at other times.
Chaos encompasses the so-called butterfly effect: a butterfly flapping its wings in the amazon could cause an ice age a thousand years later. But it also encompasses the sterile elephant effect: a herd of a million rampaging giant elephants crushing a forest could end up having virtually no effect at all a thousand years later.
That’s the fascination of chaotic systems. They’re completely deterministic, and yet completely unpredictable. What makes them so amazing is how they’re a combination of incredibly simplicity and incredible complexity. How many systems can you think of that are really much simpler to define that the logistic map? But how many have outcomes that are harder to predict?
Let’s look at one of the classic chaos examples, which demonstrates just how simple a chaotic system can be. It really doesn’t take much at all to push a system from being nice and smoothly predictable to being completely crazy.
This example comes from mathematical biology, and it generates a graph commonly known as the logistical map. The question behind the graph is, how can I predict what the stable population of a particular species will be over time?
If there was an unlimited amount of food, and there were no predators, then it would be pretty easy. You’d have a pretty straightforward exponential growth curve. You’d have a constant, R, which is the growth rate. R would be determined by two factors: the rate of reproduction, and the rate of death from old age. With that number, you could put together a simple exponential curve – and presto, you’d have an accurate description of the population over time.
But reality isn’t that simple. There’s a finite amount of resources – that is, a finite amount of food for for your population to consume. So there’s a maximum number of individuals that could possibly survive – if you get more than that, some will die until the population shrinks below that maximum threshold. Plus, there are factors like predators and disease, which reduce the available population of reproducing individuals. The growth rate only considers “How many children will be generated per member of the population?”; predators cull the population, which effectively reduces the growth rate. But it’s not a straightforward relationship: the number of individuals that will be consumed by predators and disease is related to the size of the population!
Modeling this reasonably well turns out to be really simple. You take the maximum population based on resources, Pmax. You then describe the population at any given point in time as a population ratio: a fraction of Pmax. So if your environment could sustain one million individuals, and the population is really 500,000, then you’d describe the population ratio as 1/2.
Now, you can describe the population at time T with a recurrence relation:
P(t+1)= R × P(t) × (1-P(t))
That simple equation isn’t perfect, but it’s results are impressively close to accurate. It’s good enough to be very useful for studying population growth.
So, what happens when you look at the behavior of that function as you vary R? You find that below a certain threshold value, it falls to zero. Cross that threshold, and you get a nice increasing curve, which is roughly what you’d expect. Up until you hit R=3. Then it splits, and you get an oscillation between two different values. If you keep increasing R, it will split again – your population will oscillate between 4 different values. A bit farther, and it will split again, to eight values. And then things start getting really wacky – because the curves converge on one another, and even start to overlap: you’ve reached chaos territory. On a graph of the function, at that point, the graph becomes a black blur, and things become almost completely unpredictable. It looks like the beautiful diagram at the top of this post that I copied from wikipedia (it’s much more detailed then anything I could create on my own).
But here’s where it gets really amazing.
Take a look at that graph. You can see that it looks fractal. With a graph like that, we can look for something called a self-similarity scaling factor. The idea of a SS-scaling factor is that we’ve got a system with strong self-similarity. If we scale the graph up or down, what’s the scaling factor where a scaled version of the graph will exactly overlap with the un-scaled graph/
For this population curve, the SSSF turns out to about 4.669.
What’s the SSSF for the Mandelbrot set? 4.669.
In fact, the SSSF for nearly all bifurcating systems that we see, and their related fractals, is virtually always exactly 4.669. There’s a basic structure which underlies all systems of this sort.
What’s this sort? Basically, it’s a dynamical system with a quadratic maximum. In other words, if you look at the recurrence relation for the dynamical system, it’s got a quadratic factor, and it’s got a maximum value. The equation for our population system can be written: P(t+1) = R×P(t)-P(t)2, which is obviously quadratic, and it will always produce a value between zero and one, so it’s got a fixed maximum value, and Pick any chaotic dynamical system with a quadratic maximum, and you’ll find this constant in it. Any dynamical system with those properties will have a recurrence structure with a scaling factor of 4.669.
That number, 4.669 is called the Feigenbaum constant, after Mitchell Fiegenbaum, who first discovered it. Most people believe that it’s a transcendental number, but no one is sure! We’re not really sure of quite where the number comes from, which makes it difficult to determine whether or not it’s really transcendental!
But it’s damned useful. By knowing that a system is subject to recurrence at a rate determined by Feigenbaum’s constant, we know exactly when that system will become chaotic. We don’t need to continue to observe it as it scales up to see when the system will go chaotic – we can predict exactly when it will happen just by virtue of the structure of the system. Feigenbaum’s constant predictably tell us when a system will become unpredictable.
One of the things that confused my when I started reading about chaos is easy to explain using what we’ve covered about attractors. (The image to the side was created by Jean-Francois Colonna, and is part of his slide-show here)
Here’s the problem: We know that things like N-body gravitational systems are chaotic – and a common example of that is how a gravity-based orbital system that appears stable for a long time can suddenly go through a transition where one body is violently ejected, with enough velocity to permanently escape the orbital system.
But when we look at the definition of chaos, we see the requirement for dense periodic orbits. But if a body is ejected from a gravitational system, ejection of a body from a gravitational system is a demonstration of chaos, how can that system have periodic orbits?
The answer relates to something I mentioned in the last post. A system doesn’t have to be chaotic at all points in its phase space. It can be chaotic under some conditions – that is, chaotic in some parts of the phase space. Speaking loosely, when a phase space has chaotic regions, we tend to call it a chaotic phase space.
In the gravitational system example, you do have a region of dense periodic orbits. You can create an N-body gravitational system in which the bodies will orbit forever, never actually repeating a configuration, but also never completely breaking down. The system will never repeat. Per Ramsey theory, given any configuration in its phase space, it must eventually come arbitrarily close to repeating that configuration. But that doesn’t mean that it’s really repeating: it’s chaotic, so even those infinitesimal differences will result in divergence from the past – it will follow a different path forward.
An attractor of a chaotic system shows you a region of the phase space where the system behaves chaotically. But it’s not the entire phase space. If the attractor covered the entire space, it wouldn’t be particularly interesting or revealing. What makes it interesting is that it captures a region where you get chaotic behavior. The attractor isn’t the whole story of a chaotic systems phase space – it’s just one interesting region with useful analytic properties.
So to return to the N-body gravitational problem: the phase space of an N-body gravitational system does contain an attractor full of dense orbits. It’s definitely very sensitive to initial conditions. There are definitely phase spaces for N-body systems that are topologically mixing. None of that precludes the possibility that you can create N-body gravitational systems that break up and allow escape. The escape property isn’t a good example of the chaotic nature of the system, because it encourages people to focus on the wrong properties of the system. The system isn’t chaotic because you can create gravitational systems where a body will escape from what seemed to be a stable system. It’s chaotic because you can create systems that don’t break down, which are stable, but which are thoroughly unpredictable, and will never repeat a configuration.
Sorry for the slowness of the blog; I fell behind in writing my book, which is on a rather strict schedule, and until I got close to catching up, I didn’t have time to do the research necessary to write the next chaos article. (And no one has sent me any particularly interesting bad math, so I haven’t had anything to use for a quick rip.)
Anyway… Where we left off last was talking about attractors. The natural question is, why do we really care about attractors when we’re talking about chaos? That’s a question which has two different answers.
First, attractors provide an interesting way of looking at chaos. If you look at a chaotic system with an attractor, it gives you a way of understanding the chaos. If you start with a point in the attractor basin of the system, and then plot it over time, you’ll get a trace that shows you the shape of the attractor – and by doing that, you get a nice view of the structure of the system.
Second, chaotic attractors are strange. In fact, that’s their name: strange attractors: a strange attractor is an attractor whose structure has fractal dimension, and most chaotic systems have fractal-dimension attractors.
Let’s go back to the first answer, to look at it in a bit more depth. Why do we want to look in the basin in order to find the structure of the chaotic system?
If you pick a point in the attractor itself, there’s no guarantee of what it’s going to do. It might jump around inside the attractor randomly; it might be a fixed point which just sits in one place and never moves. But there’s no straightforward way of figuring out what the attractor looks like starting from a point inside of it. To return to (and strain horribly) the metaphor I used in the last post, the attractor is the part of the black hole past the even horizon: nothing inside of it can tell you anything about what it looks like from the outside. What happens inside of a black hole? How are the things that were dragged into it moving around relative to one another, or are they moving around? We can’t really tell from the outside.
But the basin is a different matter. If you start at a point in the attractor basin, you’ve got something that’s basically orbital. You know that every path starting from a point in the basin will, over time, get arbitrarily close to the attractor. It will circle and cycle around. It’s never going to escape from that area around the attractor – it’s doomed to approach it. So if you start at a point in the basin around a strange attractor, you’ll get a path that tells you something about the attractor.
Attractors can also vividly demonstrate something else about chaotic systems: they’re not necessarily chaotic everywhere. Lots of systems have the potential for chaos: that is, they’ve got sub-regions of their phase-space where they behave chaotically, but they also have regions where they don’t. Gravitational dynamics is a pretty good example of that: there are plenty of N-body systems that are pretty much stable. We can computationally roll back the history of the major bodies in our solar system for hundreds of millions of years, and still have extremely accurate descriptions of where things were. But there are regions of the phase space of an N-body system where it’s chaotic. And those regions are the attractors and attractor basins of strange attractors in the phase space.
A beautiful example of this is the first well-studied strange attractor. The guy who invented chaos theory as we know it was named Edward Lorenz. He was a meteorologist who was studying weather using computational fluid flow. He’d implemented a simulation, and as part of an accident resulting from trying to reproduce a computation, but entering less precise values for the starting conditions, he got dramatically different results. Puzzling out why, he laid the foundations of chaos theory. In the course of studying it, he took the particular equations that he was using in the original simulation, and tried to simplify them to get the simplest system that he could that still showed the non-linear behavior.
The result is one of the most well-known images of modern math: the Lorenz attractor. It’s sort of a bent figure-eight. It’s dimensionality isn’t (to my knowledge) known precisely – but it’s a hair above two (the best estimate I could find in a quick search was in the 2.08 range). It’s not a particularly complex system – but it’s fascinating. If you look at the paths in the Lorenz attractor, you’ll see that things follow an orbital path – but there’s no good way to tell when two paths that are very close together will suddenly diverge, and one will pass on the far inside of the attractor basin, and the other will fly to the outer edge. You can’t watch a simulation for long without seeing that happen.
While searching for information about this kind of stuff, I came across a wonderful demo, which relates to something else that I promised to write about. There’s a fantastic open-source mathematical software system called sage. Sage is sort of like Mathematica, but open-source and based on Python. It’s a really wonderful system, which I really will write about at some point. On the Sage blog, they posted a simple Sage program for drawing the Lorenz attractor. Follow that link, and you can see the code, and experiment with different parameters. It’s a wonderful way to get a real sense of it. The image at the top of this post was generated by that Sage program, with tweaked parameters.
In my first chaos post, I kept talking about dynamical systems without bothering to define them. Most people who read this blog probably have at least an informal idea of what a dynamical system is. But today I’m going to do a quick walkthrough of what a dynamical system is, and what the basic relation of dynamical systems is to chaos theory.
The formal definitions of dynamical systems are dependent on the notion of phase space. But before going all formal, we can walk through the basic concept informally.
The basic idea is pretty simple. A dynamical system is a system that changes
over time, and whose behavior can be (in theory) described a function that takes
time as a parameter. So, for example, if you have a gravitational system which
has three bodies interacting gravitationally, that’s a dynamical system. If you
know the initial masses, positions, and velocities of the planets, the positions of all three bodies at any future point in time is a function of the time.
One mathematical topic that I find fascinating, but which I’ve never had a chance to study formally is chaos. I’ve been sort of non-motivated about blog-writing lately due to so many demands on my time, which has left me feeling somewhat guilty towards those of you who follow this blog. So I decided to take this topic about which I know very little, and use the blog as an excuse to learn something about it. That gives you something interesting to read, and it gives me something to motivate me to write.
I’ll start off with a non-mathematical reason for why it interests me. Chaos is a very simple idea with very complex implications. The simplicity of the concept makes it incredibly ripe for idiots like Michael Crichton to believe that he understands it, even though he doesn’t have a clue. There’s an astonishingly huge quantity of totally bogus rubbish out there, where the authors are clueless folks who sincerely believe that their stuff is based on chaos theory – because they’ve heard the basic idea, and believed that they understood it. It’s a wonderful example of my old mantra: the worst math is no math. If you take a simple mathematical concept, and render it into informal non-mathematical words, and then try to reason from the informal stuff, what you get is garbage.
So, speaking mathematically, what is chaos?
To paraphrase something my book-editor recently mentioned: in math, the vast majority of everything is bad. Most functions are non-continuous. Most topologies are non-differentiable. Most numbers are irrational. Most irrational numbers are undescribable. And most complex systems are completely unstable.
Modern math students have, to a great degree, internalized this basic idea. We pretty much expect badness, so the implications of badness don’t surprise us. We’ve grown up mathematically knowing that there are many, many interesting things that we would really like to be able to do, but that can’t be done. That realization, that there are things that we can’t even hope to do, is a huge change from the historical attitudes of mathematicians and scientists – and it’s a very recent one. A hundred years ago, people believed that we could work out simple, predictable, closed-form solutions to all interesting mathematical problems. They expected that it might be very difficult to find a solution; and they expected that it might take a very long time, but they believed that it was always possible.
For one example that has a major influence on the study of chaos: John Von Neumann believed that he could build a nearly perfect weather prediction computer: it was just a matter of collecting enough data, and figuring out the right equations. In fact, he expected to be able to do more than that: he expected to be able to control the weather. He thought that the weather was likely to be a system where there were unstable points, and that by introducing small changes at the unstable points, that weather managers would be able to push the weather in a desired direction.
Of course, Von Neumann knew that you could never gather enough data to perfectly predict the weather. But most systems that people had studied could be approximated. If you could get measurements that were correct to within, say, 0.1%, you could use those measurements to make predictions that would be extremely close to correct – within some multiple of the precision of the basic measurements. Small measurement errors would mean small changes in the results of a prediction. So using reasonably precise but far from exact or complete measurements, you could make very accurate predictions.
For example, people studied the motion of the planets. Using the kinds of measurements that we can make using fairly crude instruments, people have been able to predict solar eclipses with great precision for hundreds of years. With better precision, measuring only the positions of the planets, we can predict all of the eclipses and alignments for the next thousand years – even though the computations will leave out the effects of everything but the 8 main planets and the sun.
Mathematicians largely assumed that most real systems would be similar: once you worked out what was involved, what equations described the system you wanted to study, you could predict that system with arbitrary precision, provided you could collect enough data.
Unfortunately, reality isn’t anywhere near that simple.
Our universe is effectively finite – so many of the places where things break seem like they shouldn’t affect us. There are no irrational numbers in real experience. Nothing that we can observe has a property whose value is an indescribable number. But even simple things break down.
Many complex systems have the property that they’re easy to describe – but where small changes have large effects. That’s the basic idea of chaos theory: that in complex dynamical systems, making a minute change to a measurement will produce huge, dramatic changes after a relatively short period of time.
For example, we compute weather predictions with the Navier-Stokes equations. N-S are a relatively simple set of equations that describe how fluids move and interact. We don’t have a closed-form solution to the N-S equations – meaning that given a particular point in a system, we can’t compute the way fluid will flow around it without also computing separately how fluid will flow around the points close to it, and we can’t compute those without computing the points around them, and so on.
So when we make weather predictions, we create a massive grid of points, and use the N-S equations to predict flows at every single point. Then we use the aggregate of that to make weather predictions. Using this, short-term predictions are generally pretty good towards the center of the grid.
But if you try to extend the predictions out in time, what you find is that they become unstable. Make a tiny, tiny change – alter the flow at one point by 1% – and suddenly, the prediction for the weather a week later is dramatically different. A difference of one percent in one out of a million cells can, over the space of a month, be responsible for the difference between a beautiful calm summer day and a cat-5 hurricane.
That basic bit is called sensitivity to initial conditions is a big part of what defines chaos – but it’s only a part. And that’s where the crackpots go wrong. Just understanding the sensitivity and divergence isn’t enough to define chaos – but to people like Crichton, understanding that piece is understanding the whole thing.
To really get the full picture, you need to dive in to topology. Chaotic systems have a property called topological mixing. Topological mixing is an idea which isn’t too complex informally, but which can take a lot of effort to describe and explain formally. The basic idea of it is that no matter where you start in a space, given enough time, you can wind up anywhere at all.
To get that notion formally, you need to look at the phase space of the system. You can define a dynamical system using a topological space called the phase space of the system. Speaking very loosely, the phase space P of a system is a topological space of points where each point p∈P corresponds to one possible state of the system, and the topological neighborhoods of p are the system states reachable on some path from p.
So – image that you have a neighborhood of points, G in a phase space. From each point in G, you traverse all possible forward paths through the phase space. At any given moment t, G will have evolved to form a new neighborhood of points, Gt. For the phase space to be chaotic, it has to have the property that for any arbitrary pair of neighborhoods G and H in the space, no matter how small they are, no matter how far apart they are, there will be a time t such that Gt and Ht will overlap.
But sensitivity to initial conditions and topological mixing together still aren’t sufficient to define chaos.
Chaotic systems must also have a property called dense periodic orbits. What that means is that Chaotic systems are approximately cyclic in a particular way. The phase space has the property that if the system passes through a point p in the neighborhood P, then after some finite period of time, the system will pass through another point in P. That’s not to say that it will repeat exactly: if it did, then you would have a repeating system, which would not be chaotic! But it will come arbitrarily close to repeating. And that almost-repetition has to have a specific property: the union of the set of all of those almost-cyclic paths must be equivalent to the entire phase-space itself. (We say, again speaking very loosely, that the set of almost-cycles is dense in the phase space.)
That’s complicated stuff. Don’t worry if you don’t understand it yet. It’ll take a lot of posts to even get close to making that comprehensible. But that’s what chaos really is: a dynamical system with all three properties: sensitivity to initial conditions, overlaps in neighborhood evolution, and dense periodic orbits.
In subsequent posts, I’ll spend some more time talking about each of the three key properties, and showing you examples of interesting chaotic systems.
which maps each point x in the phase space of f to a real number, and which has the property that points that are close together in f have measure values which are close together.
. 
at an initial point 
, we’ll write it 
.
are, if you run the system for long enough starting at each point, the results will be separated by as large a value as you want. Phrased informally, that’s actually confusing; but when you formalize it, it actually gets simpler to understand:
. Then f is sensitive to initial conditions if and only if:
. (Let diff(t) be the distance between 
at time 
.)
(No matter what value you chose for G, at some point in time T, diff(T) will be larger than G.)
and 
, if 
with measure 
. And for our non-chaotic system, we’ll use 
, with 
. 
