Stardate 20030825.1859

(On Screen via long range sensors): I ran into this post over the weekend which talks about crime rates in the US. It talks about trends over time, and compares American crime rates to those reported from Europe. He is dealing with two basic issues: the belief held by many in Europe that the US has far more crime than in Europe, and the fact that when it comes to certain kinds of violent crime there are certain groups which are disproportionately likely to be involved both as perpetrators and as victims.

Specifically, young black men who live in the inner city are far more likely to commit murder by gunfire, and far more likely to be victims of murder by gunfire. The overall murder rate for the US is somewhat higher than in Europe, but that's being skewed by this particular group. Calculated without them, it turns out that our murder rate is not really much different than in Europe, and in most other kinds of crime (e.g. armed robbery, burglary, rape) it seems our crime rate is much lower.

After having read that article, I spotted this one. The latest report on American crime rates has just been issued, and the crime rate has dropped even further.

Nonetheless, it leaves open the question of why it is that inner city blacks are so much more at risk, and whether anything can be done to help them. There have been many attempts to do so, and they're something of a cause célèbre in certain circles. Their apparent failure is used by many as evidence that there are still lingering effects of slavery and discrimination, even after all this time.

Is it true? Certainly there are still such effects; things are vastly better now than they were when I was a kid, but we haven't yet reached the point of having a race-blind culture. But there's also the fact that a lot of blacks live in the suburbs, hold professional positions (some with great responsibility), make a good living, and don't seem to be any more prone to crime than anyone else. And recent immigrants from Africa and the Caribbean who are also Negroes don't seem to have the same troubles. Whatever is going on is more complicated than skin color.

Last January I wrote about some of the reasons why inequality in a competitive system tends to magnify itself. Those who are successful competitively have advantages which make it more likely that they'll be competitive in the future. Those who are behind have a harder time competing. Those who have been losing the contest until now are more likely to continue losing.

I won't rehash yet again the basic concepts of network effect, economy of scale and reinvesting the profits. If you're not familiar with them, read this. As I was reading the article about crime and the way that violent crime in particular is quite concentrated, it occurred to me that there is a tendency for people to sort themselves out.

We can move where we want, if we can afford it. There aren't any legal or bureaucratic limits. I didn't have to get a permit from the government in order to move from Boston to San Diego. And what that means is that people will tend to live where they want to (if they can do so financially).

Now combine that with the tendency of concentrations and inequality to reinforce themselves. Suppose, for instance, you have a small but very vocal number of "out" gay men living in a few places in the 1970's. Other gay men who are looking for a place to live where they think they'd feel more comfortable will hear about this, and some will decide to move there as well. As the concentration of gays rises, the area becomes more hospitable. Money talks; the businesses start catering to gay interests, and there's less discrimination against them in nearly every way. This makes these areas even more attractive, and even more immigration of like-minded individuals will take place.

It doesn't follow an infinite exponential expansion curve; it's a lot like that for a long time, but it then levels out as you get near to some sort of saturation point. (And then it's possible for it to fade back down. These things can be very complicated.) This process rarely proceeds to complete uniformity.

It isn't just gays that did this. Immigrants from certain backgrounds have a tendency to concentrate. Historically you had "Little Italy" in New York. A disproportionate number of Irish immigrants ended up in Boston. In LA there's one of the largest concentrations of Iranian immigrants in the US. When I was growing up in Oregon, there was a community of White Russians living near Woodburn. You've got the Amish, and the Mormons. Baptists are a lot more common in the South than elsewhere. There are certain places which have become particularly popular with retirees, and demographically the average age of the local population is well above the national average. There are areas where the percentage of children in the population is well below average; that happens in areas with disproportionate concentrations of retirees or gays, for obvious reasons, but it also happens in areas where you tend to get a concentration of particularly well educated people who are particularly prosperous, because they are less likely to have kids, and tend to have fewer when they do.

These kinds of things happen all the time, nearly everywhere. This nation is not homogenous, and probably can't be made homogenous short of the use of armed force. Some sort of small inequality will appear in a few places, and whatever-they-are might start to grow. That's why you got some places which had particularly great concentrations of high tech over the last 30 years. It's exactly the same kind of thing: places where there was already a lot of high tech tended to be good places to make more of it, both because of economy of scale and network effect.

Sometimes you can get reverse trends; some places can be victims of their own success. As Silicon Valley became a mecca for high tech, property values rose and so did traffic and pollution, and soon you had companies (like Intel) who decided to bail out and go somewhere else which was cheaper and less polluted. (Intel ended up moving to the Tualitan Valley in Oregon, near where I was living at the time.) There are a lot of factors which kick in.

This doesn't just happen geographically; it happens in other ways. For instance, it tends to be the case that members of some professional fields tend to not be demographically representative of the population as a whole. Americans of Chinese and Japanese descent here in California are statistically overrepresented in engineering schools, for instance, but women (and blacks) are very much underrepresented. To a great extent this is the result of individual choice based on the perception of how easy or hard a given area might be to break into. Nursing was traditionally an area where women did well (though originally all nurses were men), but other fields might not be seen as easy to get into. A young woman thinking about a career 40 years ago saw nursing as less risky relative to the potential gain, so this was self-reinforcing.

Some of the factors which influence this kind of concentration are unhealthy; some may be illegal. But many of them can't easily be changed, if they can be changed at all.

In the 1960's, the schools in many cities were highly segregated. The courts eventually ordered busing. Whatever other benefits busing may or may not have had, the one thing it didn't do was to homogenize the schools. You still have a lot of schools which are predominantly white, while others are predominantly black, or predominantly other kinds of folks who ain't either of those two. There are also schools which are quite mixed, but over time they don't tend to stay that way. That's because in general the school population reflects the overall population of the local area.

In a lot of cases, when various kinds of serious coercion was used to try to force integration in some neighborhoods, the result was "white flight". The problem was that the government might well be able to move minorities into a neighborhood, but they couldn't prevent people from leaving. The history of American cities are full of neighborhoods which started as prosperous white, became integrated, started to run down, and eventually ended up almost exclusively minority. Sometimes the initial integration was the result of direct action and sometimes it was spontaneous. And though not by any means invariably, in a lot of cases once the blacks started moving in, the white started to move out.

This is about the point where I better make clear that I'm not making any value judgment on this. I have opinions on the subject but they're not relevant to this discussion.

I'm talking about the way that competitive systems made up of a large number of independent agents who make decisions in isolation have a tendency to sort themselves and to reinforce inequality. Whether that's good or bad, it's a real phenomenon, and worth studying in its own right. Some of the motivation for the individuals involved may be contemptible, but the phenomenon itself is value neutral.

There have been cases where certain concentrations were deliberately maintained. There were realtors who would steer certain potential clients to certain neighborhoods, for example, and within my lifetime there were some states in which the power of law was used to maintain segregation.

But even without that kind of thing, these kinds of concentrations will tend to form voluntarily. I am not aware of any particular attempt by anyone to deliberately segregate gays in San Francisco and a few other locales; the gays living there do so voluntarily. And a lot of other gays don't, and have no great urge to do so.

So concentrations can form voluntarily and spontaneously. But sometimes they form because they're a residue. It isn't so much that the people there chose to live there; it's that a lot of people chose to leave. And as a group the people who leave aren't statistically identical to those who stay behind. This can work several ways.

For instance, if a neighborhood is subject to what's known as "gentrification", one result is that property values will start to rise. That means that property taxes will rise, and it means rents will rise, and it means that some long-time residents may no longer be able to afford to live there and may have to move. On the other hand, some who lived there before gentrification began may still be able to afford to live there, and may not move, and of course gentrification tends to bring in new people who are not like those who were already there. So there's a filtration process involved in who stays and who comes and who goes, and it isn't random. In the case of gentrification, the effect is to take a somewhat-nice neighborhood and make it a lot nicer. But it isn't the same people, or not all the same ones.

By the same token, a lot of the areas where the murder rate is the greatest in the US were formed as the residue of a filtration process, only in this case it was the opposite of gentrification. Call it "ghettoization", for lack of a better term.

With the creation of the suburbs in the period after WWII, and with personal transportation cheaper and easier than ever before, there was a mass movement out of the cities. But as a group those who left weren't like those who remained. And, unfortunately, it became a self-reinforcing process. As people left, businesses closed or relocated, the climate degraded, and there was even more of a tendency for people to leave and businesses to relocate if they were capable of doing so and were sufficiently motivated. Basically, that meant money and education and marketable skills.

As such places got seedier and seedier, land values declined, rents went down, and there was also immigration. Where do you think the people who are forced out of their homes by gentrification end up? (At least some of them.)

Those who remained were the ones who either weren't able to move, or didn't want to do so, and that meant that demographically they were more likely to not have a good education or good job prospects. Those who moved in were the ones who came there because they couldn't afford to go anywhere better.

The process of migration to the suburbs tended to disproportionately favor the center and right sides of the bell curve, disproportionately leaving behind those on the left side, a sigma or two out. It was life's losers who stayed behind for the most part or who moved in. And that filtration process is ongoing: when someone who lives in Watts manages to get a decent education, they don't move back there. They find a job somewhere else, somewhere better and nicer, somewhere safer and more prosperous. Most of those who stay behind can't leave, even if they wanted to.

And there's also a certain tendency for cultural drift in cases like this. Such areas become culturally isolated. Once there's any kind of local concentration of certain cultural features, that tends to increase its local dominance. Those same factors which tend to reinforce inequality apply here, too. Things like formation of gangs, or popularity of certain kinds of music or clothing styles; holding of certain attitudes. There is a tendency for the most popular ones to become even more powerful and widespread. That happens at all levels, internationally, nationally, in regions, and in neighborhoods. When for other reasons a given neighborhood is socially isolated (which tends to happen with slums) then its culture evolves in a cultural vacuum. And filtration continues to contribute: those who really dislike the resulting culture, no matter what it is, are more likely to leave or less likely to move in.

That's happening with Europe now, I think. There are people in Europe who believe in Socialism and strong central government. There are others who are more aligned to what most Americans believe, and advocate less government overall, less government regulation, lower taxes, more freedom. Ideally the advocates for both sides would duke it out politically, but for a variety of reasons there isn't much of that going on. I think one reason is that instead most of the non-Socialists are trying to come here. Rather than band together with like-minded people to change their own nations, they're trying to get out. We don't allow as many people in as want to come, but we allow a lot of them, and people keep applying, sometimes waiting years.

It's a completely reasonable decision for each individual. Why try to create a capitalist system in France, with a very low chance of succeeding, when you can move to a place where it already exists?

And since each such person is planning their future based on going somewhere else, there's little incentive for them to try to work to fix what's wrong in the nations they're trying to leave. (In one sense this is a manifestation of the tragedy of the commons.)

All of which means that there really isn't the kind of political discussion on these issues going on there that I wish was happening. The people who remain behind and have no interest in leaving are a residue, and they're not statistically identical (especially in terms of ideology) to those who have emigrated or who are trying to do so.

That also applies to the brain drain, and the way that Europe's best engineers and scientists try to come to the US. If they want to do cutting edge work, they as individuals have a lot better chance of it here than they do trying to create such an environment in Europe. There are too many hurdles and roadblocks in the way. Had all of those who emigrated remained behind, gotten organized, and worked hard at it they probably could have done so. If the US didn't exist in its current form, then they all would have stayed behind, and Europe would be the scientific and technological capital of the world.

But we do exist, and thousands of individuals are making self-interested decisions that they'd rather go somewhere to work that encourages what they want to do, instead of placing roadblocks in the way. And because of that, Europe is stagnating technologically.

Getting back to the inner cities, it isn't exclusively blacks who live in the worst areas, nor do the majority of American blacks live in such places. It's not clear that this is a "Black Problem" as such. There have always been slums, and what I think is that there always will be slums. Slums spontaneously form by a natural process of filtration, where those who are both willing and able to leave are different than those who don't, and are left behind, and those who move in, because they have no other choices.

And there will always be slum cultures. There were Irish slums, Polish slums, Chinese slums, Puerto-Rican slums; some of those still exist. There was always more crime in such places. It was always more dangerous.

And there have been slums which were not identifiably associated with any one race or culture, at least initially. Nor do these things exist only in the inner city.

You can also get large areas, approaching the size of states, which become residue as a result of this kind of filtered exodus. There are large parts of the Appalachians where nearly everyone is dreadfully poor, with few job prospects and lousy education and no hope. And every year a few of their best young people leave; they get admitted to a university somewhere, and get a decent education, and then find a good job somewhere else. Those left behind continue to have no hope. Businesses don't relocate there; there's no reason to do so. Jobs aren't created. The people sit, and stagnate.

And tend to drink a lot, or use other kinds of drugs, which of course doesn't help the situation any. That's also always been a factor in slums. In the Irish and Polish slums it was alcohol. In the Chinese slums it was opium.

Medical researchers are concerned with what are called "clusters"; these are places where there are a lot of cases of some particular disease, especially non-communicable ones. They're studied by epidemiologists, because in some cases it turns out there's actually a local cause and if they can find it and get rid of it, they can cut the rate of new cases. But researchers also know that some clusters have no causes at all; they're just the result of randomness. Sometimes it's just like that; the disease overall isn't happening at unexpectedly high rates, it's just that a lot of the cases all occurred in the same area. Probability theory says that there's a chance that this will happen without having any significance at all.

We all know that the population as a whole can evaluated in various ways, and plotted on a curve. Looked at in that way, it's clear that some people will have excellent prospects and some will have terrible prospects, just because that's how things tend to land statistically. Often it's a "power law curve" instead of a bell curve, but there's always someone on the lousy end of the curve.

But the physical filtration process I have been describing has a tendency to physically sort people according to these kinds of factors, with some people deliberately concentrating themselves, and others being concentrated by being left behind.

But when that happens, they become visible. There's a cluster. Does it actually have a proximate cause? Well, yes, but not necessarily one that has anything to do with the people who are there. They're there because they're the ones who didn't leave. They're the residue. The cluster exists not because of a factor which is present, but because of the factors which are no longer present because they walked away.

And maybe it isn't possible to make such clusters go away. Individuals can be helped, and maybe given better prospects, but then they'll go somewhere else, leaving behind a core residue which is even less able to succeed (and leave). You can gentrify a neighborhood, but you do so by changing who lives there, not by making the people who live there better.

They tried that. The idea was to build huge amounts of "public housing". You'd create new neighborhoods; they'd be much nicer than the old ramshackle buildings. The same people would live there, but they'd live in much nicer surroundings. The hope was that things would then improve in other ways, but that didn't happen.

There will always be people who are less able to succeed. There will always be a tendency for them to become concentrated in a small number of areas. And that means they'll be visible, identifiable.

And when the whole curve moves, they'll move with it, but they'll still be on the low end. Kevin points to this page which shows that since 1970 the overall homicide rate in the US has dropped, along with the general trend towards less crime in general. The homicide rate for whites (both as perpetrators and as victims) has dropped, and so have both rates for blacks. But the rate for blacks is still higher than for whites.

Worse, this disguises some realities: some white groups are more likely than others to be involved in homicide than other whites, and some groups of blacks are than other blacks. The majority of black people in the US are no more at risk than I am, but there's a core group which is skewing the average. There's such a group of whites, too, but they're being averaged into a much larger population, which dilutes the effect. To really see what's going on you'd need much more data, to dissect those averages.

Kevin's argument was that we have to face the fact of black crime squarely, which is difficult because so many others are primed to scream Racist! at any hint of such discussion. But I'm not sure we can conclude that the correlation of high crime to race is actually significant. Are we rather seeing the result of a residue of natural filtration, whose tendencies to crime are then reinforced through local concentration?

When "poor" and "ill-educated" and "crime-ridden" are relative terms, then what you're trying to do is to eradicate the low end of the curve. But there will always be a "low end of the curve", unless everything is absolutely identical for everyone, which isn't possible (and in my opinion also isn't desirable). The so-called "poverty line" in the US now is an income of about $16,000 per year, but that is a level of income which in world terms is in the top few percentile (and approaches the average per-capita GDP of several European nations which are generally considered "prosperous" by world standards). By the same token, the homicide rate for inner city American blacks is high by American standards, but well below the rate of death-by-violence in many parts of the world.

Before we can solve a problem, we have to understand what the problem is. And we have to begin by asking whether there even is a problem. Are we just seeing a physical concentration of the low end of the curve?