Saturday, July 26, 2003

Breakfast
Strawberries and Raspberries from Cortez Farms in Santa Maria. English Shelling Peas from Swank Farms in Hollister. Beef Bacon from Corralitos Meat. Co. All courtesy of the Cabrillo College Farmers' Market. Plus an Almond and Cherry granola bar.
Software - An Interview with Richard M. Stallman
I don't always agree with RMS, which is why Chrysalis uses a BSD license. But you gotta love this exchange:

GSMBOX: Therefore, you refuse the term piracy when speaking about unauthorised copies?

RMS: Piracy means attacking and burning ships and this is very very bad. But sharing copies of any information technology product over the computer for most of us is a good act, it is social cooperation. The idea of making a connection between these two acts, one absolutely immoral and the other very moral, is completely wrong. It is a term of propaganda and I do not intend to participate in the propaganda of publishers.


He also notes that "Free software is neither to the right nor the left". Is the tie to my previous post coincidence or synchronicity? Read the whole interview, courtesy of OS News.

Friday, July 25, 2003

Plagiarizing Shakespeare
Neither a liberal nor a conservative be

Thursday, July 24, 2003

Social Darwinism
In the midst of a wonderful article on Saving the Net in Linux Journal, Doc Searls makes this tangential observation:

“Liberals often are flummoxed by the way conservatives seem to love big business (including, of course, big media). Yet the reason is simple: they love winners, literally. They like to reward strength and achievement...As George Lakoff explained in Moral Politics: What Conservatives Know that Liberals Don't (University of Chicago, 1995), conservatives consider strength a "moral value". Strong is good. Weak is bad.”

Put another way, Success=Morality. This is what some of the Calvinists believed. There's a name for this point of view when it's taken to a certain extreme. It's called Social Darwinism. The last great heyday of Social Darwinism was in late 19th-Century Great Britian. Thanks to Herbert Spencer, English imperialists found a philosophical justification in their drive for empire. Some historians have made the argument that the abandonment of what had made England great - trade, commerce, and a grand strategy dedicated to maintaining the balance of power - in favor of hegemony and expansion is what led to the eventual decline of the British Empire. There's also direct connection between Spencer and the National Socialists in Germany less than a century later. In his Survival Guide to the New Renaissance, W.R. Clement posits that the current rise of Social Darwinism among Western elites is both a reaction to the rising tide of change in human society and a justification for establishing an dominant oligarchy. He also derides Social Darwinism as a dangerous, stupid, and short-sighted world view doomed to failure.

I am not equating mainstream conservatism with Social Darwinism. That's as absurd as equating mainstream liberalism with communism, or libertarianism with anarchy. My point is that Social Darwinism is a pernicious trap that elites, particularly conservative elites, are prone to fall into.


Databases - My Brilliant Career Tuning SQL
One of the first things I did when I started my first real job, way back when GWB I was prez, was to try to tune a bunch of complex SQL statements. My first task in my new job was to try to tune a bunch of complex SQL statements. So what's changed in 15 years? Well, the databases are much larger (hundreds of GBs vs. hundreds of MBs), we're now up to Oracle Version 9 (it was v5 then), PL/SQL and Java came along to change how we write programs, the net came along to change the kind of programs we write, Larry Ellison is even richer, etc., etc.

What hasn't changed? The basic nature of the problem. Conceptually, there has been no real progress. I'm still doing essentially the same thing - figuring out what the database engine should do and then finding a way to make the engine do it.

Yes, it's all bigger-faster-better, and yes, I don't have to ensure that every single statement is optimized like I did in days of yore. But the growth in the size and complexity of applications effectively cancels out whatever other progress has been made. And so I find myself back where I started. But I'm not alone. The whole database industry has the same problem.

It wasn't supposed to be this way. What was supposed to happen was that the database engine's optimizer would be able to always figure out the optimal execution plan for you. It would do this using the statistics collected in the database catalog together with a variety of expression transformation algorithms to produce the best possible access plan. No intervention required, and on to the next level of abstraction. But we haven't gotten there. And it's not looking like we will anytime soon.

Now, in the interests of full disclosure, I should tell you that SQL tuning has been very, very good to me. It's one of the reasons why I receive a handsome salary. While I consider it one of the lesser talents that I bring to the show, making something run several orders of magnitude faster than it did before I performed my magic tuning act certainly impresses the masses. And the semi-arcane knowledge required seems plenty daunting to the uninitiated. It does requires a certain feel or intuitive sense, but so do lots of other things. In short, it's an overrated skill.

But the application of said knowledge and instinct ought to be completely unnecessary. Back in the early-to-mid '80's, database researchers and gurus were just about sure that the need for manual tuning could be eliminated. And indeed it can be, in a relational database system. Because in a relational database system, every query could be reduced to a set of canonical relational algebra expressions. Potential access paths could be analyzed by looking at the statistics captured in the system catalog and the most optimal ones chosen. The set of expressions could then be sequenced in the most efficient order, and the statement would be executed. All with no human intervention required. So the effort previously devoted to manually optimizing physical data access could now be devoted to figuring out how to make applications and systems work better for the people who used and depended on them - a small example of the general way in which human progress is made, i.e., formerly labor-intensive tasks are automated, allowing us to move to the next higher level of abstraction.

And for one brief, shining moment this was actually achieved. A 1988 article in Database Programming and Design showed a comparison of several different database management systems (Oracle, Sybase, Ingres, etc.) each executing a variety of SQL queries. The twist was that all of the SQL statements were semantically equivalent - i.e., they meant the same thing and produced the same results. But due to the redundancy of SQL syntax, the same 3-table join could be expressed in a variety of different ways. So the article measured how fast each DBMS executed each variation of the statement. Ideally, every statement would take the roughly the same time to execute - because the optimizer would produce the same execution plan, and the only variable would be how long it took the optimizer to figure out the access plan.

Well, it didn't work that way. The variation in execution times ranged from about .1 seconds to over 1000 seconds. Oracle, which at this time had a syntactically based optimizer, produced both the fastest and slowest results. But the others weren't much better.

With one exception - Ingres. Every query got executed in about 2 seconds. No significant variations. A bit slow, perhaps - every other DBMS's best time was better - but almost perfectly consistent. Why? Because the Ingres optimizer produced the same execution plan every time. The other DBMS's produced different plans for different formulations of the same statement. How? Well, the secret was that Ingres originally used a different query language - QUEL - instead of SQL. QUEL was much closer to the relational calculus languages that Codd originally proposed. Consequently, it was much easier to optimize - because every QUEL statement could be transformed into a canonical relational algebra expression. Ingres was eventually forced to support SQL, but they were able to apply what they'd learned about optimization of QUEL to SQL (originally, in fact, they translated SQL to QUEL and then optimized it. By Ingres 7.X, they eliminated the translation step and optimized the SQL directly. It didn't work as well as optimizing QUEL, but it was better than what anyone else was doing).

So it can be done. Ingres was slow, but making it faster was an engineering problem, not a research problem. Engineering problems, in my definition, are ones that can be solved incrementally. Making cars more fuel-efficient is an engineering problem. Research problems, on the other hand, deal with bigger, more fundamental issues. Fusion power is a research problem. Nanotechnology is a research problem. Research leads to conceptual breakthroughs. Engineering takes those breakthroughs and produces viable, usable products. Like a relational database management system (DBMS) with a reliable optimizer. It can be done.

But it won't be, at least not by our present crop of DBMS's. That's because they use SQL. And guess what? SQL can't really be optimized. Why not? Because it breaks so many of the fundamental rules for a relational language that it can't be systematically and unambiguously optimized. One of the rules of relational algebra is that an arbitrary set of expressions can be evaluated in any order and still produce the same result. However, because SQL allows duplicates, this isn't true. Different orders of evaluation can yield different results. Further complicating this is the syntactic redundancy mentioned earlier - making the task of transforming expressions into a canonical form even more difficult. An optimizer that is unable to always transform equivalent expressions to a single form and that is unable to always arrange those expressions in the most optimal manner is not going to be an optimizer that is always able to produce the best (or even an acceptable) execution plan.

And so here we are, still hand-optimizing complex SQL queries instead of solving problems for people and businesses. One of the prevailing myths of this society is that progress moves in a straight, smooth path from one technological triumph to the next. But it doesn't. The path isn't uninterrupted, it's full of ruts and dead ends, and we seldom know where it will lead us. Sometimes, we even lose the path completely for a time. That's why Western philosophy didn't advance much beyond Aristotle and Plato for over 1000 years, that's why we're still using the internal combustion engine 150 years after its invention, and that's why we don't have real relational databases.

Wednesday, July 23, 2003

War and Peace
I would rather have seen Saddam's sons captured and tried for crimes against their people instead of killed. The equivalent of the Nuremburg trials for the Ba'athists would set a good precedent for the importance of the rule of law in a newly democratic Iraq.

Will their deaths help to end the insurgency? That sounds more like a wish from the administration's spin machine than anything else. If the present insurgency is nothing more than a series of guerilla attacks by Saddam's supporters, then it wasn't really much of a threat because it obviously lacks popular support (proof: the celebrations in the Baghdad streets upon confirmation of their deaths). But if it's something more - like a slowly growing popular rebellion against a foreign invader - it isn't going to make any real difference.
Proposition 13
Proposition 13 was passed in 1978. It cut California property taxes by 30 percent and capped the rate of increase in the future. It still arouses strong passions for and against, but the pro and con viewpoints seem to me to miss the most significant effect of Prop 13. That effect was identified by the Santa Cruz City Manager, Richard Wilson, who wrote, "In 1978 Proposition 13 reconstituted the financial structure of California's public sector. The result was system change, from one in which the State, counties, cities, special districts, and school boards all made independent financial decisions to one in which the State is the only financial decision maker of any consequence".

This is the reason why all of California's municipalities are in trouble along with the state. Since the passage of Prop 13, the state has increasingly taken revenues historically dedicated to local government for itself. During the boom, an adequate amount was returned to municipalities. Now, with the bust and concomitant deficits, that money won't be returned and local governments will be forced to make significant and painful budget cuts. So we should expect local services and infrastructure to decline noticeably in the next few years.

Yet another example of the law of unintended consequences. It's why good policy - the wonkish, obsessed with both minutiae and outcomes, boring-as-hell kind - is so important and so critical. The devil really is in the details. California's rising property taxes and spending did desperately need to be reined in the late '70's. But Prop. 13 wrecked California's school system and transferred fiscal control from local to state government. It's worth analyzing what went wrong and what better approaches could have been taken, because those lessons could be applied today. Maybe we could find a better way to hold down property taxes, revoke Prop 13, and even transfer control over local collected funds back to the localities where they were collected. It's never too late for good policy.

Oh, and the idea of deficits being concomitant with the bust isn't quite true. Our huge state deficit can't be blamed on just the economic downturn. Good policy and realistic planning would have ameliorated the effects of the bust. Unfortunately, our Governer and the Legislature elected to go on spending as if the boom would continue forever. I'd like to oppose the recall initiative because it's just a ploy by the Republicans to get Arnold into office two years early, but Gray Davis fully deserves it. Too bad there's no way to recall the legislature as well.
Iranian Blogs
Discovered a relatively new blog called Iranian Truth, written by two Iranian women living in Cairo and Los Angeles, respectively. In addition to being interesting and well-written, it also has plenty of links to other Iranian blogs and news sources. It's good to have the opportunity to read authentic voices talking about their country and their people instead of trying to discern the truth through the haze of ill-informed and ideologically-driven Western news sources. It's a pity that Afghanistan's infrastructure is so ruined and lacking that it's impossible to get first-hand information like this from there.

Monday, July 21, 2003

War and Peace
Strategic Forecasting has a story that's crystallized my thinking about this Administration's grand strategic vision (or apparent lack thereof) in the war on terror. Key quote: “Lurking in the shadows is the not fully articulated perception that the Iraq war not only began in deception but that planning for the Iraq war was incompetent -- a perception driven by the realization that the United States is engaged in a long-term occupation and guerrilla war in Iraq, and the belief that the United States neither expected nor was prepared for this.”. The deception, by itself, isn't fatal - the real problem, according to the article, is that “what bothers the American public is the idea that the lying is not designed to hide the strategy, but to hide the fact that there is no strategy.”.

And that's what one of things that's bothered me all along. I have not heard any believable or even coherent formulation of "this is how we win". Because I think the real key here is not winning the battles - that's been ridiculously easy - but winning the peace. Winning the peace is the province of grand strategy. And we don't seem to have one.

The consequences of not having an overarching and achievable grand strategy are pretty severe. Here's what B.H. Liddell Hart says in his seminal book, Strategy: “They did not look beyond the immediate strategic aim of "winning the war" and were content to assume that military victory would assure peace - an assumption contrary to the general experience of history. The outcome has been the latest of many lessons that pure military strategy needs to be guided by the longer and wider view from the higher plane of "grand strategy"”

Liddell Hart was talking about the outcome of World War II, which led directly to the Cold War. He could have just as easily been talking about the outcome of World War I. The Treaty of Versailles led directly to WWII as well as to many of the other ills of this century. Among those was the somewhat arbitrary redrawing of the map of the Middle East, which has had a wide variety of direct and indirect consequences - one of them being the invasion of Kuwait by Iraq in 1991. We won that war and then failed to win the peace, which ultimately led to our invasion of Iraq early this year. And so here we are.

I think we need the notion of Hart's “Indirect Approach” to strategy now more than ever. The neo-cons, unfortunately, are the direct heirs of Kissingerian RealPolitik and so suffer from the same ills of short-sightedness and ethically challenged expediency. Put another way, they're content to try to fix problems with sledgehammers and then immediately walk away. You can criticize their approach on all kinds of grounds, but the very worst thing you can say about it is that it's unlikely to decisively win this conflict. And we need to win; the alternative is that the Middle East and Africa succumb to a post-industrial Dark Age and drag the rest of the world down with them.

Now, I'm only an amateur student of history and politics. But it seems to me any approach at the level of grand strategy should deal with root causes and not just our immediate circumstances. Since this is a war on terror the grand strategic goal should be to remove the underlying causes of terrorism. Have we really asked ourselves what those underlying causes are? And if we identified them, how would that shape our grand strategy? What is the longer and wider view?

I have my own answers to those questions, but I'll save them for another post.