A major milestone: a machine has consistently defeated very strong humans at at the game of “go” (report here ). The caveat is that this is on a 7×7 board, which is dozens of orders of magnitude less complex than the full 19×19 game.

Crazy Stone, the program in question, used the so-called Monte Carlo technique. Basically, it plays hundreds of thousands of random games and finds which move leads to the highest winning percentage. An interesting Wired article on this approach is here. A server allows bots to play thousands of games of each other, providing real-time information on the degree of promise of various approaches.

The real question is whether this is a one-trick pony useful only for tiny boards, or whether it can be usefully extended to 19×19. Until 20 years gives us immensely more powerful computers, we need some kind of abstraction to serve as the topic of Monte Carlo simulations. Or, perhaps Monte Carlo can be another trick in the go programmer’s bag of tricks, somehow combined with the opening books and connectivity analysis and pattern matching and heuristics that serve as the basis of today’s strongest 19×19 programs.

The brain is not a computer, of course. But wait. Computers are devices that process information…and that’s certainly what the brain does, right?

As an alert reader pointed out in my post on Roger Penrose, the English mathematician and philosopher, the problem here lies in the definition of “computer” or “computer-like”.

In one sense, saying that the brain is a computer is saying exactly nothing. That’s since the word “computer” refers to any device that “computes”—processes information. That includes everything from adding machines to quantum computers . Penrose may think the brain operates on quantum principles—so fine, it may be a quantum computer, but that’s still a computer. The only possibility for negating the assertion that the brain is a computer in this extremely general sense is to hold that the brain does not even “process information”. Perhaps it is doing something with information other than processing it, or perhaps it is processing something other than information as we know it. A more likely possibility is that it might be processing information but, at the same time, doing additional, important things that cannot be interpreted as processing information—such as being conscious. In that case, we would have to say the brain is only partially “like a computer.”

We also have to be aware of hidden agendas in defining these words. For some people, saying “the brain is not (like) a computer” is a kind of code for a belief in the human “spirit”, the absolute uniqueness of our “minds”, or the ineffability of existence. These people are simply making an exclamation of a particular variety of faith.

Personally, I believe that even consciousness is a form of information processing, and thus that the brain is a computer in the tautological sense. (Not that I think it’s a quantum computer.)

In that case, in what sense of the word “computer” does the brain fail to qualify? The narrowest sense is that of von Neumann, a stored program computer , one that computes a problem sequentially and deterministically from beginning to end. Even if we include parallel processing within the von Neumann paradigm, our “neural computer” does not fit within that framework. Most basically, it seems clear our brains involve no equivalent of a “program” or “stored data” in the von Neumann sense.

A broader sense is that of the Turing machine. This is the model that Eric Baum believes the brain works under, although in his details he often seems to have a von Neumannian focus. A Turing machine executes algorithms, and serves as a model for all modern computer hardware and software architectures.

If we limit ourselves to considering the synaptic architecture of the brain, we can say that it has Turing-like aspects, and its processing can be described as being algorithmic in nature, but it’s very unlike any Turing machine you’ve ever seen, with tens of thousands of dendrites converging on individual neurons, and neuronal plasticity involved in a type of learning at the level of the architecture of the “machine” itself.

But in other important regards the brain is almost certainly not a Turing machine. First, neural functioning involves chemical and hormonal levels which are fundamentally analog in nature; such analog behavior could be simulated by a digital computer, but never reproduced exactly. Second, the brain appears to be strongly specialized to deal with intrinsic temporal flow and temporal patterns.

In summary, we can say that the brain is a semi-analog, parallel, self-modifying, temporally-specalized device for processing information. Whether or not that’s a “computer”, we’ll leave up to the reader.

Numenta is an intriguing new startup that plans to commercialize technology based a model of human memory developed by Jeff Hawkins, he of Palm Pilot and Graffiti fame, in his book On Intelligence (see also the book’s website).

According to the website:

Numenta co-founder Dileep George has created a mathematical formalism that follows and extends Hawkins’ biological theory. This formalism is a variation of Belief Propagation, a mathematical technique invented by Judea Pearl. Belief propagation explains how a tree of conditional probability functions can reach a set of mutually consistent beliefs about the world. By adding time and sequence memory to each node of the tree, belief propagation can be morphed to match Hawkins’ biological theory.

We may be seeing the very beginning of a second “AI” boom, one rooted this time in actual neurobiological research, or at least in models inspired by, or plausibly informed by, schemes of brain functioning.

Stephen Wolfram (Wikipedia article) is the child prodigy who went on to invent Mathematica, the ubiquitous software package for mathematical analysis. It’s now been three years since the publication of his A New Kind of Science (Wikipedia article) to much fanfare. The book’s main thesis is that complexity can emerge from extremely simple models, of the type that can be embodied in computer programs. He claims

My purpose in this book is to initiate a transformation in science…making it possible to make progress on a remarkable range of fundamental issues that have never successfully been addressed by any of the existing sciences before.

The book is nearly 1200 pages of dense mathematics, diagrams, and discussion. The notes alone are over 300 pages, and the book is not cheap, so I’m not recommending people read it, but it is nonetheless thought-provoking, regardless of whether you accept his grandiose claims, which many people do not. For one thing, it’s never clear whether he’s claiming that his models might generate behavior which resembles the real world, or that they are the models governing the real world.

At one level, this book is a work of philosophy. So how does Wolfram approach the hoary old philosophical problem of free will? For him, free will is related to “computational irreducibility”, one of his key concepts, which basically means that there are some types of computation which don’t allow shortcuts. Such phenomena permit no predictions about what is going to happen until it actually does. There is no future until the universe has finished computing it.

Wolfram says, “I believe that it is this kind of intrinsic process [complex, unpredictable behavior generated by simple rules] that is primarily responsible for the apparent freedom in the operation of our brains.” A novel definition of “freedom”: “free of obvious laws”, “freedom from predictability”.

In a word, Wolfram believes that free will vs. determinism is a false dichotomy. The world proceeds deterministically, but appears to be (is?) imbued with “freedom” due to its unpredictability.

(Students of language may find it interesting that for this book Wolfram invented a distinct new style of writing which he claims is specifically suited to its material. That style involves starting a large percentage of his sentences with conjunctions: “And” (to show a connected thought), “But” (to show a contrasting thought), or “For” (to show background or reason). He notes that this helps break up extremely long sentences. After a few hundred pages, however, this style becomes extremely irritating.)

New Scientist reports on statistical machine translation and the commercialization being done by Language Weaver.

When we talk about computational models of neurotheology, what do we mean?

What first springs to mind is to model an individual brain, or more likely brain/body system, to model the biological processes associated with a religious experience. Modeling transcendance, if you will. But could we tell that in fact what is being simulated is a religious experience? Humans know they are having or had religious experiences by being, at some level, conscious of them. But we can hardly build an entire mechanism of consciousness into our computer model. And even “pure” transcendant religious experiences have historical and social backgrounds, or, to put it another way, occur within the context of certain memories, which even Blue Brain could not model. All in all, a tough problem.

More tractable would be to integrate a coarse statistical model of individual religious experience with a sociological model. In other words, we would model religious experiences, large and small, but at the population level. Some percentage of religious experiences are at the breakthough level that can jumpstart an entire new religion, whereas others might suffice to rejuvenate or sustain a religion, if experienced by enough adherents.

Once a religion has started, we would apply sociological modeling techniques to model its spread and/or decline as the system of doctrines or cermonies that religions inevitably settle into, albeit leavened by periodic awakenings that serve to inject new energy into the religion for some period of time.

The model involves two distinct categories of data. The first relates to the statistical frequency, intensity, and types of human religious experience. I’m not aware of any data on that topic. Our goal would be derive hypotheses for those values, hopefully ones that could be cross-validated, either by working backward through the model from the sociological data mentioned below, or by running multiple scenarios to find one or more that are consistent with the sociological data.

The sociological data I am referring to, which should be relatively easy to capture, is primarily the distribution of sizes of religious groups over time, as well as other peripheral data such as conversion rates.

A flavor of the sociological side of the model can be gained from Simulating the Emergence of New Religious Movements, a paper which crudely models the formation and growth of religions. I can’t agree with the premise that NRM (new religious movement) founders are “rational agents who obtain various social advantages such as reputation enhancement and increased respect from other utility maximizing rational agents who buy their solutions”, but the seeds of one half of the model I propose—the sociological side—are there.

I hereby name this particular approach computational socioneurotheology™.

First Deep Blue. Then Blue Gene. Now, Blue Brain.

Solve chess, next tackle the wonders of the gene, then unravel the mysteries of the brain for an encore?

Sort of. Deep Blue is now in a museum, an ultimately unsatisfying technological tour de force that accomplished little more than demonstrating that the complexity parameters of chess put it within reach of your average supercomputer.

And Deep Gene is not really designed to do anything with genes, although it’s been used to do molecular simulations. It’s cleverly named to create the image of a family of supercomputing projects, but in fact has nothing to do with Deep Blue, and at heart is a massive science fair project to see how many teraflops you get when you string together 32K nodes.

Blue Brain is the catchy name of the latest project, a partnership with a Swiss university (EPFL ) to use Blue Gene to model the human brain.

Although this project has been widely reported, most of the commentary has been at the level of calling the project a “virtual brain”, claiming for example

the hope is that the virtual brain will help shed light on some aspects of human cognition, such as perception, memory and perhaps even consciousness.

Wow, a thinking computer that’s also conscious.

But readers of Numenware will want to understand the research plan in a bit more detail. The first project is a cellular-level model of a neocortical column. They’ll simulate 10,000 neurons and 100 million synapses (yes, there really are that many synapses on 10,000 neurons). They’re going to use 8,000 nodes, so it would seem obvious to have one node per neuron, but that doesn’t appear to be the approach. They say the simulation will run in “real time”, but shouldn’t it be able to go faster? Of course they’ll have snazzy visualization systems. Hey, can I go for a “walk” among your neocortical columns?

From there, the researchers hope to go down—and up. They’ll go down to the molecular level, and up to the level of the entire neocortex. To do the latter will require a simplified model of the neocortical columns, which they hope to be able to derive from the first project. They’ll eventually move on the subcortical parts of the brain and before you know it, your very own virtual brain.

It’s undoubtedly true that this is “one of the most ambitious research initiatives ever undertaken in the field of neuroscience,” in the words of EPFL’s Henry Markram, director of the project. But I wonder if the kind of knowledge we gather about brain functioning from this project will be the same kind of knowledge we gathered about chess from Deep Blue.

Markram has a very micro focus. For instance, he has sliced up thousands of rat brains and stained them and stimulated them and cataloged them. And this whole project has the same intensely micro focus. That’s extremely valuable, but it’s like building a supercomputer simulation of how gasoline ignites in order to understand how a car runs, when we don’t even understand the roles of the carburetor and fuel pump and combustion chamber, to borrow an overused analogy.

For instance, I’m sure Blue Brain will cast light on the mechanisms underlying memory, but when these guys say “memory” they mean synaptic plasticity. What I want to know is how I remember my beloved Shiba-ken, Wanda, who was hit and killed by a car in Kamakura.

It seems to me we don’t need supercomputers to model the brain, although I’m sure they’ll be useful; we need concepts to model. The actual model could be no bigger than Jay Forrester’s ground-breaking system dynamics model of the world’s socioeconomic system. The problem is not the technology for modeling—it’s what we model.

The same goes for neurotheology. We desperately need a computer model, but before that—we need a theory.

I just noticed that installing Google Desktop injected Google code into my TCP/IP stacks. I love Google Desktop, but do we really want Google controlling the TCP/IP on our computers?

Is “performant” a word? I came across it in an article about Microsoft Visual Studio .NET 2005:

C++ is the easiest language to use for native interop and is often the most performant.

We’ve all heard that Microsoft software is buggy. Bill G. says this is just because so many people use it they find all the bugs and so they get more attention, plus there are those who just enjoy pissing on MS. But I just had the experience, for the first time in a decade, of programming for Windows. The bottom line is that the entire environment is so buggy, flaky, and poorly documented that it’s a miracle anyone can write a program for Windows which runs at all.

Bill says that the open-source model can’t work because there’s no economic incentive to produce solid software or support it. What I realized is that he has it exactly backwards. The MS software development model can’t produce good software because it’s corrupted by the need to get the next version of the product out the door. Good software periodically and suddenly requires being rearchitected, or “refactored”, as they say, at inopportune times from the standpoint of the CFO. With the MS model, there is never a reason to take the time to go back and build the software right.

The elegance of the MS architecture has gone steadily down since Windows 3.1, which although kind of funky, at least had a weird predictability and consistency about it. Now, there are layers upon layers of additional libraries and wrappers on top of it, each
documented more poorly than the last. The only way to use the MS docs is to search them using Google, and even then it is all too often the case that you just can’t find what you are looking for, or worse—it’s wrong.

Programming in the MS world uses the approach I call “throwing mud at the wall”. Basically, you throw mud at the wall and see what sticks. You can never figure out the best way or the right way to do something in advance so you just try all the ways and use the first one that works. It’s like playing “pin the tail on the donkey”.

A good architecture has the characteristic that it surprises you from time to time with the cool things you can do easily because of its superior design. I’ve yet to come across a single thing in the entire Windows architecture that gave me this feeling.

Just a couple of quick examples, all from the browser extension world, which is what I was working on.

1. The MS docs refer to an API to manipulate your browser’s history list. They give the name of a header file you use to access the API. But this header file exists nowhere in the world, except in a non-MS version that you can find on the net that was reverse-engineered by some poor sap who had no choice.
2. A key ATL library used for internet access is just missing the wide-character version of the interface needed to read a web page off the net—making the entire app fail to link. I finally found this info in the MS knowledge base—with no work-around given,
   of course.
3. Using the technique suggested to take the user to a particular web page after running the installer—namely a one-line VB application—polluted the entire installer with “.NET-ness”, which persisted even when I removed the VB app, requiring me to rebuild the installer component from scratch.
4. The HTML DOM API provides a W3C-compatible “text range” object to represent ranges of text within a web page. But it is so buggy that something as basic as moving and endpoint of a text range forward or backward by one character doesn’t even work. And operations on a text range corrupt the DOM.
5. The DOM built by IE is not even well-formed to the extent it sometimes cannot be walked from start to end.
6. The API uses multiple interfaces for the same thing—four different interfaces for windows, for example—and of course the documentation is structured so you can never find anything unless you knew what you were looking for before you started.

And so on, multiplied by ten or a hundred.

What’s surpising, then, is not that Microsoft’s software has as many bugs as it does but that it doesn’t have many, many more; not that their software is often late, sometimes by years, but that it gets released at all. And I suspect that the high levels of profitability deriving from Microsoft’s near-monopoly in many markets is hiding the fact that it is well behind even other commercial software companies in development productivity because of the abysmal state of its architecture