Inquiry Into Inquiry

From Zipf’s Law and the category of “things that vary inversely to frequency” I got my first brush with the idea that keeping track of usage frequencies is part and parcel of building efficient codes.

In its first application the environment the Learner had to learn was the usage behavior of its user, as given by finite sequences of characters from a finite alphabet which might as well be called words and as given by finite sequences of those words which might as well be called phrases or sentences.  In other words, Job One for the Learner was the job of constructing a user model.

In that frame of mind we are not seeking anything so grand as a Universal Induction Algorithm but simply looking for any approach that gives us a leg up, complexity wise, in Interactive Real Time.

Resource

• Survey of Theme One Program

cc: Cybernetics • Ontolog Forum (1) (2) • Systems Science (1) (2)
cc: Peirce List (12-12) (18-02) (18-03) (20-09) (20-10) (21-10)
cc: FB | Theme One Program • Laws of Form (1) (2)

Sometime around 1970 John B. Eulenberg came from Stanford to direct Michigan State’s Artificial Language Lab, where I would come to spend many interesting hours hanging out all through the 70s and 80s.  Along with its research program the lab did a lot of work on augmentative communication technology for limited mobility users and the observations I made there prompted the first inklings of my Learner program.

Early in that period I visited John’s course in mathematical linguistics, which featured Laws of Form among its readings, along with the more standard fare of Wall, Chomsky, Jackendoff, and the Unified Science volume by Charles Morris which credited Peirce with pioneering the pragmatic theory of signs.  I learned about Zipf’s Law relating the lengths of codes to their usage frequencies and I named the earliest avatar of my Learner program XyPh, partly after Zipf and playing on the xylem and phloem of its tree data structures.

Resource

• Survey of Theme One Program

cc: Cybernetics • Ontolog Forum (1) (2) • Systems Science (1) (2)
cc: Peirce List (12-12) (18-02) (18-03) (20-09) (20-10) (21-10)
cc: FB | Theme One Program • Laws of Form (1) (2)

A side-effect of working on the Theme One program over the course of a decade was the measure of insight it gave me into the reasons why empiricists and rationalists have so much trouble understanding each other, even when those two styles of thinking inhabit the very same soul.

The way it came about was this.  The code from which the program is currently assembled initially came from two distinct programs, ones I developed in alternate years, at first only during the summers.

In the Learner program I sought to implement a Humean empiricist style of learning algorithm for the adaptive uptake of coded sequences of occurrences in the environment, say, as codified in a formal language.  I knew all the theorems from formal language theory telling how limited any such strategy must ultimately be in terms of its generative capacity, but I wanted to explore the boundaries of that capacity in concrete computational terms.

In the Modeler program I aimed to implement a variant of Peirce’s graphical syntax for propositional logic, making use of graph-theoretic extensions I had developed over the previous decade.

As I mentioned, work on those two projects proceeded in a parallel series of fits and starts through interwoven summers for a number of years, until one day it dawned on me how the Learner, one of whose aliases was Index, could be put to work helping with sundry substitution tasks the Modeler needed to carry out.

So I began integrating the functions of the Learner and the Modeler, at first still working on the two component modules in an alternating manner, but devoting a portion of effort to amalgamating their principal data structures, bringing them into convergence with each other, and unifying them over a common basis.

Another round of seasons and many changes of mind and programming style, I arrived at a unified graph-theoretic data structure, strung like a wire through the far‑flung pearls of my programmed wit.  But the pearls I polished in alternate years maintained their shine along axes of polarization whose grains remained skew in regard to each other.  To put it more plainly, the strategies I imagined were the smartest tricks to pull from the standpoint of optimizing the program’s performance on the Learning task I found the next year were the dumbest moves to pull from the standpoint of its performance on the Reasoning task.  I gradually came to appreciate that trade-off as a discovery …

Resource

• Survey of Theme One Program

cc: Cybernetics • Ontolog Forum (1) (2) • Systems Science (1) (2)
cc: Peirce List (12-12) (18-02) (18-03) (20-09) (20-10) (21-10)
cc: FB | Theme One Program • Laws of Form (1) (2)

The main idea behind the Theme One program is the efficient use of graph-theoretic data structures for the tasks of “learning” and “reasoning”.

I am thinking of learning in the sense of learning about an environment, in essence, gaining information about the nature of an environment and being able to apply the information acquired to a specific purpose.

Under the heading of reasoning I am simply lumping together all the ordinary sorts of practical activities which would probably occur to most people under that name.

There is a natural relation between the tasks.  Learning the character of an environment leads to the recognition of laws which govern the environment and making full use of that recognition requires the ability to reason logically about those laws in abstract terms.

Resource

• Survey of Theme One Program

cc: Cybernetics • Ontolog Forum (1) (2) • Systems Science (1) (2)
cc: Peirce List (12-12) (18-02) (18-03) (20-09) (20-10) (21-10)
cc: FB | Theme One Program • Laws of Form (1) (2)