Inquiry Into Inquiry

Memories are coming back to me more through the association of ideas than ordered by time or place.  I can sense, almost touch a tangle of thoughts interlaced with each other — the “information first” approach to ontology, the “arrows only”, element‑free angle on category theory, Peirce’s relativity of generals and individuals dispatching nominalism once and for all — but there is at core a hard knot of ideas so tightly wound it makes it difficult to articulate the links or see the untying if there is one to make.

cc: FB | Semeiotics • Laws of Form • Ma^thstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science

I was at the time working as a “scanner” in the High Energy Physics Lab at Michigan State, sitting in a darkened room measuring tracks of particle interactions projected on a lighted scanning table from reels and reels of bubble chamber photographs gathered at CERN in a massive mad dash accelerator experiment some years before.  For my part it was a menial job, 4pm to midnight every worklong day, but even a minion can imagine himself sharing in a hunt for the particle, or whatever the Grail or Questying Beastie was at the time.

Meanwhile, in another part of the grove, I was spending my daylight hours checking off the final boxes for my Bachelor’s degree, the main thing being to get my paper on Peirce, “Complications of the Simplest Mathematics”, approved as a substitute for a field study requirement.  That had taken me two years’ work in MSU’s media library, poring through the microfilm reels of Peirce’s Nachlass in search of enlightenment about a single puzzling paragraph I tripped over in his Collected Papers.

cc: FB | Semeiotics • Laws of Form • Ma^thstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science

I remember it was back in ’76 when I began to notice a subtle shift of focus in the computer science journals I was reading, from discussing X to discussing Information About X, a transformation I noted mentally as whenever I ran across it.  I suppose that small arc of revolution had been building for years but it struck me as crossing a threshold to a more explicit, self‑conscious stage about that time.

cc: FB | Semeiotics • Laws of Form • Ma^thstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science

Information Recapped

Reflection on the inverse relation between uncertainty and information led us to define the information capacity of a communication channel as the average uncertainty reduction on receiving a sign, taking the acronym auroras as a reminder of the definition.

To see how channel capacity is computed in a concrete case let’s return to the scene of uncertainty shown in Figure 5.

For the sake of the illustration let’s assume we are dealing with the observational type of uncertainty and operating under the descriptive reading of signs, where the reception of a sign says something about what’s true of our situation.  Then we have the following cases.

• On receiving the message “A” the additive measure of uncertainty is reduced from to so the net reduction is
• On receiving the message “B” the additive measure of uncertainty is reduced from to so the net reduction is

The average uncertainty reduction per sign of the language is computed by taking a weighted average of the reductions occurring in the channel, where the weight of each reduction is the number of options or outcomes falling under the associated sign.

Finally, the weighted average of the two reductions is computed as follows.

Extracting the pattern of calculation yields the following worksheet for computing the capacity of a two‑symbol channel with frequencies partitioned as

In other words, the capacity of the channel is slightly under 1 bit.  That makes intuitive sense in as much as 3 against 2 is a near‑even split of 5 and the measure of the channel capacity, otherwise known as the entropy, is especially designed to attain its maximum of 1 bit when a two‑way partition is split 50‑50, that is, when the distribution is uniform.

cc: FB | Semeiotics • Laws of Form • Ma^thstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science

Information Channeled

Suppose we find ourselves in the classification‑augmented or sign‑enhanced situation of uncertainty shown in Figure 5.  What difference does it make to our state of information regarding the objective outcome if we heed one or the other of the two signs, “A” or “B”, at least, operating on the charitable assumption we grasp the significance of each sign?

• Under the sign “A” our uncertainty is reduced from to
• Under the sign “B” our uncertainty is reduced from to

The above characteristics of the relation between uncertainty and information allow us to define the information capacity of a communication channel as the average uncertainty reduction on receiving a sign, a formula with the splendid mnemonic “AURORAS”.

cc: FB | Semeiotics • Laws of Form • Ma^thstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science

Uncertainty Moderated

In many ways the provision of information, a process which serves to reduce uncertainty, operates as an inverse process in relation to the type of uncertainty augmentation which takes place in compound decisions.  By way of illustrating the relation in question, let us return to our initial example.

A set of signs enters on a setup like that as a system of middle terms, a collection of signs one may regard, aptly enough, as constellating a medium.

The language or medium in Figure 5 is the set of signs   On the assumption the initial outcomes are equally likely it is possible to associate a frequency distribution and thus a probability distribution with the language, thereby defining a communication channel.

The most important thing at this stage of development is simply to get a first handle on the “conditions for the possibility of signs making sense”, but once we have that much in hand we can begin constructing the rough and ready pieces of information-theoretic furniture we need, like measures of uncertainty, channel capacity, and the amount of information associated with the reception or the recognition of a single sign.

cc: FB | Semeiotics • Laws of Form • Ma^thstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science

Uncertainty Multiplied

In our minds’ eyes last time we imagined ourselves coming to a fork in the road and seeing four paths diverge from that point.  Suppose a survey of the scene ahead now shows each path reaching a point where another decision has to be made, this time a choice between two alternatives.  Figure 4 gives us the picture so far.

The Figure illustrates the fact that the compound uncertainty, is the product of the two component uncertainties,   To convert that to an additive measure, one simply takes the logarithms to a convenient base, say and thus arrives at the not too astounding fact that the uncertainty of the first choice is bits, the uncertainty of the next choice is bit, and the total uncertainty is bits.

cc: FB | Semeiotics • Laws of Form • Ma^thstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science

Uncertainty Measured

As a matter of fact, at least in the discrete types of cases we are currently considering, it would be possible to use the degree of a node, the number of paths fanning out from it, as a measure of uncertainty at that point.  That would give us a multiplicative measure of uncertainty rather than the sorts of additive measures we are more used to thinking about — no doubt someone would eventually think of taking logarithms to bring measures back to familiar ground — but that is getting ahead of the story.

To illustrate how multiplicative measures of multiplicity, variety, or uncertainty would work out, let us take up a simpler example, one where the main choice point has a degree of four.  Figure 3 gives us the picture.

• cc: FB | Semeiotics • Laws of Form • Ma^thstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science