Information Recapped
Reflection on the inverse relation between uncertainty and information led us to define the information capacity of a communication channel as the average uncertainty reduction on receiving a sign, taking the acronym auroras as a reminder of the definition.
To see how channel capacity is computed in a concrete case let’s return to the scene of uncertainty shown in Figure 5.
For the sake of the illustration let’s assume we are dealing with the observational type of uncertainty and operating under the descriptive reading of signs, where the reception of a sign says something about what’s true of our situation. Then we have the following cases.
- On receiving the message “A” the additive measure of uncertainty is reduced from
to
so the net reduction is
- On receiving the message “B” the additive measure of uncertainty is reduced from
so the net reduction is
The average uncertainty reduction per sign of the language is computed by taking a weighted average of the reductions occurring in the channel, where the weight of each reduction is the number of options or outcomes falling under the associated sign.
The uncertainty reduction 
The uncertainty reduction 
Finally, the weighted average of the two reductions is computed as follows.
Extracting the pattern of calculation yields the following worksheet for computing the capacity of a two‑symbol channel with frequencies partitioned as
Capacity of a channel {“A”, “B”} bearing the odds of 60 “A” to 40 “B”
![\begin{array}{lcl} & = & \quad {1 \over n}(k_1(\log n - \log k_1) + k_2(\log n - \log k_2)) \\[4pt] & = & \quad {k_1 \over n}(\log n - \log k_1) + {k_2 \over n}(\log n - \log k_2) \\[4pt] & = & \quad - {k_1 \over n}(\log k_1 - \log n) - {k_2 \over n}(\log k_2 - \log n) \\[4pt] & = & \quad - {k_1 \over n}(\log {k_1 \over n}) - {k_2 \over n}(\log {k_2 \over n}) \\[4pt] & = & \quad - (p_1 \log p_1 + p_2 \log p_2) \\[4pt] & = & \quad - (0.6 \log 0.6 + 0.4 \log 0.4) \\[4pt] & = & \quad 0.971 \end{array}](https://s0.wp.com/latex.php?latex=%5Cbegin%7Barray%7D%7Blcl%7D++%26+%3D+%26+%5Cquad+%7B1+%5Cover+n%7D%28k_1%28%5Clog+n+-+%5Clog+k_1%29+%2B+k_2%28%5Clog+n+-+%5Clog+k_2%29%29++%5C%5C%5B4pt%5D++%26+%3D+%26+%5Cquad+%7Bk_1+%5Cover+n%7D%28%5Clog+n+-+%5Clog+k_1%29+%2B+%7Bk_2+%5Cover+n%7D%28%5Clog+n+-+%5Clog+k_2%29++%5C%5C%5B4pt%5D++%26+%3D+%26+%5Cquad+-+%7Bk_1+%5Cover+n%7D%28%5Clog+k_1+-+%5Clog+n%29+-+%7Bk_2+%5Cover+n%7D%28%5Clog+k_2+-+%5Clog+n%29++%5C%5C%5B4pt%5D++%26+%3D+%26+%5Cquad+-+%7Bk_1+%5Cover+n%7D%28%5Clog+%7Bk_1+%5Cover+n%7D%29+-+%7Bk_2+%5Cover+n%7D%28%5Clog+%7Bk_2+%5Cover+n%7D%29++%5C%5C%5B4pt%5D++%26+%3D+%26+%5Cquad+-+%28p_1+%5Clog+p_1+%2B+p_2+%5Clog+p_2%29++%5C%5C%5B4pt%5D++%26+%3D+%26+%5Cquad+-+%280.6+%5Clog+0.6+%2B+0.4+%5Clog+0.4%29++%5C%5C%5B4pt%5D++%26+%3D+%26+%5Cquad+0.971++%5Cend%7Barray%7D&bg=ffffff&fg=000000&s=0&c=20201002)
In other words, the capacity of the channel is slightly under 1 bit. That makes intuitive sense in as much as 3 against 2 is a near‑even split of 5 and the measure of the channel capacity, otherwise known as the entropy, is especially designed to attain its maximum of 1 bit when a two‑way partition is split 50‑50, that is, when the distribution is uniform.
cc: FB | Semeiotics • Laws of Form • Mathstodon • Ontolog • Academia.edu
cc: Conceptual Graphs • Cybernetics • Structural Modeling • Systems Science
Inquiry Into Inquiry 
Pingback: Survey of Pragmatic Semiotic Information • 8 | Inquiry Into Inquiry