Let us now return to the information.  The information of a term is the measure of its superfluous comprehension.  That is to say that the proper office of the comprehension is to determine the extension of the term.  For instance, you and I are men because we possess those attributes — having two legs, being rational, &c. — which make up the comprehension of man.  Every addition to the comprehension of a term lessens its extension up to a certain point, after that further additions increase the information instead.

Thus, let us commence with the term colour;  add to the comprehension of this term, that of red.  Red colour has considerably less extension than colour;  add to this the comprehension of dark;  dark red colour has still less [extension].  Add to this the comprehension of non-blue — non-blue dark red colour has the same extension as dark red colour, so that the non-blue here performs a work of supererogation;  it tells us that no dark red colour is blue, but does none of the proper business of connotation, that of diminishing the extension at all.  Thus information measures the superfluous comprehension.  And, hence, whenever we make a symbol to express any thing or any attribute we cannot make it so empty that it shall have no superfluous comprehension.

I am going, next, to show that inference is symbolization and that the puzzle of the validity of scientific inference lies merely in this superfluous comprehension and is therefore entirely removed by a consideration of the laws of information.

(Peirce 1866, p. 467)