Information

portuguese gaita again. today i’ll be continuing the subject of structure and quantification. this time, i will use our concept of thing directly in the calculation of structure.

we first saw that a thing of layer l , i.e., a thing is made of things. for each thing, its information (quantification of structure), is a function of the arrangement of its constituents, . note this is only true if there is no mutual information between the constituents. if there is, we can just conjure up a higher level “middle layer” that hides this mutual information as a single entity. this makes the structure of a thing of any layer quantifiable and independent of its constituents, only their quantity and arrangement. this means we do not need to know the information of constituents to know the information of a higher level system. this is a bit mind boggling, for sure, but this means our layers are irrelevant for the calculation of information, what matters is the quantity of sub-elements.

how is this relevant? this means that to properly quantify any structure, we can either quantify it in a single layer, just like the real world, or quantify it including explicitly all the information of the parts (as we, sentient beings, define them). if we choose to quantify and include the parts, we get . infinite recursion, just as we would expect from a seemingly fractal definition. what does this lead to? let’s calculate it using induction (philosophers start panicking!).

grouping,

we can now group this beast using sum and product operators. this will only disguise the big beast this is.

let’s hope nobody notices how big these numbers are. it’s obvious it is a divergent series and grows with N.

what this means is that we can expand or contract our “zoom” to define structure at any level and accounting for any layer whatsoever. we’ll need this in the future. for now, let’s let these concepts settle. when structure is quantified over a hierarchy as ours, the quantity always depends on the zoom chosen. this paradoxical result is well known from fractal mathematics. i guessed our structure was fractal, this is the proof. fractals are a consequence of abstraction. for nature, since she is abstraction-less, there is only one value for the structure of the universe: the value for the structure of the universe. nothing like ending a philosophical text with a tautology

back to portuguese pipes and modern tracks with omiri

just a brief comment on my last post. some dating sites that don’t follow the rules i said work too, so don’t take my previous post too seriously. stats are out there and they tell many different stories. back to heavy topics, and please correct me if you find an error, today’s math is extra fancy

a long time ago, we began discussing structure as property of things. as previously summarized, we can think of things as letters in a gigantic soup called reality. structure is when, through stirring this soup, words come up. but how does one quantify structure? this is a big challenge that i’m currently embracing but haven’t put into good numbers yet. so for now, we will deal with our abstract quantity thing and not apply it to the physical reality, until this develops further. since we do not have any laws of physics that change our distribution of things, we will assume we are dealing with an entirely abstract system. this means letters have all equally likelihood of banding up with any other letters, and nobody adds letters or eats letters from the soup.

this means we can apply basic information theory. no special distributions, just letters, means that this specific arrangement (our message) of the letters (our message space) is equal to . note that the base of the logarithm can be chosen, i chose 2 so we could use the SI unit bits. how does this arrangement compare to another, less specific, such as ? using the same math, we’ll get . here we can see that this quantity is intuitively coherent with the abstract idea of structure. to us, abc is more specific, more structured, than just ab. this is a key concept. note that in a miniverse like this one, there are no minds, so it is impossible for ab to represent any other concept and carry more information than itself.

now if we feed the concept of thing discussed previously as the letter of the above equation, we can apply it to (virtually) any structure in any layer of abstraction. but wait, didn’t i say layers were an illusion? yes. but i also discussed the need for compression. for example, to compute all the possible arrangements of all the atoms in a brick that is used to build a cathedral, one would have to calculate the information of the system as a whole. this would be the real quantity for the system. but since it is impossible to know all the possible states of all these atoms at a single moment in time, we will use things to solve the problem. here’s how.

consider that the information of brick a and brick b are, respectively, Ia and Ib. if they do not mix, i.e., their constituents aren’t switched at any point, we can assume that their information is independent. note that quantum physics tells us that this isn’t true, but for the sake of my margin of error, i won’t add the probability of an electron of a brick showing up in another brick. since they are independent, there is no mutual information, and therefore the total information of the system It is Ia + Ib. the total information of the two bricks is It, but not together. why? because bricks are being seen from another system, the cathedral which uses bricks as its constituents. therefore, bricks are the letters of a new message space. so this information, It, is the information that each brick has on its own, but not the entire system. let’s try to calculate the total information of a cathedral then. let’s conceive a very simple cathedral, with only 3 bricks and enough space for each brick in any orientation. if we now calculate all possible positions of the 3 bricks versus the single set of positions for the cathedral, we will obtain a new quantity, the information for the cathedral, which is, again, . though it is possible to do the math, it already seems a bit more complex. the probability of a brick occupying a certain volume is (where b is brick and s is space). but the brick can be in any position, so we need to count the probability of a position versus all possible positions. let’s consider rotations around its own axis. we get a total rotation for , so a single orientation in all of these is where  is the smallest section of motion (let’s say it’s as small as planck’s constant). the likelihood of a position and orientation is, therefore, . this is for only one brick. for all three, it is now . the information for our tiny cathedral is therefore (the numerator is always bigger than the denominator). also, obviously, we consider the bricks don’t move around and that the whole thing isn’t zero (that would make it explode to infinity).

now for the prestige. if we accept abstraction as a part of our model of reality, the total information of a Cathedral made of N bricks is , where is the information of a brick and is the information of a whole cathedral, both greater than 0. the whole is bigger than the sum of its parts. we can also simplify it, if we assume all bricks have the same information, then .

but let’s be critical of this. the whole is only bigger than the sum of its parts if and only if the constituents of a system are seen from another system, i.e., if concepts and abstraction exist (or we use recursive things in my definition). if we consider nature, it has no concept of a cathedral, therefore, it is impossible to define what a cathedral is. for a mindless universe, or a mindless system, the whole is equal to the sum of its parts because there is only one set of symbols (the message space is all letters in the universe) and only arrangements of these symbols (the particular message is a local arrangement of these letters).

i know that this is a bit confusing, but this is the proof of how, depending on your axiomatic structure, you can end up with emergence or reductionism. as you can see, this is a simple proof, whose only “leap” is considering the bricks as constituents of another system. this, as we saw when we analyzed the concept ouroboros, is a consequence of our own way of dealing with the world, that requires us to use compression to fit information in our tiny minds.

so we could extract a quantity from the equations above, and if we use things, we can even quantify bigger, macroscopic structures and compare them to each other. as we saw above, a brick is less structured than a cathedral (has less information) for example. i will be building upon this from now on, and though i favor reductionism, both are compatible as it has been demonstrated.

this is a first in a series about information, entropy and us. here’s today’s tune, on modern gaita transmontana by the band of some of the makers of my pipes, Roncos do Diabo. i’ll be posting with music and thoughts together, so you can have background music while you read.

of all the natural laws we learn in school (discovered through empirical evidence), there is usually a group of them that is regarded as ugly, inelegant or simply uninteresting.

thermodynamics, namely the second law, states in a crude way that entropy (or “disorder”) always increases in a system. there are a few exceptions to this but they are not relevant to our macroscopic everyday life (though they might be for our very existence, quantum fluctuations that is).

now, what is observed is that the more states are available to something, the bigger its entropy, and it’s a fact of nature that entropy increases. this is why you can’t open your fridge to cool down your house (in fact, you would end up heating it up), why you can’t “unbreak” an egg, and even more so, why time moves forward and not backward. consider the possible futures versus the single causal past. the future has much more entropy than the past (if i can put it this simply, what i just said is incredibly profound).

lurking in this is the notion first of thing and then of state. by thing usually we refer to atoms of a non reactive gas. but the thing can be expanded from that to any other “things” (after all, things are made of things, right?). take your bedroom for example. there are relatively few states for “clean”, but infinitely more states for “dirty”. that’s why your room ends up being dirty most of the time, and to keep it “clean” you need to work for it (more on this on a later post).

a state is no more than an “arrangement” of sorts. your socks in the drawer, your books in the shelf in alphabetical order. note that any state could be the clean state. the problem is that it would remain a single state, against the many possible places you could leave your socks and your books.

so entropy itself requires “minds” to assert whether a state exists or not, and whether it happened or not. nature itself has no mind to say which one is the ordered and which one is the disordered. we can use our minds to develop metrics to assert order and observe it exists (in surplus on earth for example). what is fed to us by nature is that order is both highly improbable and a requirement for complexity.

the study of entropy was mostly driven by industrial necessities and the development of engines. later, information theory emerged on an entirely different field of work: telecommunications.

information can be physically quantified (in bits), and bits themselves are things. so they are subject to entropy and order. take a book. any book. it is an arrangement of n things (letters) repeated m times (words, phrases, etc) where every thing must be in the right place, plus or minus a typo. if your letters are just 1 and 0, like everything online, you’re dealing with physical information (yes! information is physical and can be quantified!).

so communication itself must be subject to entropy. in fact, the two laws in their statistical definition are very similar. the reasoning i used here is not the usual one, it is a very simplified one. information theory and thermodynamics are very different with very different applications. what i’m writing about is the underlying fact, not the applicability of the specific laws.

i believe information, or ordered things, is strong enough of a concept to explain most of what is around us, including in complex systems like human beings, and the laws of entropy and energy are good enough to work with.

i know i’m risking generalizing too much but don’t see this as a simplification. i’m not saying human beings are information. that would be like saying a book is just a collection of letters from a to z. understanding underlying rules (like alphabets, grammar, etc) does not remove any value at all from the information itself (the books), and their emergent qualities (not all sequences of an alphabet hold ideas, only certain arrangements).

what i’m saying is we should accept the idea that things come to an end, permanently, by definition, and what we humans do, trying to reverse entropy all the time, building, surviving, fighting to be heard and remembered, is both useless and impossible.

it might be useless to paint a pretty portrait, but it doesn’t mean it can’t be beautiful. the japanese have a term for this kind of beauty, Mono no aware, an appreciation for the ephemeral beauty of things, and its bitter taste.

in a way, thanks to the 2nd law, we’re left with a universe permanently Mono no aware to us, leaving us in a permanent bitter awe, cooling asymptotically to absolute zero.

closing up with another tune, from Mono no Aware (an electronic music artist).