Physics, Topology, Logic and Computation: A Rosetta Stone #categorytheory
arXiv:0903.0340 [quant-ph]
In physics, Feynman diagrams are used to reason about quantum processes. In the 1980s, it became clear that underlying these diagrams is a powerful analogy between quantum physics and topology: namely, a linear operator behaves very much like a "cobordism". Similar diagrams can be used to reason about logic, where they represent proofs, and computation, where they represent programs. With the rise of interest in quantum cryptography and quantum computation, it became clear that there is extensive network of analogies between physics, topology, logic and computation. In this expository paper, we make some of these analogies precise using the concept of "closed symmetric monoidal category". We assume no prior knowledge of category theory, proof theory or computer science.
#categorytheory, #physics, #topology
"A few exciting words": information and entropy revisited | Lyn Robinson and David Bawden - Academia.edu
A review is presented of the relation between information and entropy, focusing on two main issues: the similarity of the formal definitions of physical entropy, according to statistical mechanics, and of information, according to information theory; and the possible subjectivity of entropy considered as missing information. The paper updates the 1983 analysis of Shaw and Davis. The difference in the interpretations of information given respectively by Shannon and by Wiener, significant for the information sciences, receives particular consideration. Analysis of a range of material, from literary theory to thermodynamics, is used to draw out the issues. Emphasis is placed on recourse to the original sources, and on direct quotation, to attempt to overcome some of the misunderstandings and oversimplifications that have occurred with these topics. While it is strongly related to entropy, information is neither identical with it, nor its opposite. Information is related to order and pattern, but also to disorder and randomness. The relations between information and the “interesting
complexity,” which embodies both patterns and randomness, are worthy of attention.
#informationtheory #entropy
First, let's unpack your question a bit. What we're really doing is taking mathematical form, function, and theory and apply it in some sort of unifying manner to unify the areas of information theory and statistical thermodynamics.
You can also ask your question in the reverse direction to apply the mathematics of information to thermodynamics as well as there should be a mathematical duality of sorts, if in fact the two are directly related.
First, let's unpack your question a bit. What we're really doing is taking mathematical form, function, and theory and apply it in some sort of unifying manner to unify the areas of information theory and statistical mechanics and thermodynamics.
You can also ask your question in the reverse direction to apply the mathematics of information to thermodynamics as well as there should be a mathematical duality of sorts, if in fact the two are directly related.
The original link between the two areas stems from a paper by E.T. Jaynes entitled "Information Theory and Statistical Mechanics" (downloadable from http:/
Since then, in my opinion, the best source tying the fields together mathematically is A Farewell To Entropy: Statistical Thermodynamics Based on Information by Arieh Ben-Naim [http:/
As for other connections, I've not seen anything as direct, primarily as there aren't any other major concepts with the exact same name. However, because of the underlying mathematics, I strongly suspect that there are many similar 'concepts'. A few in particular, I have a feeling will come out of the more rigorous mathematical frameworks based on Lieb and Yngvason’s concptualization of thermodynamics (see Resource Convertibility (Part 1)[https:/
Another fruitful area of conceptual similarities is slowly grown out of quantum information theory and the statistical mechanics of black holes, though there is some heavily rarified air surrounding these given the required background needed to operate with them both simultaneously.
@mrgunn @MendeleyTips Is there an easy way to use Mendeley as a linkblog? eg. when adding a paper via bookmarklet, the title and link to paper are sent as a tweet?
I know that I could create a group, and add papers to it via the bookmarklet, but then I'd have to put the details into the group status as a second step (or vice-versa). Is there a way to make it a one step process so that I can bookmark it both to a group and simultaneously send out a tweet with the title/URL? I'd imagine that I could do this via API, but would think that you guys may have already built something like this.
Response to Quora Question: Are there any popular science books on information theory?
There are a small handful that may meet your requirements:
Information, Entropy, Life and the Universe: What We Know and What We Do Not Know by Arieh Ben-Naim
http:/
An Introduction to Information Theory: Symbols, Signals and Noise by John R. Pierce
http:/
The Information: A History, A Theory, A Flood by James Gleick
http:/
Decoding the Universe: How the New Science of Information Is Explaining Everything in the Cosmos, from Our Brains to Black Holes by Charles Seife
http:/
Complexity: A Guided Tour by Melanie Mitchell
http:/
Programming the Universe: A Quantum Computer Scientist Takes on the Cosmos by Seth Lloyd
http:/
The Mathematical Theory of Communication by Claude E. Shannon and Warren Weaver
http:/
The Ben-Naim book is coming out at the end of May, and is probably the best popular science book on the topic that you're likely to find (I've read most of an early draft). Following that it's still very hard to beat Pierce's presentation all these years later. Seife wasn't too bad while Gleick comes at things from the most broad viewpoint and covers more history than anything else. Lloyd's book mentions it tangentially. Mitchell's book is excellent, and though it only has one chapter on information theory, some of the other areas it covers are tangential to the subject and very interesting in their own right.
Finally, after Ben-Naim and Robinson, I'd really recommend the UofI Press edition of Shannon/Weaver. Though you can get a copy of Shannon's paper for free online, the introduction by Warren Weaver is really excellent (and similar to Robinson's presentation) and then one can delve into a bit of Shannon's actual math, which isn't too difficult and should be able to be followed by most reasonable high school students. Shannon's text is the only one I've listed that has any mathematical equations, but the Weaver half of the book is certainly a popular science presentation without any math.
Your opening reminded me a bit of the motivating chapter of Andre Thess's book <a href="http://www.amazon.com/gp/product/3642133487/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=3642133487&linkCode=as2&tag=itbio-20&linkId=PVX5BVQGWFXLAGZZ">The Entropy Principle (Springer, 2011)</a> in which he uses the fable of Hans who manages to waste away a block of gold until he has nothing.
The book was an interesting textbook-like conceptualization of Lieb and Yngvason's work, which I always thought was due for a more thorough mathematical framework. I'm glad to see you not only reference them, but you appear to do what I've always wanted. Can't wait to delve further into your actual paper.
Interestingly, Erwin Schrödinger's seminal paper "What is Life?" came out in 1944, almost contemporaneously with the creation of information theory in 1948. Unbeknownst to most Claude Shannon's (the researcher who created the field of information theory and whose Masters thesis literally launched the digital revolution) Ph.D. thesis (1940) was entitled: "An Algebra for Theoretical Genetics," so he could certainly be said to be the god-father of the entire area.
Since then, certainly dozens of researchers have looked at the potential of using information theory to create a better hard core definition of what life really is (certainly the place where any self-respecting Platonist would begin.)
One of the best definitions and frameworks I've seen thus far has to be that of Christoph Adami. To start, and depending on your level of sophistication, take a look at his recent arXiv paper (Information-theoretic considerations concerning the origin of life - http:/
For further references, I maintain a nice list of resources at Information Theory and Biology Resources (http:/
If you really want to blow the top off of your definitions, you might also consider taking a much broader look at the universe (a distant reading, so-to-speak) and read through David Christian's conceptualization (Maps of Time: An Introduction to Big History: David Christian, William H. McNeill http:/
This sounds a tad like the debate in early Greek philosophy between the relativists (pre-Socratics) and the Platonists.
I'm curious what your calculation would look like to prove that they either contain the same information or that Mount Fuji contains more?
One of the subtleties of Shannon's original paper on information theory is in the second paragraph where he explicitly states that he's leaving out the semantic processing of the message by the receiver and just concentrating on the sending and receipt of the signal. (Put another way we can play the game telephone and you can hear and repeat the exact words I say, but concepts like <i>double entendre</i> and nuance may prevent you from understanding exactly what I meant to say. Shannon is leaving the second problem understanding out of the picture and solely working on the question of did you hear the words I actually said.) The way you're framing your question, it would appear that you're only focusing on the interpretation of the information after it's been received, in which case Shannon's information doesn't really have anything to say.
Now, to take a look at it from a purely mathematical standpoint within a Shannon framework, once could count the totality of the number of atoms and their states in Fuji and compare it to that of Rushmore and the one with more information is simply going to be the larger one.
If we're looking at photos of the two, then the one with more information is going to be the one that doesn't compress down as much under whatever compression algorithm we might choose. (ie, it won't have anything to do with the "information" contained in the faces and whose faces they are, which again is a semantic issue and not a mathematical one.)