Biomedical and Electrical Engineer with interests in information theory, evolution, genetics, abstract mathematics, microbiology, big history, IndieWeb, mnemonics, and the entertainment industry including: finance, distribution, representation
Chaos and randomness: An equivalence proof of a generalized version of the Shannon entropy and the Kolmogorov–Sinai entropy for Hamiltonian dynamical systems | Roman Frigg - Academia.edu
Chaos is often explained in terms of random behaviour; and having positive Kolmogorov–Sinai entropy (KSE) istaken to be indicative of randomness. Although seemly plausible, the association of positive KSE with random behav-iour needs justification since the definition of the KSE does not make reference to any notion that is connected to ran-domness. A common way of justifying this use of the KSE is to draw parallels between the KSE and Shannon's information theoretic entropy. However, as it stands this no more than a heuristic point, because no rigorous connec-tion between the KSE and Shannon's entropy has been established yet. This paper fills this gap by proving that the KSEof a Hamiltonian dynamical system is equivalent to a generalized version of Shannon's information theoretic entropyunder certain plausible assumptions.
In a Search for a Structure, Part 1: On Entropy. Category theory is the appropriate language for describing entropy.
Mathematics is about ”interesting structures”. What make a structure interesting is an abundance of interesting problems; we study a structure by solving these problems.
The worlds of science, as well as of mathematics itself, is abundant with gems (germs?) of simple beautiful ideas. When and how may such an idea direct you toward beautiful mathematics?
I try to present in this talk a few suggestive examples.
#categorytheory #entropy
"A few exciting words": information and entropy revisited | Lyn Robinson and David Bawden - Academia.edu
A review is presented of the relation between information and entropy, focusing on two main issues: the similarity of the formal definitions of physical entropy, according to statistical mechanics, and of information, according to information theory; and the possible subjectivity of entropy considered as missing information. The paper updates the 1983 analysis of Shaw and Davis. The difference in the interpretations of information given respectively by Shannon and by Wiener, significant for the information sciences, receives particular consideration. Analysis of a range of material, from literary theory to thermodynamics, is used to draw out the issues. Emphasis is placed on recourse to the original sources, and on direct quotation, to attempt to overcome some of the misunderstandings and oversimplifications that have occurred with these topics. While it is strongly related to entropy, information is neither identical with it, nor its opposite. Information is related to order and pattern, but also to disorder and randomness. The relations between information and the “interesting
complexity,” which embodies both patterns and randomness, are worthy of attention.
#informationtheory #entropy
The Homological Nature of Entropy
We propose that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. Three cases are presented: (1) classical probabilities and random variables; (2) quantum probabilities and observable operators; (3) dynamic probabilities and observation trees. This gives rise to a new kind of topology for information processes, that accounts for the main information functions: entropy, mutual-informations at all orders, and Kullback–Leibler divergence and generalizes them in several ways. The article is divided into two parts, that can be read independently. In the first part, the introduction, we provide an overview of the results, some open questions, future results and lines of research, and discuss briefly the application to complex data. In the second part we give the complete definitions and proofs of the theorems A, C and E in the introduction, which show why entropy is the first homological invariant of a structure of information in four contexts: static classical or quantum probability, dynamics of classical or quantum strategies of observation of a finite system.
Shannon information; #homology theory; #entropy; #quantuminformation; homotopy of links; mutual informations; Kullback–Leiber divergence; trees; monads; partitions; #informationtheory