Biomedical and Electrical Engineer with interests in information theory, evolution, genetics, abstract mathematics, microbiology, big history, IndieWeb, mnemonics, and the entertainment industry including: finance, distribution, representation
@Bizarro1303 I think the definition (both then and now) is that set forward by Claude Shannon in 1948 in "A Mathematical Theory of Communication". Shannon was an early colleague of N. Weiner who pioneered the version Kolmogorov references.
http:/
@Storify @StorifyHelp, how am I supposed to download stories when they don't seem to be available? This one has many pieces but only one is visible and I can't follow the archive instructions because the web interface isn't working. https:/
@natalieasis Probably not much you don't know already. There's LOTS of philosophy, 1/6 Greene minus string theory (it covers core theory/quantum fields--with no equations, naturally); 1/6 Bayesian theory, a little bit of entropy, no IT really, 1/6 complexity and some basic biology, and then even more philosophy. You'll probably enjoy Part 4, but most of the rest will be review of what you already know. My (extended) review will cover the few pieces you'll be interested in without eating up as much time. Overall, it's probably better as something readable for your parents to understand some of what you're studying.
DM me your email with a preference for .mobi or .epub format and it's in your lap.
There's apparently a growing movement in #complexity & #entropy overlapping the #indieweb: @michielbdejong @chrisaldrich
Chaos and randomness: An equivalence proof of a generalized version of the Shannon entropy and the Kolmogorov–Sinai entropy for Hamiltonian dynamical systems | Roman Frigg - Academia.edu
Chaos is often explained in terms of random behaviour; and having positive Kolmogorov–Sinai entropy (KSE) istaken to be indicative of randomness. Although seemly plausible, the association of positive KSE with random behav-iour needs justification since the definition of the KSE does not make reference to any notion that is connected to ran-domness. A common way of justifying this use of the KSE is to draw parallels between the KSE and Shannon's information theoretic entropy. However, as it stands this no more than a heuristic point, because no rigorous connec-tion between the KSE and Shannon's entropy has been established yet. This paper fills this gap by proving that the KSEof a Hamiltonian dynamical system is equivalent to a generalized version of Shannon's information theoretic entropyunder certain plausible assumptions.
Relative Entropy in Biological Systems by John Baez and Blake Pollard https:/
A general foothold into the overlap of maximum entropy methods and biology:
John Harte's work on applying the mathematical theory of maximum entropy to ecology is certainly one of the better known examples of the application of this area of mathematics to science, in part because he literally wrote the textbook: [Maximum Entropy and Ecology: A Theory of Abundance, Distribution, and Energetics (Oxford Series in Ecology and Evolution)][1]
To be clear, maximum entropy (also known as MaxEnt in some of the literature, though most/all researchers use the longer form in publications) is a mathematical tool stemming from the fields of probability theory, statistics, and information theory. It's use is classically most often seen in thermodynamics, statistical thermodynamics, physics, and information theory, primarily because these were the areas in which E.T. Jaynes was working when he posited the idea. [Wikipedia has links to his two seminal papers.][2]But because of it's mathematical form, it can be applied in a multitude of areas, typically where one can utilize probabilistic methods.
If you're looking for additional areas of application, simply google the phrase "applied maximum entropy" and you'll find a [wealth of areas][3] including: econometrics, natural language processing, nuclear medicine, queuing systems, mass spectrometry, image processing, machine learning, and many others.
For ecology related work, a cross search on maximum entropy and "genetics", "evolution", "species", and similar words will provide a wealth of papers like ["A maximum entropy approach to species distribution modeling"][4].
Given the generic nature of your question, I might suggest that you'll find E.T. Jaynes' paper ["On the Rationale of Maximum-Entropy Methods" (IEEE, 1982)][5] useful.
Those generally interested in the broader applications of information theoretic methods to biology will likely appreciate some of the work that came out of last year's [NIMBioS Workshop on Information and Entropy in Biological Systems][6] (which Harte both attended and presented at), the [BIRS Workshop Biological and Bio-Inspired Information Theory][7], and the 2014 [CECAM Entropy in Biomolecular Systems][8]. The NIMBios Workshop was organized by John Baez, a physicist, who has worked with MaxEnt methods and explored them on his blog "[Azimuth][9]".
Those with a more sophisticated mathematical background (including measure theory, functional analysis, etc.) may appreciate Henryk Gzyl's text [The Method of Maximum Entropy (World Scientific: Series on Advances in Mathematics for Applied Sciences, Vol 29, 1995)][10].
[1]: http:/
[2]: https:/
[3]: https:/
[4]: http:/
[5]: ftp:/
[6]: http:/
[7]: http:/
[8]: http:/
[9]: https:/
[10]: http:/
Paul, thanks for the provocative piece, though the state of the art is certain much further along that your piece intimates. For the general reader, I would suggest reading MIT professor Cesar Hidalgo's recent book Why Information Grows (MIT Press, 2015) for some general structure and philosophy.
One of the best definitions and frameworks I've seen thus far has to be that of Christoph Adami. To start, and depending on your level of sophistication, take a look at his recent arXiv paper (Information-theoretic considerations concerning the origin of life - http:/
For further references, I maintain a nice list of resources at Information Theory and Biology Resources [http:/
For those who like to watch video material, I'll refer them to some videos from the NIMBioS Workshop on Information and Entropy in Biological Systems [http:/
Christian is without a doubt a historian through and through, and is quite upfront about his general lack of scientific expertise and background. He has however spent quite a bit of time working with and consulting physicists, chemists, biologists, and other scientists to supplement the appropriate portions of his bigger thesis. I would say though, that he's got firm footing in both of C.P. Snow's "Two Cultures."
Christian references Prigogine only once, though includes two Prigogine related footnotes in the last quarter of the text. He's not as Prigogine-centric as [author:César Hidalgo|13831217] is in his recent [book:Why Information Grows: The Evolution of Order, from Atoms to Economies|25472587], which touches on some of the related physics of information theory and entropy (and general complexity theory - although I don't recall him using this specific term) as they relate to economics. I'd classify Why Information Grows as a "big history" book, though Hidalgo wasn't aware of the conceptualization of "big history" when he wrote it.
I wrote a slightly longer review of Christian's book(s) on my blog: http:/
#bighistory
Only 1 copy of Arieh Ben-Naim's new book Information, Entropy, Life & the Universe left on day 1 http:/
In a Search for a Structure, Part 1: On Entropy. Category theory is the appropriate language for describing entropy.
Mathematics is about ”interesting structures”. What make a structure interesting is an abundance of interesting problems; we study a structure by solving these problems.
The worlds of science, as well as of mathematics itself, is abundant with gems (germs?) of simple beautiful ideas. When and how may such an idea direct you toward beautiful mathematics?
I try to present in this talk a few suggestive examples.
#categorytheory #entropy