.@NIMBioS call for applicants for tutorial on Game Theory Modeling of Evolution in Populations, full funding! https://twitter.com/NIMBioS/status/666287597782745089

Some additional thoughts on information theory and complexity for models in ecology.

In addition to some of my commentary on <a href="http://biology.stackexchange.com/a/40555/6629">StackExchange</a>, I'll include a few additional thoughts which may be of help/assistance.

First for those who don't have the background, I <i>highly</i> recommend reading the two seminal papers on information theory and statistical mechanics by ET Jaynes and the standard text on information theory <i><a href="http://amzn.to/1NThEjd">Elements of Information Theory</a></i> by Thomas M. Cover and Joy A. Thomas.

In addition to the information theoretic related areas, you might want to take a look at the discipline of complexity theory, which has primarily grown out of the Santa Fe Institute over the past several decades and which includes information theory as part of its disciplines. If you're unfamiliar with the broader topic, Melanie Mitchell has an excellent overview with her book <i><a href="http://amzn.to/1YbcUZ4">Complexity: A Guided Tour</a></i>. Also related to complexity is the area of cellular automata which one could view as a very base model of more complex ecological systems. Here, perhaps Stephen Wolfram's <i><a href="http://amzn.to/1kwIAtM">A New Kind of Science</a></i> or <i><a href="http://amzn.to/1Ybd3Me">Cellular Automata and Complexity</a></i> will be enlightening. The broader theories coming out of these primarily mathematical areas may be useful to you.

In particular, given the types of models in ecosystems, I might suggest taking a look at some of the mathematical modeling going on at the intersection of complexity and economics. For a relatively simple introduction to this area, one could look at the relatively introductory text <i><a href="http://amzn.to/1kwJ6Ic">Complexity and the Economy</a></i> by W. Brian Arthur which is very interesting. The economy is essentially a very specific type of ecology dealing with human beings, assets, and the monetary system.

Another area which I've seen a lot of literature over the last few years is applicable to the ideas of resiliency and complexity in cities, for assisting in designing more robust city planning. This really isn't that far from the naturally evolving systems being looked at in ecology settings.

For those looking for researchers in the area of complexity, I have a <a href="https://twitter.com/ChrisAldrich/lists/complexity/members">list of many who are on twitter</a> in a variety of sub-areas. In addition to individuals, it also includes a number of institutes and related organizations as well.

I'd also suggest, that for the broadest theoretical setting, one could actually start with the topic known as "Big History" which takes the broadest approach of looking at history and the evolution of the cosmos over 13.7 billion years since the big bang. This conceptualization includes ideas like evolution, complexity, and emergence on the biggest scales, a set of theories that could be similarly applied to ecologies both large and small. For this viewpoint, I would suggest two works by David Christian including <i><a href="http://amzn.to/1kwJPJy">Maps of Time: An Introduction to Big History</a></i> and <i><a href="http://amzn.to/1YbdExx">Big History: The Big Bang, Life on Earth, and the Rise of Humanity</a></i>.

In essence, with many of these topics and viewpoints, one is treating individual animals or even entire species as elementary particles and then using the mathematical models of statistical thermodynamics to tease out specific types of data or trends. As layers of overlapping "particles" interact with each other, they cause emergent types properties, and then these resultant emergent properties combine to create further layers of emergent properties, none of which might have been necessarily deduced from the initial conditions. Within Big History, these types of emergence go from the big bang and basic particles in the early universe to the ultimate evolution of humankind by way of a variety of stages.

My reply to @PeterCoffee "Information should be more ‘force’ than ‘mass’" | Diginomica

You're certainly asking the right types of questions here, but doing so in a somewhat flimsy framework in what is already a fairly solid mathematical theory. Mathematicians would call this a "hand-waving argument." Some of your conceptualization is clouded by mistaking the semantics of the commonly accepted definition of information with Claude Shannon's formal mathematical definition, something which he admonishes the reader about in the opening paragraphs of his seminal paper.

I would suggest that you take a look at Cesar Hidalgo's recent and very accessible book Why Information Grows: The Evolution of Order, from Atoms to Economies (MIT Press, 2015) [http://amzn.to/1MrA7Pf] which provides a more rigorous grounding of what information is and what it means, particularly within the context of economics and global business. In it, he re-frames your questions and provides a firmer theoretical platform for seeing farther and moving faster. The framework he provides will provide a more solid grounding for what is happening within the growing digital economy and proliferation of areas like big data.

For those who want to take the economics piece a step further, one can then delve into the broader topics at the intersection of fields like complexity theory and economics similar to those framed by the Santa Fe Institute over the past two decades. (W. Brian Arthur's Complexity and the Economy, (Oxford, 2014) [http://amzn.to/1MrB5ex] is a fairly good starting point without getting too deep too quickly for most).

From a business perspective, these theories underpin many of the ideas expounded by Judith Rodin's The Resilience Dividend: Being Strong in a World Where Things Go Wrong (Public Affairs, 2014) or many of the probabilistic arguments made in Nassim Nicholas Taleb's various works.

A general foothold into the overlap of maximum entropy methods and biology:

John Harte's work on applying the mathematical theory of maximum entropy to ecology is certainly one of the better known examples of the application of this area of mathematics to science, in part because he literally wrote the textbook: [Maximum Entropy and Ecology: A Theory of Abundance, Distribution, and Energetics (Oxford Series in Ecology and Evolution)][1]

To be clear, maximum entropy (also known as MaxEnt in some of the literature, though most/all researchers use the longer form in publications) is a mathematical tool stemming from the fields of probability theory, statistics, and information theory. It's use is classically most often seen in thermodynamics, statistical thermodynamics, physics, and information theory, primarily because these were the areas in which E.T. Jaynes was working when he posited the idea. [Wikipedia has links to his two seminal papers.][2]But because of it's mathematical form, it can be applied in a multitude of areas, typically where one can utilize probabilistic methods.

If you're looking for additional areas of application, simply google the phrase "applied maximum entropy" and you'll find a [wealth of areas][3] including: econometrics, natural language processing, nuclear medicine, queuing systems, mass spectrometry, image processing, machine learning, and many others.

For ecology related work, a cross search on maximum entropy and "genetics", "evolution", "species", and similar words will provide a wealth of papers like ["A maximum entropy approach to species distribution modeling"][4].

Given the generic nature of your question, I might suggest that you'll find E.T. Jaynes' paper ["On the Rationale of Maximum-Entropy Methods" (IEEE, 1982)][5] useful.

Those generally interested in the broader applications of information theoretic methods to biology will likely appreciate some of the work that came out of last year's [NIMBioS Workshop on Information and Entropy in Biological Systems][6] (which Harte both attended and presented at), the [BIRS Workshop Biological and Bio-Inspired Information Theory][7], and the 2014 [CECAM Entropy in Biomolecular Systems][8]. The NIMBios Workshop was organized by John Baez, a physicist, who has worked with MaxEnt methods and explored them on his blog "[Azimuth][9]".

Those with a more sophisticated mathematical background (including measure theory, functional analysis, etc.) may appreciate Henryk Gzyl's text [The Method of Maximum Entropy (World Scientific: Series on Advances in Mathematics for Applied Sciences, Vol 29, 1995)][10].


[1]: http://amzn.to/1SnNB2c
[2]: https://en.wikipedia.org/wiki/Principle_of_maximum_entropy
[3]: https://scholar.google.com/scholar?q=applied%20maximum%20entropy
[4]: http://dl.acm.org/citation.cfm?id=1015412
[5]: ftp://129.240.33.108/pub/outgoing/IMN/Prediction%20modelling%20artikler%20fra%20Anders%20K%20W/Jaynes%201982,%20On%20the%20rational%20of%20Maximum-Entropy%20models.pdf
[6]: http://boffosocko.com/2015/05/20/videos-from-nimbios-workshop-on-information-and-entropy-in-biological-systems/
[7]: http://www.birs.ca/events/2014/5-day-workshops/14w5170
[8]: http://www.cecam.org/workshop-1014.html
[9]: https://johncarlosbaez.wordpress.com/?s=maximum%20entropy
[10]: http://amzn.to/1MFrrIC

Paul, thanks for the provocative piece, though the state of the art is certain much further along that your piece intimates. For the general reader, I would suggest reading MIT professor Cesar Hidalgo's recent book Why Information Grows (MIT Press, 2015) for some general structure and philosophy.

One of the best definitions and frameworks I've seen thus far has to be that of Christoph Adami. To start, and depending on your level of sophistication, take a look at his recent arXiv paper (Information-theoretic considerations concerning the origin of life - http://arxiv.org/abs/1409.0590 ) and then take a crack at this popular press article about it in Medium https://medium.com/the-physics-arxiv-blog/information-theory-and-the-origin-of-life-4cf6b93d156c). If it's something that blows your skirt up, then you can certainly begin to delve more deeply into some of his journal articles over the past decade or so.

For further references, I maintain a nice list of resources at Information Theory and Biology Resources [http://boffosocko.com/itbio/], as well as a "journal club" of sorts at Mendeley: ITBio: Information Theory, Microbiology, Evolution, and Complexity [https://www.mendeley.com/groups/2545131/itbio-information-theory-microbiology-evolution-and-complexity/].

For those who like to watch video material, I'll refer them to some videos from the NIMBioS Workshop on Information and Entropy in Biological Systems [http://boffosocko.com/2015/05/20/videos-from-nimbios-workshop-on-information-and-entropy-in-biological-systems/] organized by physicist John Carlos Baez. The Banff International Research Station also hosted a relatively recent week long workshop on Biological and Bio-Inspired Information Theory [http://www.birs.ca/events/2014/5-day-workshops/14w5170] which covered some interesting related ground with videos of many of the talks there as well.

Christian is without a doubt a historian through and through, and is quite upfront about his general lack of scientific expertise and background. He has however spent quite a bit of time working with and consulting physicists, chemists, biologists, and other scientists to supplement the appropriate portions of his bigger thesis. I would say though, that he's got firm footing in both of C.P. Snow's "Two Cultures."

Christian references Prigogine only once, though includes two Prigogine related footnotes in the last quarter of the text. He's not as Prigogine-centric as [author:César Hidalgo|13831217] is in his recent [book:Why Information Grows: The Evolution of Order, from Atoms to Economies|25472587], which touches on some of the related physics of information theory and entropy (and general complexity theory - although I don't recall him using this specific term) as they relate to economics. I'd classify Why Information Grows as a "big history" book, though Hidalgo wasn't aware of the conceptualization of "big history" when he wrote it.

I wrote a slightly longer review of Christian's book(s) on my blog: http://boffosocko.com/2012/06/17/big-history/. (Perhaps I'll have to move more detail over into my GoodReads review.)

#bighistory

Over the past several years, there's been a growing movement of "citizen science" and a handful of related games which send data back to scientists to assist in various areas of work, including primarily genetics. Googling for "games" and "citizen science" will bring back some interesting possibilities for you. Many should be integrateable into a big history program, particularly the genetics related ones which explore some of the evolutionary related space along with curricula in biology, chemistry, and physics. In particular, students may be able to experience first hand how physics influences evolution in the mid-level thresholds from the start of life onwards.

Here's a particular example that was recently in Scientific American: http://www.scientificamerican.com/citizen-science/play-to-cure-genes-in-space/

#BigHistory

I've just come across two Coase references in as many days... The first was by way of Cesar Hidalgo's new book "Why Information Grows: The Evolution of Order, from Atoms to Economies" which posits a relatively simple framework for economies based on our favorite topic of information theory. http://www.amazon.com/gp/product/0465048994/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0465048994&linkCode=as2&tag=itbio-20&linkId=AJJXNTLI3JJ6577T Though I wish it had some significant mathematics in the book to further validate the examples, on it's face his thesis has a lot of general merit and seems logically sound. I'll hope to have a short review of it up shortly.
Thanks for the further references and the podcast.