I think one of the limitations of the traditional formulation of information theory is that the entropy of a signal doesn't obviously tell you how much useful information is in the signal.
This paper provides an interesting perspective on this issue. "The Information Bottleneck Method" Bialek, Tishby and Pereira
Complexity, Entropy and the Physics of gzip: http://www.cscs.umich.edu/~crshalizi/notebooks/cep-gzip.html
I think one of the limitations of the traditional formulation of information theory is that the entropy of a signal doesn't obviously tell you how much useful information is in the signal.
This paper provides an interesting perspective on this issue. "The Information Bottleneck Method" Bialek, Tishby and Pereira
http://www.princeton.edu/~wbialek/our_papers/tishby+al_99.pd...