"It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience."
“I also remember a remark of Albert Einstein, which certainly applies to music. He said, in effect, that everything should be as simple as it can be, but not simpler.”
--Roger Sessions in the New York Times (8 January 1950)
"Things should be made as simple as possible, but no simpler."
--misquote misattributed to Albert Einstein, and rather ironic considering the sentiment expressed
All quotes taken from Wikiquote.
A Short Definition of Entropy
"Entropy is a measure of how evenly energy is distributed in a system. In a physical system, entropy provides a measure of the amount of energy that cannot be used to do work." This quote from the Introduction to entropy page of Wikipedia is a short definition that belies the depths that the definition actually implies.
I'd like to argue that a different definition of entropy might be provided: the amount of work to be done, or energy to be expended, to achieve a given aim through a given methodology; entropy then depends on how high we aim in any given context, and what methodologies we have available to us to give effective and efficient shape to our efforts.
Entropy provides a measure of the amount of energy available that cannot be used to complete the task to which it might otherwise be committed, or energies that are committed to tasks not of the best choosing.