Let's say you receive a "message" (this is a highly abstract example) - the letter A. Then you receive another "A" from the same source. Then another, and another, and so on. It's easy to describe the complete message as "continuous A's". Each new "A" does not add any additional information - it's still "continuous A's". If you got a "B" after, say, three hundred and twelve A's, and then got A's again, you would have to say the message consists of "312 A's, 1 B, and then continuous A's". In other words, it takes more information to describe the message, because the message itself contains more information (in the Shannon information theory sense).
I just finished reading George Orwell's 1984 (finally!), and I realized that one of the goals of Big Brother (the totalitarian government in the book) is to maintain a zero information level. In the book, Big Brother has complete control over every human being - there is even "Thought Police" and "crimethink". Big Brother's Ministry of Truth is in charge of maintaining this zero-information level by literally rewriting the past to suit Big Brother's needs in the present. For example, if Big Brother makes a prediction that turns out to be false, all references to that incorrect prediction are collected and altered. Nothing that contradicts Big Brother is allowed to exist, ever. Citizens who fail to alter their own internal records (their memory) to comply with Big Brother's are "vanished".
In terms of the previous example, Big Brother, upon receiving a "B" message, either destroys it or changes all the previous A's into B's, so that the message does not add any information. That way, no new message could increase information within the society. Orwell understood intuitively that changing old information can be equivalent to destroying new information.
And now I'm off to watch some prolefeed.
p.s. Fine - I'll make a political point: when the administration lies about lying about WMD's in Iraq, they are attempting, in a very Orwellian manner, to destroy the potency of new information (no WMD's in Iraq after all) by rewriting the past ("we never said that/we've always said that"). Much of 1984 hits far too close to home these days.