Bugs are inevitable, but suppose a user notices a bug that causes strange behavior in the system. Suppose that the user then works out how that behavior could be used to advantage. Is this hacking and is it punishable by law, the Computer Fraud and Abuse Act in particular?

This is not a theoretical argument. A compulsive machine poker player, John Kane, played so many hours of poker that he noticed by accident an obscure bug. If he played a machine until he won then he had the option of doubling up by pressing a button and inserting some money. At this point, if the player hits the "amount" button again it becomes possible to alter the jackpot payout figure. So playing a $1 machine to a win, pressing the double up, entering some more money and the player can modify the payout amount to say $10 and then cash out immediately - so winning ten times that original payout. 

So there is a bug and you can exploit it - but is this hacking? Is it covered by the Computer Fraud and Abuse Act (CFAA)?

The CFAA contains a clause that makes exceeding authorized access to a machine a crime. It is this clause that Aaron Swartz was being prosecuted under, for an entirely different set of circumstances, and led to the campaign for Aaron's Law which would remove it. 

The poker bug was exploited to the tune of hundreds of thousands of dollars before casino staff noticed. The local district attorney prosecuted on lots of different charges but wire fraud and computer hacking were the important ones. The accused most certainly felt that no wrong doing had occurred and compared exploiting the bug to card counting - which isn't illegal even if it is banned by casinos. The relevant charge is that the machine wasn't hacked from the outside but the player exceeded there legitimate access to obtain alter information in the computer that the accessors is not entitled so to obtain or alter. The argument for the defense is that the players worked within the rules imposed by the machine - bug and all. 

At the end of last year a federal magistrate found that the CFAA doesn't apply and recommended that the hacking charge be dropped. The case is now in front of the U.S. District Court waiting for a final verdict. An earlier case has been cited as a precedent that the CFAA doesn't apply in a situation of computer misuse and the judge has asked both sides to reconsider. 

You can get a much fuller version of the details as reported by Wired but the basic issues are fairly clear. The CFAA has been applied in cases of computer misuse rather then clear manipulative hacking. If a bug is present then it most probably will be exploited by users who discover it if there is any reward.

More complex is whether or not exploiting a bug is a crime?

If an ATM malfunctions and issues a thousand times the amount of money you requested then keeping that money is, in most countries a crime. If the ATM has a bug and you know the key combination to make the machine pay out a thousand time the amount of money then this too must be a crime. However, the gambling machine isn't an ATM - it is a machine that plays a game and if a bug modifies the rules of that game then can the player claim that exploiting it is a fair strategy in playing the game. Are bugs sometimes features?