USB backup battery has weird spiky output

Bought this Anker USB battery to power and Arduino project.  Since the Arduino takes a USB connection, using rechargeable cell phone backup batteries with a USB plug is an easy way to get a high capacity portable power source.

I had an older Duracell battery that I used for this purpose but when I pulled it out after a long period of not using it, it was totally dead and wouldn't take a charge.

I found and bought this one, <$25 and 3200mAH.  Seemed pretty hefty.  It has a standard USB plug on top.

Anker® SlimTalk 3200mAh Backup External Battery Pack Power Bank Charger with Embedded Micro-USB and Flashlight for iPhone 5 4S 4 3GS, iPod (OEM CABLE REQUIRED for Apple); Android Smartphones: HTC Sensation, One series, EVO 4G / Samsung Galaxy S3, S2, Galaxy Note 2 / Motorola Razr/ LG Optimus 4X, PSP, and Many More Mobile Devices - Black [Ultra Slim 0.4 Inch Profile]

However when I plugged it into my project,  the LCD flickered and the project went nuts.
http://blog.workingsi.com/2012/12/i-was-inspired-by-slightly-misguided.html

I put it's output on the scope, and wow!  This thing must have a switching regulator or charge pump to create the 5V.   The scope shows 250mV spikes at 500MHz follwed by a linear ramp.  The output goes up to about 5.40V on the spikes and there are occasional negative spikes of about 200mV which are hard to see in this photo.  One is about 1/3 of the way from the left.    I was kind of surprised by this result from what I thought was just a DC battery.

I think I can fix the problem by just adding some cap to my board, I guess I was lazy in the first place and didn't automatically drop a uF electrolytic on it.  But if I had I wouldn't have learned this fun fact!  I added a 100uF cap to my prototype board on top the Arduino and it is able to function properly with this battery as the supply.    Sorry to waste everyone's time :).

The manufacturer emailed and called me when I posted a comment on Amazon about this.  To their credit they attempted to see if I had a problem and offered to replace the battery.  I don't think they understood what I was saying at all and thought the battery was defective.  Since I'm using the battery for an off book application, I hold nothing against them.   So to be totally clear, there is nothing wrong with this battery when used for it's advertised purpose.  If you are charging your phone you wont care at all about these spikes.   The battery still works great for charging my phone.