Biz & IT —

AWS console breach leads to demise of service with “proven” backup plan

Code Spaces closes shop after attackers destroy Amazon-hosted customer data.

AWS console breach leads to demise of service with “proven” backup plan

A code-hosting service that boasted having a full recovery plan has abruptly closed after someone gained unauthorized access to its Amazon Web Service account and deleted most of the customer data there.

Wednesday's demise of Code Spaces is a cautionary tale, not just for services in the business of storing sensitive data, but also for end users who entrust their most valuable assets to such services. Within the span of 12 hours, the service experienced the permanent destruction of most Apache Subversion repositories and Elastic Block Store volumes and all of the service's virtual machines. With no way to restore the data, Code Spaces officials said they were winding down the operation and helping customers migrate any remaining data to other services.

"Code Spaces will not be able to operate beyond this point," a note left on the front page of codespaces.com said. "The cost of resolving this issue to date and the expected cost of refunding customers who have been left without the service they paid for will put Code Spaces in a[n] irreversible position both financially and in terms of on going credibility. As such at this point in time we have no alternative but to cease trading and concentrate on supporting our affected customers in exporting any remaining data they have left with us."

A copy of the Code Spaces website still cached on Bing shows that the website promoted its ability to protect customer data from catastrophic events as a key benefit.

"Backing up data is one thing, but it is meaningless without a recovery plan, not only that [but also] a recovery plan—and one that is well-practiced and proven to work time and time again," the cache stated. "Code Spaces has a full recovery plan that has been proven to work and is, in fact, practiced."

Wednesday's advisory said the unauthorized access to the Amazon-hosted Code Spaces data came on the heels of a distributed denial-of-service attack, presumably in an attempt to extort money from the service. Code Spaces officials wrote:

Dear Customers,
On Tuesday the 17th of June 2014 we received a well orchestrated DDOS against our servers, this happens quite often and we normally overcome them in a way that is transparent to the Code Spaces community. On this occasion however the DDOS was just the start.

An unauthorised person who at this point is still unknown (All we can say is that we have no reason to think [it's] anyone who is or was employed with Code Spaces) had gained access to our Amazon EC2 control panel and had left a number of messages for us to contact them using a hotmail address

Reaching out to the address started a chain of events that revolved [around] the person trying to extort a large fee in order to resolve the DDOS.

Upon realisation that somebody had access to our control panel we started to investigate how access had been gained and what access that person had to the data in our systems, it became clear that so far no machine access had been achieved due to the intruder not having our Private Keys.

At this point we took action to take control back of our panel by changing passwords, however the intruder had prepared for this and had already created a number of backup logins to the panel and upon seeing us make the attempted recovery of the account he proceeded to randomly delete artifacts from the panel. We finally managed to get our panel access back but not before he had removed all EBS snapshots, S3 buckets, all AMI's, some EBS instances and several machine instances.

In summary, most of our data, backups, machine configurations and offsite backups were either partially or completely deleted.

This took place over a 12 hour period which I have condensed into this very brief explanation, which I will elaborate on more once we have managed our customers' needs.

The advisory didn't say exactly how the hacker gained entry to the Code Space panel hosted on AWS. People who host their services on Amazon should avail themselves of the full spectrum of multifactor protections.

Channel Ars Technica