“Biggest threat to a system’s security is virtually always the people using the system.”. It is imperative to empower people with tools that secures and traces cloud transactions.
Curious case of our customer
One our customer recently had his Amazon AWS account hacked. I am reproducing the conversation I had with my customer. I am writing this post with my customer’s permission to help others falling prey to the hackers like one of our customers did.
Mark, client representative recounts, “Online security didn’t bother me too much until I signed up to Amazon AWS, then my problems started. I was amazed when hackers raided my credit card of $600 in charges in just eight hours. I was dumbfounded, how could such devastation occur in just eight hours? This was an account on which I hadn’t even spent any money yet.”
My first reaction was why someone would hack an Amazon AWS account? As customer mentioned, he was only experimenting with the account and definitely wouldn’t spend any money here. I was basically just using my one year of free membership.
The problem was though that a condition of sign-up to an Amazon AWS account is that you must provide your credit card details. This is why the hackers were interested in the account.
Amazon has some sensible measures to hide credit card information stored on the account, in case of breaches. But amazingly Amazon had no safeguards to prevent unauthorised purchases that hackers placed on my account. It turned out the hackers were more interested in the processing power of my credit card than anything else.
After hacking my account, the hackers created 50+ extra-large EC2 instances directly from my Amazon AWS account. These EC2 instances were then converted into cash accessible by them through Bitcoin mining.
Now let’s discuss “the how”. How in the first place did the hackers compromise the Amazon AWS account?
Usually Internet login to accounts is through the typical user entered username / password login method. I don’t want this to become another lesson in password security, let’s instead discuss Amazon AWS login through their system of access keys.
If you have never heard of this, I will explain in simple English. The Amazon AWS system’s access keys use 2 parts – i) an identity key and ii) a private key. Just think of this as fairly long, totally randomized user provided names with passwords which enable access via a program to the account.
These access keys provide full access to your entire account; the user doesn’t need to login through the usual browser screens either. The hackers had knowledge of my access keys – why you might ask? Well I’ll explain that next.
Mark says. “I had made a colossal mistake. I had previously been working on a project with some friends and was using PHP as the software language. As a part of this project I used my access keys to verify the functionality of Amazon S3 bucket. I then pushed this code complete with my access keys to GitHub”.
I didn’t think at the time, yes I know this was a horrendous error. Obviously I needed to remove my access key information. Unfortunately I needed to learn in a very painful way, always “sterilize” code before sharing with the public. My error had effectively published my access keys online and I was entirely open to hacking – so I was hacked!
It was a honest and simple mistake that costed Mark. Being part of his security experts, as a first step, we created a fake GitHub account. I added some bogus details, then I wrote some scraper code in order to find them. I can’t share the scraper code (it’s obvious why!) but I can say it took less than 1-hour for me to find their details. I didn’t think an hour was too bad, I asked some hackers I knew (with in our team included), it took them just ten minutes. So this just goes to prove – always, 100% of the time, sterilize code – always.
“Prevention is better than cure”. Please don’t think I am blaming Amazon here. I don’t want any one including Mark and his company to avoid Amazon AWS or Cloud adoption out of fear of getting hacked. My motivation for writing this piece to help customers to ensure your Amazon account is secure and not easily hackable. I love Amazon and all the other Cloud providers; I want to expose the seriousness of these threats to both users and the reputation of Amazon alike. Irrespective of the safeguards that Amazon and cloud providers may have in place, it’s ultimately up to you to practice self-help.
AWS IDENTITY AND ACCESS MANAGEMENT (IAM):
The first part of our security regime is the AWS Identity and Access Management service (IAM). Here we can set-up access keys but with certain conditions which restrict users privileges and regions. The main strategy here is to prevent hackers from gaining 100% entire control of your Amazon account.
In Marks’s case as an example the Amazon AWS access key (which was used to hack my account) , he only needed to upload / download files from an existing bucket within the eastern region of the US. If he had restricted the privileges to just this then all the hackers would have been able to do would be to download just a few images.
To read through Amazon’s suggested best practices click here.
ENABLE MULTI-FACTOR AUTHENTICATION:
The next tool in our armory is to enable Amazon’s multi-factor authentication. Ensure this is actioned for all IAM users who are given access to your account.
It’s highly recommended that you provide role-based privileges which are constrained wherever this can be sensibly achieved. The golden rule is not use root type credentials to enable access to your Amazon account.
Amazon AWS multi-factor authentication
Using Amazon CloudTrail won’t prevent you getting hacked, but it will help very much in cleaning your Amazon account up after experiencing a compromised account.
CloudTrail provides lots of useful log information, such as IP addresses used to access the account, the activities undertaken and the time of account access. When using CloudTrail remember that it might not be used in all the regions in your Amazon account. Therefore use IAM access keys which are always monitored by CloudTrail.
AMAZON BILLING ALERTS:
The next Amazon tool to use is Amazon’s billing alerts service; this is a very useful and also free tool. This tool enables you to set-up alerts by e-mail when your account spending limits are exceeded.
These billing alerts can be set-up to automatically notify you weekly. You will then know that if you are notified early that there is a likely issue with your account.
The tools which have been listed above are recommended to prevent attacks. Ultimately though the best method is to sterilize all code before it is placed online. This has to be the best technique to protect an account from hackers.
If your Amazon account has been hacked it is vital that you react quickly and totally close down access to your account. I’ve listed a series of steps below in the suggested order that you should undertake to thwart the hackers. It’s still important though to liaise with an Amazon AWS agent to verify that all account damage has been closed down.
1. Especially if the root credentials are comprimised, then the entire set of securty credentials are accessible the hacker. We at Xervmon are so careful, we do not share the access keys/passwords over internal and public collaboration system such as Skype, Google Hangout, Email etc.,
2. Always as in 100% of the time, remove your access keys from published code. Place it in a credentials file; make sure the file is in your .gitignore list. This is a great tip from one who’s learned, prevention is better than cure!
1.DELETE ALL OF YOUR AWS AND IAM ACCESS KEYS
This first step is vital to prevent further access of your account from the hackers. Next go to your Amazon account settings and delete every one of your Amazon AWS access keys. Don’t just disable, you must entirely delete them.
Next proceed to IAM, and then delete every one of the groups as well as any access keys which are there too.
2.STOP ALL EC2 SPOT BIDS
A favourite of Amazon bitcoin hackers is the EC2 spot bids tool; this is used to keep a steady flow at all times of EC2 instances. On occasions, hackers still continue to use this tool to create lots more instances even when account access has been lost. Always remember, Amazon splits its range of services into a variety of regions, you therefore need to stop every spot bid across all regions.
3.TERMINATE ALL ACTIVE EC2 INSTANCES
Next work through all of your Amazon regions, detail all of the instances that you definitely want to retain. Now terminate every one of the EC2 instances which are operational on your account. Ensure that you keep full details of at least one of the hacker instances which were created, you may want to do further forensic investigation work later on.
4.CHECK ALL OTHER SERVICES FOR CHANGES AND SHUT THEM DOWN
It’s really important that you work through all of the AWS service regions to check whether the hackers have tampered with any of these. My suggested method would be to view your billing details and analyse all charges that have been made during the past month.
5.CONTACT AMAZON AWS CUSTOMER SUPPORT
The final step is to visit Amazon AWS’ customer support page. Fill out the form and then ask to speak directly to an Amazon agent.
From my experience Amazon agents will call back within just a few hours of you completing the form. Although Amazon providing a phone number for an instant response would be nice, my experience was that there support staff are very knowledgable and helpful. All you need to do is explain the situation calmly and clearly and it’s their job to ensure all damage on your account is fixed.
Keeping calm is important, let the agent help you to work through your case as quickly as possible. “Count to ten”, deep inward breathing – whatever it takes, just ensure you give the agent all the information they need. I am pleased to say in my case Amazon agreed to waive all charges, so I was not “out of pocket”.
I honestly believe if you use the information in this blog you will be much better protected from hackers accessing your Amazon AWS or any othe cloud provider account. Of course “where there’s a will – there’s a way”, but these are practical, common sense suggestions. If you really want avoid security concerns, threats and vulnerabilities, you should avoid sharing the secure keys and passwords over social collaboration and communication platforms such as skype, google hangouts and public email system.
If you need any further help, or have some questions then please get in touch. Don’t risk getting hacked by not asking a few questions, hacking is far worse an experience!
Try us to be your managed services provider by Signing Up.
Contact Us to discuss your managed services, monitoring, management, deployments, optimizations or consulting needs. We will make sure your infrastructure comply with best practices based on our experience that reduces costs, optimizes usage and improves reliability of service.