Most of the posts highlight what I am focused on and express work and personal experience. Reason I put them here - recall later or help someone else with similar challenge.
Few week ago in one of my previous posts I mentioned lots of free AWS services either forever free or during first twelve months. If you missed that post, I do encourage you to check it out. This article is my first episode (since I've been in backup industry for the last 8 years). Let's talk about free tiers around storage and see how exactly we can use it.
If you have new account in AWS you can enjoy first 12 month the following free services:
The first two are great for keeping data, the last is actually CDN (content delivery network) id you plan to distribute your files globally you can use one of the AWS edge location for faster delivery to your clients (your data is being cached and delivered based on geo-locaiton of requester).
AWS Glacier offers (non-expiring) free 10 GB standard data retrievals (since Glacier class is super chip, but you pay huge when you restore, this is really great to have). There are few retrieval options, based on your specific case you may want to select one of available (if standard does not work for you and you need faster). Check this guide for further details.
I'd like to concentrate on S3 storage. Another important thing - AWS storage classes, that we can apply to our objects (by the way, officially it is explained here). It is great opportunity to cut costs and pay less for the data we keep in the cloud. Again, this is applicable to the case, where you need above 5 GB available free tier (well, someone would say "Eugene, what is 5 GB? It is nothing today.. "). Agree, 5 GB in AWS is just starter package, if you need to keep something small (not your media set or archives) free tier should work, if not - read further and get more details on storage classes and how you can save.
By default you get your objects placed in Standard S3 class and you start paying (again if you are not eligible for free tier) ~ $ 0.023 (actually this is different in each AWS region). I am using Ireland region, but you may want to check how much you are going to pay. From that we now can start saving using either Infrequent Access or even Glacier storage classes.
Look at the below table! You are going to save almost x2 times with IA and x6 times with Glacier! But, wait! You should think about your restore before going to this storage classes! Since additional costs applied to your retrieval (that's why I mentioned Glacier free tier)!
Standard Storage Standard - Infrequent Access Storage Glacier Storage First 50 TB / month $0.023 per GB $0.0125 per GB $0.004 per GB
Apart storage pricing we are paying for API calls. Since we work with object storage and in order to create, read, update or delete (CRUD operations) object in the storage we do HTTP call. Those calls are not all free (some of them like
DELETE method is free for S3 Standard), other methods are not and varies from $ 0.004 per 1,000 requests (for
GET and all other Requests) up to $0.055 per 1,000 requests (Lifecycle Transition Requests into Glacier). Lifecycle transition is the way to change storage class on the object. As you can see for Glacier it is the most expansive. Consider single call per object (per file). If you are large set of small files in your backup folder, grab calculator and make sure you it suits it (by the way, it is not the same like I mentioned above with retrieval free tier for Glacier, it is lifecycle transition, which is another type of calls, so it is not free!).
Great news! Amazon does not charge for data IN! So all your backup uploads go for free! However if you need to download (data OUT) take into account first 1 GB download per month goes for free, next 10 TB goes with $0.090 per GB and full price is here. Great news, Cloud Front (CDN) comes for free as well, but data transfer between regions $0.020 per GB!
I'd love to cover two other things here since they are important for AWS S3.
Transfer accelerator utilizes existing edge location infrastructure and improves the speed of upload (it makes sense when we are really far from the destination bucket). For Example we have bucket in US West, but we are geographically uploading from Ireland. So we want to do this over edge location, since it can make this upload few times faster (there is great resource to compare results with and without transfer acceleration feature). However the drawback is increased price by $0.04/GB.
You are going to consider snowball (network attached storage) optimized to work with AWS recourse when you need to upload a lot of data and you are locked in with slow provider. There are many ways to decide, AWS has some recommendations around this. But basically you have the following and you need to attach it to your computer and start upload (if you need to seed data for S3). Look at the picture below - two devices ready for data offload.
Now, when it is super clear with all storage services (at least majority) let's see backup. With AWS CLI you can do the following:
#!/bin/sh aws s3 cp /path/to/my/files/* s3://my-bucket-goes-here/
This is really primitive, however it helps you to upload files. With
--recursive flag you can automate some of you backup activities (include or exclude). Cron can help to automate the process.
If you are looking at something better and want to get out of the box schedule, filter (for example file extension), purger, incremental and block level features, you probably want to look at CloudBerry Backup. It is cross-platform backup software available for Windows, Linux, Mac (there is also extension for VMs, - VMware and Hyper-V agent-less solution). If you are service provider, start with free sign up with Managed Backup Service! Completely free version of CloudBerry Backup is available for Desktop (Windows, 7, 8, 10 etc), check the comparison, may be it worth paying few extra dollars for image based or Server OS! Good luck and bless your-self (and your data)!
That's it for the episode, if you liked it, please hit "heart" down left and leave your comments below! Twits are appreciated!
Check this video, you should get better understanding around all AWS storage services.