Here, I want to share my experiences with configuration management of cloud infrastructure using an infrastructure-as-a-code service called “Terraform”!
I was working in a start-up company with a small team and a huge load of tasks as a typical startup! Besides my other responsibilities, I was in charge of cloud configuration and management. We had several thousand dollars credits to use AWS that received in the early days. On the contrary to many of my peers, I am in favor of being vendor neutrality even though that vendor is Amazon. I have two reasons for that.
First, It would be costly if you lock-in yourself to a specific vendor by using native cloud services. If you lock in yourself to a vendor, you will not have any control on the monthly bill and you can not minimize it easily. The monthly bills can become crazy! For example, if you will be received credits to use other cloud services it would not be easy for you to migrate services to the new host. So, my recommendation is to think twice before using, for example, CloudFormation to configure your cloud.
Second, there is no company that can claim to have the industry standards for all the cloud services that you need. Big companies like Amazon or Google have their own solutions for each module; however, they are not necessarily the best-in-class especially for small companies who need to develop fast. For example, AWS has Amazon ECR for its Docker Registry solution, but which one you will use: Amazon ECR or DockerHub?
It is not finished yet 🙂
Several weeks after we set up our cloud services on AWS, we received an email from Amazon that one of the EC2 instances in our cluster will be shut down permanently due to a hardware issue. We noted that issue in a JIRA card, and we put that in our backlog but it slipped from our attention easily. On a Monday, we encountered with an error in our cloud services which we finally found that it raised due to the EC2 instance shutdown. Thank to Terraform, we were able to rebuild our EC2 clusters within a matter of minutes reliably and quickly.
One of the best practices in building cloud service is to have Development, Staging, and Production environments. According to the definition, the Staging and Production environments must be identical in every level including the cloud infrastructure or deployment configuration. Using an infrastructure-as-a-code service is the best way to ensure that two cloud infrastructures are identical (re: EC2 instances, orchestrator, security groups, and etc.).
At some point in our development process, we decided to improve the security of our service. For that reason, we needed to store DockerHub credentials in the corresponding EC2 instances. We, first, add them manually but we then manage to copy that file in the stage of creating using the Terraform. It would not be possible to do that manually each time that you make any change to your cloud infrastructure.
All of my experiences just makes me more excited about infrastructure-as-a-code services; especially Terraform. If you want to build a high-quality industry solution you don’t have any choice other than using this powerful service.