The Implementation of Auto Scale Deployment in Ruby on Rails
When a website is up and running, different resources are required depending on the number of users, functions provided, and the backbone architecture design. High traffic websites usually require many more resources on the backend, which may mean more server machines or a larger database.
How to choose the appropriate resources for a website has turned into a science. Allocating too many resources would incur additional costs, while insufficient resources that are unable to support the traffic volume on the website could lead to website service interruptions and stoppage altogether.
However, depending on the circumstances, different resources may be required even for the same website. For example, supplying the same amount of resources used to handle dense traffic in midmorning for sparse traffic late at night leads to waste. This is precisely the problem that the Auto Scale framework aims to resolve. It allows systems to automatically adjust required resources based on the usage level. For example, when the traffic volume increases, new server machines are added to share the traffic flow. This strategy helps to achieve a reasonable distribution of resources.
When Ruby on Rails Meets Auto Scale
Ruby on Rails, or most deployment tools frequently used for framework development, are not particularly suited for Auto Scale. For instance, Capistrano is a deployment tool often used for Ruby on Rails development. Its major function is to automate the deployment script, but becomes problematic when it is used to install system packages or kits. Therefore, the development of the Auto Scale framework requires the selection of suitable tools as well as proper integration of such tools. As a result, how to complete the task of Auto Scale become a worthy topic for research.
This article introduces how 5xRuby integrates the services provided by AWS, Packer, Ansible and GitLab, to complete the Auto Scale deployment in the Ruby on Rails project. Using Ansible, we execute automated scripts on the server machines, install required packages and programs, and set up the system environment requirements. Then Packer is used to package the environment into an image file (AWS AMI). Using the AWS Systems Manager, the target machine group is updated with the image file to complete the Auto Scale architecture.
To “automate” the above process, such that the new image file is updated simultaneously and the deployment is completed whenever an engineer updates the source code, we will use GitLab programming storage service related functions, to achieve a turn-key deployment for the system.
Increasing the coverage rate of automation in the workflow not only enhances development efficiency, but also lowers the rate of problems resulting from operator errors.
The Automated Deployment Execution Process of Ansible
The Ansible scripts are based on the unit of roles, separating functions into different roles executed by different scripts. For example, one role is specifically designed for setting up system users. Another role is for installation of a website server (Nginx). Internally, 5xRuby has designed a public version of a set of Ansible scripts. All the roles inside are enough to cover every step required of a machine in order to activate Ruby on Rails. This set of scripts are organized based on the role format. The set of roles are available for different projects to select or modify as necessary. In this manner, the design of roles can help to speed up the development process while retaining the flexibility of customization.
When it is running, Ruby on Rails requires several environment variables, such as the password required for connecting to the database, or the certificate required by a third-party package. For environment variables requiring encryption, here at 5xRuby we have chosen Ansible’s Vault function for encryption purposes.
Ansible Vault works by encrypting sensitive data before placing such data in the version control system, to avoid direct exposure of sensitive data or leakage. For relatively small-scale projects, the methodology may lower the cost for building extra cryptographic key management service or hardware purchase. It is a sensible compromise to achieve a certain level of protection without too much cost.
Packer Packages the Environment Image Files
Packer is a tool for creating machine images. Common cloud machines such as the EC2 and GCP compute engines of AWS are all supported by machine images created by Packer. The image files we need to create include the Ruby on Rails environment, and all usable Provision settings of Packer. Provision refers to the script execution requirement prior to creating an image file. The script may be directly written in Shell Script, or a script in the Ansible script format.
After applying the Ansible script we just wrote to open an image file, the machine will contain Ansible installation packages and source code, which prepares the machine directly into the “ready” state. This methodology saves 5 to 20 minutes compared to the method of turning on the machine before package installation. The faster Auto Scale can be completed, the faster the machine becomes more capable to carry the load of an elevated traffic flow. How to effectively shorten the time required by Auto Scale is another important factor in the selection of a deployment tool.
Integration of AWS Cloud Service
When an (AWS AMI) image file is available, we may directly apply the AWS Lambda service to upgrade the machine group with a new image file. If we want to verify whether an image file can function properly before updating the machine group, we can utilize the AWS System Manager Service, which can inspect and control multiple AWS services. In the automated service, we can set up to activate a machine using a machine image, and only export the image file after determining that everything is normal. Finally, the AWS Lambda service is invoked to update the target machine group.
*The following image shows the AWS Systems Manager Execution Detail page
GitLab Integrates multi-tool programming source code
At 5xRuby, most of our program source code is managed using GitLab. By setting up the automated GitLab script (GitLab CI/CD), when we want to update the source code, we can automatically execute test programs and program analysis. It can even complete the deployment of source code by deploying the latest code onto machines to facilitate manual testing, or perform an update on the official environment.
The GitLab Trigger function is quite valuable. When the Ruby on Rails project makes a Merge Request, it can be set up to trigger another GitLab automated script in the Packer source code storage. This function allows integration of automated scripts with different source code to complete the task of deployment.
* The Image below shows the GitLab Trigger function setup page
Challenges and Optimizations
While working on a Double-Eleven event for a well-known e-commerce platform in 2019, 5xRuby once added Ansible through Packer to implement the deployment of Auto Scale functions within a GCP environment. At that time, I was honored to participate in the project, and took on the challenge of transforming the client’s project from the GCP to the AWS environment. Referencing previous designs, we also set up a Ruby on Rails environment with the Auto Scale function.
In the GCP environment, the latest machine image may be updated to the target machine group directly via the command method. In the AWS environment, however, it is necessary to have an image serial number (AMI ID) before updating an image file to the machine group. However, in the GitLab automated script, different stages of scripts are used for Packer to create image files, and for the AWS Systems Manager to update the machine group. Therefore, the use of GitLab CI’s artifacts function is required to take the image ID and pass onto the script in the next stage in the form of environmental variables.
Among the many required steps to execute an Ansible script, static file packaging is a very time-consuming action item. Static file packaging is required once during the testing phase and once during the deployment phase. We could save a few minutes to dozens of minutes if it is possible to perform one static file packaging to complete testing and deployment. Therefore, we take the static file packaged during the testing phase and upload to the AWS Object Storage Service (S3), and then download the packaged static files directly during the deployment phase. This strategy helps to save a lot of time in the process of automated deployment.
Summary
The description above is how we implemented the deployment of Auto Scale in Ruby on Rails. The deployment allows the most suitable automatic resource adjustment corresponding to website traffic fluctuations. As the scale of projects grow, originally feasible system architecture often becomes inadequate to support an increased load. In recent years, website technology has been changing continuously. The selection and introduction of appropriate tools has become a complex inquiry and an important professional skill for company development.
With professional knowledge and experience over many years, 5xRuby can provide technical and consulting service to assist you in adjusting and optimizing website system architecture, to create the best solution for your products. Stop wasting corporate resources, and spend every penny where it counts.