Who can take advantage of it?
If you wish to know the criteria for using Stateful Support, this is a perfect place. Below are important points to consider while choosing this option.
- Does your application has a replication factor more than 1 and is either sharded or clustered? If yes, this is a perfect fit.
- Would you need to retain the data of the terminated instance to a replacement instance? If no, you do not need to enable this option. If yes, this is a perfect fit.
- Do you have any auditory purpose to retain crucial data such as Application logs? If yes, this is a perfect fit.
- Have you verified if the node can be taken away from such a cluster suddenly and it keeps working? If yes, this is a perfect fit.
- The data stored on such instances within OptimoGroup can be either the same(such as Autoscaling groups) or can be different(such as NoSQL databases like Cassandra or Cockroach DB).
How do we do it?
The principle behind this feature is very simple. Data on AWS Instances are stored in EBS volumes(Root and Externally Mounted Data Partitions). If you want to persist data on Terminated Instance, it means that we need to migrate all volumes of the terminated instance to a replacement instance. Sometimes you also need to preserve the Internal IP Address of the terminated instance for the application to work properly. Our Support for Stateful Applications can be enabled by enabling two below options. For these options to appear, you need to select Application Architecture as “Stateful”. There are some corner cases which we will discuss in detail.
- Preserve All Volumes
- Preserve Private IP Address.
Preserve All Volumes:
When the Spot Instance is terminated, we do below things to achieve Data Persistence:
- When a new OptimoGroup is being configured, you need to enable the option “Preserve All Volumes”.
- When this option is turned on, we ensure that all volumes(including root and data volumes) will be migrated to a replacement node at the same mount point. As the data from the terminated instances will be available, it might just seem that the original instance was just restarted.
- The instance selection will be done from the cheapest instance type from any Availability Zone in case Preserve IP Option is not enabled.
Preserve Private IP Address:
Sometimes Stateful Applications require internal IP Address to be preserved as they are hardcoded in the configuration. E.g. if you are running a single-node MySQL server for QA, the web application that connects to such MySQL node will require IP Address to remain the same. This option ensures that your replacement Stateful Spot Instance retains the original internal IP Address.
- When a new OptimoGroup is being configured, you need to enable the option “Preserve Private IP Address”.
- When this option is turned on, any replacement node will retain the original IP Address of terminated spot instance.
- As the IP Address is bound to subnet, the instance selection will happen for the cheapest instance type from the same Availability Zone to which IP Address belongs. Please note that this might result in increased cost as we might have to launch On-Demand instances in case we do not get Spot Instances. It is also recommended to configure more instance types in case you wish to select this option.
Use Cases:
If you have a Stateful Application that can handle node failure like, you can take advantage of this feature to reduce your cost using Spot Instances. You shall enable the “Preserve Data Volumes” option during OptimoGroup configuration. As a precautionary note, we strongly advise doing a litmus test which verifies that any node can be taken away without disrupting the cluster when more than 1 instance are running.
- NoSQL or Distributed / Sharded Databases:
Most NoSQL Databases(like Cassandra / Scylla DB, Cockroach DB etc.) and even distributed RDBMS like MySQL Cluster can take advantage of this feature. It will appear as if the terminated node was restarted. This can also ensure that replication and data redistribution will catch up sooner than it should thereby be available quickly for serving. This also reduces your IOPS spending.
- Development / QA environment:
You can take advantage of Stateful Application Support to reduce the cost of your Development / QA environment where occasional downtimes can be tolerated. Data Persistence can also be used to run a single node database in non-critical environments. MySQL / PostgreSQL are one of the most popular databases which many companies use. For production deployment, we strongly advise running your RDBMS( like MySQL or PostgreSQL) in Hybrid Deployment Model. RDBMS can employ Master-Slave configuration where Master node runs on On-Demand instance and Slave nodes run on Persistent Spot Instances.
