Supporting processes around the data refresh from production to lower environments adds to the complexity of implementation. The data environment at CURO consisted of: 

On premise database  

• 95% of SQL Server Version 15.0 

• 5% on the version below SQL Server 15.0. 

• Data size: ~ 150 TBs 

• Server storage: ~250 TBs 

• Annual growth: 20% 

• Number of tables: ~ 10000 (In multiple environments) 

• Number of databases: ~600 (In multiple environments) 

• Number of environments : 5 Non-prod and 1 Prod environment

Assessment:

• The data landscape assessment was broken down into two phases. The first phase used Cloudamize to gather data on the existing on-premise infrastructure that covers all servers, including the database servers. The second phase took a deeper dive into the gaps in the data captured by Cloudamize. This was done using the RDSTool provided by AWS, allowing CURO to examine the SQLServer landscape in a few days, instead of weeks. The tool assessed the health of SQLServer infrastructure, features in use and

• Based on data analysis, more than 90% of on-premise DB instances were deemed able to move to RDS, with the remaining 10% on EC2 instances.

• Migration pattern for the SQL Server can broadly be categorized into 3 buckets:

  • Older servers (Version below 12)

  • Clustered servers

  • Standalone servers

Planning:

• Based on the assessment, the migration was planned into four waves. The first phase was inclusive of a pilot that would determine the speed of migration for the waves and the sequencing of activities involved in the migration.

• The migration depended on the structure of the current data landscape, how many environments are being used and migration speeds for databases.

Execution:

• Executed the waves of migration with increased automation over time.

• Complete end to end migration involved database migration along with scheduled backups and disaster recovery solutions deployed.

• Ensured security and compliance requirements are satisfied. improved automation using IAC tools and integrated approach to support current and future migrations.

• Built the pattern for migrating on-premise to RDS and to EC2 based SQL Server, with increased automation based on templates

• Setup storage gateway to move huge data from on-prem to S3 and tuned the performance to match the speed of migration.  

• Migration of database using Log shipping approach. This involved setting up log shipping and cutover, with minimal downtime to bring the respective databases on RDS/EC2.

• Migrate jobs & users/roles/permissions

• Setup daily backup jobs.

• Created jobs to refresh non-prod environments from prod backup and scheduled the jobs on a weekly basis. 

• Setup S3 lifecycle policy to meet security and compliance requirements on data backup.

• Implement Disaster recovery using AWS RDS.

• Setup a migration plan to move to support multi-region DR strategy in the future.  

03

The Outcome

• Migrated the database footprint to the cloud in a controlled and aligned manner with validation and continuous integration. Minimize the downtime required for migration.  

• Overall TCO reduced primarily, saving licensing costs with more flexibility needed for their future visions.

• Increased resiliency helped to enhance business continuity

• Built on a scalable architecture that provides agility and helps focus on core business.  

• Overall availability, durability, and disaster recovery enhanced across Multi AZs

• Improved functional stability enhancing the user’s trust and experience 

• In the initial phases the process was done manually, and overtime shifted to

• Automation which resulted in a maintenance window executed in less than 3 hrs.

• The migration execution for individual databases was tuned in the initial phase and resized to accommodate the migration for various databases; this prevented the necessity for additional outages during the migration.