In a move to optimise for multi-tenant support, improve scalability, and prepare for future regional deployments, the

MyPetParlor App

team has successfully completed a massive migration of its

Azure

infrastructure. The migration, which spanned 38 days from the 21st of February to the 30th of March, involved a comprehensive redesign of the system's architecture to enable shared Azure resources across all tenants through the Deployment Stamps pattern.

Laying the Groundwork: Database Consolidation

Before embarking on the Azure migration, the MyPetParlor App team undertook a critical step: consolidating their databases. Previously, multiple database instances were created, each dedicated to specific tenants or services, posing challenges in terms of operational complexity, resource utilization, and scalability.

By consolidating their databases into a unified PostgreSQL environment the team laid the foundation for a simplified multi-tenant architecture. This involved migrating data from MongoDB to PostgreSQL for a shared service pivotal to the platform's daily operations. The migration was successfully implemented on February 20th, 2024, around 5 PM (GMT), marking the final step in the team's journey away from MongoDB-based services.

Click here

to read more about it, titled MyPetParlor App Completes Consolidation of Databases for Enhanced Efficiency and Performance.

More Groundwork: Row-based Sharding

Previously, the team had a

schema-based sharding

model, where each tenant had a dedicated schema within the database. While this approach provided data isolation, it became increasingly challenging to manage and maintain as the customer base grew, limiting scalability to around 10,000 tenants.

To address these scalability limitations and enable more flexibility, the team migrated to a row-based sharding model using the distributed tables capability in Citus. In this new architecture, all tenant data is stored in a shared set of tables with an additional tenant_id column acting as the distribution key. Each table is hash-distributed across nodes based on the tenant_id value, ensuring that all data for a given tenant is co-located on the same nodes for optimal query performance.

This row-based sharding architecture offers several key benefits:

• Improved Scalability: The team can now easily scale to hundreds of thousands or even millions of tenants by distributing across more nodes as needed.

• Cross-Tenant Queries and Reporting: With all data co-located by tenant_id, complex queries, reporting, and analytics can be performed efficiently across all tenants.

• Customization Flexibility: If a tenant requires custom tables or a unique data model, it can be easily provided within their isolated tenant_id data subset.

• Row-Level Security: Using the tenant_id column, row-level security policies can be implemented to restrict tenants to only their data.

The migration to the row-based sharding architecture was completed during a scheduled maintenance window on March 16th, 2024, between 12:00 AM and 2:00 AM SAST, breaking through previous scalability limits and allowing the MyPetParlor App to efficiently serve over 1 million customers.

Click here

to read more about it, titled MyPetParlor App Database Scales to Millions of Customers with Row-Based Sharding Architecture.

Optimising for Multi-Tenancy Support: Shared Resources with Tenant Isolation

With the database consolidation complete, the team could proceed with implementing an architecture optimised for multi-tenancy on Azure. This involved adopting the

Deployment Stamps pattern

, a calculated approach that facilitates resource sharing while maintaining tenant isolation.

To achieve this, the team implemented several key tasks:

1. Tenant Identification: A new service was created to handle tenant identification. Constants were defined for tenant ID query parameters and headers, and middleware was implemented to extract the tenant ID from requests and set it in the context.

2. Database Restructuring: The PostgreSQL database was restructured to support a row-based sharding model using the Citus extension. Each table was modified to include a tenant_id column as part of the primary key, foreign keys, and uniqueness constraints. Queries were updated to filter and join based on the tenant_id.

3. API Updates: All API endpoints were updated to execute the TenantMiddleware and incorporate the tenant ID from the request context into their queries and operations. This ensured that data access and manipulation were scoped to the appropriate tenant.

Benefits of the Migration

The successful migration to the optimised multi-tenant architecture brings several key benefits to the MyPetParlor App platform:

1. Multi-Tenant Support: MyPetParlor App can now serve multiple tenants from a single running instance, improving resource utilization and enabling more efficient scaling.

2. Improved Scalability: With shared resources and tenant isolation, MyPetParlor App can scale its infrastructure more efficiently, accommodating growth in user traffic and tenant demand without the need for extensive resource provisioning for each new tenant.

3. Simplified Operations: By consolidating resources and implementing a centralized tenant management system, the operations team can streamline management and monitoring tasks, reducing the overall operational complexity of the platform.

4. Regional Expansion Readiness: The new architecture lends itself well to more complex queries and deployments, paving the way for MyPetParlor App's planned expansion into regions like the United States of America and the EU.

The migration was a complex undertaking, involving careful planning, execution, and coordination. The MyPetParlor App team's dedication and expertise in Azure technologies ensured a smooth transition, minimizing downtime and ensuring uninterrupted service for existing users.

Looking Ahead

With the successful completion of this migration, MyPetParlor App is well-positioned to handle increasing demand, support multiple tenants efficiently, and deliver a seamless experience to its growing user base. The team is now focused on leveraging the newly implemented architecture to enhance the platform's capabilities further and prepare for the upcoming launch in the American and EU regions.