Latency-Consistency Tradeoffs in Real-Time Personalization Serving: A Comparative Architectural Study

Abstract

Personalization systems must continually provide and display features that are dynamic: user, product, inventory, pricing and contextual. With e-consumers demanding a lightning-fast response, it's not a simple task to get a site to respond quickly, while maintaining the desired consistency and data freshness of the business. This research paper will compare the different approaches to the design of personalization-serving architectures, distributed caching, cache invalidation strategies, the pros and cons of using either eventual consistency or strong consistency, and the way in which features can be cached as individual features or as sequences. These are compared to other important characteristics of the service (such as serving latency, throughput, feature freshness, stale read rate, scalability, cache efficiency, and infrastructure requirements). Three axis classification are provided, as a function of freshness requirement, read/written ratio and consistency tolerance to consider the implications of workload and feature properties on architectural choices. The framework is based on differentiating features based on their need for very up-to-the-moment information, read-heavy workload, availability of limited staleness periods. The analysis shows that there isn't a single consistency and serving architecture for all personalization workloads. Rather, architectural decisions must be made based on the operational needs of individual features and their workloads characteristics. The proposed framework provides suggestions of what actionable steps should be taken by the back-end developers and system architects of personalization systems to design infrastructure that can balance latency, freshness, consistency, scalability, and resource efficiency.