Procedure

---

To evaluate our approach, simulations were performed using three different caching configurations: (i) no application-level caching (NO), (ii) application-level caching manually designed and implemented by developers (DEV); and (iii) our approach (AP). To assess performance, we used three metrics: throughput (number of requests handled per second), hit ratio and the total number of hits, because they are well-known in the context of web applications and cache performance tests. Our simulation emulated client sessions to exercise applications and evaluate decision criteria. Simulations consisted of variations of 1, 5, 10, 25 and 50 simultaneous users constantly navigating through the application, always at a limit of 500 requests per user. Each simulation was repeated ten times, and the mean of each metric was collected. For all the executions of AP, caches were bounded by a cache size and configured with a TTL, which were both specified and used by developers of our target systems (DEV). As the results may be influenced by the cache policy adopted, in addition to our standard technique based on TTL, we also evaluated our approach combined with popular replacement policies, namely least recently used (LRU) or least frequently used (LFU).

Target Systems

Our evaluation was performed with three open-source web applications, presented in table below, which summarizes the general characteristics of each target system.

Project name	Domain	LOC
Cloud Store	e-Commerce based on TPC-W benchmark	7.6K
Pet Clinic	Sample application	6.3K
Shopizer	E-commerce	111.3K

RQ1. What is the performance improvement provided by our automated caching approach?

---

Figures below show the throughput obtained for each target application in simulations with different amount of simultaneous users and setups.

---

Tables below presents the complete results for every combination of our setup.