The paper sets out to make a “real life“ comparison of web applications.
Effectively using a production application from Berger-Levrault.
Two implementations of the same application are looked at:
- Legacy (Java with Swing Frontend)
- Modern (Spring Boot with Angular Frontend)
Additionally the data layer was compared between using JSON and ProtoBuf implementations.
Results
The modern implementation used far less energy in the frontend compared to the legacy one.
Conversely the modern implementation used more energy in the backend than the legacy application.
ProtoBuf was far more effcient in terms of CPU and energy than JSON.
Review
The study takes an interesting case by using a real world web application and not a watered down example of e.g. a wordpress single page application or similar.
However the research approach and also the technical implementation make it hard to derive useful meaning from the data:
- CPU utilization is used without describing hardware configurations (Was frequency fixed? Hyper Threading turned on etc.)
- Energy is not directly measured but the SDIA Model is used. This model is linear and works only on CPU Utilization as input. Effectively providing no further insight than CPU% itself
- They describe that no identical hardware setups could be used due to internal deployment restrictions. This makes values inherently incomparable
- Values are captured from VMs without making clear if CPUs where oversubscribed or not
- Workloads are not made public / No Reproduction package
Especially for the CPU Utilization problem we have a nice deep dive if you want to further understand how helpful this metric is: CPU Utilization - A useful metric? | green-coding.io
Also Intel has a nice introduction (https://www.intel.com/content/www/us/en/developer/articles/tool/performance-counter-monitor.html) why CPU Utilization should not be used as a metric for comparsion without implementing strong constraints.
Link to the paper: Energy Consumption of Web Applications: Measurement Challenges in Practice - Inria - Institut national de recherche en sciences et technologies du numérique