A Temporal and Geographic Evaluation Concerning the Performance Variability of Function-as-a-Service

The serverless paradigm emerges as a new computing resource in the field of cloud computing. Cloud Providers (CPs) promote this new way of sharing resources and advertise it as an easily scalable, cost-efficient, and easy-tomaintain way for enterprises and academics to utilize the Cloud. Function-asa-Service (FaaS) offerings are a specific field of serverless and provide users a pragmatic way to focus on only supplying and running the business logic and leave maintenance and operation of runtime to the CPs. With the increased shift toward the CPs, operational characteristics like runtime performance become the responsibility of the CPs. This change of responsibility raises concerns about the continuous delivery of the same performance. Previous works have already indicated performance characteristics variability, such as runtime performance over time and for different regions. Our work provides the first longitudinal and large-scale study that observes and evaluates FaaS for the three biggest CPs: Amazon Web Services (AWS), Microsoft, and Google. We observe 29 regions for two months and find significant differences for the CPs, regions, and substantial variability across time-based on performance analysis using a benchmark-based approach. We see reoccurring patterns that resemble usage patterns (e.g., peak and off-peak usage) for commodity resources such as electricity.