eFaaS

# eFaaS: A Lightweight, Kernel-Bypass Serverless Framework for HPC Workflows

eFaaS is designed to bridge the gap between the fine-grained elasticity of cloud-native computing and the extreme performance demands of High-Performance Computing (HPC). By re-architecting the serverless execution model from the kernel up, it eliminates structural bottlenecks encountered when handling massive, bursty concurrency.

## Core Features

* **Lock-Free Resource Isolation**: Replaces traditional cgroup mechanisms—which suffer from heavy lock contention—with pluggable eBPF policy hooks in critical kernel paths (CPU scheduling, memory allocation, and block I/O).

* **End-to-End Zero-Copy Communication**: Leverages XDP/TC and Shared HugePages to create a data plane that bypasses the traditional TCP/IP stack, achieving near-hardware-level throughput.

* **Efficient Control Plane (eSocket)**: Introduces a kernel-resident, event-driven signaling mechanism that uses 16-byte descriptors and Sockmap redirection to trigger functions with microsecond-level latency.

* **Workflow-Centric Security**: Enforces verifiable boundaries between different workflow chains at both the physical memory level (via DPDK-style HugePage namespaces) and logical signaling level.

## System Architecture

The framework decouples the control and data planes:

* **Data Plane**: Follows the "minimize data movement" principle, ensuring that once data enters the system, it remains in fixed physical locations to avoid redundant copies and context switches.

* **Control Plane**: Uses an eBPF-based event-driven model to answer scheduling and readiness queries with minimal system perturbation.

## Getting Started

### Hardware Requirements

* **CPU**: Intel Xeon Gold 6248R or equivalent.

* **Network**: 100 GbE NICs (e.g., NVIDIA Mellanox ConnectX-6 Dx) supporting XDP.

* **Memory**: 512 GB DDR4 ECC, with HugePages enabled.

### Software Prerequisites

* **OS**: Ubuntu 22.04 LTS.

* **Kernel**: Linux kernel v6.6 with specific eBPF and HugeTLB patches.

* **Compiler**: Clang/LLVM 16 (for eBPF C code) and Go v1.20 (for the runtime agent).

* **Libraries**: libbpf v1.6.

### Deployment

1.  **Kernel**: Apply the provided patches and recompile the kernel to enable custom eBPF mount points.

2.  **eBPF**: Compile the C-based isolation and communication programs.

3.  **Agent**: Build the Go-based runtime component and deploy it as a DaemonSet to bridge Kubernetes with the kernel BPF subsystem.

## Evaluation Results

The framework has been evaluated using representative scientific workflows, achieving significant speedups:

* **1000Genome**: Achieves up to 3.74x P99 speedup by eliminating startup jitter under extreme fan-out.

* **Epigenomics**: Provides up to 2.79x P99 speedup in I/O-intensive multi-stage pipelines.

* **SRAsearch**: Enhances stability and throughput for sustained, long-running high loads.

## Implementation Details

The prototype comprises approximately 9,925 lines of code:

* **Kernel Patches**: 845 lines.

* **eBPF C Code**: 2,310 lines.

* **Go Orchestration**: 5,620 lines.

## License

This project is licensed under the GPL License.