eFaaS

# eFaaS: A Lightweight, Kernel-Bypass Serverless Framework for HPC Workflows

eFaaS is designed to bridge the gap between the fine-grained elasticity of cloud-native computing and the extreme performance demands of High-Performance Computing (HPC). By re-architecting the serverless execution model from the kernel up, it eliminates structural bottlenecks encountered when handling massive, bursty concurrency.

## Core Features

* **Lock-Free Resource Isolation**: Replaces traditional cgroup mechanisms—which suffer from heavy lock contention—with pluggable eBPF policy hooks in critical kernel paths (CPU scheduling, memory allocation, and block I/O).

* **End-to-End Zero-Copy Communication**: Leverages XDP/TC and Shared HugePages to create a data plane that bypasses the traditional TCP/IP stack, achieving near-hardware-level throughput.

* **Efficient Control Plane (eSocket)**: Introduces a kernel-resident, event-driven signaling mechanism that uses 16-byte descriptors and Sockmap redirection to trigger functions with microsecond-level latency.

* **Workflow-Centric Security**: Enforces verifiable boundaries between different workflow chains at both the physical memory level (via DPDK-style HugePage namespaces) and logical signaling level.

## System Architecture

The framework decouples the control and data planes:

* **Data Plane**: Follows the "minimize data movement" principle, ensuring that once data enters the system, it remains in fixed physical locations to avoid redundant copies and context switches.

* **Control Plane**: Uses an eBPF-based event-driven model to answer scheduling and readiness queries with minimal system perturbation.

## Getting Started

### Hardware Requirements

* **CPU**: Intel Xeon Gold 6248R or equivalent.

* **Network**: 100 GbE NICs (e.g., NVIDIA Mellanox ConnectX-6 Dx) supporting XDP.

* **Memory**: 512 GB DDR4 ECC, with HugePages enabled.

### Software Prerequisites

* **OS**: Ubuntu 22.04 LTS.

* **Kernel**: Linux kernel v6.6 with specific eBPF and HugeTLB patches.

* **Compiler**: Clang/LLVM 16 (for eBPF C code) and Go v1.20 (for the runtime agent).

* **Libraries**: libbpf v1.6.

### Deployment

1.  **Kernel**: Apply the provided patches and recompile the kernel to enable custom eBPF mount points.

2.  **eBPF**: Compile the C-based isolation and communication programs.

3.  **Agent**: Build the Go-based runtime component and deploy it as a DaemonSet to bridge Kubernetes with the kernel BPF subsystem.

## Evaluation Results

The framework has been evaluated using representative scientific workflows, achieving significant speedups:

* **1000Genome**: Achieves up to 3.74x P99 speedup by eliminating startup jitter under extreme fan-out.

* **Epigenomics**: Provides up to 2.79x P99 speedup in I/O-intensive multi-stage pipelines.

* **SRAsearch**: Enhances stability and throughput for sustained, long-running high loads.

## Implementation Details

The prototype comprises approximately 9,925 lines of code:

* **Kernel Patches**: 845 lines.

* **eBPF C Code**: 2,310 lines.

* **Go Orchestration**: 5,620 lines.

## License

This project is licensed under the GPL License.

# eFaaS：面向高性能计算工作流的轻量级内核旁路无服务器框架

eFaaS旨在弥合云原生计算的细粒度弹性与高性能计算（High-Performance Computing, HPC）的极致性能需求之间的鸿沟。通过从内核层面重新架构无服务器执行模型，该框架消除了处理大规模突发性并发时遇到的结构性瓶颈。

## 核心特性

* **无锁资源隔离**：在关键内核路径（CPU调度、内存分配与块I/O）中引入可插拔的eBPF（eBPF）策略钩子，替代传统存在严重锁竞争的控制组（cgroup）机制。
* **端到端零拷贝通信**：借助XDP/TC与共享大页（Shared HugePages）构建绕过传统TCP/IP协议栈的数据平面，实现接近硬件级的吞吐量。
* **高效控制平面（eSocket）**：提出一种驻留于内核的事件驱动信令机制，通过16字节描述符与Sockmap重定向，以微秒级延迟触发函数执行。
* **面向工作流的安全机制**：在物理内存层面（通过数据平面开发套件（DPDK）风格的大页命名空间）与逻辑信令层面，为不同工作流链强制划定可验证的边界。

## 系统架构

该框架将控制平面与数据平面解耦：
* **数据平面**：遵循“最小化数据移动”原则，确保数据进入系统后始终保留在固定物理位置，避免冗余拷贝与上下文切换。
* **控制平面**：采用基于eBPF的事件驱动模型，以最小的系统扰动响应调度与就绪查询。

## 快速上手

### 硬件要求

* **CPU**：英特尔至强Gold 6248R或同等规格处理器。
* **网络**：支持XDP的100 GbE网卡（例如NVIDIA Mellanox ConnectX-6 Dx）。
* **内存**：512 GB DDR4 ECC内存，需启用大页（HugePages）。

### 软件依赖

* **操作系统**：Ubuntu 22.04 LTS。
* **内核**：Linux内核v6.6，需搭载特定eBPF与HugeTLB补丁。
* **编译器**：Clang/LLVM 16（用于编译eBPF C代码）与Go v1.20（用于运行时代理）。
* **依赖库**：libbpf v1.6。

### 部署步骤

1. **内核配置**：应用提供的补丁并重新编译内核，以启用自定义eBPF挂载点。
2. **eBPF组件编译**：编译基于C语言的隔离与通信程序。
3. **代理部署**：构建基于Go语言的运行时组件，并以DaemonSet（守护进程集）形式部署，以连通Kubernetes与内核BPF子系统。

## 评估结果

该框架已通过代表性科学工作流进行评估，实现了显著的性能加速：
* **1000Genome**：在极端扇出场景下消除启动抖动，最高可实现3.74倍的第99百分位（P99）延迟加速。
* **表观基因组学（Epigenomics）**：在I/O密集型多阶段流水线中，最高可实现2.79倍的第99百分位（P99）延迟加速。
* **SRAsearch**：提升了持续长时间高负载场景下的稳定性与吞吐量。

## 实现细节

该原型代码总量约9925行：
* **内核补丁**：845行。
* **eBPF C代码**：2310行。
* **Go语言编排代码**：5620行。

## 许可证

本项目采用GPL许可证。