It Takes Two to Tango: Accelerating Code Generation via Cache-Inspired Hierarchical Speculative Decoding

This repository contains code and the distilled checkpoints of the draft models for CodeLlama-34b and Deepseek-Code-33b for the paper “It Takes Two to Tango: Accelerating Code Generation via Cache-Inspired Hierarchical Speculative Decoding”. CacheCoder.zip contains the source code of this research. checkpoints.zip contains the distilled weights of the draft model used in this research.