Research
Papers from the Kafila project, newest first.
PAPERS
-
Kafila: Serving Large Language Models on a Trusted Set of Heterogeneous Commodity Machines
A bounded session of trusted, unequal machines serves one model. Dividing it by what each device can actually serve shortens the slowest stage by 4.2× against uniform division, measured in three countries.