FATE-LLM：利用联邦学习赋能大型语言模型 | KC23

FATE-LLM: Empowering Large Language Models with Federated Learning - Fangchi Wang & Layne Peng

FATE-LLM：利用联邦学习赋能大型语言模型 | FATE-LLM: Empowering Large Language Models with Federated Learning - Fangchi Wang & Layne Peng, VMware

大型语言模型（LLMs）在颠覆我们生活的各个方面显示出巨大潜力。但是，高质量公共数据的枯竭以及对隐私和治理的担忧可能会阻碍它们的成功。联邦学习（FL）通过在多个数据源之间实现协作模型训练而不共享原始数据，提供了一种有希望的解决方案。本次演讲介绍了FATE，一个工业级FL平台，以及其专门设计用于支持LLMs（包括ChatGLM、LLaMA等）在FL范式中的FATE-LLM模块。将LLMs与FL集成在一起会带来自身的复杂性，涉及效率和安全性，这是由于它们资源密集型和复杂的算法所致。为了解决这些问题，FATE-LLM包含了几个专门设计的组件，本次演讲将介绍这些组件，以及其关键的技术和实际考虑因素。此外，还将分享和讨论关于FATE-LLM的真实世界实验、评估和未来路线图。

Large language models (LLMs) have shown great potential in disrupting various aspects of our lives. But challenges such as the depletion of high-quality public data and concerns over privacy and governance can hinder their success. Federated learning (FL) offers a promising solution by enabling collaborative model training across multiple data sources without sharing raw data. This talk introduces FATE, an industrial FL platform, and its FATE-LLM module, designed specifically to support LLMs including ChatGLM, LLaMA, and others in the FL paradigm. Integrating LLMs with FL presents its own complexities around efficiency and security, due to their resource-intensive nature and complex algorithms. To address them, FATE-LLM incorporates several specially designed components, which will be covered in this talk, along with its key technical and practical considerations. Additionally, real-world experiments, evaluations, and future roadmap concerning FATE-LLM will be shared and discussed.

CNCF概况（幻灯片）

扫描二维码联系我们！

CNCF (Cloud Native Computing Foundation)成立于2015年12月，隶属于Linux  Foundation，是非营利性组织。 

CNCF（云原生计算基金会）致力于培育和维护一个厂商中立的开源生态系统，来推广云原生技术。我们通过将最前沿的模式民主化，让这些创新为大众所用。请关注CNCF微信公众号。