Human-centered LLMs are products of the multifaceted technical processes used to create them. NLP techniques determine not only what models can do but also the boundaries of what they cannot. These limitations can have particular consequences as users across diverse linguistic and cultural contexts interact with LLMs.
Prior survey papers cover the technical practicalities and details of NLP methods for LLMs (Minaee et al., 2024ReferenceMinaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., & Gao, J. (2024). Large language models: A survey. arXiv Preprint arXiv:2402.06196.; Zhao et al., 2023ReferenceZhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., & others. (2023). A Survey of Large Language Models. arXiv Preprint arXiv:2303.18223.). In this chapter, we instead focus on the human-centered considerations across the language model training pipeline. We have already discussed pre-training practices in Data for HCLLMs and will focus on post-training techniques in this chapter. Although post-training recipes differ across models, two core components include a supervised fine-tuning (SFT) stage (Supervised Fine-tuning for HCLLMs) and a reinforcement learning stage that incorporates human preferences (Learning from Human Preferences). We next discuss how the predominant paradigm of scaling applies to human-centered objectives (Scaling Human Centered LLMs). Finally, we conclude by discussing three currently open challenges and future research directions for HCLLMs, covering personalization (Personalization), pluralistic alignment (Pluralism), and multilinguality (Multilinguality). For a roadmap, see the figure.
Figure. This chapter applies human-centered considerations to existing post-training techniques like SFT and RLHF (Supervised Fine-tuning for HCLLMs-Learning from Human Preferences), and explores the limitations of scaling for human-centered outcomes (Scaling Human Centered LLMs). Finally, we cover open challenges in personalization (Personalization), pluralistic alignment (Pluralism), and multilinguality (Multilinguality).