Summer in the Lab @ Light

This repository contains my research work at my summer internship at LiGHT lab (EPFL), focusing on scaling the MultiMeditron training pipeline for 70B-parameter models. MultiMeditron is a multimodal medical language model based on the LLaVA architecture. This project explores optimization techniques to efficiently train large-scale versions using distributed computing and DeepSpeed.

Contents

• MultiMeditron_Simon_Lefort_Summer25.typ - Complete research report (Typst format)
• MultiMeditron_Simon_Lefort_Summer25.pdf - Generated PDF report
• scripts/ - Training scripts for RCP and CSCS clusters
• assets/ - Supporting images and diagrams

Authors

Simon Lefort, supervised by Michael Zhang and Annie Hartley at LiGHT Lab, EPFL.