Large language models (LLMs) have transformed artificial intelligence with their superior performance on various tasks, including natural language understanding and complex reasoning. However, adapting these models to new tasks is a significant challenge, as traditional fine-tuning methods involve large labeled datasets and heavy computational resources. Existing methods for combining multiple LLMs lack the required flexibility and face challenges in generalizing to new tasks. Moreover, gradient-based optimization dependence limits efficiency and scalability, making real-time adaptation impossible. Therefore, there is an urgent need for a more effective approach that allows LLMs to dynamically adapt, need minimal data to adapt, and improve performance without paying a heavy computational price.
Several methods have been proposed to boost LLM adaptation, yet each has essential drawbacks. Expert Fusion merges fine-tuned models by averaging their parameters according to pre-defined rules, yet the method cannot adapt dynamically to a specific task. Other methods, such as LoraHub and Model Swarms, use evolutionary algorithms such as genetic programming or particle swarm optimization to adaptively combine models. They, however, need labeled adaptation data and tend to degrade in performance when scaling across multiple tasks. Parameter merging methods such as DARE, TIES, and Pack of LLMs align and merge weights from various models yet cannot adapt dynamically to changing demands. The shared drawbacks of these methods are high computational complexity, fixed adaptation mechanisms, and weak generalization ability in zero-shot settings. These drawbacks emphasize the need for a more sophisticated solution that can continuously evolve and adapt to tasks without needing extensive retraining.
Researchers from Northeastern University and Shanghai Artificial Intelligence Laboratory propose GENOME (GENetic Optimization for Model Evolution), a population-based evolutionary framework designed to enhance LLM adaptation. The approach uses mechanisms from genetics, i.e., crossover, mutation, selection, and succession, to evolve a population of models dynamically. Unlike conventional fine-tuning based on gradient-based learning, GENOME enables successful evolution under sparse data. Crossover allows the merging of high-performing models to create offspring with better capabilities, while mutation adds randomness to discover new capabilities. Selection retains only the most efficient models and rejects suboptimal ones, and succession allows knowledge transfer by enabling newly developed models to inherit the strengths and weaknesses of earlier versions. A variant called GENOME+ adds an ensemble mechanism that combines the predictions of top-performing models to improve robustness and accuracy. These innovations allow LLMs to adapt rapidly to new tasks while minimizing computational resources simultaneously, offering an improved and scalable alternative to the conventional model adaptation approaches.
The evolutionary model is applied by a population of LLMs seeded from gemma-2-2b-it, fine-tuned on 10 domains using the Tulu-v2-SFT dataset. Evolutionary operations are iteratively applied over 10 generations, fine-tuning model parameters by optimizing them using a fitness function that monitors validation accuracy. The system runs with population sizes between 10 and 40 models, with a crossover rate of 30% and a mutation probability of 20%. Accuracy, exact match, F1-score, and BLEU score are used as evaluation metrics for multilingual tasks. The approach is computationally efficient and executed on a single RTX 4090 GPU, making it a practical and viable alternative to traditional fine-tuning approaches.
Large-scale evaluations confirm that this approach surpasses state-of-the-art model adaptation and combination methods on several benchmarks in terms of enhanced accuracy, reasoning capacity, and scalability. The system achieves an average 24.06% gain over the top-performing single expert model and 10.75% over Model Swarms, with notable gains on resource-heavy reasoning tasks. In contrast to other adaptive methods, which are handicapped when faced with multiple tasks, this evolutionary approach exhibits consistent performance on a broad range of domains. In addition, it achieves strong zero-shot generalization, successfully transferring learned representations to new tasks without the need for additional training data. Scalability experiments confirm that increasing the population size from 10 to 40 models leads to further improvement in performance. Ablation studies confirm the importance of each evolutionary process, with selection and ensemble mechanisms playing a critical role in overall effectiveness.
By implementing population-based evolution to LLMs, this paper presents a gradient-free, adaptive, and scalable optimization method that allows continuous improvement in low-data conditions. Following the principles of genetic algorithms, the method allows models to evolve dynamically, perform better than traditional adaptation methods, and generalize well to novel tasks. On cost-effective implementation on current hardware, GENOME+ presents an economical and real-world solution to traditional fine-tuning and model fusion methods, enabling AI systems to improve and adapt continuously.
Check out the Paper. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 80k+ ML SubReddit.
Aswin AK is a consulting intern at MarkTechPost. He is pursuing his Dual Degree at the Indian Institute of Technology, Kharagpur. He is passionate about data science and machine learning, bringing a strong academic background and hands-on experience in solving real-life cross-domain challenges.
Leave a comment