LLC Analysis of Jailbreak Susceptibility in Language Models

This project aims to explore the relationship between Local Learning Coefficient (LLC) dynamics and a language model’s susceptibility to jailbreaks. We’ll investigate whether LLC analysis can provide insights into model vulnerabilities and potentially predict or detect jailbreak attempts.

Key research questions:

1. How do LLC trajectories differ between models that are more or less susceptible to jailbreaks?
2. Can LLC dynamics during fine-tuning for safety predict a model’s resistance to jailbreaks?
3. Is there a relationship between LLC and the model’s ability to maintain consistent behavior under adversarial prompts?
4. Can LLC analysis provide insights into the effectiveness of different safety training techniques?

Methodology:

1. Train or fine-tune language models with various safety techniques, tracking LLC throughout the process.
2. Develop a suite of jailbreak attempts, including prompt injection and adversarial examples.
3. Evaluate models’ susceptibility to jailbreaks and analyze the relationship with LLC trajectories.
4. Investigate LLC behavior during inference when exposed to jailbreak attempts.
5. Explore the use of LLC for detecting potential jailbreak attempts in real-time.
6. Compare LLC dynamics across different model sizes, architectures, and safety training methods.
7. Analyze how LLC changes in different components of the model (e.g., specific layers or attention heads) during safety training and jailbreak attempts.

Expected outcomes:

1. Characterization of LLC dynamics in language models with different levels of jailbreak susceptibility.
2. Insights into the relationship between LLC trajectories and model robustness against jailbreaks.
3. Potential development of LLC-based metrics for predicting or detecting jailbreak susceptibility.
4. Better understanding of how safety training affects model complexity from an SLT perspective.
5. Possible identification of critical phases or components in safety training, as reflected in LLC dynamics.

This research could provide valuable insights into the nature of language model vulnerabilities and potentially lead to more robust safety training techniques. It may also contribute to the development of real-time jailbreak detection methods.