[ICML 2025] Beyond Bradley-Terry Models: A General Preference Model for Language Model Alignment (https://arxiv.org/abs/2410.02197)

alignment large-language-models rlhf preference-modeling preference-optimization

• Updated Nov 5, 2026
• Python

PipelineLLM 是一个系统性的大语言模型（LLM）后训练学习项目，涵盖从监督微调（SFT）到偏好优化（DPO）、强化学习（RLHF/PPO/GRPO）再到持续学习（Continual Learning)的完整技术栈。

reinforcement-learning lora fine-tuning post-training continual-learning sft rlhf llm-reasoning preference-optimization llm-infrastructure llm-processing

• Updated Jan 16, 2026
• Python

[TMLR] Triple Preference Optimization

alignment large-language-models rlhf preference-optimization

• Updated Feb 19, 2025
• Python

A lightweight post-training framework for LLMs and VLMs. 51 algorithms, 38 verified models. Scales with DeepSpeed, vLLM, and Ray.

reinforcement-learning alignment post-training dpo deepspeed rlhf vllm llm-training preference-optimization grpo

• Updated Aug 4, 2026
• Python

[ACL 2025] Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQL

text-to-sql nl2sql dpo text2sql preference-optimization

• Updated Oct 9, 2025
• Python

Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images

alignment-algorithms vision-language-models preference-optimization

• Updated Jun 4, 2025
• Python

A weekly updated awesome list of RL, RLHF, DPO, GRPO, reward models, and preference optimization for image and video diffusion generation.

reinforcement-learning image-generation awesome-list video-generation diffusion-models rlhf preference-optimization grpo

• Updated Oct 19, 2026

Code for ICLR 2025 Paper: Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data

audio audio-classification synthetic-data audio-generation large-language-models preference-optimization

• Updated Mar 31, 2025
• Python

🔬 Official implementation of ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detection (ICLR 2026). Novel multimodal RL approach for interpretable and explainable content moderation.

multimodal-learning explainable-ai content-moderation vision-language-models preference-optimization grpo iclr-2026 hateful-meme-detection multimodal-rl

• Updated Mar 1, 2026
• Python

[TMLR] Dual Caption Preference Optimization

alignment diffusion-models rlhf preference-optimization

• Updated Feb 12, 2025
• Python

[ICML 2025] TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization

alignment preference-learning large-language-models llm rlhf preference-alignment direct-preference-optimization preference-optimization

• Updated Jul 15, 2025
• Python

Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model

alignment large-language-models preference-optimization

• Updated Apr 23, 2025
• Python

Convex Optimization for Alignment and Preference Learning on a Single GPU

convex-optimization convex preference-learning llms preference-optimization

• Updated Jul 4, 2026
• Python

A curated list of reinforcement learning, preference optimization, and reward-driven post-training and alignment methods for video generation.

awesome reinforcement-learning awesome-list video-generation dpo diffusion-models text-to-video rlhf preference-optimization grpo

• Updated Oct 6, 2026

This is my personal home rig for serious LLM experimentation. I built it to test models head-to-head, create custom evaluation rubrics, automatically improve prompts based on the previous run’s results, and generate high-quality synthetic training data. Everything runs locally first (Ollama by default), with optional cloud support. logged locally.

frontend-web ab-testing evaluation-metrics human-in-the-loop evaluation-framework grading-system local-first synthetic-data-generation dataset-curation ollama llm-evaluation prompt-optimization preference-optimization rubric-based-evaluation

• Updated Jun 15, 2026
• Python

DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents (project page; code coming soon)

dpo llm-agents context-compression preference-optimization long-horizon-agents agent-loops

• Updated Oct 22, 2026
• HTML

DPO/SafeDPO/OPAD training + eval for teaching tool-using LLMs to refuse falsely-benign MCP exploits

mcp alignment red-teaming dpo trl rlhf llm-safety jailbreak-defense preference-optimization model-context-protocol

• Updated Jul 8, 2026
• Python

🩺 Token-Level Flow-Guided Preference Optimization for Efficient Test-Time Alignment (AAAI 2026)

transformers pytorch alignment lora llm qwen preference-optimization aaai2026 test-time-alignment

• Updated Jan 17, 2026
• Python

Execution-guided Retrieved-Schema Text-to-SQL post-training pipeline with SQLite feedback, Column-Minimal Repair, LoRA SFT/DPO, preference mining, pair-quality filtering, and clean-split Spider evaluation.

spider lora sql-agent post-training sft dpo self-repair llm preference-optimization test2sql

• Updated Jul 9, 2026
• Python

One library for the frontier LLM training stack — pre-training, CPT/DAPT, SFT, DPO/IPO/KTO/ORPO/SimPO/CPO, RLHF, RLVR+GRPO, agentic RL, distillation & self-play, LoRA/QLoRA — behind one config-driven, registry-based API. import trainall is torch-free.

reinforcement-learning deep-learning transformers pytorch lora knowledge-distillation fine-tuning ppo peft self-play sft dpo large-language-models llm rlhf qlora agentic-ai preference-optimization grpo rlvr

• Updated Jun 30, 2026
• Python