ACToR is an Adaptive Critical Token-aware Retrieval framework designed for repository-level code generation tasks.
ACToR/
āāā README.md # Project documentation
āāā classifiers/ # Critical Token classifiers
āāā weights/ # Position-aware weights
āāā datasets/ # Benchmarks and training datasets
āāā repositories/ # Code repositories for training and testing
āāā src/ # Source code directory
āāā pipeline.py
āāā critoken.py
āāā train.py
āāā data/ # Data processing pipeline
ā āāā __init__.py
ā āāā repo.py
ā āāā task.py
ā āāā process/ # Data processing modules
ā āāā __init__.py
ā āāā data.py
ā āāā window.py
ā āāā vector.py
ā āāā search.py
ā āāā prompt.py
ā āāā utils.py
āāā server/ # Model server integration
āāā __init__.py
āāā classifier.py
āāā llm.py
Requirements:
- Python 3.12.11 (recommended)
Installation:
pip install -r requirements.txt
Organize code repositories in the repositories/ directory following this structure:
repositories/ āāā codereval/ ā āāā python/ # CoderEval Python repositories āāā repoexec/ ā āāā test-app/ # RepoExec test repositories āāā repost/ āāā train/ # RepoST training repositories
Setup Instructions:
- CoderEval: Clone CoderEval Python repositories to
repositories/codereval/python/ - RepoExec: Clone RepoExec test repositories to
repositories/repoexec/test-app/ - RepoST: Clone RepoST training repositories to
repositories/repost/train/(for training reproduction)
Note: More Details are available in
src/data/process/utils.py
- Repo Context Prep
python src/pipeline.py repo --benchmark 'repoexec' # alternative 'codereval-python', 'repost_train'
- (Optional) Training Data Prep
python src/pipeline.py train --task_type 'data' --model_name 'model_name' # e.g. codellama-7b-hf
- (Optional) Classifier Training
python src/pipeline.py train --task_type 'classifier' --model_name 'model_name' # e.g. codellama-7b-hf
- Run Adaptive Critical Token-aware Retrieval augmented generation.
python src/pipeline.py task token --model_name 'model_name' --benchmark 'repoexec'
