# Capacity-Aware Inference

## Requirements

To install the required dependencies, run:

``bash
pip install -r requirements.txt 
··

## Usage

1. **Token Drop**  
   Capacity‑constrained Hugging Face models are located in  
   `./model_hf/`

2. **Expanded Drop (Megatron deployment)**  
   For Megatron‑LM users, an Expanded Drop implementation is provided in  
   `Megatron-LM-main-Expanded-Drop/`

3. **Evaluation**  
   We benchmark models with the following suites:  
   - [lm‑evaluation‑harness](https://github.com/EleutherAI/lm-evaluation-harness) – standard NLP tasks  
   - [VLMEvalKit](https://github.com/opencompass/VLMEvalKit) – multimodal tasks
