# LocFT-BF

This repository hosts the code and data for the paper: **Fine-tuning Done Right in Model Editing**.

#### Requirements:

- **Environment**: `requirements.txt` (Please use Python 3.9+ for this repository)

  ```shell
  pip install -r requirements.txt
  ```

- **Large Language Models to Edit**: 

  You have three options to load LLMs for editing:

  1. Download the LLMs you want to edit and put them in `./hugging_cache/` 

  2. Specify the path to your existing LLMs in the configuration files, e.g.,  `./hparams/FT/llama-7b.yaml`:

     ```yaml
     model_name: "your/path/to/LLMs"
     ```

  3. Provide the model name in the configuration files and the program will automatically employ `from_pretrained` to load the model:

     ```yaml
     model_name: "meta-llama/Llama-2-7b"
     ```

- **Datasets**: The data of ZsRE, COUNTERFACT, and WikiBigEdit are provided in `./data/`

#### Perform Baseline Editing Methods

```shell
python edit.py --editing_method ROME --hparams_dir ./hparams/ROME/llama3-8b.yaml --data_path ./data/zsre_mend_eval_3k.json --datatype zsre --ds_size 3000 --batch_edit True --device 0 --down_eval True --save_model_dir memit_llama3_wiki3k_bs3k
```

#### Perform Our LocFT-BF

```shell
python locft-bf.py --model llama3-8b --layer 22 --rewrite_module model.layers.{}.mlp.down_proj.weight --batch_size 32 --device 0 --save_model_dir ft_llama3_zsre3k_bs32
```
