Platform For AI:Fine-tuning Llama3-8B

Background

Llama3 is the latest model in the Llama series, released by Meta in April 2024. Trained on over 15 trillion tokens — more than seven times the Llama2 dataset — it supports an 8K context length and uses an improved tokenizer with a 128K token vocabulary for more precise handling of complex contexts and specialized terminology.

Llama3 is available in 8B and 70B parameter sizes, each with a pretrained version and an instruction-tuned version.

• 8B version

  Ideal for consumer-grade GPUs, this version enables rapid deployment on systems with limited computing resources and suits applications that require fast response times and cost-effectiveness.

  • Meta-Llama-3-8b: The 8B pretrained model.

  • Meta-Llama-3-8b-instruct: The instruction-tuned version of the 8B model.

• 70B version

  Designed for large-scale AI applications, this version handles highly complex tasks that demand superior model performance.

  • Meta-Llama-3-70b: The 70B pretrained model.

  • Meta-Llama-3-70b-instruct: The instruction-tuned version of the 70B model.

Prerequisites

• Create a workspace. For more information, see Create and manage workspaces.

• Create a DSW instance. Configure the following key parameters. For more information, see Create a DSW instance.

  • Instance type: Use a GPU instance with at least 16 GB of GPU memory, such as a V100.

  • Python environment: Use Python 3.9 or later.

  • Image: In the Image URL field, enter dsw-registry-vpc.REGION.cr.aliyuncs.com/pai-training-algorithm/llm_deepspeed_peft:v0.0.3. Replace REGION with the ID of the region where your DSW instance is located, such as cn-hangzhou or cn-shanghai. For a complete list of regions and their corresponding region IDs, see the following table.

    Region	Region ID
    China (Hangzhou)	cn-hangzhou
    China (Shanghai)	cn-shanghai
    China (Beijing)	cn-beijing
    China (Shenzhen)	cn-shenzhen

• Before you use the Llama3 large language model, read the official Meta license.

  Note

  If you cannot access the page, you may need to configure a proxy and try again.

Step 1: Download the model

Method 1: Download in DSW

1. Open the DSW development environment.

   1. Log on to the PAI console.

   2. In the upper-left corner, select the region where your DSW instance is located.

   3. In the navigation pane on the left, click Workspaces. On the Workspaces page, click the name of the default workspace.

   4. In the navigation pane on the left, choose Model Training > Data Science Workshop (DSW).

   5. Find the instance that you want to use and click Open in the Actions column to enter the DSW development environment.

2. On the Launcher page, in the Notebook section, click Python 3.

3. Run the following code to download the model files. The code automatically selects an appropriate download source and downloads the model files to the current directory.

   ! pip install modelscope==1.32.0 transformers==4.37.0

   from modelscope.hub.snapshot_download import snapshot_download
   snapshot_download('LLM-Research/Meta-Llama-3-8B-Instruct', cache_dir='.', revision='master')

Method 2: Download from Meta

Go to the Meta website to request and download the model.

Step 2: Prepare the dataset

This example uses an English poetry dataset to fine-tune the Llama 3 model and improve its poetry generation. Run the following command in a DSW Notebook to download the training dataset.

!wget https://atp-modelzoo-sh.oss-cn-shanghai.aliyuncs.com/tutorial/llm_instruct/en_poetry_train.json

You can also prepare your own dataset in the same format, adapted to your specific use case.

Step 3: Fine-tune the model

Lightweight LoRA training

This tutorial uses the training script /ml/code/sft.py to perform lightweight Low-Rank Adaptation (LoRA) training. After training, the model parameters are quantized to reduce GPU memory usage for inference.

When you run the accelerate launch command, it starts the specified Python script with the given parameters and performs training based on the settings in the multi_gpu.yaml configuration file, using the available compute resources.

! accelerate launch --num_processes 1 --config_file /ml/code/multi_gpu.yaml /ml/code/sft.py \
    --model_name  ./LLM-Research/Meta-Llama-3-8B-Instruct/ \
    --model_type llama \
    --train_dataset_name en_poetry_train.json \
    --num_train_epochs 3 \
    --batch_size 8 \
    --seq_length 128 \
    --learning_rate 5e-4 \
    --lr_scheduler_type linear \
    --target_modules k_proj o_proj q_proj v_proj \
    --output_dir lora_model/ \
    --apply_chat_template \
    --use_peft \
    --load_in_4bit \
    --peft_lora_r 32 \
    --peft_lora_alpha 32 

Merge LoRA weights

Run the following command to merge the LoRA weights with the pretrained model.

! RANK=0 python /ml/code/convert.py \
    --model_name ./LLM-Research/Meta-Llama-3-8B-Instruct/ \
    --model_type llama \
    --output_dir trained_model/ \
    --adapter_dir lora_model/

The following describes the parameters used in this example:

• RANK=0: The RANK environment variable indicates the rank of the current process in distributed training. A value of 0 means a single process or the main process.

• python /ml/code/convert.py: Runs the convert.py script for weight conversion.

• --model_name ./LLM-Research/Meta-Llama-3-8B-Instruct/: The path to the pretrained model.

• --model_type llama: The model type.

• --output_dir trained_model/: The directory for the converted model and weights.

• --adapter_dir lora_model/: The directory containing the LoRA adapter weights.

Step 4: Run inference

Run the following code to perform inference and verify the fine-tuning results. In this example, the model is asked to write a poem about spring:

import torch, transformers

# model_id = "./LLM-Research/Meta-Llama-3-8B-Instruct/"
model_id = "./trained_model/"
pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device="cuda",
)

messages = [
    {"role": "user", "content": "Write a poem on a topic 'spring' "},
]

prompt = pipeline.tokenizer.apply_chat_template(
        messages, 
        tokenize=False, 
        add_generation_prompt=True
)

terminators = [
    pipeline.tokenizer.eos_token_id,
    pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

outputs = pipeline(
    prompt,
    max_new_tokens=1024,
    eos_token_id=terminators,
    do_sample=True,
    temperature=0.6,
    top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])

The following is a sample response. The fine-tuned model can now generate poetry:

Here's a poem on the topic of "Spring":

As winter's chill begins to fade,
The earth awakens from its shade,
And spring's sweet breath begins to blow,
Bringing life to all that's cold and slow.

The trees regain their vibrant hue,
And flowers bloom, both old and new,
Their petals dancing in the breeze,
As sunshine warms the world with ease.

The air is filled with sweet perfume,
As blossoms burst forth in their room,
And robins sing their morning song,
As spring's awakening is strong.

The world is fresh, and new, and bright,
As spring's warm light begins to take flight,
And all around, new life unfolds,
As winter's grip begins to grow old.

So let us bask in spring's warm rays,
And let our spirits soar and sway,
For in this season, we're reborn,
And all around, new life is sworn.

I hope you enjoy it!

Step 5: Deploy the model

You can upload the fine-tuned model weights to Object Storage Service (OSS) and deploy the model as a service using Elastic Algorithm Service (EAS) ChatLLM. For more information, see Deploy LLM applications in EAS.

Appendix: Run Llama3 from DSW Gallery

DSW Gallery provides pre-built Llama3 Notebook examples. You can open an example in your DSW instance and run it directly, or modify it to suit your needs. For more information, see Notebook Gallery.

FAQ

References

For important ChatLLM-WebUI release information, see the ChatLLM-WebUI release notes.