Plan

模型选择

Qwen/Qwen2.5-7B-Instruct-1M

环境和工具

环境

Python3.10

虚拟环境

Conda

Accelerated PyTorch training on Mac

由于 Mac 无法使用 NVIDIA 的 GPU 加速，即 CUDA，不过好在 Apple 官网有训练加速的帮助文档

curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh

sh Miniconda3-latest-MacOSX-arm64.sh

训练工具

Transformers

Install

从 Conda 的 conda-forge 频道安装

conda install conda-forge::transformers

Hugging Face 的加速库，用于在单机器多 GPU or 多机器多 GPU 分布式训练

conda install -c conda-forge accelerate

Pytorch

Install

conda install pytorch torchvision torchaudio -c pytorch-nightly

train

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Qwen/Qwen2.5-7B-Instruct-1M"

model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype="auto", device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Give me a short introduction to large language model."
messages = [
    {
        "role": "system",
        "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant.",
    },
    {"role": "user", "content": prompt},
]
text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(**model_inputs, max_new_tokens=512)
generated_ids = [
    output_ids[len(input_ids) :]
    for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]