name | about | labels |
---|---|---|
Bug Report | Use this template for reporting a bug | kind/bug |
在微调模型时候,出现了因为BaseTunerLayer类的get_base_layer()方法的循环引用,导致调用Python对象时超过的最大递归深度。
Ascend
/GPU
/CPU
) / 硬件环境: GPUPlease delete the backend not involved / 请删除不涉及的后端:
/device ascend/GPU/CPU/kirin/等其他芯片
Software Environment / 软件环境 (Mandatory / 必填):
-- MindSpore version (e.g., 1.7.0.Bxxx) :2.2.11
-- Python version (e.g., Python 3.7.5) :3.9
Excute Mode / 执行模式 (Mandatory / 必填)(PyNative
/Graph
):
对MindNLP进行编包装包
然后在mindnlp/llm/peft/里面选择一个模型放到一个新的目录下
使用有MindNLP包的环境执行里面的train.py进行微调训练,例子如下:
aiyuxin@mindspore:/data1/yy/train_gpt_bigcode$ ./run_peft.sh
# 出问题的内部代码
class BaseTunerLayer(ABC):
def get_base_layer(self) -> nn.Cell:
base_layer = self
while hasattr(base_layer, "base_layer"):
base_layer = base_layer.base_layer
return base_layer
@property
def weight(self) -> Tensor:
# This is required for some transformers code, e.g. for T5, weight is accessed as:
# self.wo.weight
# where "wo" is the adapter layer.
# https://github.com/huggingface/transformers/blob/78f6ed6c70b29c1560780e3869a7ad4c6b3d2710/src/transformers
# /models/t5/modeling_t5.py#L292
base_layer = self.get_base_layer()
weight = base_layer.weight
return weight
aiyuxin@mindspore:/data1/yy/train_gpt_bigcode$ ./run_peft.sh
(yy_env) daiyuxin@mindspore:/data1/yy/train_gpt_bigcode$ ./run_peft.sh
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Loading model cost 0.614 seconds.
Prefix dict has been built successfully.
/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/datasets/load.py:2516: FutureWarning: 'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=<use_auth_token>' instead.
warnings.warn(
Size of the train set: 5875. Size of the validation set: 30
0%| | 0/400 [00:00<?, ?it/s]Token indices sequence length is longer than the specified maximum sequence length for this model (7483 > 2048). Running this sequence through the model will result in indexing errors
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 400/400 [00:03<00:00, 112.99it/s]
The character to token ratio of the dataset is: 3.31
FIM is not supported by tokenizer, disabling FIM
FIM is not supported by tokenizer, disabling FIM
Traceback (most recent call last):
File "/data1/yy/train_gpt_bigcode/train.py", line 187, in
main(args)
File "/data1/yy/train_gpt_bigcode/train.py", line 171, in main
model = get_peft_model(model, peft_config)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/mapping.py", line 97, in get_peft_model
return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](model, peft_config, adapter_name=adapter_name)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/peft_model.py", line 390, in init
super().init(model, peft_config, adapter_name)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/peft_model.py", line 80, in init
self.base_model = PEFT_TYPE_TO_MODEL_MAPPING[peft_config.peft_type](
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/lora.py", line 161, in init
super().init(model, config, adapter_name)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/tuners_utils.py", line 100, in init
self.inject_adapter(self.model, adapter_name)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/tuners_utils.py", line 250, in inject_adapter
self._create_and_replace(peft_config, adapter_name, target, target_name, parent, **optionnal_kwargs)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/lora.py", line 262, in _create_and_replace
new_module = self._create_new_module(lora_config, adapter_name, target, **kwargs)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/lora.py", line 539, in _create_new_module
new_module = Linear(adapter_name, in_features, out_features, has_bias=bias, **kwargs)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/lora.py", line 691, in init
nn.Dense.init(self, in_features, out_features, **kwargs)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/injection.py", line 587, in init
self.weight = Parameter(initializer(
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindspore/nn/cell.py", line 966, in setattr
self._set_attr_for_parameter(name, value)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindspore/nn/cell.py", line 859, in _set_attr_for_parameter
self.insert_param_to_cell(name, value)
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindspore/nn/cell.py", line 1224, in insert_param_to_cell
if hasattr(self, param_name) and param_name not in self._params:
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/tuners_utils.py", line 329, in weight
weight = base_layer.weight
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/tuners_utils.py", line 329, in weight
weight = base_layer.weight
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/tuners_utils.py", line 329, in weight
weight = base_layer.weight
[Previous line repeated 974 more times]
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/tuners_utils.py", line 328, in weight
base_layer = self.get_base_layer()
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindnlp/peft/tuners/tuners_utils.py", line 317, in get_base_layer
while hasattr(base_layer, "base_layer"):
File "/home/daiyuxin/miniconda3/envs/yy_env/lib/python3.9/site-packages/mindspore/nn/cell.py", line 440, in getattr
"The '{}' object has no attribute '{}'.".format(type(self).name, name)
RecursionError: maximum recursion depth exceeded while calling a Python object
Please assign maintainer to check this issue.
请为此issue分配处理人。
@fangwenyi @chengxiaoli @Shawny
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
感谢您的提问,您可以评论//mindspore-assistant更快获取帮助:
登录 后才可以发表评论