# graph-4.0 **Repository Path**: itluma2008/graph-4.0 ## Basic Information - **Project Name**: graph-4.0 - **Description**: 111111111111111111111111111 - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: main - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 2 - **Forks**: 1 - **Created**: 2026-07-23 - **Last Updated**: 2026-08-03 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README 1、面试准备：Transformer架构 1.1 为什么会出现：NLP->RNN->LSTM,注意力机制 1.2 编码器和解码器架构：Seq2Seq 编码器 解码器 1.3 核心组件 自注意力机制：QKV 多头注意力 位置编码 前馈神经网络FFN+残差连接+层归一化 { "id":1, "text":"小米12S Ultra 骁龙8+旗舰处理器 徕卡光学镜头 2K超视感屏 120Hz高刷 67W快充 12GB+256GB 冷杉绿 5G手机", "Comments":[], "entities":[ { "id":236, "label":"品牌","start_offset":0,"end_offset":2 }, {"id":237,"label":"商品","start_offset":0,"end_offset":11}, {"id":238,"label":"运行内存","start_offset":50,"end_offset":54}, {"id":239,"label":"机身内存","start_offset":55,"end_offset":60}, {"id":240,"label":"版本","start_offset":50,"end_offset":60}, {"id":241,"label":"颜色","start_offset":61,"end_offset":64}, {"id":242,"label":"品类","start_offset":67,"end_offset":69}], "relations":[ {"id":18,"from_id":237,"to_id":241,"type":"颜色"}, {"id":19,"from_id":237,"to_id":242,"type":"品类"}, {"id":20,"from_id":237,"to_id":236,"type":"品牌"}, {"id":21,"from_id":237,"to_id":238,"type":"运行内存"}, {"id":22,"from_id":237,"to_id":239,"type":"机身内存"}, {"id":23,"from_id":237,"to_id":240,"type":"版本"} ] } { "content": "购买客户尊享会员服务免||换|保7天无理由退货质量问题15天换新下单享3年质保认准官方旗舰店买得放心,用得更加安心!正连官方正品全国联保", "result_list": [], "prompt": "尺码" } { "content": "小米【国家补贴】REDMI Turbo4 Pro 新品上市 小米手机 小米5G手机红米turbo 7550mAh长续航电池 白色 12GB+256GB", "result_list": [ { "text": "小米", "start": 0, "end": 2}, { "text": "REDMI", "start": 8, "end": 13 }, { "text": "小米", "start": 30, "end": 32 }, { "text": "小米", "start": 35, "end": 37 }, { "text": "红米", "start": 41, "end": 43 } ], "prompt": "品牌"} select sku_info.id as sku_id, sku_info.sku_name, spu_info.spu_name, bc3.name category3_name, bc2.name category2_name,bc1.name category1_name,bt.tm_name trademark_name from sku_info left join spu_info on sku_info.spu_id=spu_info.id left join base_category3 bc3 on spu_info.category3_id = bc3.id left join base_category2 bc2 on bc3.category2_id=bc2.id left join base_category1 bc1 on bc2.category1_id=bc1.id left join base_trademark bt on spu_info.tm_id=bt.id 开始意图分类模型微调... 数据: d:\LLM\graph-2.0\data\intent_classify\data.csv ssify_bert.py 开始意图分类模型微调... 数据: d:\LLM\graph-2.0\data\intent_classify\data.csv 开始意图分类模型微调... 数据: d:\LLM\graph-2.0\data\intent_classify\data.csv 本地模型: d:\LLM\graph-2.0\pretrained\bert-base-chinese 地模型校验通过: d:\LLM\graph-2.0\pretrained\bert-base-chinese 有效数据: 2737 条 训练集: 2189, 测试集: 548 Some weights of BertForSequenceClassification were not initialized from the model checkpoint at d:\LLM\graph-2.0\pretrained\bert-base-chinese and are newly initialized: ['classifier.bias', 'classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. d:\LLM\graph-2.0\src\scripts\train_intent_classify_bert.py:155: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `Trainer.__init__`. Use `processing_class` instead. trainer = Trainer( 训练开始... {'loss': 2.8262, 'grad_norm': 76.39459228515625, 'learning_rate': 2.6086956521739132e-06, 'epoch': 0.07} {'loss': 2.7149, 'grad_norm': 9.616747856140137, 'learning_rate': 5.507246376811595e-06, 'epoch': 0.15} {'loss': 2.6788, 'grad_norm': 16.78179168701172, 'learning_rate': 8.405797101449275e-06, 'epoch': 0.22} {'loss': 2.5041, 'grad_norm': 10.073838233947754, 'learning_rate': 1.1304347826086957e-05, 'epoch': 0.29} {'loss': 2.3202, 'grad_norm': 8.908248901367188, 'learning_rate': 1.420289855072464e-05, 'epoch': 0.36} {'loss': 2.1231, 'grad_norm': 14.957330703735352, 'learning_rate': 1.710144927536232e-05, 'epoch': 0.44} {'loss': 1.8936, 'grad_norm': 11.752448081970215, 'learning_rate': 2e-05, 'epoch': 0.51} {'loss': 1.6136, 'grad_norm': 12.445443153381348, 'learning_rate': 1.9675324675324675e-05, 'epoch': 0.58} {'loss': 1.3976, 'grad_norm': 8.819336891174316, 'learning_rate': 1.9350649350649354e-05, 'epoch': 0.66} {'loss': 1.1092, 'grad_norm': 12.430877685546875, 'learning_rate': 1.9025974025974027e-05, 'epoch': 0.73} {'loss': 0.8649, 'grad_norm': 8.518926620483398, 'learning_rate': 1.8701298701298704e-05, 'epoch': 0.8} {'loss': 0.8042, 'grad_norm': 8.577935218811035, 'learning_rate': 1.8376623376623377e-05, 'epoch': 0.88} {'loss': 0.7106, 'grad_norm': 9.354266166687012, 'learning_rate': 1.8051948051948053e-05, 'epoch': 0.95} {'eval_loss': 0.4480431079864502, 'eval_accuracy': 0.9343065693430657, 'eval_f1_macro': 0.9271924129054662, 'eval_f1_weighted': 0.9348259667882889, 'eval_runtime': 19.1877, 'eval_samples_per_second': 28.56, 'eval_steps_per_second': 1.824, 'epoch': 1.0} {'loss': 0.5514, 'grad_norm': 5.206634044647217, 'learning_rate': 1.772727272727273e-05, 'epoch': 1.02} {'loss': 0.4911, 'grad_norm': 14.6822509765625, 'learning_rate': 1.7402597402597403e-05, 'epoch': 1.09} {'loss': 0.3435, 'grad_norm': 7.826479434967041, 'learning_rate': 1.707792207792208e-05, 'epoch': 1.17} {'loss': 0.3313, 'grad_norm': 5.271589756011963, 'learning_rate': 1.6753246753246756e-05, 'epoch': 1.24} {'loss': 0.2643, 'grad_norm': 6.203798294067383, 'learning_rate': 1.642857142857143e-05, 'epoch': 1.31} {'loss': 0.2687, 'grad_norm': 1.9527854919433594, 'learning_rate': 1.6103896103896105e-05, 'epoch': 1.39} {'loss': 0.2154, 'grad_norm': 5.274769306182861, 'learning_rate': 1.577922077922078e-05, 'epoch': 1.46} {'loss': 0.2278, 'grad_norm': 3.9496371746063232, 'learning_rate': 1.5454545454545454e-05, 'epoch': 1.53} {'loss': 0.2123, 'grad_norm': 3.3523082733154297, 'learning_rate': 1.512987012987013e-05, 'epoch': 1.61} {'loss': 0.1428, 'grad_norm': 1.093186616897583, 'learning_rate': 1.4805194805194807e-05, 'epoch': 1.68} {'loss': 0.1593, 'grad_norm': 8.381776809692383, 'learning_rate': 1.4480519480519482e-05, 'epoch': 1.75} {'loss': 0.1282, 'grad_norm': 1.4463309049606323, 'learning_rate': 1.4155844155844157e-05, 'epoch': 1.82} {'loss': 0.1605, 'grad_norm': 10.933579444885254, 'learning_rate': 1.3831168831168833e-05, 'epoch': 1.9} {'loss': 0.1496, 'grad_norm': 14.949271202087402, 'learning_rate': 1.3506493506493508e-05, 'epoch': 1.97} {'eval_loss': 0.13161982595920563, 'eval_accuracy': 0.968978102189781, 'eval_f1_macro': 0.9663999380659873, 'eval_f1_weighted': 0.9690621417318153, 'eval_runtime': 19.0286, 'eval_samples_per_second': 28.799, 'eval_steps_per_second': 1.839, 'epoch': 2.0} {'loss': 0.1241, 'grad_norm': 5.827308177947998, 'learning_rate': 1.3181818181818183e-05, 'epoch': 2.04} {'loss': 0.0835, 'grad_norm': 1.7436103820800781, 'learning_rate': 1.2857142857142859e-05, 'epoch': 2.12} {'loss': 0.0743, 'grad_norm': 11.61787223815918, 'learning_rate': 1.2532467532467534e-05, 'epoch': 2.19} {'loss': 0.0567, 'grad_norm': 0.597287118434906, 'learning_rate': 1.2207792207792208e-05, 'epoch': 2.26} {'loss': 0.0808, 'grad_norm': 6.228579044342041, 'learning_rate': 1.1883116883116883e-05, 'epoch': 2.34} {'loss': 0.0345, 'grad_norm': 1.8944127559661865, 'learning_rate': 1.155844155844156e-05, 'epoch': 2.41} {'loss': 0.0628, 'grad_norm': 1.1042351722717285, 'learning_rate': 1.1233766233766234e-05, 'epoch': 2.48} {'loss': 0.0412, 'grad_norm': 0.596706748008728, 'learning_rate': 1.0909090909090909e-05, 'epoch': 2.55} {'loss': 0.0326, 'grad_norm': 0.4639027416706085, 'learning_rate': 1.0584415584415585e-05, 'epoch': 2.63} {'loss': 0.0925, 'grad_norm': 1.129813551902771, 'learning_rate': 1.025974025974026e-05, 'epoch': 2.7} {'loss': 0.0251, 'grad_norm': 0.20756179094314575, 'learning_rate': 9.935064935064936e-06, 'epoch': 2.77} {'loss': 0.0696, 'grad_norm': 0.3769126534461975, 'learning_rate': 9.610389610389611e-06, 'epoch': 2.85} {'loss': 0.0446, 'grad_norm': 0.4931008219718933, 'learning_rate': 9.285714285714288e-06, 'epoch': 2.92} {'loss': 0.0196, 'grad_norm': 0.27558162808418274, 'learning_rate': 8.96103896103896e-06, 'epoch': 2.99} {'eval_loss': 0.0857277438044548, 'eval_accuracy': 0.9835766423357665, 'eval_f1_macro': 0.9822139790437375, 'eval_f1_weighted': 0.9834985642138303, 'eval_runtime': 18.6494, 'eval_samples_per_second': 29.384, 'eval_steps_per_second': 1.877, 'epoch': 3.0} {'loss': 0.0154, 'grad_norm': 0.2209237962961197, 'learning_rate': 8.636363636363637e-06, 'epoch': 3.07} {'loss': 0.0405, 'grad_norm': 0.3526376485824585, 'learning_rate': 8.311688311688313e-06, 'epoch': 3.14} {'loss': 0.0157, 'grad_norm': 0.20186173915863037, 'learning_rate': 7.987012987012988e-06, 'epoch': 3.21} {'loss': 0.0382, 'grad_norm': 0.155490905046463, 'learning_rate': 7.662337662337663e-06, 'epoch': 3.28} {'loss': 0.0172, 'grad_norm': 4.450871467590332, 'learning_rate': 7.3376623376623375e-06, 'epoch': 3.36} {'loss': 0.0143, 'grad_norm': 0.1667102873325348, 'learning_rate': 7.012987012987014e-06, 'epoch': 3.43} {'loss': 0.0264, 'grad_norm': 0.12170907855033875, 'learning_rate': 6.688311688311689e-06, 'epoch': 3.5} {'loss': 0.0377, 'grad_norm': 3.6297457218170166, 'learning_rate': 6.363636363636364e-06, 'epoch': 3.58} {'loss': 0.0157, 'grad_norm': 1.4221009016036987, 'learning_rate': 6.03896103896104e-06, 'epoch': 3.65} {'loss': 0.0138, 'grad_norm': 0.13010960817337036, 'learning_rate': 5.7142857142857145e-06, 'epoch': 3.72} {'loss': 0.0119, 'grad_norm': 0.22644568979740143, 'learning_rate': 5.38961038961039e-06, 'epoch': 3.8} {'loss': 0.0354, 'grad_norm': 0.1394914835691452, 'learning_rate': 5.064935064935065e-06, 'epoch': 3.87} {'loss': 0.0121, 'grad_norm': 0.17470230162143707, 'learning_rate': 4.74025974025974e-06, 'epoch': 3.94} {'eval_loss': 0.083995021879673, 'eval_accuracy': 0.9835766423357665, 'eval_f1_macro': 0.9822744149024382, 'eval_f1_weighted': 0.9835517940765971, 'eval_runtime': 18.5474, 'eval_samples_per_second': 29.546, 'eval_steps_per_second': 1.887, 'epoch': 4.0} {'loss': 0.0225, 'grad_norm': 10.152770042419434, 'learning_rate': 4.415584415584416e-06, 'epoch': 4.01} {'loss': 0.0107, 'grad_norm': 0.1512569934129715, 'learning_rate': 4.0909090909090915e-06, 'epoch': 4.09} {'loss': 0.035, 'grad_norm': 8.450490951538086, 'learning_rate': 3.7662337662337666e-06, 'epoch': 4.16} {'loss': 0.0107, 'grad_norm': 0.12015271931886673, 'learning_rate': 3.4415584415584418e-06, 'epoch': 4.23} {'loss': 0.0101, 'grad_norm': 0.11243987083435059, 'learning_rate': 3.116883116883117e-06, 'epoch': 4.31} {'loss': 0.0099, 'grad_norm': 0.10172305256128311, 'learning_rate': 2.7922077922077925e-06, 'epoch': 4.38} {'loss': 0.0247, 'grad_norm': 0.09920848906040192, 'learning_rate': 2.4675324675324676e-06, 'epoch': 4.45} {'loss': 0.0355, 'grad_norm': 0.17327748239040375, 'learning_rate': 2.1428571428571427e-06, 'epoch': 4.53} {'loss': 0.0096, 'grad_norm': 0.12368848919868469, 'learning_rate': 1.8181818181818183e-06, 'epoch': 4.6} {'loss': 0.013, 'grad_norm': 0.12540975213050842, 'learning_rate': 1.4935064935064936e-06, 'epoch': 4.67} {'loss': 0.0095, 'grad_norm': 0.8675905466079712, 'learning_rate': 1.168831168831169e-06, 'epoch': 4.74} {'loss': 0.0142, 'grad_norm': 0.12608473002910614, 'learning_rate': 8.441558441558442e-07, 'epoch': 4.82} {'loss': 0.0092, 'grad_norm': 0.17575152218341827, 'learning_rate': 5.194805194805196e-07, 'epoch': 4.89} {'loss': 0.0101, 'grad_norm': 0.16502442955970764, 'learning_rate': 1.948051948051948e-07, 'epoch': 4.96} {'eval_loss': 0.08764933049678802, 'eval_accuracy': 0.9835766423357665, 'eval_f1_macro': 0.982275284815621, 'eval_f1_weighted': 0.9834981451971787, 'eval_runtime': 19.0986, 'eval_samples_per_second': 28.693, 'eval_steps_per_second': 1.833, 'epoch': 5.0} {'train_runtime': 1502.8492, 'train_samples_per_second': 7.283, 'train_steps_per_second': 0.456, 'train_loss': 0.4171249226116351, 'epoch': 5.0} 100%|███████████████████████████████| 685/685 [25:02<00:00, 2.19s/it] 训练完成: {'train_runtime': 1502.8492, 'train_samples_per_second': 7.283, 'train_steps_per_second': 0.456, 'total_flos': 118176831335268.0, 'train_loss': 0.4171249226116351, 'epoch': 5.0} 测试集评估: 100%|███████████████████████████████| 35/35 [00:18<00:00, 1.88it/s] eval_loss: 0.0876 eval_accuracy: 0.9836 eval_f1_macro: 0.9823 eval_f1_weighted: 0.9835 eval_runtime: 19.1249 eval_samples_per_second: 28.6540 eval_steps_per_second: 1.8300 epoch: 5.0000 最佳模型已保存: d:\LLM\graph-2.0\checkpoint\intent_classify\best_model