1 Star 5 Fork 3

之江天枢 / Atlas_APP

Create your Gitee Account
Explore and code with more than 6 million developers,Free private repositories !:)
Sign up
Clone or download
Cancel
Notice: Creating folder will generate an empty file .keep, because not support in Git
Loading...
README.md

KAmalEngine

KAmalEngine (KAE) aims at building a lightweight algorithm package for Knowledge Amalgamation and Transferability Estimation.

Features

  • Knowledge amalgamation and distillation algorithms

  • Easy-to-use Interfaces for multi-tasking training

  • Deep model transferability estimation based on attribution maps

  • Predefined callbacks & metrics for evaluation and visualization

Algorithms

Student Becoming the Master (Task Branching)

Knowledge amalgamation for multiple teachers by feature projection.
Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More (CVPR 2019)

Common Feature Learning

Extract common features from multiple teacher models.
Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning (IJCAI 2019)

Feature Space Common Space
cfl-feature-space cfl-feature-space

Amalgamating Knowledge towards Comprehensive Classification

Layer-wise amalgamation
Amalgamating Knowledge towards Comprehensive Classification (AAAI 2019)

Recombination

Build a new multi-task model by combining & pruning weight matrixs from distinct-task teachers.

Deep model transferability from attribution maps

Estimate model transferability using attribution map.

DEPARA: Deep Attribution Graph for Deep Knowledge Transferability

Constructing attribution graph for model transferability estimation.

Transferability graph on classification models

Team

Developed by Zhejiang Lab and VIPA Lab from Zhejiang University.

Citation

@inproceedings{shen2019amalgamating,
  author={Shen, Chengchao and Wang, Xinchao and Song, Jie and Sun, Li and Song, Mingli},
  title={Amalgamating Knowledge towards Comprehensive Classification},
  booktitle={AAAI Conference on Artificial Intelligence (AAAI)},
  pages={3068--3075},
  year={2019}
}
@inproceedings{ye2019student,
  title={Student Becoming the Master: Knowledge Amalgamation for Joint Scene Parsing, Depth Estimation, and More},
  author={Ye, Jingwen and Ji, Yixin and Wang, Xinchao and Ou, Kairi and Tao, Dapeng and Song, Mingli},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={2829--2838},
  year={2019}
}
@inproceedings{luo2019knowledge,
  title={Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning},
  author={Luo, Sihui and Wang, Xinchao and Fang, Gongfan and Hu, Yao and Tao, Dapeng and Song, Mingli},
  booktitle={Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2019},
}
@inproceedings{shen2019customizing,
  author={Shen, Chengchao and Xue, Mengqi and Wang, Xinchao and Song, Jie and Sun, Li and Song, Mingli},
  title={Customizing student networks from heterogeneous teachers via adaptive knowledge amalgamation},
  booktitle={The IEEE International Conference on Computer Vision (ICCV)},
  year={2019}
}
@inproceedings{Ye_Amalgamating_2019,
  year={2019},
  author={Ye, Jingwen and Wang, Xinchao and Ji, Yixin and Ou, Kairi and Song, Mingli},
  title={Amalgamating Filtered Knowledge: Learning Task-customized Student from Multi-task Teachers}
  booktitle={Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2019},
}
@inproceedings{song2020depara,
  title={DEPARA: Deep Attribution Graph for Deep Knowledge Transferability},
  author={Song, Jie and Chen, Yixin and Ye, Jingwen and Wang, Xinchao and Shen, Chengchao and Mao, Feng and Song, Mingli},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={3922--3930},
  year={2020}
}
@inproceedings{song2019deep,
  title={Deep model transferability from attribution maps},
  author={Song, Jie and Chen, Yixin and Wang, Xinchao and Shen, Chengchao and Song, Mingli},
  booktitle={Advances in Neural Information Processing Systems},
  pages={6182--6192},
  year={2019}
}

Comments ( 1 )

Sign in for post a comment

About

模型炼知框架:构建了炼知平台和重组引擎,通过知识重组图谱实现深度模型可迁移性度量,同时通过重组算法实现新模型生成,重新定义了模型生产方式。 spread retract
Cancel

Releases

No release

Atlas_APP

Contributors

All

Activities

load more
can not load any more
Python
1
https://gitee.com/zhijiangtianshu/Atlas_APP.git
git@gitee.com:zhijiangtianshu/Atlas_APP.git
zhijiangtianshu
Atlas_APP
Atlas_APP
master

Search