multi-engine/performance_numbers.md · CV爱好者/djl-demo - Gitee.com

4月12日模力方舟 AI 应用沙龙 · 杭州站报名开放，产研前线第一手干货，AI 开发者必冲！

加入 Gitee

与超过 1200万开发者一起发现、参与优秀开源项目，私有仓库也完全免费：）

文件

克隆/下载

performance_numbers.md 2.94 KB

## Performance Numbers

These numbers were run on a Mac Pro 2019 (2.8 Ghz Intel Core i7)

Because of the warning above, here are some benchmarking numbers to get an idea of the performance of loading in
multiple engines vs loading in only one engine.

Benchmarking Numbers for only having one engine loaded in.

|             | Engine Name | Model         | Throughput                         | Total p50  | Total p90  | inference p50 | inference p90  |
| ----------- | ----------- | ------------- | ---------------------------------- | ---------- | ---------- | ------------- | -------------- |
| 1 Engine    | MXNet       | resnet 50     | 10.19, 5000 iteration / 490602 ms. | 95.520 ms  | 103.606 ms | 90.120 ms     | 97.954 ms      |
| MultiEngine | MXNet       | resnet 50     | 10.17, 5000 iteration / 491656 ms. | 96.084 ms  | 104.198 ms | 90.696 ms     | 98.692 ms      |
|             |             |               |                                    |            |            |               |                |
| 1 Engine    | TensorFlow  | resnet 50     | 7.15, 2500 iteration / 349777 ms.  | 123.493 ms | 142.029 ms | 110.592 ms    | 126.268 ms     |
| MultiEngine | TensorFlow  | resnet 50     | 6.84, 2500 iteration / 365355 ms.  | 124.743 ms | 160.793 ms | 110.935 ms    | 144.142 ms     |
|             |             |               |                                    |            |            |               |                |
| 1 Engine    | PyTorch     | resnet 50     | 6.87, 5000 iteration / 728242 ms.  | 140.246 ms | 155.400 ms | 133.463 ms    | 147.721 ms     |
| MultiEngine | PyTorch     | resnet 50     | 6.59, 5000 iteration / 759261 ms.  | 144.358 ms | 168.535 ms | 137.336 ms    | 160.346 ms     |
|             |             |               |                                    |            |            |               |                |
| 1 Engine    | MXNet       | resnet 18     | 22.04, 5000 iteration / 226893 ms. | 43.487 ms  | 52.077 ms  | 38.105 ms     | 45.660 ms      |
| MultiEngine | MXNet       | resnet 18     | 22.64, 5000 iteration / 220807 ms. | 42.750 ms  | 47.923 ms  | 37.482 ms     | 42.109 ms      |
|             |             |               |                                    |            |            |               |                |
| 1 Engine    | Tensorflow  | MobileNet 1.0 | 30.45, 5000 iteration / 164225 ms. | 31.190 ms  | 34.780 ms  | 19.983 ms     | 22.836 ms      |
| MultiEngine | Tensorflow  | MobileNet 1.0 | 29.17, 5000 iteration / 171395 ms. | 31.860 ms  | 38.047 ms  | 20.631 ms     | 24.798 ms      |
|             |             |               |                                    |            |            |               |                |
| 1 Engine    | PyTorch     | resnet 18     | 14.90, 5000 iteration / 335509 ms. | 64.268 ms  | 73.352 ms  | 57.620 ms     | 65.733 ms      |
| MultiEngine | PyTorch     | resnet 18     | 13.54, 5000 iteration / 369255 ms. | 71.959 ms  | 81.688 ms  | 64.580 ms     | 73.439 ms      |

一键复制编辑原始数据按行查看历史

提交于 5年前 . MultipleEngines Example (#34)

Performance Numbers

Performance Numbers

These numbers were run on a Mac Pro 2019 (2.8 Ghz Intel Core i7)

Because of the warning above, here are some benchmarking numbers to get an idea of the performance of loading in multiple engines vs loading in only one engine.

Benchmarking Numbers for only having one engine loaded in.

	Engine Name	Model	Throughput	Total p50	Total p90	inference p50	inference p90
1 Engine	MXNet	resnet 50	10.19, 5000 iteration / 490602 ms.	95.520 ms	103.606 ms	90.120 ms	97.954 ms
MultiEngine	MXNet	resnet 50	10.17, 5000 iteration / 491656 ms.	96.084 ms	104.198 ms	90.696 ms	98.692 ms

1 Engine	TensorFlow	resnet 50	7.15, 2500 iteration / 349777 ms.	123.493 ms	142.029 ms	110.592 ms	126.268 ms
MultiEngine	TensorFlow	resnet 50	6.84, 2500 iteration / 365355 ms.	124.743 ms	160.793 ms	110.935 ms	144.142 ms

1 Engine	PyTorch	resnet 50	6.87, 5000 iteration / 728242 ms.	140.246 ms	155.400 ms	133.463 ms	147.721 ms
MultiEngine	PyTorch	resnet 50	6.59, 5000 iteration / 759261 ms.	144.358 ms	168.535 ms	137.336 ms	160.346 ms

1 Engine	MXNet	resnet 18	22.04, 5000 iteration / 226893 ms.	43.487 ms	52.077 ms	38.105 ms	45.660 ms
MultiEngine	MXNet	resnet 18	22.64, 5000 iteration / 220807 ms.	42.750 ms	47.923 ms	37.482 ms	42.109 ms

1 Engine	Tensorflow	MobileNet 1.0	30.45, 5000 iteration / 164225 ms.	31.190 ms	34.780 ms	19.983 ms	22.836 ms
MultiEngine	Tensorflow	MobileNet 1.0	29.17, 5000 iteration / 171395 ms.	31.860 ms	38.047 ms	20.631 ms	24.798 ms

1 Engine	PyTorch	resnet 18	14.90, 5000 iteration / 335509 ms.	64.268 ms	73.352 ms	57.620 ms	65.733 ms
MultiEngine	PyTorch	resnet 18	13.54, 5000 iteration / 369255 ms.	71.959 ms	81.688 ms	64.580 ms	73.439 ms

Loading...

马建仓 AI 助手

尝试更多

代码解读

代码找茬

代码优化

Java

1

https://gitee.com/cvfan/djl-demo.git

git@gitee.com:cvfan/djl-demo.git

cvfan

djl-demo

djl-demo

master