serving 发行版 - Gitee.com

最新版

v1.8.0

38a9a86

2022-08-03 09:23

MindSpore Serving 1.8.0 Release Notes

最后提交信息为： !464 Modify the release note files

v1.7.0

9be5921

2022-04-20 14:18

MindSpore Serving 1.7.0 Release Notes

最后提交信息为： !432 Serving, fix deadlock when backend is Ascend310/710

v1.6.0

1a8b539

2022-01-21 09:39

MindSpore Serving 1.6.0 Release Notes

### Major Features and Improvements

- [STABLE] We can use existing interfaces(`decalre_model` and `add_stage`) that define single-model services to define
  multi-model composite services.
- [STABLE] When the number of occupied devices is fixed, additional worker processes(using parameter
  `num_parallel_workers`) are supported to accelerate Python functions such as preprocessing and postprocessing,
  improving device utilization.
- [STABLE] The interface `Model.call` is a stable feature, and can be used to define complex model invocation processes
  in the Serving server, such as looping and conditional branching.
- [STABLE] The new interfaces `Context`, `CPUDeviceInfo`, `GPUDeviceInfo`, `AscendDeviceInfo` are provided to set
  user-defined device information. The original interfaces `GpuOptions` and `AclOptions` are deprecated.
- [BETA] We support MindSpore Lite as the MindSpore Serving inference backend, for more detail see
  [MindSpore Serving backend](https://www.mindspore.cn/serving/docs/en/master/serving_install.html#installation).

最后提交信息为： !416 Serving, fix LD_LIBRARY_PATH

v1.5.0

7f5f345

2021-10-25 15:45

MindSpore 1.5.0 Release Notes

### Major Features and Improvements

- [STABLE] To support multi-model orchestration (to be released in version 1.6), a set of APIs (`decalre_model`
  and `add_stage`) is added. The new APIs will be used in single-model and multi-model scenarios. The old
  APIs(`register.declare_servable`,`call_servable`,`call_preprocess`,`call_postprocess`) used in single-model scenarios
  are deprecated.
- [BETA] When the number of occupied devices is fixed, additional worker processes are supported to accelerate Python
  functions such as preprocessing and postprocessing, improving device utilization.
- [BETA]`Model.call` interface is added to support invoking models in Python functions.

### API Change

#### API Incompatible Change

##### Python API

###### New set of APIs for single-model and multi-model scenarios

To support multiple models(will be officially released in version 1.6), a set of APIs (`decalre_model` and `add_stage`)
is added. The single-model and multi-model scenarios will use the same set of APIs.

New APIs are recommended in single-model scenarios. Old APIs (`declare_servable`,`call_servable`,`call_preprocess`,
`call_postprocess`) are deprecated.

```python
from mindspore_serving.server import register

register.declare_servable(servable_file="resnet.mindir",
                          model_format="MindIR")

def resnet_preprocess(image):
    ....

def resnet_postprocess(scores):
    ....

@register.register_method(output_names=["label"])
def predict(image):
    x = register.call_preprocess(resnet_preprocess, image)
    x = register.call_servable(x)
    x = register.call_postprocess(resnet_postprocess, x)
    return x
```

</td>
<td>

```python
from mindspore_serving.server import register

resnet_model = register.declare_model(model_file="resnet.mindir",
                                      model_format="MindIR")

def resnet_preprocess(image):
    ....

def resnet_postprocess(scores):
    ....

@register.register_method(output_names=["label"])
def predict(image):
    x = register.add_stage(resnet_preprocess, image, outputs_count=1)
    x = register.add_stage(resnet_model, x, outputs_count=1)
    x = register.add_stage(resnet_postprocess, x, outputs_count=1)
    return x
```

</td>
</tr>
</table>

#### New features

##### Python API

###### Additional worker processes are supported to accelerate Python functions(preprocessing and postprocessing)

Parameter `num_parallel_workers` is added to class `ServableStartConfig` to configure the total number of workers. The
number of workers occupying devices is determined by the length of parameter `device_ids`. Additional worker processes
use worker processes that occupy devices for model inference.

```python
class ServableStartConfig:
    def __init__(self, servable_directory, servable_name, device_ids, version_number=0, device_type=None,
                 num_parallel_workers=0, dec_key=None, dec_mode='AES-GCM')
```

Start the serving server that contains the `resnet50` servable. The `resnet50` servable has four worker
processes(`num_parallel_workers`), one of which occupies the device(`device_ids`).

```python
import os
import sys
from mindspore_serving import server

def start():
    servable_dir = os.path.dirname(os.path.realpath(sys.argv[0]))
    # Total 4 worker, one worker occupy device 0, the model inference tasks of other workers are forwarded to the worker
    # that occupies the device.
    config = server.ServableStartConfig(servable_directory=servable_dir,
                                        servable_name="resnet50", device_ids=0,
                                        num_parallel_workers=4)
    server.start_servables(config)

server.start_grpc_server("127.0.0.1:5500")
    server.start_restful_server("127.0.0.1:1500")

if __name__ == "__main__":
    start()
```

###### Model.call interface is added to support invoking models in Python functions

```python
from mindspore_serving.server import register

add_model = register.declare_model(model_file="tensor_add.mindir",
                                   model_format="MindIR")

def add_func(x1, x2, x3, x4):
    instances = []
    instances.append((x1, x2))
    instances.append((x3, x4))
    output_instances = add_model.call(instances)  # for multi instances
    y1 = output_instances[0][0]  # instance 0 output 0
    y2 = output_instances[1][0]  # instance 1 output 0
    y = add_model.call(y1, y2)  # for single instance
    return y

@register.register_method(output_names=["y"])
def predict(x1, x2, x3, x4):
    y = register.add_stage(add_func, x1, x2, x3, x4, outputs_count=1)
    return y
```

#### Deprecations

##### Python API

- `register.declare_servable`,`call_servable`,`call_preprocess`,`call_postprocess`,`call_preprocess_pipeline`
  and`call_postprocess_pipeline` are now deprecated in favor of`register.declare_model` and`add_stage`, as shown above.
  Deprecated interfaces will be deleted in the future.
- Beta interfaces`PipelineServable` and`register_pipeline` introduced in version 1.3 will be deleted and replaced
  with`Model.call`.

### Contributors

Thanks goes to these wonderful people:

chenweifeng, qinzheng, xuyongfei, zhangyinxia, zhoufeng.

Contributions of any kind are welcome!

最后提交信息为： !368 Serving, update mindspore commit