diff --git a/docs/preformance-imporvement-instructions.md b/docs/preformance-imporvement-instructions.md new file mode 100644 index 0000000000000000000000000000000000000000..2c259f511adfb501ad7467178a03558447e70ad0 --- /dev/null +++ b/docs/preformance-imporvement-instructions.md @@ -0,0 +1,462 @@ +##
OpenHarmony**相机性能提升开发指导**
+ +# 概述 + +​ 相机启动性能受限于底层器件上点、流程Pipeline初始化等耗时操作影响，本文档旨在为开发者提供更进一步功能，提升相机启动速度以及拍照返回缩略图速度。相关能力与底层器件相关，不代表所有硬件均支持相关特性。 + +​ 相关特性分别在打开相机设备过程、配流过程以及拍照过程中。本文档针对三个场景分别进行介绍 + +# 预热启动 + +## 场景描述 + +​ 普通情况下相机应用的启动是用户通过点击桌面相机图标触发的。桌面应用感知用户点击相机图标，然后通知应用管理器启动对应的相机应用（进程），这个过程是比较耗时的。进入相机应用后，开始进入相机启动流程。经典的相机启动过程会经过，“相机设备打开”，“配置数据流”，“启动数据流等”，这也是一个耗时的过程。 + +​ 相机启动方案是把“相机设备打开”这个动作提前到相机应用启动之前，即在用户点击相机图标，还没等相机应用启动的时候，触发相机设备打开的动作，从而缩短相机应用内启动相机的流程，加速相机启动。使用预热启动前后的相机应用流程对比如下： + +![image-20230524105824546](../figures/prelaunch-scene.png) + + + +## 接口描述 + +### isPreLaunchSupported + +isPreLaunchSupported(camera: CameraDevice) : boolean + +在setPreLaunchConfig接口使用前调用，用于判断指定cameraDevice是否支持预热启动。 + +**系统能力：** SystemCapability.Multimedia.Camera.Core + +**参数：** + +| 参数名 | 类型 | 必填 | 说明 | +| ------ | ------------ | ---- | ---------- | +| camera | CameraDevice | 是 | 相机信息。 | + +**返回值：** + +| 类型 | 说明 | +| ------- | ------------------------------------ | +| boolean | 返回指定cameraDevice是否支持预热启动 | + +**错误码：** + +不涉及 + +**示例：** + +```js +import camera from '@ohos.multimedia.camera' + +this.cameraManager = camera.getCameraManager(globalThis.abilityContext); +let cameras = this.cameraManager.getSupportedCameras() +if(this.cameraManager.isPreLaunchSupported(cameras[0])) { + this.cameraManager.setPreLaunchConfig({cameraDevice: cameras[0]}); +} +``` + +### setPreLaunchConfig + +setPreLaunchConfig(camera: CameraDevice) : boolean + +配置相机预热参数，指定相机设备（待扩展流参数信息等）。 +用户操作会产生下一次进入相机的预热配置变化，或是退出相机时，向相机服务发送预热配置参数。 + +**需要权限：** ohos.permission.MANAGE_CAMERA_CONFIG + +**系统能力：** SystemCapability.Multimedia.Camera.Core + +**参数：** + +| 参数名 | 类型 | 必填 | 说明 | +| ------ | ------------ | ---- | ---------- | +| camera | CameraDevice | 是 | 相机信息。 | + +**返回值：** + +| 类型 | 说明 | +| ------- | ------------------------------------ | +| boolean | 返回设置预热配置参数是否成功 | + +**错误码：** + +| 类型 | 说明 | +| ------- | ------------------------------------ | +| BusinessError | 7400101 缺少参数或者参数类型错误
7400102 权限校验失败 | + +**示例：** + +```js +import camera from '@ohos.multimedia.camera' + +this.cameraManager = camera.getCameraManager(globalThis.abilityContext); +let cameras = this.cameraManager.getSupportedCameras() +if(this.cameraManager.isPreLaunchSupported(cameras[0])) { + try { + this.cameraManager.setPreLaunchConfig({cameraDevice: cameras[0]}); + } catch (error) { + HiLog.e(TAG, `catch error: ${JSON.stringify(error)}`) + } +} +``` +```json +"requestPermissions": [ + { + "name": "ohos.permission.CAMERA" + } +] +``` + +### preLaunch +preLaunch() : void + +用户点击系统相机图标，拉起相机应用同时调用，下发预热请求，使能相机预热启动。 + +**需要权限：** ohos.permission.MANAGE_CAMERA_CONFIG + +**系统能力：** SystemCapability.Multimedia.Camera.Core + +**参数：** + +无 + +**返回值：** + +无 + +**错误码：** + +| 类型 | 说明 | +| ------- | ------------------------------------ | +| BusinessError | 7400102 权限校验失败 | + +**示例：** + +```js +import camera from '@ohos.multimedia.camera' + +this.cameraManager = camera.getCameraManager(globalThis.abilityContext); +try { + this.cameraManager.preLaunch(); +} catch (error) { + HiLog.e(TAG, `catch error: ${JSON.stringify(error)}`) +} +``` +```json +"requestPermissions": [ + { + "name": "ohos.permission.CAMERA" + } +] +``` + +### PreLaunchConfig +新增参数类型，预热配置参数，当前sensor级别预热，待扩展流预热参数 + +**示例：** + +```js +this.cameraManager = camera.getCameraManager(globalThis.abilityContext); +let cameras = this.cameraManager.getSupportedCameras() +this.preLaunchConfig: PreLaunchConfig = { + cameraDevice: cameras[0] +} +if(this.cameraManager.isPreLaunchSupported(cameras[0])) { + this.cameraManager.setPreLaunchConfig(this.preLaunchConfig); +} +``` + + +## 调用流程 + +启动方案调用流程图建议如下： + +![image-20230524100713046](../figures/prelaunch-sequence-diagram.png) + +## 注意事项 + +* 预热启动方案相关API均为SystemAPI，如若使用，需申请系统应用权限，app-feature级别设置为hos_system_app。 + + + +# 延时配流 + +## 场景描述 + +经典的相机启动过程经过“相机设备打开”、“配置数据流”、“启动数据流”等流程，而配流启流之前需要得到图形组件的surfaceId。 + +延时配流方案是把配流启流与surface解耦，在组件尚未给应用surface之前，可以先进行配流启流，只需要在启流结束之前提供surface，可以提升启动速度，防止影响其他启动优化方案的落地。 + +![image-20230529093514951](../figures/deferred-surface-scene.png) + +## 接口描述 + +### createDeferredPreviewOutput + +createDeferredPreviewOutput(profile: Profile): Promise\ + +创建延迟预览输出对象，在配流时替代普通的预览输出对象加入数据流。 + +**系统能力：** SystemCapability.Multimedia.Camera.Core + +**参数：** + +| 类型 | 说明 | +| ------- | -------------------- | +| Profile | 相机预览流的配置文件 | + +**返回值：** + +| 类型 | 说明 | +| ------------- | ------------------ | +| PreviewOutput | 返回预览输出对象。 | + +**错误码：** + +不涉及 + +**示例：** + +```js +import camera from '@ohos.multimedia.camera'; + +function getDeferredPreviewOutput(context: Context, previewProfile: camera.Profile): Promise { + const cameraManager = camera.getCameraManager(context); + const output: Promise = cameraManager.createDeferredPreviewOutput(previewProfile); + return output; +} +``` + +### addDeferredSurface + +addDeferredSurface(surfaceId: string): Promise\ + +配置延迟预览的Surface，可以在session.commitConfig()配流和session.start()启流之后运行。 + +**系统能力：** SystemCapability.Multimedia.Camera.Core + +**参数：** + +| 类型 | 说明 | +| ------ | --------------------- | +| string | 预览使用的surfaceId。 | + +**返回值：** + +无 + +**错误码：** + +| 类型 | 说明 | +| ------------- | --------------------- | +| BusinessError | 7400104 session未配置 | + +**示例**： + +```js +import camera from '@ohos.multimedia.camera'; + +function async preview(context: Context, cameraInfo: camera.Device, previewProfile: camera.Profile, photoProfile: camera.Profile, surfaceId: string): Promise { + const cameraManager: camera.CameraManager = camera.getCameraManager(context); + const cameraInput camera.CameraInput = await cameraManager.createCameraInput(cameraInfo) + const previewOutput: camera.PreviewOutput = await cameraManager.createDeferredPreviewOutput(previewProfile); + const photoOutput: camera.PhotoOutput = await cameraManager.createPhotoOutput(photoProfile); + const session: camera.CaptureSession = await this.mCameraManager.createCaptureSession(); + await session.beginConfig(); + await session.addInput(cameraInput); + await session.addOutput(previewOutput); + await session.addOutput(photoOutput); + await session.commitConfig(); + await session.start(); + await previewOutput.addDeferredSurface(surfaceId); +} +``` + +## 调用流程 + +![image-20230529103448268](../figures/deferred-surface-sequence-diagram.png) + +## 注意事项 + +* 延时配流方案相关API均为SystemAPI，如若使用，需申请系统应用权限，app-feature级别设置为hos_system_app。 + + + +# 快速缩略图 +## 场景描述 + +相机拍照性能依赖算法处理的速度，算法链越复杂、效果就越好，但同时处理时间就越长。要能够从拍照流程上进行优化，既满足后处理算法处理的要求，又不要阻塞前台的拍照速度。 + +通过相机快速缩略图技术，相机拍照可单独输出拇指缩略图，在真图没有上来前，提前上报一张缩略图给应用去显示，提升shot2see用户感知拍照速度。 + +## 接口描述 + +### isQuickThumbnailSupported + +isQuickThumbnailSupported() : boolean + +是否支持快速缩略图。 + +在CaptureSession.addOutput(photoOutput： PhotoOutput)、CaptureSession.addInput(cameraInput： CameraInput)之后，CaptureSession.commitConfig()之前生效。 + +**系统能力：** SystemCapability.Multimedia.Camera.Core + +**参数：** + +无 + +**返回值：** + +| 类型 | 说明 | +| ------- | ------------------------------------------------ | +| boolean | 返回当前CaptureSession中拍照流是否支持快速缩略图 | + +**错误码：** + +| 类型 | 说明 | +| ------------- | --------------------- | +| BusinessError | 7400104 session未配置 | + +**示例：** + +```js +import camera from '@ohos.multimedia.camera' + +this.cameraManager = camera.getCameraManager(globalThis.abilityContext); +let cameras = this.cameraManager.getSupportedCameras() +// 创建CaptureSession实例 +this.captureSession = await this.cameraManager.createCaptureSession() +// 开始配置会话 +await this.captureSession.beginConfig() +// 把CameraInput加入到会话 +this.mCameraInput = await this.cameraManager.createCameraInput(cameras[0]) +await this.cameraInput.open() +await this.captureSession.addInput(this.cameraInput) +// 把PhotoOutPut加入到会话 +this.photoOutPut = await this.cameraManager.createPhotoOutput(photoProfile, surfaceId) +await this.captureSession.addOutput(this.photoOutPut) + +boolean isSupported = this.photoOutPut.isQuickThumbnailSupported() +``` + +### enableQuickThumbnail + +enableQuickThumbnail (enabled:bool): void + +使能/去使能快速缩略图。 + +在CaptureSession.addOutput(photoOutput： PhotoOutput)、CaptureSession.addInput(cameraInput： CameraInput)之后，CaptureSession.commitConfig()之前生效。 + +**系统能力：** SystemCapability.Multimedia.Camera.Core + +**参数：** + +| 参数名 | 类型 | 必填 | 说明 | +| ------- | ------- | ---- | ------------------ | +| enabled | boolean | 是 | 是否使能快速缩略图 | + +**返回值：** + +无 + +**错误码：** + +| 类型 | 说明 | +| ------- | ------------------------------------ | +| BusinessError | 7400104 session未配置 | + +**示例：** + +```js +import camera from '@ohos.multimedia.camera' + +this.cameraManager = camera.getCameraManager(globalThis.abilityContext); +let cameras = this.cameraManager.getSupportedCameras() +// 创建CaptureSession实例 +this.captureSession = await this.cameraManager.createCaptureSession() +// 开始配置会话 +await this.captureSession.beginConfig() +// 把CameraInput加入到会话 +this.cameraInput = await this.cameraManager.createCameraInput(cameras[0]) +await this.cameraInput.open() +await this.captureSession.addInput(this.cameraInput) +// 把PhotoOutPut加入到会话 +this.photoOutPut = await this.cameraManager.createPhotoOutput(photoProfile, surfaceId) +await this.captureSession.addOutput(this.photoOutPut) +boolean isSupported = this.photoOutPut.isQuickThumbnailSupported() +if (isSupported) { + // 使能快速缩略图 + this.photoOutPut.enableQuickThumbnail(true) +} +``` + +### on +on (type: "quickThumbnail", callback: AsyncCallback): void + +配置相机缩略图回调接口 + +在enableQuickThumbnail(true)使能快速缩略图之后监听生效。 + +**系统能力：** SystemCapability.Multimedia.Camera.Core + +**参数：** + +| 参数名 | 类型 | 必填 | 说明 | +| ------ | ------ | ---- | ------------ | +| type | String | 是 | 监听事件类型 | +| callback | Function | 是 | 回调函数，返回PixelMap | + +**返回值：** + +无 + +**错误码：** + +不涉及 + +**示例：** + +```js +import camera from '@ohos.multimedia.camera' + +this.cameraManager = camera.getCameraManager(globalThis.abilityContext); +let cameras = this.cameraManager.getSupportedCameras() +// 创建CaptureSession实例 +this.captureSession = await this.cameraManager.createCaptureSession() +// 开始配置会话 +await this.captureSession.beginConfig() +// 把CameraInput加入到会话 +this.cameraInput = await this.cameraManager.createCameraInput(cameras[0]) +await this.cameraInput.open() +await this.captureSession.addInput(this.cameraInput) +// 把PhotoOutPut加入到会话 +this.photoOutPut = await this.cameraManager.createPhotoOutput(photoProfile, surfaceId) +await this.captureSession.addOutput(this.photoOutPut) +boolean isSupported = this.photoOutPut.isQuickThumbnailSupported() +if (isSupported) { + // 使能快速缩略图 + this.photoOutPut.enableQuickThumbnail(true) +} +this.photoOutPut.on('quickThumbnail', (err, pixelmap) => { + if (err || pixelmap === undefined) { + Logger.error(this.tag, 'photoOutPut on thumbnail failed ') + return + } + // 显示或保存pixelmap + this.showOrSavePicture(pixelmap) +}) +``` + + +## 调用流程 + +启动方案调用流程图建议如下： + +![image-20230524113602445](../figures/quick-thumbnail-sequence-diagram.png) + +## 注意事项 + +* 快缩缩略图方案相关API均为SystemAPI，如若使用，需申请系统应用权限，app-feature级别设置为hos_system_app。 +* isQuickThumbnailSupported及enableQuickThumbnail 接口需要在CaptureSession.addOutput(photoOutput： PhotoOutput)、CaptureSession.addInput(cameraInput： CameraInput)之后，CaptureSession.commitConfig()之前生效。 +* on (type: "quickThumbnail", callback: AsyncCallback)接口需要在enableQuickThumbnail (true)之后生效。 \ No newline at end of file diff --git a/figures/deferred-surface-scene.png b/figures/deferred-surface-scene.png new file mode 100644 index 0000000000000000000000000000000000000000..b2aa93f5ddf16e1b3b9bfe56d661b37a8ea3902e Binary files /dev/null and b/figures/deferred-surface-scene.png differ diff --git a/figures/deferred-surface-sequence-diagram.png b/figures/deferred-surface-sequence-diagram.png new file mode 100644 index 0000000000000000000000000000000000000000..9fb90507a7983f8fdc61dcb04a51b739bbe9be98 Binary files /dev/null and b/figures/deferred-surface-sequence-diagram.png differ diff --git a/figures/prelaunch-scene.png b/figures/prelaunch-scene.png new file mode 100644 index 0000000000000000000000000000000000000000..b0c522aa2a3e3c4c1fbc161b1138d76023be8110 Binary files /dev/null and b/figures/prelaunch-scene.png differ diff --git a/figures/prelaunch-sequence-diagram.png b/figures/prelaunch-sequence-diagram.png new file mode 100644 index 0000000000000000000000000000000000000000..dce519d89acdb4596caa9cf62b521e9a2c3923df Binary files /dev/null and b/figures/prelaunch-sequence-diagram.png differ diff --git a/figures/quick-thumbnail-sequence-diagram.png b/figures/quick-thumbnail-sequence-diagram.png new file mode 100644 index 0000000000000000000000000000000000000000..dab8badd11d2c20167010562e1dd4aa8f952f13f Binary files /dev/null and b/figures/quick-thumbnail-sequence-diagram.png differ diff --git a/frameworks/js/camera_napi/src/input/camera_manager_napi.cpp b/frameworks/js/camera_napi/src/input/camera_manager_napi.cpp index 605c4f9767471803dd7505f2dbbb774c3b1cff0e..e8541ec84dded4fe84d71e4767bba395b589ea09 100644 --- a/frameworks/js/camera_napi/src/input/camera_manager_napi.cpp +++ b/frameworks/js/camera_napi/src/input/camera_manager_napi.cpp @@ -402,6 +402,10 @@ napi_value CameraManagerNapi::CreatePreviewOutputInstance(napi_env env, napi_cal napi_value CameraManagerNapi::CreateDeferredPreviewOutputInstance(napi_env env, napi_callback_info info) { MEDIA_INFO_LOG("CreateDeferredPreviewOutputInstance is called"); + if (!CameraNapiUtils::CheckSystemApp(env)) { + MEDIA_ERR_LOG("SystemApi CreateDeferredPreviewOutputInstance is called!"); + return nullptr; + } napi_status status; napi_value result = nullptr; napi_value resource = nullptr; @@ -816,6 +820,10 @@ napi_value CameraManagerNapi::On(napi_env env, napi_callback_info info) napi_value CameraManagerNapi::IsPreLaunchSupported(napi_env env, napi_callback_info info) { MEDIA_INFO_LOG("IsPreLaunchSupported is called"); + if (!CameraNapiUtils::CheckSystemApp(env)) { + MEDIA_ERR_LOG("SystemApi IsPreLaunchSupported is called!"); + return nullptr; + } napi_status status; napi_value result = nullptr; @@ -848,6 +856,10 @@ napi_value CameraManagerNapi::IsPreLaunchSupported(napi_env env, napi_callback_i napi_value CameraManagerNapi::PrelaunchCamera(napi_env env, napi_callback_info info) { MEDIA_INFO_LOG("PrelaunchCamera is called"); + if (!CameraNapiUtils::CheckSystemApp(env)) { + MEDIA_ERR_LOG("SystemApi PrelaunchCamera is called!"); + return nullptr; + } napi_value result = nullptr; CameraManager::GetInstance()->PrelaunchCamera(); MEDIA_INFO_LOG("PrelaunchCamera"); @@ -858,6 +870,10 @@ napi_value CameraManagerNapi::PrelaunchCamera(napi_env env, napi_callback_info i napi_value CameraManagerNapi::SetPreLaunchConfig(napi_env env, napi_callback_info info) { MEDIA_INFO_LOG("SetPrelaunchConfig is called"); + if (!CameraNapiUtils::CheckSystemApp(env)) { + MEDIA_ERR_LOG("SystemApi SetPreLaunchConfig is called!"); + return nullptr; + } napi_status status; napi_value result = nullptr; size_t argc = ARGS_ONE; diff --git a/frameworks/js/camera_napi/src/output/photo_output_napi.cpp b/frameworks/js/camera_napi/src/output/photo_output_napi.cpp index 04fba0d8b3e575939cad2f4a26de184aecd8a936..3ff0695fc39e02d780b0bd05cdb31a41a2ffa90d 100644 --- a/frameworks/js/camera_napi/src/output/photo_output_napi.cpp +++ b/frameworks/js/camera_napi/src/output/photo_output_napi.cpp @@ -814,6 +814,10 @@ napi_value PhotoOutputNapi::IsMirrorSupported(napi_env env, napi_callback_info i napi_value PhotoOutputNapi::IsQuickThumbnailSupported(napi_env env, napi_callback_info info) { + if (!CameraNapiUtils::CheckSystemApp(env)) { + MEDIA_ERR_LOG("SystemApi IsQuickThumbnailSupported is called!"); + return nullptr; + } napi_status status; napi_value result = nullptr; size_t argc = ARGS_ZERO; @@ -889,6 +893,10 @@ napi_value PhotoOutputNapi::SetMirror(napi_env env, napi_callback_info info) napi_value PhotoOutputNapi::EnableQuickThumbnail(napi_env env, napi_callback_info info) { + if (!CameraNapiUtils::CheckSystemApp(env)) { + MEDIA_ERR_LOG("SystemApi EnableQuickThumbnail is called!"); + return nullptr; + } napi_status status; napi_value result = nullptr; size_t argc = ARGS_ONE; @@ -948,6 +956,10 @@ napi_value PhotoOutputNapi::On(napi_env env, napi_callback_info info) if (eventType == OHOS::CameraStandard::thumbnailRegisterName) { // create thumbnail listener when eventType is thumbnail + if (!CameraNapiUtils::CheckSystemApp(env)) { + MEDIA_ERR_LOG("SystemApi quickThumbnail on is called!"); + return nullptr; + } if (!obj->isQuickThumbnailEnabled_) { MEDIA_ERR_LOG("quickThumbnail is not enabled!"); return undefinedResult; diff --git a/frameworks/js/camera_napi/src/output/preview_output_napi.cpp b/frameworks/js/camera_napi/src/output/preview_output_napi.cpp index e454edb0a6ea823ede75f2c97e7420b90a5e0d07..8d4b76046f4900b5d2e6a3df227805d02cbca379 100644 --- a/frameworks/js/camera_napi/src/output/preview_output_napi.cpp +++ b/frameworks/js/camera_napi/src/output/preview_output_napi.cpp @@ -484,6 +484,10 @@ napi_status PreviewOutputNapi::CreateAsyncTask(napi_env env, napi_value resource napi_value PreviewOutputNapi::AddDeferredSurface(napi_env env, napi_callback_info info) { MEDIA_DEBUG_LOG("AddDeferredSurface is called"); + if (!CameraNapiUtils::CheckSystemApp(env)) { + MEDIA_ERR_LOG("SystemApi AddDeferredSurface is called!"); + return nullptr; + } napi_status status; napi_value result = nullptr; const int32_t refCount = 1; diff --git a/services/camera_service/src/hcamera_service.cpp b/services/camera_service/src/hcamera_service.cpp index 1a30429ee1e61fd1e3a3d94c3147d33ed8428985..bcf410a26c1dc9a83b25fee1d16139ff143f42d3 100644 --- a/services/camera_service/src/hcamera_service.cpp +++ b/services/camera_service/src/hcamera_service.cpp @@ -572,14 +572,6 @@ int32_t HCameraService::MuteCamera(bool muteMode) int32_t HCameraService::PrelaunchCamera() { CAMERA_SYNC_TRACE; - OHOS::Security::AccessToken::AccessTokenID callerToken = IPCSkeleton::GetCallingTokenID(); - std::string permissionName = OHOS_PERMISSION_CAMERA; - int32_t ret = CheckPermission(permissionName, callerToken); - if (ret != CAMERA_OK) { - MEDIA_ERR_LOG("HCameraService::PrelaunchCamera failed permission is : %{public}s", permissionName.c_str()); - return ret; - } - MEDIA_INFO_LOG("HCameraService::PrelaunchCamera"); if (preCameraId_.empty()) { std::vector cameraIds_; @@ -588,7 +580,7 @@ int32_t HCameraService::PrelaunchCamera() } MEDIA_INFO_LOG("HCameraService::PrelaunchCamera preCameraId_ is: %{public}s", preCameraId_.c_str()); CAMERA_SYSEVENT_STATISTIC(CreateMsg("Camera Prelaunch CameraId:%s", preCameraId_.c_str())); - ret = cameraHostManager_->PreLaunch(preCameraId_); + int32_t ret = cameraHostManager_->PreLaunch(preCameraId_); if (ret != CAMERA_OK) { MEDIA_ERR_LOG("HCameraService::PreLaunch failed"); }