# UnityAndroidNativeCamera

**Repository Path**: KouMeanSun/unity-android-native-camera

## Basic Information

- **Project Name**: UnityAndroidNativeCamera
- **Description**: This is a unity plugin projection,unity texture2d show android camera2 frame.
一个unity3d端显示android 原生相机 图像 的工程。
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 1
- **Forks**: 0
- **Created**: 2021-07-16
- **Last Updated**: 2022-12-05

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

#unity3d 的Texture2D 显示 android 原生相机返回的图像

##1.个人对实现原理的简单理解：
####首先c#要创建一个Texture2D,然后通过GetNativeTexturePtr()把指针丢给c++层，c#代码：

```
        Texture2D tex = new Texture2D(1280, 720, TextureFormat.RGBA32, false);
        // Set point filtering just so we can see the pixels clearly
        tex.filterMode = FilterMode.Point;
        // Call Apply() so it's actually uploaded to the GPU
        tex.Apply();
        //displayMaterial.mainTexture = tex;
        rawImage.texture = tex;
        SetTextureFromUnity(tex.GetNativeTexturePtr());
        EnablePreview(true);
```

####SetTextureFromUnity 是个c++端声明给c#调用的native方法，c#代码片段为：

```
[DllImport("NativeCameraPlugin")]
    private static extern void SetTextureFromUnity(IntPtr texture);
```

####SetTextureFromUnity 的c++层代码为：

```
extern "C" void UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API SetTextureFromUnity(void *texturePtr)
{
    // A script calls this at initialization time; just remember the texture pointer here.
    // Will update texture pixels each frame from the plugin rendering event (texture update
    // needs to happen on the rendering thread).
    g_TexturePointer = texturePtr;
    LOGD("########################## SetTextureFromUnity texturePtr=%p\n", g_TexturePointer);
}
```

####这样c++层就拿到了这个c#层的Texture2D的指针，方便c++继续对此2D纹理进行操作。
接下来2D纹理要显示内容，需要unity端 不断调用 GL.IssuePluginEvent(GetRenderEventFunc(), 1); 实时拿到android原生相机的图像，显示到texture上。
GetRenderEventFunc函数是一个c++的native方法。所以c#需要引入对应的库，代码片段

```
extern "C" UnityRenderingEvent UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API GetRenderEventFunc()
{
    return OnRenderEvent;
}
```

####上面会返回一个函数OnRenderEvent，OnRenderEvent的主要代码为:

```
static void UNITY_INTERFACE_API OnRenderEvent(int eventID)
{
    if (g_TexturePointer)
    {
        int status;
        JNIEnv *env;
        int isAttached = 0;
        if (!gCallbackObject)
            return;
        if ((status = gJavaVM->GetEnv((void **)&env, JNI_VERSION_1_6)) < 0)
        {
            if ((status = gJavaVM->AttachCurrentThread(&env, NULL)) < 0)
            {
                return;
            }
            isAttached = 1;
        }
        jclass cls = env->GetObjectClass(gCallbackObject);
        if (!cls)
        {
            if (isAttached)
                gJavaVM->DetachCurrentThread();
            return;
        }
        jmethodID method = env->GetMethodID(cls, "requestJavaRendering", "(I)V");
        if (!method)
        {
            if (isAttached)
                gJavaVM->DetachCurrentThread();
            return;
        }
        GLuint gltex = (GLuint)(size_t)(g_TexturePointer);
        env->CallVoidMethod(gCallbackObject, method, (int)gltex);
        if (isAttached)
            gJavaVM->DetachCurrentThread();
    }
}
```

####仔细看这个函数的基本原理是通过JNI调用一个android 的java方法:requestJavaRendering,所以我们继续看一下requestJavaRendering代码:

```
private void requestJavaRendering(int texturePointer) {
        if (this._update) {
            int[] imageBuffer = new int[0];
            if (this._conversionScript != null) {
                imageBuffer = this._conversionScript.getOutputBuffer();
            }
            if (imageBuffer.length > 1) {
                GLES20.glBindTexture(GL_TEXTURE_2D, texturePointer);
                GLES20.glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, this._previewSize.getWidth(), this._previewSize.getHeight(), GL_RGBA, GL_UNSIGNED_BYTE, IntBuffer.wrap(imageBuffer));
            }
        }
    }
```

####java方法requestJavaRendering的核心意思是通过_conversionScript得到imageBuffer数组，这个int数组的具体值就是像素值，如果不为空，则通过
OpenGL 更新到对应的纹理上去。核心代码为 GLES20.glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, this._previewSize.getWidth(), this._previewSize.getHeight(), GL_RGBA, GL_UNSIGNED_BYTE, IntBuffer.wrap(imageBuffer));
接下来我们继续看下this._conversionScript从何而来，创建代码为：

```
this._conversionScript = new YuvToRgb(this._renderScript, this._previewSize, CONVERSION_FRAME_RATE);
this._conversionScript.setOutputSurface(this._imagePreviewReader.getSurface());
this._previewSurface = this._conversionScript.getInputSurface();
```

####我这个功能是在android camera2的基础上实现的，所以上面的代码片段是在启动相机的时候执行的。
这样就实现了主要的c#显示android camera2的帧数据的功能，具体的权限判断等基础代码我没有贴出来，大佬们可以自行注意，最下面我会贴出Demo地址。
还有一个要注意c++所以来的一些Unity需要的头文件：

####CMakeLists.txt代码为：

```
# For more information about using CMake with Android Studio, read the
# documentation: https://d.android.com/studio/projects/add-native-code.html
# Sets the minimum version of CMake required to build the native library.
cmake_minimum_required(VERSION 3.4.1)
# Creates and names a library, sets it as either STATIC
# or SHARED, and provides the relative paths to its source code.
# You can define multiple libraries, and CMake builds them for you.
# Gradle automatically packages shared libraries with your APK.
add_library( # Sets the name of the library.
             NativeCameraPlugin
             # Sets the library as a shared library, otherwise it won't be
             # packaged with AAR.
             SHARED
             # Provides a relative path to your source file(s).
             RenderingPlugin.cpp )
# Searches for a specified prebuilt library and stores the path as a
# variable. Because CMake includes system libraries in the search path by
# default, you only need to specify the name of the public NDK library
# you want to add. CMake verifies that the library exists before
# completing its build.
find_library( # Sets the name of the path variable.
              dl
              GLESv2
              # Specifies the name of the NDK library that
              # you want CMake to locate.
              log )
# Specifies libraries CMake should link to your target library. You
# can link multiple libraries, such as libraries you define in this
# build script, prebuilt third-party libraries, or system libraries.
target_link_libraries( # Specifies the target library.
                       NativeCameraPlugin
                       log
                       # Links the target library to the log library
                       # included in the NDK.
                       ${log-lib} )
```

##注意:unity的Texture2D要和android camera2的预览尺寸要一致，我这里都是1280*720
还有一个地方要注意注意再注意！！！！，我们看到上面的CMakeLists代码的find_library 配置的是GLESv2，所以对应unity工程也要配置成OpenGL2！如图：

具体代码可以参考码云仓库；
[码云链接](https://gitee.com/KouMeanSun/unity-android-native-camera)

详细博客后续更新会在语雀上，附上链接
[语雀链接](https://www.yuque.com/yidenghou/gd0t1h/vq2k6g)