# lagou-file-upload **Repository Path**: nicholas_lcj/lagou-file-upload ## Basic Information - **Project Name**: lagou-file-upload - **Description**: 作业一： 业务场景：用户上传的头像图片大小不一、手机和PC等设备显示尺寸也存在差异，因此需要能根据http请求指定的尺寸参数对图片进行动态压缩，以达到屏幕适配的作用。 功能需求：搭建FastDFS图片服务器，通过http请求访问服务器中图片时，显示动态缩略图。具体要求如下： 在Linux系统中安装FastDFS服务器 可以使用FastDFS自带的工具将文件上传到FastDFS 通过http访问某个图片时，FastDFS通过GraphicsMagick工具生成缩略图，将动态缩略图响应输出 作业要求： 提供Linux下的安装步骤文档 作业提示： 1. GraphicsMagick选择1.3.18版本，安装完成之后测试是否支持png、jpg，以及验证gm是否可用 2. 安装LuaJIT-2.0.4、lua-nginx-module-0.8.10和ngx_devel_kit-0.2.18 3. 安装Nginx选择nginx-1.4.2版本，安装完成后测试lua环境是否支持，比如输出hello world 4. 在nginx中配置l - **Primary Language**: Java - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 2 - **Forks**: 0 - **Created**: 2020-10-04 - **Last Updated**: 2021-11-03 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # lagou-file-upload ## 一、介绍 ### 1.1、视频讲解地址 链接：https://pan.baidu.com/s/1lKGeMgm-Mue6hjzAsbZSIg 提取码：l76s ### 1.2、作业内容介绍 **作业一：** 业务场景：用户上传的头像图片大小不一、手机和PC等设备显示尺寸也存在差异，因此需要能根据http请求指定的尺寸参数对图片进行动态压缩，以达到屏幕适配的作用。 功能需求：搭建FastDFS图片服务器，通过http请求访问服务器中图片时，显示动态缩略图。具体要求如下： - 在Linux系统中安装FastDFS服务器 - 可以使用FastDFS自带的工具将文件上传到FastDFS - 通过http访问某个图片时，FastDFS通过GraphicsMagick工具生成缩略图，将动态缩略图响应输出 作业要求： - 提供Linux下的安装步骤文档 作业提示： \1. GraphicsMagick选择1.3.18版本，安装完成之后测试是否支持png、jpg，以及验证gm是否可用 \2. 安装LuaJIT-2.0.4、lua-nginx-module-0.8.10和ngx_devel_kit-0.2.18 \3. 安装Nginx选择nginx-1.4.2版本，安装完成后测试lua环境是否支持，比如输出hello world \4. 在nginx中配置lua脚本，在lua脚本中使用gm命令完成图片压缩 **作业二：** 使用SpringBoot和OSS实现图片的上传、下载和删除功能， 具体要求如下 - 可以使用postman 发送上传请求 /oss/upload ，实现图片上传到OSS对应的Bucket中 类型检查：必须为 jpg、png 或者 jpeg类型，其它图片类型返回错误提示信息 大小检查：必须小于5M，大于5M时返回错误提示信息 图片名称：生成的文件名，必须保持唯一 - 可以使用postman 发送下载请求 /oss/download，实现图片下载 可以根据图片名进行文件的下载 - 可以使用postman 发送删除请求/oss/delete，实现图片删除 可以根据图片名进行文件的删除 \-------------------------------------------------------------------- 1、提供资料：说明文档，验证及讲解视频。 2、讲解内容包含：题目分析、实现思路、环境介绍。 \------------------------------------------------------------------- ## 二、FastDFS文件存储与裁剪 ### 2.1、安装FastDFS集群 #### 2.1.1、环境准备 3台CentOS 7虚拟机 3个tracker 3个Storage | 192.168.0.227 | 192.168.0.228 | 192.168.0.229 | | --------------- | --------------- | --------------- | | tracker | tracker | tracker | | storage(group1) | storage(group1) | storage(group2) | | nginx | nginx | nginx | 安装单机安装好三台机器，然后配置集群。 #### 2.1.2、安装libfastcommon基础库 注意：如果出现Peer reports incompatible or unsupported protocol version.的git错误：执行yum update -y nss curl libcurl ```shell [root@192 ~]# mkdir /root/fastdfs [root@192 ~]# cd /root/fastdfs/ [root@192 fastdfs]# git clone https://github.com/happyfish100/libfastcommon.git --depth 1 [root@192 fastdfs]# cd libfastcommon/ [root@192 libfastcommon]# ./make.sh && ./make.sh install ``` #### 2.1.3、安装FastDFS ```shell [root@192 libfastcommon]# cd /root/fastdfs/ [root@192 fastdfs]# wget https://github.com/happyfish100/fastdfs/archive/V5.11.tar.gz [root@192 fastdfs]# ls libfastcommon V5.11.tar.gz [root@192 fastdfs]# tar -zvxf V5.11.tar.gz [root@192 fastdfs]# cd fastdfs-5.11/ [root@192 fastdfs-5.11]# ./make.sh && ./make.sh install ``` 配置fdfs 准备配置文件 ```shell cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage.conf cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf cp /root/fastdfs/fastdfs-5.11/conf/http.conf /etc/fdfs cp /root/fastdfs/fastdfs-5.11/conf/mime.types /etc/fdfs ``` tracker.conf修改 ``` [root@192 fdfs]# vim tracker.conf # the base path to store data and log files # 需要保证这个目录存在 base_path=/home/fastdfs ``` storage.conf修改 ``` [root@192 fdfs]# vim storage.conf port=23000 base_path=/home/fastdfs # 数据和日志文件存储根目录 store_path0=/home/fastdfs # 第一个存储目录 tracker_server=192.168.0.229:22122 # http访问文件的端口(默认8888,看情况修改,和nginx中保持一致) http.server_port=8888 ``` 启动 启动之前保证文件存储的路径存在： ```shell [root@192 fdfs]# mkdir /home/fastdfs -p ``` 启动tracker和storage： ```shell /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart /usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart 查看所有运行的端口 netstat -ntlp ``` 安装fastdfs-nginx-module 下载模块之后解压，并修改config文件，nginx模块所包含的位置ngx_module_incs以及核心库CORE_INCS的位置。 ```shell cd /root/fastdfs wget https://github.com/happyfish100/fastdfs-nginx-module/archive/V1.20.tar.gz 解压 tar -xvf V1.20.tar.gz cd fastdfs-nginx-module-1.20/src vim config 修改第5 行 和 15 行 修改成 ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/" CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/" ``` 将mod_fastdfs.conf文件拷贝到/etc/fdfs文件夹下： ```shell cp mod_fastdfs.conf /etc/fdfs/ ``` 修改模块配置mod_fastdfs.conf： 其中url_have_group_name表示url是否带组名。这里我们是带组名的。 ```shell vim /etc/fdfs/mod_fastdfs.conf #需要修改的内容如下 tracker_server=192.168.211.136:22122 url_have_group_name=true store_path0=/home/fastdfs ``` nginx目录 ```shell mkdir -p /var/temp/nginx/client ``` 安装Nginx 将fastdfs的磁盘路径发布为http可以访问的路径。 ```shell cd /root/fastdfs wget http://nginx.org/download/nginx-1.15.6.tar.gz tar -zxvf nginx-1.15.6.tar.gz cd nginx-1.15.6/ yum -y install pcre-devel openssl openssl-devel # 添加fastdfs-nginx-module模块 ./configure --add-module=/root/fastdfs/fastdfs-nginx-module-1.20/src 编译安装 make && make install 查看模块是否安装上 /usr/local/nginx/sbin/nginx -V ``` 修改nginx配置，将group的路径交给ngx_fastdfs_module处理。ngx_fastdfs_module指定了文件的存储路径store_path0，这样就可以定位到文件。 ```shell vim /usr/local/nginx/conf/nginx.conf #添加如下配置 server { listen 8888; server_name localhost; location ~/group[0-9]/ { ngx_fastdfs_module; } } ``` 启动nginx ```shell [root@192 local]# /usr/local/nginx/sbin/nginx ngx_http_fastdfs_set pid=34105 ``` #### 2.1.4、配置tracker集群 vim /etc/fdfs/tracker.conf ```properties store_lookup=0 # 0是轮询，1是指定组,2是剩余存储空间多的group优先 ``` 在第一步时，我们采用2，这里我们改为0，做轮询测试。实际开发中根据业务需求进行配置。这里为了方便测试。 #### 2.1.5、配置storage集群 vim /etc/fdfs/storage.conf ```properties tracker_server=192.168.0.227:22122 tracker_server=192.168.0.228:22122 tracker_server=192.168.0.229:22122 group_name=group1 #注意组名 192.168.0.229 配置是 group2 port=23000 #storage 的端口号,同一个组的 storage 端口号必须相同 ``` 修改完之后重启三台机器： ```shell /usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart /usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart ``` #### 2.1.6、检查日志 cat /home/fastdfs/logs/storaged.log 访问FastDFS时 可以把Tracker 理解成对等的,但实际底层多个Tracker时在运行过程中会选择其中一个 作为Leader，由该Leader执行一些唯一性的操作。在早期版本中Tracker-Leader有两个作用，分别 是：为新加入的Storage分配一个源Storage；为开启合并存储的Group选择Trunk-Server。但是在最新 的版本中实际上只有第二个作用，也就是选择Trunk-Server。 查看之后 如果连接不上 注意防火墙 。 systemctl stop firewalld 此外注意第一次安装时，可以kill-9 然后重启进程。 #### 2.1.7、查看集群信息 [root@localhost ~]# /usr/bin/fdfs_monitor /etc/fdfs/storage.conf [2020-09-30 14:31:18] DEBUG - base_path=/home/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=3, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0 server_count=3, server_index=0 tracker server is 192.168.0.227:22122 group count: 2 Group 1: group name = group1 disk total space = 8905 MB disk free space = 1531 MB trunk free space = 0 MB storage server count = 2 active server count = 2 storage server port = 23000 storage HTTP port = 8888 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0 Storage 1: id = 192.168.0.227 ip_addr = 192.168.0.227 (anantes-651-1-49-net.w2-0.abo.wanadoo.fr) ACTIVE http domain = version = 5.11 join time = 2020-09-30 10:07:36 up time = 2020-09-30 14:30:09 total storage = 8905 MB free storage = 1560 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 1 connection.max_count = 1 total_upload_count = 2 success_upload_count = 2 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 122544 success_upload_bytes = 122544 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 61272 success_sync_in_bytes = 61272 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 3 success_file_open_count = 3 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 3 success_file_write_count = 3 last_heart_beat_time = 2020-09-30 14:31:09 last_source_update = 2020-09-30 10:47:09 last_sync_update = 2020-09-30 11:28:36 last_synced_timestamp = 1970-01-01 08:00:00 (never synced) Storage 2: id = 192.168.0.228 ip_addr = 192.168.0.228 (anantes-651-1-49-net.w2-0.abo.wanadoo.fr) ACTIVE http domain = version = 5.11 join time = 2020-09-30 10:08:33 up time = 2020-09-30 11:28:04 total storage = 8905 MB free storage = 1531 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 1 connection.max_count = 1 total_upload_count = 1 success_upload_count = 1 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 61272 success_upload_bytes = 61272 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 122544 success_sync_in_bytes = 122544 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 3 success_file_open_count = 3 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 3 success_file_write_count = 3 last_heart_beat_time = 2020-09-30 14:30:57 last_source_update = 2020-09-30 10:15:20 last_sync_update = 2020-09-30 11:28:56 last_synced_timestamp = 2020-09-30 10:47:10 (-1s delay) Group 2: group name = group2 disk total space = 8905 MB disk free space = 1463 MB trunk free space = 0 MB storage server count = 1 active server count = 1 storage server port = 23000 storage HTTP port = 8888 store path count = 1 subdir count per path = 256 current write server index = 0 current trunk file id = 0Storage 1: id = 192.168.0.229 ip_addr = 192.168.0.229 (anantes-651-1-49-net.w2-0.abo.wanadoo.fr) ACTIVE http domain = version = 5.11 join time = 2020-09-30 11:02:18 up time = 2020-09-30 14:25:44 total storage = 8905 MB free storage = 1463 MB upload priority = 10 store_path_count = 1 subdir_count_per_path = 256 storage_port = 23000 storage_http_port = 8888 current_write_path = 0 source storage id = if_trunk_server = 0 connection.alloc_count = 256 connection.current_count = 0 connection.max_count = 0 total_upload_count = 0 success_upload_count = 0 total_append_count = 0 success_append_count = 0 total_modify_count = 0 success_modify_count = 0 total_truncate_count = 0 success_truncate_count = 0 total_set_meta_count = 0 success_set_meta_count = 0 total_delete_count = 0 success_delete_count = 0 total_download_count = 0 success_download_count = 0 total_get_meta_count = 0 success_get_meta_count = 0 total_create_link_count = 0 success_create_link_count = 0 total_delete_link_count = 0 success_delete_link_count = 0 total_upload_bytes = 0 success_upload_bytes = 0 total_append_bytes = 0 success_append_bytes = 0 total_modify_bytes = 0 success_modify_bytes = 0 stotal_download_bytes = 0 success_download_bytes = 0 total_sync_in_bytes = 0 success_sync_in_bytes = 0 total_sync_out_bytes = 0 success_sync_out_bytes = 0 total_file_open_count = 0 success_file_open_count = 0 total_file_read_count = 0 success_file_read_count = 0 total_file_write_count = 0 success_file_write_count = 0 last_heart_beat_time = 2020-09-30 14:31:15 last_source_update = 1970-01-01 08:00:00 last_sync_update = 1970-01-01 08:00:00 last_synced_timestamp = 1970-01-01 08:00:00 #### 2.1.8、测试上传 ```shell vim /etc/fdfs/client.conf tracker_server=192.168.0.227:22122 tracker_server=192.168.0.228:22122 tracker_server=192.168.0.229:22122 ``` 上传，可以看见是轮询group1和group2： ```shell [root@192 ~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /root/fastdfs/1.png group1/M00/00/00/wKgA4190KTWAMniWAADvWAVnr0Y666.png [root@192 ~]# /usr/bin/fdfs_upload_file /etc/fdfs/client.conf /root/fastdfs/1.png group2/M00/00/00/wKgA5V90KVeAJZtlAADvWAVnr0Y978.png ``` #### 2.1.9、Nginx集群配置 实现在网页上访问上传的图片。 修改/etc/fdfs/mod_fastdfs.conf文件 ```shell vim /etc/fdfs/mod_fastdfs.conf tracker_server=192.168.0.227:22122 tracker_server=192.168.0.228:22122 tracker_server=192.168.0.229:22122 group_name=group1 #注意组名 如果是group2 则一定要改 ``` 通过任意一台机器访问之前上传的文件： http://192.168.0.227:8888/group2/M00/00/00/wKgA5V90KVeAJZtlAADvWAVnr0Y978.png http://192.168.0.228:8888/group2/M00/00/00/wKgA5V90KVeAJZtlAADvWAVnr0Y978.png http://192.168.0.229:8888/group2/M00/00/00/wKgA5V90KVeAJZtlAADvWAVnr0Y978.png 227和228是同一个组，那么两个均存在wKgA4190KTWAMniWAADvWAVnr0Y666.png文件： ```shell [root@192 ~]# find / -name wKgA4190KTWAMniWAADvWAVnr0Y666.png /home/fastdfs/data/00/00/wKgA4190KTWAMniWAADvWAVnr0Y666.png ``` ### 2.2、安装GraphicsMagick #### 2.2.1、下载GraphicsMagick 下载地址：https://master.dl.sourceforge.net/project/graphicsmagick/graphicsmagick-history/1.3/GraphicsMagick-1.3.18.tar.gz 通过XFtp上传到三台FastDFS服务器的/usr/local目录下。 #### 2.2.2、安装相关依赖 ```shell yum install -y gcc gcc-c++ zlib zlib-devel openssl openssl-devel pcre pcre-devel gd-devel #更新libvirt-client yum -y update libvirt-client yum install -y libpng libjpeg libpng-devel libjpeg-devel ghostscript libtiff libtiff-devel freetype freetype-devel readline-devel ncurses-devel ``` #### 2.2.3、安装GraphicsMagick ```shell tar -zvxf GraphicsMagick-1.3.18.tar.gz cd /usr/local/GraphicsMagick-1.3.18 ./configure --prefix=/usr/local/GraphicsMagick-1.3.18 --enable-shared make && make install ln -s /usr/local/GraphicsMagick-1.3.18 /usr/local/GraphicsMagick ``` #### 2.2.4、测试安装效果 （1）查看GM版本 ```shell [root@localhost local]# /usr/local/GraphicsMagick/bin/gm version GraphicsMagick 1.3.18 2013-03-10 Q8 http://www.GraphicsMagick.org/ Copyright (C) 2002-2013 GraphicsMagick Group. Additional copyrights and licenses apply to this software. See http://www.GraphicsMagick.org/www/Copyright.html for details. Feature Support: Thread Safe yes Large Files (> 32 bit) yes Large Memory (> 32 bit) yes BZIP no DPS no FlashPix no FreeType yes Ghostscript (Library) no JBIG no JPEG-2000 no JPEG yes Little CMS no Loadable Modules no OpenMP yes (201107) PNG yes TIFF yes TRIO no UMEM no WMF no X11 yes XML no ZLIB yes ``` （2）查看图片信息 ```shell [root@localhost local]# gm identify /home/fastdfs/data/00/00/wKgA5V90KVeAJZtlAADvWAVnr0Y978.png /home/fastdfs/data/00/00/wKgA5V90KVeAJZtlAADvWAVnr0Y978.png PNG 1218x745+0+0 DirectClass 8-bit 59.8K 0.000u 0:01 ``` （3）生成缩略图 ```shell gm convert -resize 300x300 ./input.png output.png [root@localhost local]# gm identify output.png output.png PNG 300x183+0+0 DirectClass 8-bit 20.0K 0.000u 0:01 gm convert -strip -thumbnail 383x483 input.png output_383x483.jpg ``` 将原图像转换成了300x300大小的图片。 #### 2.2.5、常用命令 - `benchmark`: 测量和报告实用程序命令的性能 - `batch`：在交互式或批处理模式中发出多个命令 - `convert`：转换图像或图像序列，模糊，裁剪，驱除污点，抖动，临近，图片上画图片，加入新图片，生成缩略图等 - `identify`：描述一个或较多图像文件的格式和特性 - `mogrify`：变换一个图像或图像序列，模糊，裁剪，抖动等，Mogrify改写最初的图像文件然后写到一个不同的图像文件 - `composite`：将多个图片组合一起 - `montage`：从不同的图像创建一个复合图像（在一个网格中） - `compare`：比较两个图像使用统计或视觉差 - `display`：在运行X server 的工作站上显示图像 - `animate`：在运行X服务器的任何工作站上显示一个图像序列 - `import`：在X server或任何可见的窗口上输出图片文件，你可以捕获单一窗口，整个的荧屏或任何荧屏的矩形部分 - `conjure`：解释执行 MSL (Magick Scripting Language) 写的脚本 - `time`: 执行一个实用程序命令的时间 - `version`: 报告graphicsmagick版本，功能，和编译选项。 #### 2.2.6、常用参数 - `-crop` x{+-}{+-}{%} 宽x高+起点横坐标+起点纵坐标：裁剪图像的大小和位置 - `-resize` x{%}{@}{!}{<}{>} 宽x高!：改变尺寸，如果使用惊叹号，表示不保留视觉比例，强行改变尺寸匹配给定的宽和高；如果仅给定宽或者高，如“宽x”或“x高”形式的参数(“x高”与“宽x高”的效果是一样的)，则以已知参数为基准按比例改变尺寸。 - `-colors` 颜色数：设定图片采用的颜色数，如果是生成png或gif图片应指定这个参数 - `-quality` JPEG/MIFF/PNG/TIFF的压缩级别，设定图片输出质量，推荐采用80，省略的话默认质量是95，生成图片过大 - `+profile "*"`：图片中不存储Exif信息，去掉图片中所有的配置信息 - `-strip`: 从映像中删除所有配置文件和文本属性，可以很大降低图片大小 - `-thumbnail` 快速裁剪图片 ### 2.3、安装LuaJIT-2.0.4、lua-nginx-module-0.8.10和ngx_devel_kit-0.2.18 #### 2.3.1、安装Lua5.3.1 下载： ```shell [root@localhost local]# wget http://www.lua.org/ftp/lua-5.3.1.tar.gz ``` 安装： ```shell [root@localhost local]# tar -zvxf lua-5.3.1.tar.gz [root@localhost local]# cd lua-5.3.1/ [root@localhost lua-5.3.1]# make linux && make install ``` #### 2.3.2、安装LuaJIT-2.0.4 下载： ``` wget http://luajit.org/download/LuaJIT-2.0.4.tar.gz ``` 安装： ```shell [root@localhost local]# tar -zvxf LuaJIT-2.0.4.tar.gz [root@localhost local]# cd LuaJIT-2.0.4/ [root@localhost LuaJIT-2.0.4]# make && make install [root@localhost nginx-1.4.2]# ln -s /usr/local/lib/libluajit-5.1.so.2 /lib64/libluajit-5.1.so.2 [root@localhost ~]# cat /etc/ld.so.conf include ld.so.conf.d/*.conf [root@localhost ~]# echo "/usr/local/lib" >> /etc/ld.so.conf ``` 配置环境变量： ```shell vim /etc/profile #配置环境变量 export LUAJIT_LIB=/usr/local/lib export LUAJIT_INC=/usr/local/include/luajit-2.0 source /etc/profile ln -s /usr/local/lib/libluajit-5.1.so.2 /lib64/libluajit-5.1.so.2 ``` #### 2.3.3、安装lua-nginx-module-0.8.10和ngx_devel_kit_0.2.18插件 ngx_devel_kit_0.2.18下载： https://codeload.github.com/vision5/ngx_devel_kit/tar.gz/v0.2.18 lua-nginx-module-0.8.10下载： https://github.com/chaoslawful/lua-nginx-module/archive/v0.8.10.tar.gz 将两个源码包下载到/usr/local/src目录下并解压： ```shell tar -zxvf ngx_devel_kit-0.2.18.tar.gz tar -zxvf lua-nginx-module-0.8.10.tar.gz ``` nginx1.4.2下载： wget http://nginx.org/download/nginx-1.4.2.tar.gz 安装Nginx： ```shell yum -y install pcre-devel openssl openssl-devel wget http://nginx.org/download/nginx-1.4.2.tar.gz tar -zvxf nginx-1.4.2.tar.gz cd nginx-1.4.2/ ./configure --prefix=/usr/local/nginx --add-module=/root/fastdfs/fastdfs-nginx-module-1.20/src --add-module=/usr/local/src/lua-nginx-module-0.8.10 --add-module=/usr/local/src/ngx_devel_kit-0.2.18 # 编译 make && make install # 查看Nginx是否安装上 [root@localhost nginx-1.4.2]# /usr/local/nginx/sbin/nginx -V nginx version: nginx/1.4.2 built by gcc 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) configure arguments: --prefix=/usr/local/nginx --add-module=/root/fastdfs/fastdfs-nginx-module-1.20/src --add-module=/usr/local/src/lua-nginx-module-0.8.10 --add-module=/usr/local/src/ngx_devel_kit-0.2.18 ``` 测试lua环境： 修改nginx.conf配置文件，增加hello请求时返回lua打印的内容： ```shell vim /usr/local/nginx/conf/nginx.conf server { listen 8888; server_name localhost; location /hello { default_type 'text/plain'; content_by_lua 'ngx.say("hello,lua")'; } } ``` 启动nginx: ```shell [root@localhost nginx-1.4.2]# /usr/local/nginx/sbin/nginx ``` curl访问该hello，打印测试内容证明lua环境没有问题： ```shell [root@localhost nginx-1.4.2]# curl http://192.168.0.229:8888/hello hello,lua ``` ### 2.4、nginx中配置lua脚本 #### 2.4.1、配置Nginx的lua脚本 ```shell location ~/group[0-9]/M00 { alias /home/fastdfs/data; set $image_root "/home/fastdfs/data"; if ($uri ~ "/([a-zA-Z0-9]+)/([a-zA-Z0-9]+)/([a-zA-Z0-9]+)/([a-zA-Z0-9]+)/(.*)") { set $image_dir "$image_root/$3/$4/"; set $image_name "$5"; set $file "$image_dir$image_name"; } if (!-f $file) { # 关闭lua代码缓存，方便调试lua脚本 #lua_code_cache off; content_by_lua_file "/usr/local/nginx/conf/lua/fastdfs.lua"; } ngx_fastdfs_module; } ``` 改变目录权限： ```shell [root@localhost lua]# chmod -R 777 /home/fastdfs/data/ ``` #### 2.4.2、配置fastdfs.lua脚本 下载nginx-lua-fastdfs-GraphicsMagick脚本： ```shell [root@localhost fastdfs]# git clone https://github.com/hpxl/nginx-lua-fastdfs-GraphicsMagick.git ``` 修改fastdfs.lua脚本内容，并拷贝到/usr/local/nginx/conf/lua目录下： ```shell [root@localhost lua]# mkdir /usr/local/nginx/conf/lua/ -p [root@localhost fastdfs]# cd nginx-lua-fastdfs-GraphicsMagick/lua [root@localhost lua]# vim fastdfs.lua [root@localhost lua]# cp fastdfs.lua /usr/local/nginx/conf/lua/ ``` fastdfs.lua修改两个地方，一处为tracker的ip地址，另外一处为gm的工作路径。 ```lua -- 写入文件 local function writefile(filename, info) local wfile=io.open(filename, "w") --写入文件(w覆盖) assert(wfile) --打开时验证是否出错 wfile:write(info) --写入传入的内容 wfile:close() --调用结束后记得关闭 end -- 检测路径是否目录 local function is_dir(sPath) if type(sPath) ~= "string" then return false end local response = os.execute( "cd " .. sPath ) if response == 0 then return true end return false end -- 检测文件是否存在 local file_exists = function(name) local f=io.open(name,"r") if f~=nil then io.close(f) return true else return false end end local area = nil local originalUri = ngx.var.uri; local originalFile = ngx.var.file; local index = string.find(ngx.var.uri, "([0-9]+)x([0-9]+)"); if index then originalUri = string.sub(ngx.var.uri, 0, index-2); area = string.sub(ngx.var.uri, index); index = string.find(area, "([.])"); area = string.sub(area, 0, index-1); local index = string.find(originalFile, "([0-9]+)x([0-9]+)"); originalFile = string.sub(originalFile, 0, index-2) end -- check original file if not file_exists(originalFile) then local fileid = string.sub(originalUri, 2); -- main local fastdfs = require('restyfastdfs') local fdfs = fastdfs:new() fdfs:set_tracker("192.168.0.229", 22122) fdfs:set_timeout(1000) fdfs:set_tracker_keepalive(0, 100) fdfs:set_storage_keepalive(0, 100) local data = fdfs:do_download(fileid) if data then -- check image dir if not is_dir(ngx.var.image_dir) then os.execute("mkdir -p " .. ngx.var.image_dir) end writefile(originalFile, data) end end -- 创建缩略图 local image_sizes = {"80x80", "800x600", "300x300", "40x40", "60x60"}; function table.contains(table, element) for _, value in pairs(table) do if value == element then return true end end return false end if table.contains(image_sizes, area) then local command = "/usr/local/GraphicsMagick/bin/gm convert " .. originalFile .. " -thumbnail " .. area .. " -background gray -gravity center -extent " .. area .. " " .. ngx.var.file; os.execute(command); end; if file_exists(ngx.var.file) then --ngx.req.set_uri(ngx.var.uri, true); ngx.exec(ngx.var.uri) else ngx.exit(404) end ``` #### 2.4.3、测试裁剪效果 postMan输入http://192.168.0.229:8888/group2/M00/00/00/wKgA419z8a6AJ3xDAADvWAVnr0Y719.png_300x300.png并发起get请求。 ![image-20201009153156914](images/image-20201009153156914.png) ### 2.5、集群其它节点安装裁剪功能 分别在其它连个节点安装GraphicsMagick以及Lua环境，并且重新编译1.4.2版本的Nginx。 安装完成之后，对group1和group2组的图片分别裁剪功能测试。 ## 三、OSS文件上传与下载 ### 3.1、环境准备 #### 3.1.1、创建OSS账号并生成accessKey 登录阿里云服务器，注册账号并开通OSS服务。 创建lagou-lcj-imgs的bucket。 通过账号的AccessKey管理功能生成accessKey。 ```properties accessKeyId=LTAI4GLCEjnN1JJn9Lfy3Wsz accessKeySecret=R3W1TDzVjv6jN2Enx0YH7PoGuiZ8yR ``` #### 3.1.2、创建SpringBoot工程整合OSS ```xml 4.0.0 com.lagou.edu lagou-oss-fileupload 1.0-SNAPSHOT org.springframework.boot spring-boot-starter-parent 2.1.0.RELEASE org.springframework.boot spring-boot-starter-test test org.springframework.boot spring-boot-starter-web com.aliyun.oss aliyun-sdk-oss 3.8.0 org.apache.commons commons-lang3 3.7 org.projectlombok lombok 1.18.4 joda-time joda-time 2.9.9 ``` #### 3.1.3、配置阿里云环境以及初始化OSSClient对象 新增aliyun.properties文件，配置阿里云信息，并构建AliyunConfig 配置类装配OSSClient客户端以及bucket信息。 ```properties aliyun.endpoint=https://oss-cn-hangzhou.aliyuncs.com aliyun.accessKeyId=LTAI4GLCEjnN1JJn9Lfy3Wsz aliyun.accessKeySecret=R3W1TDzVjv6jN2Enx0YH7PoGuiZ8yR aliyun.bucketName=lagou-lcj-imgs aliyun.urlPrefix=https://lagou-lcj-imgs.oss-cn-hangzhou.aliyuncs.com/ ``` AliyunConfig配置接口： ```java package com.lagou.edu.oss.fileupload.config; import com.aliyun.oss.OSSClient; import com.aliyun.oss.OSSClientBuilder; import lombok.Data; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.PropertySource; @Configuration @PropertySource("classpath:aliyun.properties") @ConfigurationProperties(prefix = "aliyun") @Data public class AliyunConfig { private String endpoint; private String accessKeyId; private String accessKeySecret; private String bucketName; private String urlPrefix; @Bean public OSSClient ossClient(){ return (OSSClient)new OSSClientBuilder().build(endpoint,accessKeyId,accessKeySecret); } } ``` #### 3.1.4、application.yaml ```yaml server: port: 8899 ``` #### 3.1.5、SpringBoot启动类 ```java package com.lagou.edu.oss.fileupload; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication public class OssFileuploadApplication { public static void main(String[] args) { SpringApplication.run(OssFileuploadApplication.class,args); } } ``` ### 3.2、业务实现 #### 3.2.1、UpLoadResult返回结果对象 ```java package com.lagou.edu.oss.fileupload.bean; import lombok.Data; @Data public class UpLoadResult { // 文件唯一标识 private String uid; // 文件名 private String name; // 状态有：uploading done error removed private String status; // 服务端响应内容，如：'{"status": "success"}' private String response; } ``` #### 3.2.2、FileUpLoadService接口 ```java package com.lagou.edu.oss.fileupload.service; import com.aliyun.oss.OSSClient; import com.aliyun.oss.model.OSSObject; import com.lagou.edu.oss.fileupload.bean.UpLoadResult; import com.lagou.edu.oss.fileupload.config.AliyunConfig; import org.apache.commons.lang3.StringUtils; import org.joda.time.DateTime; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import org.springframework.web.multipart.MultipartFile; import java.io.*; import java.util.UUID; @Service public class FileUpLoadService { @Autowired private AliyunConfig aliyunConfig; @Autowired private OSSClient ossClient; // 允许上传的格式 private static final String[] IMAGE_TYPE = new String[]{".bmp", ".jpg", ".jpeg", ".gif", ".png"}; /** * 上传文件 * @param uploadFile * @return */ public UpLoadResult upload(MultipartFile uploadFile){ // 校验图片格式 boolean isLegal = false; for (String type : IMAGE_TYPE){ if (StringUtils.endsWithIgnoreCase(uploadFile.getOriginalFilename(), type)) { isLegal = true; break; } } UpLoadResult upLoadResult = new UpLoadResult(); if (!isLegal) { upLoadResult.setStatus("error"); return upLoadResult; } String fileName = uploadFile.getOriginalFilename(); String filePath = getFilePath(fileName); try { ossClient.putObject(aliyunConfig.getBucketName(),filePath , new ByteArrayInputStream(uploadFile.getBytes())); } catch (IOException e) { e.printStackTrace(); // 上传失败 upLoadResult.setStatus("error"); return upLoadResult; } upLoadResult.setStatus("done"); upLoadResult.setName(this.aliyunConfig.getUrlPrefix() +filePath); upLoadResult.setUid(String.valueOf(System.currentTimeMillis())); return upLoadResult; } /** * 调用getObject方法，将 * @param firstKey * @return */ public byte[] download(String firstKey){ try{ OSSObject object = ossClient.getObject(aliyunConfig.getBucketName(), firstKey); InputStream objectContent = object.getObjectContent(); ByteArrayOutputStream outputStream = new ByteArrayOutputStream(); byte[] buffer = new byte[1024]; int length = 0; while ((length = objectContent.read(buffer))!=-1){ outputStream.write(buffer,0,length); } objectContent.close(); outputStream.flush(); outputStream.close(); return outputStream.toByteArray(); }catch (Exception e){ return null; } } /** * 调用deleteObject删除文件 * @param fileKey */ public UpLoadResult delete(String fileKey){ UpLoadResult upLoadResult = new UpLoadResult(); boolean objectExist = ossClient.doesObjectExist(aliyunConfig.getBucketName(), fileKey); if (!objectExist) { upLoadResult.setStatus("removed"); upLoadResult.setResponse("File " + fileKey + " is not exists!"); return upLoadResult; } ossClient.deleteObject(aliyunConfig.getBucketName(),fileKey); upLoadResult.setStatus("done"); upLoadResult.setUid(String.valueOf(System.currentTimeMillis())); return upLoadResult; } private String getFilePath(String sourceFileName) { DateTime dateTime = new DateTime(); return "images/" + dateTime.toString("yyyy") + "/" + dateTime.toString("MM") + "/" + dateTime.toString("dd") + "/" + UUID.randomUUID().toString()+ "." + StringUtils.substringAfterLast(sourceFileName,"."); } } ``` #### 3.2.3、UpLoadController ```java package com.lagou.edu.oss.fileupload.controller; import com.lagou.edu.oss.fileupload.bean.UpLoadResult; import com.lagou.edu.oss.fileupload.service.FileUpLoadService; import org.apache.commons.lang3.StringUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Controller; import org.springframework.web.bind.annotation.*; import org.springframework.web.multipart.MultipartFile; import javax.servlet.ServletOutputStream; import javax.servlet.http.HttpServletResponse; import java.io.IOException; import java.io.UnsupportedEncodingException; import java.net.URLEncoder; @RequestMapping("/oss") @Controller public class UpLoadController { @Autowired private FileUpLoadService fileUpLoadService; /** * 文件上传 * @param multipartFile 上传文件 * @return */ @PostMapping("/upload") @ResponseBody public UpLoadResult upload(@RequestParam("file") MultipartFile multipartFile){ return this.fileUpLoadService.upload(multipartFile); } /** * 文件下载 * @param fileKey 文件路径 * @param response * @throws UnsupportedEncodingException */ @GetMapping("/download") public void download(String fileKey, HttpServletResponse response) throws UnsupportedEncodingException { byte []bytes = this.fileUpLoadService.download(fileKey); if (bytes == null) { throw new RuntimeException("404"); } response.setHeader("Content-disposition", "attachment;filename=" + URLEncoder.encode(StringUtils.substringAfterLast(fileKey,"/"), "UTF-8")); response.setCharacterEncoding("UTF-8"); ServletOutputStream outputStream = null; try { outputStream = response.getOutputStream(); outputStream.write(bytes); } catch (IOException e) { e.printStackTrace(); } finally { try { outputStream.flush(); outputStream.close(); } catch (IOException e) { e.printStackTrace(); } } } /** * 删除文件 * @param fileKey 文件路径 * @return */ @PostMapping("/delete") @ResponseBody public UpLoadResult delete(String fileKey){ return this.fileUpLoadService.delete(fileKey); } } ``` ### 3.3、测试 #### 3.3.1、上传文件 ![image-20201010172153151](images/image-20201010172153151.png) 上传结果： ```json { "uid": "1602321665093", "name": "https://lagou-lcj-imgs.oss-cn-hangzhou.aliyuncs.com/images/2020/10/10/b115c9ee-e34d-46d9-ae57-3019b07747a5.png", "status": "done", "response": null } ``` #### 3.3.2、下载文件 ![image-20201010172456949](images/image-20201010172456949.png) #### 3.3.3、删除文件 ![image-20201010172723176](images/image-20201010172723176.png)