1 Star 4 Fork 0

Charming / SecKill

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

Java秒杀系统优化-Redis缓存-分布式session-RabbitMQ异步下单-页面静态化

项目介绍

基于SpringBoot+Mybatis搭建的秒杀系统,并且针对高并发场景进行了优化,保证线程安全的同时极大地提高了服务器的吞吐量,主要优化手段有页面静态化、Redis缓存(页面缓存、对象缓存)、RabbitMQ异步下单,项目实现的主要功能为 登录-->商品列表浏览-->秒杀-->付款或返回。本项目利用压测工具Jmeter对优化前后的性能做了详细的评测,附带完整的测试报告以及整个系统的设计思路报告。

软件说明

整个项目的结构为严格的Maven项目结构,源码包下的包名即为其主要作用的缩写名,易于理解,概不赘述。

本文重点阐述计思路及优化方案,并附在Jmeter下压测的完整结果报告。

设计方案

以下分功能模块阐述:

一、登录功能及session

登录的设计并不复杂,主要思路为两次MD5+盐化,首先为前端用固定salt+password做一次MD5然后传至后端,后端逻辑首先对用户存在与否做判断,然后取出该用户的salt值,和前端传来的已经一次MD5的password再次MD5,然后比较即可,然后生成session并存储在redis缓存中,返回cookie。值得注意的是在user表中加入随机的salt可极大降低彩虹表攻击的风险,还有就是将session值存储于redis缓存中也极大地降低了对数据库的访问。

二、商品列表、订单详情、商品详情功能

这部分功能逻辑十分简单,即为查数据,然后展示,无须赘述,需注意的是,这部分功能的优化策略,详见后文。

三、秒杀功能

该功能为系统的核心功能,既要保证程序的线程安全性,又要满足高并发场景的需求。 在未优化前,其主要的逻辑为:查库存-->查是否秒杀-->秒杀,又因为该逻辑理论上应为一个原子的操作,所以加锁,或者作为事务,但是这样会大大影响程序的并发性能,所以需做优化。

四、数据库

数据库的主要设计为五张表:goods、miaosha_goods、miaosha_user、miaosha_order、order_info 具体数据表的主要结构,在./sql/中有sql文件,并且./java/util/中有用于生成测试用户的脚本,有兴趣的读者可以查看。

以上即为主要功能,接下来阐述对应的一些优化手段。

优化方案

一、页面缓存

页面缓存的主要思路为,将一些用户经常请求的页面,例如/goods/to_list--商品列表页面,存储到redis缓存中,在用户请求的时候直接在缓存中获取并返回,如果取缓存失败,则利用thymeleaf的手动渲染,渲染后存入缓存,并且返回。我们可以很明显的知道,不使用页面缓存的请求,每次都先访问数据库,然后经thymeleaf渲染,然后返回,其中渲染的过程可能需要从磁盘中读取html模板,而使用页面缓存以后,直接在内存缓存中读取,无需查库和渲染,只有失效的情况下才需要查库渲染,所以在些用户经常请求的页面中使用页面缓存优化,可大大降低对数据库和服务器的压力。(需要注意的是合理的设置页面缓存的有效期)。

二、对象缓存

相对于页面缓存,对象缓存是个更细粒度的缓存,比如说在登录模块中的session中,我们把session对应的user对象存储到redis缓存中,那么在需要user对象的页面中,既不需要登录,也不需要更具cookie去查找数据库,只需要通过cookie在redis中获取user对象,即可使用,同理,这样类型的缓存也会减小对数据库的压力。

三、页面静态化

上述的两种缓存,都是利用redis缓存服务器来实现的,虽然可以降低对数据库和服务器的压力,但是,redis服务器的容量和处理能力也是有限的,所以我们可以考虑将页面模板直接缓存到用户的浏览器,那么每次请求用户只需要请求用于渲染的对象即可,这不仅仅减轻了redis服务器的压力,同时也减少了带宽的消耗,此即为页面静态化。 在本项目中,主要实现的是商品详情、订单详情页面、秒杀页面的静态化,主要方法是利用ajax的异步加载,请求渲染需要的对象,并且通过配置 ####### spring.resources的相关参数来告诉浏览器是否缓存,缓存有效时间等等。

四、静态资源优化

主要手段包括JS/CSS压缩,CDN等,此项目中并没有尝试,但不失为优化的另外一些好的思路。

以上部分的优化手段主要为缓存、页面优化等于前端比较接近的手段,对于后端接口的优化将在以下部分阐述

五、接口优化

主要思路为Redis预减库存+RabbitMQ异步下单 具体流程如下:

1、系统初始化,加载库存到redis缓存
2、收到请求,预减库存
3、判断库存,若剩余,则入队列,否则秒杀失败
4、出队下单

分析:

一、通过将库存加载到redis中,使得每次判断、减少库存直接从内存中读取,无需访问数据库

二、收到请求预见库存,然后判断

注意这一顺序非常重要,保证了线程安全

分析:因为redis封装的decr()等函数是线程安全的,无需外加同步,所以你通过decr()减少库存后获取到的库存永远都是你刚刚减少后得到的库存,本身就是个原子操作,不会存在线程安全问题,然后根据这个库存来入队,不符合条件的秒杀请求直接返回失败,极大地减少了服务器的压力,而且整个后台逻辑中,需要保证原子性的也仅仅是decr()这一个操作,并且由于redis经过了乐观锁优化,所以整个系统的并发性相对于自己首先同步代码而言,并发性得到了极大的提高。

三、完成了上述的操作,再去实现接下来的逻辑就很简单了,唯一需要注意的是,从队列中出来的请求执行秒杀过程是一个事务,需完整执行,否则回滚。 同时,订单的详情页面做一个静态化优化,前端轮询秒杀结果,得到结果后进行渲染即可。

以上即为接口优化的阐述,接下来是压测报告。

测试参数

服务器:(Mysq、Redis、RabbitMQ等服务也均安装在本机上
CPU: Hasse/战神Z7m Intel-i7-6700HQ 2.6GHz-3.5Ghz 四核心八线程 三级cache 6M
内存: 8G DDR3L
磁盘: 5400转/s 1TB

Java版本:1.8.0_161 Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
MySQL:5.7.20-log MySQL Community Server (GPL)
Redis:4.0.10 容量--100M
RabbitMQ:3.7.7

Mybatis、Druid等具体配置参数见./resource/application.properties文件

测试方法

测试工具使用Jmeter并发测试工具,为保证无非相关变量的影响,每次测试的并发线程数均设为1000,并发时间1s,并发循环次数为10次,如图所示:
测试线程组

/goods/to_list接口测试

该接口为商品列表接口,对该接口做了页面缓存的优化,分别对优化前,优化后做压测,结果如下:
http请求设置
优化前的测试结果
优化前
优化后的测试结果
优化后

对于上图的两个测试,我们比较关注的是聚合报告里的 ThoughPut 的值,其值可以作为吞吐量的一个较好估计。
由图可见:
优化前,系统吞吐量为:921.1/sec
优化后,系统吞吐量为:4046.9/sec
也就意味着,加速比为:4.39。

/goods/detail/{goodsId}接口测试

该接口为商品详情接口,对该接口实现了页面静态化的处理,分别对优化前,优化后做压测,结果如下:
优化前的测试结果
优化前
优化后的测试结果
优化后

对比以上两图,会发现,似乎区别并不明显,很容易让人得出优化无效的结论,其实不然
据本人分析,这种情况应该是由Jmeter自身导致的,因为Jmeter做测试的时候并不会依赖于其他浏览器,只是发起http请求,而浏览器所具备的一些功能,他并没有,比如,缓存,所有,利用Jmeter进行测试并不能得出一个如意的结果,但是,我们可以通过浏览器来大致的了解页面静态化后的一些改变。如图:
页面静态化后
可以清楚的看到,这个页面大部分的内容都被“已缓存”,只有400个字节左右的对象被请求并传输,这也就达到了我们优化前的目的了。

秒杀接口优化

对于接口测试,我实现准备了1000个user和cookie,具体脚本见./java/util/下代码,在Jmeter中使用参数方法可自行百度,如图:
token

对于秒杀接口,我们只对优化后的接口进行压测,测试的情况分为两类:
1、秒杀库存充足

2、秒杀库存不足
对于秒杀库存充足的情况,我们设置初始的库存为200,然后,并发量和其他测试设置一致,结果如图:
库存充足竞争
对于秒杀库存不足的情况,我们设置初始的库存为0,然后,并发量和其他测试设置一致,结果如图:
库存不足竞争

对于处理逻辑比较复杂的接口而言,最好和最坏的情况能达到这样的一个吞吐量,同时保证线程安全性,比较满意。

以上为全部测试报告,读者若有不明之处,欢迎提问。

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

基于SpringBoot+Mybatis的秒杀系统优化 -Redis缓存-分布式session-RabbitMQ异步下单-页面静态化 展开 收起
Java
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Java
1
https://gitee.com/chenyamin11/SecKill.git
git@gitee.com:chenyamin11/SecKill.git
chenyamin11
SecKill
SecKill
master

搜索帮助