1 Star 0 Fork 2

panza-open / jvm-sandbox-repeater

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

logo

Build Status License GitHub issues

基于JVM-Sandbox的录制/回放通用解决方案

jvm-sandbox-repeaterJVM-Sandbox生态体系下的重要模块,它具备了JVM-Sandbox的所有特点,插件式设计便于快速适配各种中间件,封装请求录制/回放基础协议,也提供了通用可扩展的各种丰富API。

目标人群 - 面向测试开发工程师

  • 线上有个用户请求一直不成功,我想在测试环境Debug一下,能帮我复现一下吗?
  • 压测流量不知道怎么构造,数据结构太复杂,压测模型也难以评估,有什么好的办法吗?
  • 不想写接口测试脚本了,我想做一个流量录制系统,把线上用户场景做业务回归,可能会接入很多服务系统,不想让每个系统都进行改造,有好的框架选择吗?
  • 我想做一个业务监控系统,对线上核心接口采样之后做一些业务校验,实时监控业务正确性。

如果你有以上的想法或需求,jvm-sandbox-repeater 都将是你的不二选择方案;框架基于JVM-Sandbox,拥有JVM-Sandbox的一切特性,同时封装了以下能力:

  • 录制/回放基础协议,可快速配置/编码实现一类中间件的录制/回放
  • 开放数据上报,对于录制结果可上报到自己的服务端,进行监控、回归、问题排查等上层平台搭建

项目简介

repeater的核心能力是什么?

1. 通用录制/回放能力

  • 无侵入式录制HTTP/Java/Dubbo入参/返回值录制能力(业务系统无感知)
  • 基于TTL提供多线程子调用追踪,完整追踪一次请求的调用路径
  • 入口请求(HTTP/Dubbo/Java)流量回放、子调用(Java/Dubbo)返回值Mock能力

2. 快速可扩展API实现

  • 录制/回放插件式架构
  • 提供标准接口,可通过配置/简单编码实现一类通用插件

3. standalone工作模式

  • 无需依赖任何服务端/存储,可以单机工作,提供录制/回放能力

repeater的可以应用到哪些场景?

1. 业务快速回归

  • 基于线上流量的录制/回放,无需人肉准备自动化测试脚本、准备测试数据

2. 线上问题排查

  • 录制回放提供"昨日重现"能力,还原线上真实场景到线下做问题排查和Debug
  • 动态方法入参/返回值录制,提供线上快速问题定位

3. 压测流量准备

  • 0成本录制HTTP/Dubbo等入口流量,作为压测流量模型进行压测

4. 实时业务监控

  • 动态业务监控,基于核心接口数据录制回流到平台,对接口返回数据正确性进行校验和监控

核心原理

流量录制

对于Java调用,一次流量录制包括一次入口调用(entranceInvocation)(eg:HTTP/Dubbo/Java)和若干次子调用(subInvocations)。流量的录制过程就是把入口调用和子调用绑定成一次完整的记录,框架抽象了基础录制协议,调用的组装由调用插件(InvokePlugin)来完成,需要考虑解决的核心问题:

  • 快速开发和适配新插件
  • 绑定入口调用和子调用(解决多线程上下文传递问题)
  • invocation唯一定位,保障回放时精确匹配
  • 自定义流量采样、过滤、发送、存储

框架的核心逻辑录制协议基于JVM-Sandbox的BEFORERETRUNTHROW事件机制进行录制流程控制,详见DefaultEventListener

基于TTL解决跨线程上下文传递问题,开启RepeaterConfig.useTtl之后支持多线程子调用录制

开放插件定义enhance埋点/自定义调用组装方式快速实现插件适配

Invocation抽象Identity统一定位由插件自己扩展实现

基于Tracer实现应用内链路追踪、采样;同时支持多种过滤方式,插件可自由扩展;

public void onEvent(Event event) throws Throwable {
    try {
        /*
         * event过滤;针对单个listener,只处理top的事件
         */
        /** -------- **/
        /*
         * 初始化Tracer开启上下文追踪[基于TTL,支持多线程上下文传递]
         */
        /** -------- **/
        /*
         * 执行基础过滤
         */
        /** -------- **/
        /*
         * 执行采样计算
         */
        /** -------- **/
        /*
         * processor filter
         */
        /** -------- **/
        /*
         * 分发事件处理
         */
    } catch (ProcessControlException pe) {
        /*
         * sandbox流程干预
         */
    } catch (Throwable throwable) {
    	 /*
    	  * 统计异常
    	  */
    } finally {
        /*
         * 清理上下文
         */
    }
}

流量回放

流量回放,获取录制流量的入口调用入参,再次发起调用。注意:读接口或者幂等写接口可以直接回放,否则在生产环境请谨慎使用,可能会造成脏数据;用户可自行选择mock回放或者非mock,回放过程要解决的核心问题:

  • 多种入口(HTTP/Dubbo/Java)的回放发起
  • 自定义回放流量数据来源、回放结果的上报
  • 自定义mock/非mock回放、回放策略
  • 开放回放流程关键节点hook

回放过程通过异步EventBus方式订阅回放请求;基于FlowDispather进行回放流量分发,每个类型回放插件实现RepeaterSPI完成回放请求发起;每次回放请求可决定本地回放是否mock,插件也自由实现mock逻辑,mock流程代码

mock回放:回放流量子调用(eg:mybatis/dubbo)不发生真实调用,从录制子调用中根据 MockStrategy 搜索匹配的子调用,利用JVM-Sandbox的流程干预能力,有匹配结果,进行throwReturnImmediately返回,没有匹配结果则抛出异常阻断流程,避免重复调用污染数据

public void doMock(BeforeEvent event, Boolean entrance, InvokeType type) throws ProcessControlException {
    /*
     * 获取回放上下文
     */
    RepeatContext context = RepeatCache.getRepeatContext(Tracer.getTraceId());
    /*
     * mock执行条件
     */
    if (!skipMock(event, entrance, context) && context != null && context.getMeta().isMock()) {
        try {
            /*
             * 构建mock请求
             */
            final MockRequest request = MockRequest.builder()
                    ...
                    .build();
            /*
             * 执行mock动作
             */
            final MockResponse mr = StrategyProvider.instance().provide(context.getMeta().getStrategyType()).execute(request);
            /*
             * 处理策略推荐结果
             */
            switch (mr.action) {
  					...
            }
        } catch (ProcessControlException pce) {
            throw pce;
        } catch (Throwable throwable) {
            ProcessControlException.throwThrowsImmediately(new RepeatException("unexpected code snippet here.", throwable));
        }
    }
}

合作共建的公司团队

v

已支持的插件列表

Java生态中间件及各种框架众多,各公司技术选型差异较大没办法统一适配,目前适配了几款常用插件作为示例,如有需求可以通过issue方式提交,同时也欢迎大家来贡献开发插件

插件类型 录制 回放 Mock 支持时间 贡献者
http-plugin × 201906 zhaoyb1990
dubbo-plugin × 201906 zhaoyb1990
ibatis-plugin × 201906 zhaoyb1990
mybatis-plugin × 201906 ztbsuper
java-plugin 201906 zhaoyb1990
redis-plugin × 201910 ElesG
hibernate × 201910 zhaoyb1990
spring-data-jpa × 201910 zhaoyb1990

相关文档

钉钉交流群

pic

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright 1999-2019 Alibaba Group Holding Ltd. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

克隆自 https://github.com/alibaba/jvm-sandbox-repeater.git 展开 收起
Java 等 5 种语言
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/panza-open/jvm-sandbox-repeater.git
git@gitee.com:panza-open/jvm-sandbox-repeater.git
panza-open
jvm-sandbox-repeater
jvm-sandbox-repeater
master

搜索帮助

344bd9b3 5694891 D2dac590 5694891