1 Star 0 Fork 259

金学 / gobrs-async

forked from dromara / gobrs-async 
加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

资源索引

本框架是什么

Gobrs-Async 是一款功能强大、配置灵活、带有全链路异常回调、内存优化、异常状态管理于一身的高性能异步编排框架。为企业提供在复杂应用场景下动态任务编排的能力。 针对于复杂场景下,异步线程复杂性、任务依赖性、异常状态难控制性; Gobrs-Async 为此而生。

能解决什么问题

能解决 CompletableFuture 所不能解决的问题。 怎么理解呢?

传统的FutureCompleteableFuture一定程度上可以完成任务编排,并可以把结果传递到下一个任务。如CompletableFuture有then方法,但是却无法做到对每一个执行单元的回调。譬如A执行完毕成功了,后面是B,我希望A在执行完后就有个回调结果,方便我监控当前的执行状况,或者打个日志什么的。失败了,我也可以记录个异常信息什么的。

此时,CompleteableFuture就无能为力了。

Gobrs-Async框架提供了这样的回调功能。并且,如果执行成功、失败、异常、超时等场景下都提供了管理线程任务的能力!

场景概述

场景一

场景一

说明 任务A 执行完了之后,继续执行 B、C、D

场景二

场景二

说明 任务A 执行完了之后执行B 然后再执行 C、D

场景三

场景二

说明 任务A 执行完了之后执行B、E 然后按照顺序 B的流程走C、D、G。 E的流程走F、G

还有更多场景,如果你想详细理解任务编排的概念, 请仔细阅读文档,或者通过资源索引导航到官网了解全貌!

为什么写这个项目

在开发复杂中台业务过程中,难免会遇到调用各种中台业务数据, 而且会出现复杂的中台数据依赖关系,在这种情况下。代码的复杂程度就会增加。 如下图所示: 1.1

在电商平台业务中, 各中台数据可能依赖 商品Product 数据,而且需要依赖特殊属性中 Item的数据。(有朋友会问,为什么Product 数据不和 Item数据出自同一个中台呢?中台业务发展是多样性的,不同业务中台设计方式不同 , 难道我们就不对接了吗?所以我们要针对于这种复杂多变的中台业务数据提供技术支撑才是一个合格的开发者应该做的)而且Item数据是HTTP的服务,但Product 是RPC服务。 如果按照Future的 开发方式。我们可能会这样开发


    // 并行处理任务 Product 、 Item 的任务
    @Resource
    List<ParaExector> paraExectors;

    // 依赖于Product 和 Item的 任务
    @Resource
    List<SerExector> serExectors;

    public void testFuture(HttpServletRequest httpServletRequest) {
        DataContext dataContext = new DataContext();
        dataContext.setHttpServletRequest(httpServletRequest);
        List<Future> list = new ArrayList<>();
        for (AsyncTask asyncTask : paraExectors) {
            Future<?> submit = gobrsThreadPoolExecutor.submit(() -> {
                asyncTask.task(dataContext, null);
            });
            list.add(submit);
        }
        for (Future future : list) {
            try {
                future.get();
            } catch (InterruptedException e) {
                e.printStackTrace();
            } catch (ExecutionException e) {
                e.printStackTrace();
            }
        }

        List<Future> ser = new ArrayList<>();
        for (AsyncTask asyncTask : serExectors) {
            Future<?> submit = gobrsThreadPoolExecutor.submit(() -> {
                asyncTask.task(dataContext, null);
            });
            ser.add(submit);
        }
        for (Future future : ser) {
            try {
                future.get();
            } catch (InterruptedException e) {
                e.printStackTrace();
            } catch (ExecutionException e) {
                e.printStackTrace();
            }
        }
    }

存在的问题

以上示例中,Product数据是通过RPC 方式获取, Item是通过HTTP服务获取,大家都知道, RPC性能要高于HTTP性能。 但是通过Future 的方式, get会阻塞等待 Item数据返回后才会往下执行。 这样的话, 图书音像、装修数据、限购数据等都要等待Item数据返回,但是这些中台并不依赖Item返回的数据, 所以会产生等待时间影响系统整体QPS。

起源

起源

  • 作者通过对开源中间件的源码详细阅读和二次开发的经验和使用心得总结而来。
  • 用户的一些使用体验 包括业务的需求

Gobrs-Async 核心能力

核心能力

业界对比

在开源平台找了挺多任务异步编排框架,发现都不是很理想,唯一一款现阶段比较好用的异步编排框架就数asyncTool比较好用。但是在使用时发现,API不是很好用。而且需要频繁的创建 WorkerWrapper 对象 用起来有点不爽。对于业务比较复杂的场景,在开发时需要写较多的 WorkerWrapper 代码,而且框架不能对全局异常进行拦截。只能通过任务的 result 方法捕捉单任务的异常。 不能在任意任务出现异常后 停止全局的异步任务。同时无法在发生全局异常的时候进行异常拦截。如果需要实现在发生需停止全局任务流的时候,发送报警邮件的功能。 asyncTool就显得力不从心了。

asyncTool 本身已经功能很强大了,本人与asyncTool 作者 都是在京东担任研发工作。 涉及到的场景大同小异。 会有很多业务场景,很复杂的中台接口调用关系。所以针对当前业务场景。需要探索更多的技术领域。本身技术是应该服务于业务,落地业务场景。

想给项目起一个简单易记的名字,类似于 Eureka、Nacos、Redis;经过再三考虑后,决定命名:Gobrs-Async

功能 asyncTool Gobrs-Async sirector
多任务处理
单任务异常回调
全局异常中断
可配置任务流
自定义异常拦截器
内存优化
可选的任务执行

它解决了什么问题

在请求调用各大中台数据时,难免会出现多个中台数据互相依赖的情况,现实开发中会遇到如下场景。

并行常见的场景 1 客户端请求服务端接口,该接口需要调用其他N个微服务的接口

譬如 请求我的购物车,那么就需要去调用用户的rpc、商品详情的rpc、库存rpc、优惠券等等好多个服务。同时,这些服务还有相互依赖关系,譬如必须先拿到商品id后,才能去库存rpc服务请求库存信息。 最终全部获取完毕后,或超时了,就汇总结果,返回给客户端。

2 并行执行N个任务,后续根据这1-N个任务的执行结果来决定是否继续执行下一个任务

如用户可以通过邮箱、手机号、用户名登录,登录接口只有一个,那么当用户发起登录请求后,我们需要并行根据邮箱、手机号、用户名来同时查数据库,只要有一个成功了,都算成功,就可以继续执行下一步。而不是先试邮箱能否成功、再试手机号……

再如某接口限制了每个批次的传参数量,每次最多查询10个商品的信息,我有45个商品需要查询,就可以分5堆并行去查询,后续就是统计这5堆的查询结果。就看你是否强制要求全部查成功,还是不管有几堆查成功都给客户做返回

再如某个接口,有5个前置任务需要处理。其中有3个是必须要执行完毕才能执行后续的,另外2个是非强制的,只要这3个执行完就可以进行下一步,到时另外2个如果成功了就有值,如果还没执行完,就是默认值。

3 需要进行线程隔离的多批次任务。

如多组任务, 各组任务之间彼此不相关,每组都需要一个独立的线程池,每组都是独立的一套执行单元的组合。有点类似于hystrix的线程池隔离策略。

4 单机工作流任务编排。

5 其他有顺序编排的需求。


它有什么特性

Gobrs-Async 在开发时考虑了众多使用者的开发喜欢,对异常处理的使用场景。并被运用到电商生产环境中,在京东经历这严酷的高并发考验。同时框架中 极简灵活的配置、全局自定义可中断全流程异常、内存优化、灵活的接入方式、提供SpringBoot Start 接入方式。更加考虑使用者的开发习惯。仅需要注入GobrsTask的Spring Bean 即可实现全流程接入。

Gobrs-Async 项目目录及其精简

  • gobrs-async-example:Gobrs-Async 接入实例,提供测试用例。
  • gobrs-async-starter:Gobrs-Async 框架核心组件

Gobrs-Async 在设计时,就充分考虑了开发者的使用习惯, 没有依赖任何中间件。 对并发框架做了良好的封装。主要使用 CountDownLatchReentrantLockvolatile 等一系列并发技术开发设计。

整体架构


1.0

任务触发器

任务流的启动者, 负责启动任务执行流

规则解析引擎

负责解析使用者配置的规则,同时于Spring结合,将配置的 Spring Bean 解析成 TaskBean,进而通过解析引擎加载成 任务装饰器。进而组装成任务树

任务启动器

负责通过使用解析引擎解析的任务树。结合 JUC 并发框架调度实现对任务的统一管理,核心方法有

  • trigger 触发任务加载器,为加载任务准备环境

任务加载器

负责加载任务流程,开始调用任务执行器执行核心流程

  • load 核心任务流程方法,在这里阻塞等待整个任务流程
  • getBeginProcess 获取子任务开始流程
  • completed 任务完成
  • errorInterrupted 任务失败 中断任务流程
  • error 任务失败

任务执行器

最终的任务执行,每一个任务对应一个TaskActuator 任务的 拦截、异常、执行、线程复用 等必要条件判断都在这里处理

  • prepare 任务前置处理
  • preInterceptor 统一任务前置处理
  • task 核心任务方法,业务执行内容
  • postInterceptor 统一后置处理
  • onSuccess 任务执行成功回调
  • onFail 任务执行失败回调

任务总线

任务流程传递总线,包括 请求参数、任务加载器、 响应结果, 该对象暴露给使用者,拿到匹配业务的数据信息,例如: 返回结果、主动中断任务流程等功能 需要任务总线(TaskSupport)支持

核心类图

核心类图

加群沟通

对于这个项目,是否有什么不一样看法,欢迎在 Issue 一起沟通交流; 群二维码七天会失效,可以添加作者微信进交流群





Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

🚀🔥🚀 一款功能强大、配置灵活、带有全链路异常回调、内存优化、异常状态管理的高性能异步编排框架(多线程管理)。为企业(电商)提供在复杂应用场景下动态任务编排的能力。 针对于复杂场景下,异步线程复杂性、任务依赖性、异常状态难控制性; Gobrs-Async 为此而生。 展开 收起
Java
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Java
1
https://gitee.com/kimxue/gobrs-async.git
git@gitee.com:kimxue/gobrs-async.git
kimxue
gobrs-async
gobrs-async
master

搜索帮助