8 Star 61 Fork 29

阿库玛 / dataql

Create your Gitee Account
Explore and code with more than 12 million developers,Free private repositories !:)
Sign up
Clone or Download
contribute
Sync branch
Cancel
Notice: Creating folder will generate an empty file .keep, because not support in Git
Loading...
README
Apache-2.0

DataQL 数据查询引擎

  DataQL(Data Query Language)DataQL 是一种查询语言。旨在通过提供直观、灵活的语法来描述客户端应用程序的数据需求和交互。

  数据的存储根据其业务形式通常是较为简单的,并不适合直接在页面上进行展示。因此开发页面的前端工程师需要为此做大量的工作,这就是 DataQL 极力解决的问题。

  例如:下面这个 DataQL 从 user 函数中查询 id 为 4 的用户相关信息并返回给应用。

return userByID({'id': 4}) => {
    'name',
    'sex' : (sex == 'F') ? '' : '' ,
    'age' : age + ''
}

返回结果:

{
  'name' : '马三',
  'sex'  : '男',
  'age'  : '25岁'
}

特性

  1. 层次结构:多数产品都涉及数据的层次结构,为了保证结构的一致性 DataQL 结果也是分层的。
  2. 数据为中心:前端工程是一个比较典型的场景,但是 DataQL 不局限于此(后端友好性)。
  3. 弱类型定义:语言中不会要求声明任何形式的类型结构。
  4. 简单逻辑:具备简单逻辑处理能力:表达式计算、对象取值、条件分支、lambda和函数。
  5. 编译运行:查询的执行是基于编译结果的。
  6. 混合语言:允许查询中混合任意的其它语言代码,典型的场景是查询中混合 SQL 查询语句。
  7. 类 JS 语法:类JS语法设计,学习成本极低。

样例

public class UserByIdUdf implements Udf {
    public UserInfo call(Hints readOnly, Object[] params) {
        ...
    }
}

public class ConsoleDemo {
    public static void main(String[] args) {
        AppContext appContext = Hasor.create().build((QueryModule) apiBinder -> {
            apiBinder.addShareVarInstance("userByID", new UserByIdUdf());

        });
        DataQL dataQL = appContext.getInstance(DataQL.class);
        QueryResult queryResult = dataQL.createQuery("return userByID({'id': 4}) => {" +
                                                     "    'name'," +
                                                     "    'sex' : (sex == 'F') ? '男' : '女' ," +
                                                     "    'age' : age + '岁'" +
                                                     "}").execute();
        DataModel dataModel = queryResult.getData();
    }
}

子项目:Dataway 数据接口配置服务

  依托 DataQL 服务聚合能力,为应用提供一个 UI 界面。并以 jar 包的方式集成到应用中。 通过 Dataway 可以直接在界面上配置和发布接口。

  这种模式的革新使得开发一个接口不必在编写任何形式的代码,只需要配置一条 DataQL 查询即可完成满足前端对接口的需求。 从而避免了从数据库到前端之间一系列的开发配置任务,例如:Mapper、DO、DAO、Service、Controller 统统不在需要。

  Dataway特意采用了 jar包集成的方式发布,这使得任意的老项目都可以无侵入的集成 Dataway。 直接改进老项目的迭代效率,大大减少企业项目研发成本。

avatar

  如上图所示 Dataway 在开发模式上提供了巨大的便捷。 虽然工作流程中标识了由后端开发来配置 DataQL 接口,但这主要是出于考虑接口责任人。 但在实际工作中根据实际情况需要,配置接口的人员可以是产品研发生命周期中任意一名角色。


主打场景

  主打场景并不是说 Dataway 适用范围仅限于此,而是经过多次项目实践。我们认为下面这些场景会有非常好的预期效果。 比如说 取数据 在一些报表、看板项目中即便是取数据逻辑在复杂。我们依然做到了真正的 零 开发,所有取数逻辑全部通过 DataQL + SQL 的方式满足。 对比往期项目对于后端技术人员的需求从 3~5 人的苦逼通宵加班,直接缩减为 1人配置化搞定。

  再比如,某个内部类 ERP 项目,20多个表单页面,后端部分仅有 1000 行左右的核心代码。其它数据存取逻辑全部配置化完成。

  1. 取数据
    • 如果你只想从数据库或者服务中获取某类数据,不需要: VO、BO、Convert、DO、Mapper 这类东西。
  2. 存数据
    • 如果是从页面表单递交数据到数据库或者服务,免去 BO、FormBean、DO、Mapper 这类东西。
  3. 数据聚合
    • 基于服务调用结果经过结构转换并响应给前端。
    • 将数据库和服务等多个结果进行汇聚然后返回给前端。

技术架构

avatar

  刚一接触 DataQL 可能会有一种错觉认为 DataQL 是一个高级别的 ORM 工具。 这一点需要澄清。DataQL 的竞品应是 GraphQL,而非 ORM 框架。

  ORM 类框架有一个最大的特点是具有 Mapping 过程,然后通过框架在进行 CURD 操作。 例如:Mybatis、Hibernate。其中有一些甚至做到了更高级的界面化例如: apijson,但其本质依然是 ORM。

  而 DataQL 有很大不同。虽然 DataQL 提供了非常出色的基于 SQL 数据存取能力。但从技术架构上来审视它并不是 ORM 框架。 它没有 ORM 中最关键的 Mapping 过程。DataQL 专注的是:结果转换、数据和服务的聚合查询。

  造成 ORM 错觉的是由于 DataQL 充分利用 UDF 和 Fragment 组合,提供了更便捷的数据库访问配置化而已。


Spring 中使用 Dataway

  Dataway 是 Hasor 生态中的一员,使用 Dataway 第一步需要通过 hasor-spring 打通两个生态。

<!-- 引入依赖 -->
<dependency>
    <groupId>net.hasor</groupId>
    <artifactId>hasor-spring</artifactId>
    <version>4.2.2</version>
</dependency>
<dependency>
    <groupId>net.hasor</groupId>
    <artifactId>hasor-dataway</artifactId>
    <version>4.2.2</version>
</dependency>
@EnableHasor()      // 在Spring 中启用 Hasor
@EnableHasorWeb()   // 将 hasor-web 配置到 Spring 环境中,Dataway 的 UI 是通过 hasor-web 提供服务。

  然后第二步,在应用的 application.properties 配置文件中启用 Dataway

# 启用 Dataway 功能(默认不启用)
HASOR_DATAQL_DATAWAY=true
# 开启 ui 管理功能(注意生产环境必须要设置为 false,否则会造成严重的生产安全事故)
HASOR_DATAQL_DATAWAY_ADMIN=true

# (可选)API工作路径
HASOR_DATAQL_DATAWAY_API_URL=/api/
# (可选)ui 的工作路径,只有开启 ui 管理功能后才有效
HASOR_DATAQL_DATAWAY_UI_URL=/interface-ui/

  第三步,初始化 dataway 的必要数据库表。

CREATE TABLE interface_info (
    api_id          varchar(64)  NOT NULL COMMENT 'ID',
    api_method      varchar(12)  NOT NULL COMMENT 'HttpMethod:GET、PUT、POST',
    api_path        varchar(512) NOT NULL COMMENT '拦截路径',
    api_status      varchar(4)   NOT NULL COMMENT '状态:-1-删除, 0-草稿,1-发布,2-有变更,3-禁用',
    api_comment     varchar(255) NOT NULL COMMENT '注释',
    api_type        varchar(24)  NOT NULL COMMENT '脚本类型:SQL、DataQL',
    api_script      mediumtext   NOT NULL COMMENT '查询脚本:xxxxxxx',
    api_schema      mediumtext   NOT NULL COMMENT '接口的请求/响应数据结构',
    api_sample      mediumtext   NOT NULL COMMENT '请求/响应/请求头样本数据',
    api_option      mediumtext   NOT NULL COMMENT '扩展配置信息',
    api_create_time varchar(32)  NOT NULL COMMENT '创建时间',
    api_gmt_time    varchar(32)  NOT NULL COMMENT '修改时间',
    PRIMARY KEY (api_id),
    UNIQUE KEY uk_interface_info (api_path)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Dataway 中的API';

CREATE TABLE interface_release (
  pub_id           varchar(64)  NOT NULL COMMENT 'Publish ID',
  pub_api_id       varchar(64)  NOT NULL COMMENT '所属API ID',
  pub_method       varchar(12)  NOT NULL COMMENT 'HttpMethod:GET、PUT、POST',
  pub_path         varchar(512) NOT NULL COMMENT '拦截路径',
  pub_status       varchar(4)   NOT NULL COMMENT '状态:-1-删除, 0-草稿,1-发布,2-有变更,3-禁用',
  pub_comment      varchar(255) NOT NULL COMMENT '注释',
  pub_type         varchar(24)  NOT NULL COMMENT '脚本类型:SQL、DataQL',
  pub_script       mediumtext   NOT NULL COMMENT '查询脚本:xxxxxxx',
  pub_script_ori   mediumtext   NOT NULL COMMENT '原始查询脚本,仅当类型为SQL时不同',
  pub_schema       mediumtext   NOT NULL COMMENT '接口的请求/响应数据结构',
  pub_sample       mediumtext   NOT NULL COMMENT '请求/响应/请求头样本数据',
  pub_option       mediumtext   NOT NULL COMMENT '扩展配置信息',
  pub_release_time varchar(32)  NOT NULL COMMENT '发布时间(下线不更新)',
  PRIMARY KEY (pub_id),
  KEY idx_interface_release_api  (pub_api_id),
  KEY idx_interface_release_path (pub_path)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='Dataway API 发布历史。'

  最后一步,将 Spring 使用的数据源导入到 Hasor 环境共 Dataway 使用。

@DimModule
@Component
public class ExampleModule implements SpringModule {
    @Autowired
    private DataSource dataSource = null;

    @Override
    public void loadModule(ApiBinder apiBinder) throws Throwable {
        // .DataSource form Spring boot into Hasor
        apiBinder.installModule(new JdbcModule(Level.Full, this.dataSource));
        // .custom DataQL
        //apiBinder.tryCast(QueryApiBinder.class).loadUdfSource(apiBinder.findClass(DimUdfSource.class));
        //apiBinder.tryCast(QueryApiBinder.class).bindFragment("sql", SqlFragment.class);
    }
}

使用UI配置接口

  启动工程,访问 http://127.0.0.1:8080/interface-ui/ 就可以看到下面页面了。

avatar

avatar

发布接口需要先进行冒烟测试,冒烟通过之后就可以点亮发布按钮。接口只有在发布之后才能在 Api List 页面中调用,前端才可以正常访问。

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright 2020 @ 赵永春<zyc@hasor.net> Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

About

DataQL(Data Query Language)DataQL 是一种查询语言。旨在通过提供直观、灵活的语法来描述客户端应用程序的数据需求和交互。 expand collapse
Java and 6 more languages
Apache-2.0
Cancel

Releases

No release

Contributors

All

Activities

Load More
can not load any more
1
https://gitee.com/zycgit/dataql.git
git@gitee.com:zycgit/dataql.git
zycgit
dataql
dataql
dev

Search