同步操作将从 王老邪/drunkard 强制同步,此操作会覆盖自 Fork 仓库以来所做的任何修改,且无法恢复!!!
确定后同步将在后台操作,完成时将刷新页面,请耐心等待。
简书-首页:平常写的内容都在上面
Gitee首页:Gitee首页
Git.IO首页:Git.IO首页
Github首页:Github首页
Weathertop:其他开源的部分,已停止维护
drunkard: 英[ˈdrʌŋkəd] 美[ˈdrʌŋkərd] 酒鬼; 醉鬼;
醉酒状态下,可以忘却很多烦恼,倒是很羡慕那些 酒鬼
们,每天都可以忘记好多事情。再者与同道中的人喝
起来酒越喝越暖,也可以说 酒逢知己千杯少
吧,中华古文化中对酒
可以说有很多名人名句。信手拈来的有曹孟德对酒当歌,人生几何
这样的焦虑;也有东坡先生 明月几时有?把酒问青天
怀情;再有杜牧先生的 借问酒家何处有,牧童遥指杏花村
的念酒
。
以前听说魏晋时期的 竹林七贤
中有两人嗜酒如命 嵇康
、刘伶
,小时候还见到酒的品牌就以这两人名字命名的,甚是觉得好玩不得了。
言归正传,一直依赖,自己就想做一套后台管理系统,在此期间主要前端参考的开源项目 RuoYi
,谁让自己前端知识储备不够,实在是写不出来那样的效果,没办法,只能抄。
本项目是基于SpringBoot开发的脚手架模块,已经集成 MyBatis
作为持久层组件,为了方便提供两种不同的主键生成策略,一种是利用 Redis
;另一种是利用数据库自身的ID策略,两种方式各有差异,主要针对不同的场景。
同时本人也将日常学习的一些材料也整理出来,放到 github.io 的个人网站上,希望对大家有帮助。
|-- akkad-base ------------基包
| |-- base-utils ------------通用工具包
| |-- persistence-jpa ------------基于JPA封装持久层
| |-- persistence-mybatis ------------基于MyBatis封装持久层
| | |-- mybatis-base ------------Mybatis抽象基类封装
| | | |-- resources ------------
| | | | |-- conf ------------Mybatis自动刷新配置路径
| | |-- mybatis-no-pk ------------依赖DB主键版本
| | |-- mybatis-pk-redis ------------依赖Redis生成主键的版本
|-- akkad-springboot ------------springboot集成开源组件
| | |-- springboot-dist-lock ------------分布式锁例子
| | |-- springboot-dist-trans ------------分布式事物
| | |-- springboot-elasticsearch ------------集成ES
| | |-- springboot-jwt ------------JWT示例
| | |-- springboot-mq ------------集成消息MQ组件
| | |-- springboot-nosql ------------集成oauth2的版本
|-- akkad-war3 ------------常用的示例
| | |-- war3-area ------------获取行政区域的版本
| | |-- war3-infi ------------集成 RuoYi 前端实现一个通用管理后台
| | |-- war3-oauth2 ------------集成oauth2的版本
|-- README.md
|-- LICENSE
SpringBoot、Maven、Jnuit、MySQL、JDK8+
在pom文件中加上generator
依赖,在 configurationFile
中指定generatorConfig.xml
文件的位置,剩下一些数据库连接以及配置项,都在配置文件generatorConfig.xml
中指定即可,方便又实用。
<build>
<plugin>
<groupId>org.mybatis.generator</groupId>
<artifactId>mybatis-generator-maven-plugin</artifactId>
<version>1.3.2</version>
<configuration>
<configurationFile>${basedir}/src/main/resources/generator/generatorConfig.xml</configurationFile>
<overwrite>true</overwrite>
<verbose>true</verbose>
</configuration>
</plugin>
</plugins>
</build>
这里我将数据库配置信息独立出来在另外一个文件,即generator.properties
,table
节点自定义,可以一次性写个表。
<generatorConfiguration>
<properties resource="generator/generator.properties"></properties>
<!--数据库驱动-->
<classPathEntry location="${jdbc.driverLocation}"/>
<context id="DB2Tables" targetRuntime="MyBatis3" defaultModelType="flat">
<commentGenerator>
<property name="suppressDate" value="true"/>
<property name="suppressAllComments" value="true"/>
</commentGenerator>
<!--数据库链接地址账号密码-->
<jdbcConnection driverClass="${jdbc.driverClass}" connectionURL="${jdbc.connectionURL}"
userId="${jdbc.userId}" password="${jdbc.password}">
</jdbcConnection>
<javaTypeResolver>
<property name="forceBigDecimals" value="false"/>
</javaTypeResolver>
<!--生成Model类存放位置-->
<javaModelGenerator targetPackage="${targetPackage}.entity" targetProject="${targetProject}">
<property name="enableSubPackages" value="true"/>
<property name="trimStrings" value="true"/>
</javaModelGenerator>
<!--生成映射文件存放位置-->
<sqlMapGenerator targetPackage="${targetPackage}.mapping" targetProject="${targetProject}">
<property name="enableSubPackages" value="true"/>
</sqlMapGenerator>
<!--生成Dao类存放位置-->
<javaClientGenerator type="XMLMAPPER" targetPackage="${targetPackage}.mapper" targetProject="${targetProject}">
<property name="enableSubPackages" value="true"/>
</javaClientGenerator>
<table tableName="register_user_his" schema="RegisterUserHis"
enableCountByExample="false" enableUpdateByExample="false"
enableDeleteByExample="false" enableSelectByExample="false"
selectByExampleQueryId="false">
</table>
</context>
</generatorConfiguration>
指定数据库驱动路径以及数据库URL、用户名、密码等即可
jdbc.driverLocation=G:\\Devp_Repository\\Jar\\mysql-connector-java-5.1.46.jar
jdbc.driverClass=com.mysql.jdbc.Driver
jdbc.connectionURL=jdbc:mysql://localhost:3306/addbook?characterEncoding=utf8&useSSL=true
jdbc.userId=root
jdbc.password=123456
targetProject=D:/data
targetPackage=xyz.wongs.drunkard.domain.addbook
// Generator
generator.targetProject=src/main/java
最后就是执行阶段,两种方式,我使用的maven
插件命令方式来执行,直接双击。
分页组件是继承开源com.github.pagehelper
<dependency>
<groupId>com.github.pagehelper</groupId>
<artifactId>pagehelper-spring-boot-starter</artifactId>
<version>${Last Version}</version>
</dependency>
在应用中PageHelper.startPage即可,后面需要紧跟着持久层语句,否则分页失效
public PageInfo<T> selectPage(PaginationInfo pgInfo, T t) {
PageHelper.startPage(pgInfo.getPageNum(), pgInfo.getPageSize());
List<T> lt = getMapper().getList(t);
PageInfo<T> pageInfo = new PageInfo<T>(lt);
return pageInfo;
}
测试类型 | 代码目录 | 测试内容 |
---|---|---|
单元测试 | src/test/java | 包含核心领域模型(包含领域对象和Factory类)的测试 |
组件测试 | src/componentTest/java | 用于测试一些核心的组件级对象,比如Repository |
API测试 | src/apiTest/java | 模拟客户端调用API |
项目运行时所依赖的外部集成方;
不知为什么在以前的一段时间内,我特别喜欢用 JPA
,它给我印象就是小巧灵便,为我省去了很多不必要的编码,带给我不一样的代码输出效率,因为业务在垂直划分过程中都相对来说封闭,要求在编码过程中相对来说实体关联没有那么复杂,而且项目本身的交付周期特别短,我就选择 JPA
作为我们某个特定项目的专用持久层框架,当然 JPA
自身的优势就不说啦。
项目架构目的就是为了高质量、低成本、更便捷的交付,也是我那段时间里秉承的思路。
下面言归正传,我们用我之前项目在springboot集成 JPA
来做一个演示。
pom中引入 spring-boot-starter-data-jpa
依赖,注意版本。
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<version>2.3.4.RELEASE</version>
</dependency>
JPA
默认工厂类 JpaRepositoryFactoryBean
,并不能满足我们实际要求,比如我们想在插入或者修改的适合做些事情,默认工厂类就不会支持,所以我们重写一个自己的,继承 JpaRepositoryFactoryBean
即可。
继承后重写 createRepositoryFactory
方法,指定我们自己的,这里我用一个内静态类来,这个内部类 BaseRepositoryFactory
也需要继承 JpaRepositoryFactory
工厂,此处我们指定自己真实的工厂类实现 BaseRepositoryImpl
。
@Override
protected RepositoryFactorySupport createRepositoryFactory(EntityManager entityManager) {
return new BaseRepositoryFactory(entityManager);
}
private static class BaseRepositoryFactory<T,I extends Serializable> extends JpaRepositoryFactory{
private final EntityManager em;
public BaseRepositoryFactory(EntityManager em) {
super(em);
this.em = em;
}
@Override
protected JpaRepositoryImplementation<?, ?> getTargetRepository(RepositoryInformation information, EntityManager entityManager) {
return new BaseRepositoryImpl<T, I>((Class<T>) information.getDomainType(), em);
}
/**
* 设置具体的实现类的class
* @param metadata
* @return
*/
@Override
protected Class<?> getRepositoryBaseClass(RepositoryMetadata metadata) {
return BaseRepositoryImpl.class;
}
}
下面就可以在我们自己的类做我们的自己事情,同时我们实现类中抽象出来一个接口暴露对外,这个接口定义我们需要共用的方法,具体实现我们额外实现。这样设计有个好处,就是耦合度降低,扩展方便。
@NoRepositoryBean
@Transactional(readOnly=true,rollbackFor = Exception.class)
public interface BaseRepository<T, ID extends Serializable> extends JpaRepository<T, ID> {
/**
* 根据主键删除
*
* @param ids
*/
void delete(ID[] ids);
/**
*
* @param sql
* @return
*/
List<Object[]> listBySQL(String sql);
public Long getTargetId(String sql);
/**
*
* @param sql
* @param args
*/
@Transactional(rollbackFor = Exception.class)
void updateBySql(String sql,Object...args);
@Transactional(rollbackFor = Exception.class)
void updateByHql(String hql,Object...args);
Page<T> findCriteria(Specification<T> spec, Pageable pageable);
int batchInsert(String sql);
@SuppressWarnings({"unchecked"})
public class BaseRepositoryImpl<T, ID extends Serializable> extends SimpleJpaRepository<T, ID> implements BaseRepository<T, ID> {
//
private final EntityManager entityManager;
public BaseRepositoryImpl(Class<T> domainClass, EntityManager em) {
super(domainClass, em);
this.entityManager=em;
}
@Override
public void delete(ID[] ids) {
}
@Override
public Long getTargetId(String sql) {
Query query = entityManager.createNativeQuery(sql);
return Long.valueOf(query.getSingleResult().toString());
}
@Override
public void updateBySql(String sql, Object... args) {
Query query = entityManager.createNativeQuery(sql);
int i = 0;
for(Object arg:args) {
query.setParameter(++i,arg);
}
query.executeUpdate();
}
@Override
public void updateByHql(String hql, Object... args) {
Query query = entityManager.createQuery(hql);
int i = 0;
for(Object arg:args) {
query.setParameter(++i,arg);
}
query.executeUpdate();
}
@Override
public List<Object[]> listBySQL(String sql) {
return entityManager.createNativeQuery(sql).getResultList();
}
@Override
public int batchInsert(String sql) {
Query query = entityManager.createNativeQuery(sql);
return query.executeUpdate();
}
public Page<T> find(Class rootCls, CriteriaQuery<T> criteria, int pageNo, int pageSize) {
//count
CriteriaBuilder builder = entityManager.getCriteriaBuilder();
CriteriaQuery criteriaC = builder.createQuery();
Root root = criteriaC.from(rootCls);
criteriaC.select(builder.count(root));
criteriaC.where(criteria.getRestriction());
List<Long> totals = entityManager.createQuery(criteriaC).getResultList();
Long total = 0L;
for (Long element : totals) {
total += element == null ? 0 : element;
}
//content
TypedQuery<T> query = entityManager.createQuery(criteria);
query.setFirstResult((pageNo - 1) * pageSize);
query.setMaxResults(pageSize);
List<T> content = total > query.getFirstResult() ? query.getResultList() : Collections.<T> emptyList();
Sort sort = Sort.by(Sort.Direction.DESC, Constant.DEFAULT_SORT);
Pageable pageable = PageRequest.of(pageNo, pageSize, sort);
Page<T> pageRst = new PageImpl<T>(content, pageable, total);
return pageRst;
}
@Override
public Page<T> findCriteria(Specification<T> spec, Pageable pageable){
return super.findAll(spec,pageable);
}
}
实例基类,必须要实现一个接口 Persistable
,这个接口只定义ID主键。当然我们自己的基类也会定义这个,但是这并不冲突。
@MappedSuperclass
public abstract class AbsEntity<ID extends Serializable> extends AbstractEntity<ID> implements Persistable<ID> {
private static final long serialVersionUID = 1L;
@Override
public abstract ID getId();
/**
* Sets the id of the entity.
* @param id the id to set
*/
public abstract void setId(final ID id);
//......
}
抽象 Service 提供基础业务类的功能。
/** Service基类
* @ClassName BaseService
* @Description
* @author WCNGS@QQ.COM
* @Github <a>https://github.com/rothschil</a>
* @date 20/12/18 11:05
* @Version 1.0.0
*/
public abstract class BaseService<T extends AbsEntity<?>, ID extends Serializable> {
protected JpaRepository<T, ID> jpaRepository;
public BaseService() {}
/** 重要 **/
public abstract void setJpaRepository(JpaRepository<T, ID> jpaRepository);
public boolean retBoolFindByExample(T t){
ExampleMatcher matcher = ExampleMatcher.matching();
List<String> fields = new ArrayList<String>();
Reflections.getField(t,fields);
for (String fld: fields){
matcher.withMatcher(fld,ExampleMatcher.GenericPropertyMatchers.exact());
}
Example<T> example = Example.of(t,matcher);
if(jpaRepository.findAll(example).size()>0){
return true;
}
return false;
}
public boolean retBoolSave(T t ){
try{
this.save(t);
return true;
}catch (RuntimeException re){
return false;
}
}
public List<T> findByProperty(T t,String propertyName, Object value) {
try {
Class<?> cls = t.getClass();
Field[] fields = cls.getDeclaredFields();
Method[] methods = cls.getDeclaredMethods();
for (int i=0;i<fields.length;i++) {
if(fields[i].getName().equals(propertyName)){
String fieldSetName = StringUtils.parSetName(fields[i].getName());
for (Method met : methods) {
if (fieldSetName.equals(met.getName())) {
met.invoke(t,value);
}
}
}
}
} catch (SecurityException e) {
e.printStackTrace();
} catch (NumberFormatException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
return findByEntity(t);
}
public boolean retBoolDelete(T t) {
try {
this.delete(t);
return true;
} catch (RuntimeException re) {
return false;
}
}
public List<T> findByEntity(T t) {
ExampleMatcher matcher = ExampleMatcher.matching();
List<String> fields = new ArrayList<String>();
Reflections.getField(t,fields);
for (String fld: fields){
matcher.withMatcher(fld,ExampleMatcher.GenericPropertyMatchers.exact());
}
/* .withMatcher("username", ExampleMatcher.GenericPropertyMatchers.startsWith())
.withMatcher("address" ,ExampleMatcher.GenericPropertyMatchers.contains())
.withIgnorePaths("password");*/
Example<T> example = Example.of(t,matcher);
return jpaRepository.findAll(example);
}
public Page<T> findPageByEntity(int page, int size, T t) {
size=size==0?10:size;
// TODO Auto-generated method stub
Pageable pageable = PageRequest.of(page, size);
ExampleMatcher matcher = ExampleMatcher.matching();
List<String> fields = new ArrayList<String>();
Reflections.getField(t,fields);
for (String fld: fields){
matcher.withMatcher(fld,ExampleMatcher.GenericPropertyMatchers.exact());
}
/* .withMatcher("username", ExampleMatcher.GenericPropertyMatchers.startsWith())
.withMatcher("address" ,ExampleMatcher.GenericPropertyMatchers.contains())
.withIgnorePaths("password");*/
Example<T> example = Example.of(t,matcher);
return jpaRepository.findAll(example,pageable);
}
/**
* 保存单个实体
*
* @param t 实体
* @return 返回保存的实体
*/
public T save(T t) {
return jpaRepository.save(t);
}
public T saveAndFlush(T t) {
t = save(t);
jpaRepository.flush();
return t;
}
/**
* 根据主键删除相应实体
*
* @param id 主键
*/
public void delete(ID id) {
jpaRepository.delete(findOne(id));
}
/**
* 删除实体
*
* @param t 实体
*/
public void delete(T t) {
jpaRepository.delete(t);
}
/**
* 按照主键查询
*
* @param id 主键
* @return 返回id对应的实体
*/
public T findOne(ID id) {
return jpaRepository.getOne(id);
}
/**
* 实体是否存在
* @method exists
* @author WCNGS@QQ.COM
* @version
* @see
* @param id id 主键
* @return boolean 存在 返回true,否则false
* @exception
* @date 2018/7/3 22:08
*/
public boolean exists(ID id) {
return findOne(id)==null?true:false;
}
/**
* 统计实体总数
* @method count
* @author WCNGS@QQ.COM
* @version
* @see
* @param
* @return long
* @exception
* @date 2018/7/3 22:07
*/
public long count() {
return jpaRepository.count();
}
/**
* 查询所有实体
* @method findAll
* @author WCNGS@QQ.COM
* @version
* @see
* @param
* @return java.utils.List<T>
* @exception
* @date 2018/7/3 22:07
*/
public List<T> findAll() {
return jpaRepository.findAll();
}
/**
* 按照顺序查询所有实体
* @method findAll
* @author WCNGS@QQ.COM
* @version
* @see
* @param sort
* @return java.utils.List<T>
* @exception
* @date 2018/7/3 22:06
*/
public List<T> findAll(Sort sort) {
return jpaRepository.findAll(sort);
}
/**
* 分页及排序查询实体
*
* @param pageable 分页及排序数据
* @return
*/
public Page<T> findAll(Pageable pageable) {
return jpaRepository.findAll(pageable);
}
public Page<T> findEntityNoCriteria(Integer page, Integer size) {
Pageable pageable = PageRequest.of(page, size);
return findAll(pageable);
}
}
用 lombok
一目了然,对应字段名以及列与属性对应关系,代码整体简洁。
@EqualsAndHashCode(callSuper=false)
@Builder(toBuilder=true)
@Data
@AllArgsConstructor
@NoArgsConstructor
@Entity
@Table(name="tb_locations")
public class Location extends AbsEntity<Long> {
@Id
@Column(name = "id", nullable = false)
private Long id;
@Column(name = "flag")
private String flag;
@Column(name = "local_code")
//省略....
}
直接继承就完事,方法命名的书写,有些讲究,这里就不单独说明,有兴趣童鞋自行恶补。
public interface LocationRepository extends BaseRepository<Location, Long>,JpaSpecificationExecutor<Location> {
List<Location> findByLv(int lv);
//省略....
}
需要引入我们 locationRepository
,这一步很关键,当然我看有的人将引入的 Repository
弄在类泛型中,效果一样。
@Service(value="locationService")
@Transactional(readOnly = true)
public class LocationService extends BaseService<Location, Long> {
private LocationRepository locationRepository;
@Autowired
@Qualifier("locationRepository")
@Override
public void setJpaRepository(JpaRepository<Location, Long> jpaRepository) {
this.jpaRepository=jpaRepository;
this.locationRepository =(LocationRepository)jpaRepository;
}
public List<Location> getLocationListByLevel(int lv){
return locationRepository.findByLv(lv);
}
}
因为我们自定义了工厂类,所以我们的启动类需要将我们工厂类引入进来,所以 repositoryFactoryBeanClass
属性需要了解下。
这样子 springboot
整合 JPA
就完啦。
@EnableSwagger2
@EnableJpaAuditing
@SpringBootApplication
@EnableJpaRepositories(basePackages = {"xyz.wongs.drunkard"},
repositoryFactoryBeanClass = BaseRepositoryFactoryBean.class//Specify your own factory class
)
public class MoonApplication {
public static void main(String[] args) {
SpringApplication.run(MoonApplication.class,args);
}
}
使用Mybatis过程中,很多时候修改了XML文件需要整个项目重新启动,比较耗时,如果没什么业务数据状态还好,有数据状态可就惨啦,所以XML自动线下更新就很有必要。手写一个简单实现,大家参考下。
我的实现思路就是利用一个额外线程扫描mybatis
XML文件,更新到 Spring
中的 上下文ApplicationContext
中。
我们定义一套刷新时间和周期频次的配置文件在路径 persistence-mybatis\mybatis-base\src\main\resources\conf\mybatis-refresh.properties
中,里面内容如下:
enabled=true
delaySeconds=30
sleepSeconds=10
mappingPath=mapper
核心类需要实现上下文接口 ApplicationContextAware
。
setApplicationContext
方法SqlSessionFactory
进行额外配置关键性步骤如下:
// 1、从上下文容器获取 SqlSessionFactory
SqlSessionFactory sessionFactory = applicationContext.getBean(SqlSessionFactory.class);
// 2、获取Configuration
Configuration configuration = sessionFactory.getConfiguration();
this.configuration = configuration;
// 3、扫描Locations
mapperLocations = getResource(basePackage,XML_RESOURCE_PATTERN);
// 4、启动线程执行
exeTask();
核心类在akkad-base\persistence-mybatis\mybatis-base\src\main\java\xyz\wongs\drunkard\base\persistence\mybatis\loader\MapperAutoRefresh.java
下,而且行数太长,代码就不贴。
在多线程处理这块有需要注意有一定的线程使用基础,看官自行学习。
很多地方需要用到 统计用区划和城乡划分代码
这块以国家统计局的权威数据为准,但是人家是一个网页。
虽然Python解析起来很快,但我还是想用 Java
写一套,打发时间也好,无聊也罢,学习学习。
首先要做的就是分析网页的内容特点,进行数据建模和构建框架。
我本机MySQL运行的,图个方便也没用Oracle或者服务器类,一切从简。
CREATE TABLE tb_locations (
id bigint(20) NOT NULL,
flag varchar(6) DEFAULT NULL,
local_code varchar(30) DEFAULT NULL,
local_name varchar(100) DEFAULT NULL,
lv int(11) DEFAULT NULL,
sup_local_code varchar(30) DEFAULT NULL,
url varchar(60) DEFAULT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
先说下我的实现思路:自上而下,逐级递归。
统计用区划和城乡可以想象成一个树形结构,主干就是省、直辖市、自治区。逐级解析html
文本内容,再拼装成完整URI
路径作为下一级路径解析依据。
这里用到两个技术点:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>akkad-war3</artifactId>
<groupId>xyz.wongs.drunkard</groupId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>war3-area</artifactId>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>xyz.wongs.drunkard</groupId>
<artifactId>mybatis-pk-redis</artifactId>
<exclusions>
<exclusion>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
</exclusion>
<exclusion>
<artifactId>HikariCP</artifactId>
<groupId>com.zaxxer</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.9.2</version>
</dependency>
<dependency>
<groupId>net.sourceforge.htmlunit</groupId>
<artifactId>neko-htmlunit</artifactId>
<version>2.30</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>4.5.5</version>
</dependency>
<dependency>
<groupId>com.gargoylesoftware</groupId>
<artifactId>htmlunit</artifactId>
<version>2.3</version>
</dependency>
<dependency>
<groupId>net.sourceforge.htmlunit</groupId>
<artifactId>htmlunit-core-js</artifactId>
<version>2.31</version>
</dependency>
<dependency>
<groupId>com.gargoylesoftware</groupId>
<artifactId>htmlunit-cssparser</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>xalan</groupId>
<artifactId>xalan</artifactId>
<version>2.7.2</version>
</dependency>
<dependency>
<groupId>xerces</groupId>
<artifactId>xercesimpl</artifactId>
<version>2.11.0</version>
</dependency>
<dependency>
<groupId>javax.persistence</groupId>
<artifactId>javax.persistence-api</artifactId>
<version>2.2</version>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.mybatis.generator</groupId>
<artifactId>mybatis-generator-maven-plugin</artifactId>
<version>1.3.2</version>
<configuration>
<configurationFile>${basedir}/src/main/resources/generator/generatorConfig.xml</configurationFile>
<overwrite>true</overwrite>
<verbose>true</verbose>
</configuration>
</plugin>
</plugins>
</build>
</project>
package xyz.wongs.drunkard.war3.web.area.task.impl;
import lombok.extern.slf4j.Slf4j;
import org.apache.http.HttpEntity;
import org.apache.http.client.config.CookieSpecs;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.ConnectTimeoutException;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Attribute;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;
import xyz.wongs.drunkard.base.constant.Constant;
import xyz.wongs.drunkard.war3.domain.entity.Location;
import xyz.wongs.drunkard.war3.domain.service.LocationService;
import xyz.wongs.drunkard.war3.web.util.IdClazzUtils;
import xyz.wongs.drunkard.war3.web.util.AreaCodeStringUtils;
import xyz.wongs.drunkard.war3.web.area.task.ProcessService;
import java.io.IOException;
import java.sql.Time;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Random;
import java.util.concurrent.TimeUnit;
/**
* @ClassName: JsoupProcessServiceImpl
* @Description:TODO(这里用一句话描述这个类的作用)
* @author: <a href="wcngs@qq.com">WCNGS</a>
* @date: 2017年7月28日 上午11:31:30 *
* @Copyright: 2017 WCNGS Inc. All rights reserved.
*/
@Slf4j
@Service("processService")
public class ProcessServiceImpl implements ProcessService {
@Autowired
@Qualifier("locationService")
LocationService locationService;
@Override
public void initLevelOne(String url, Location parentLocation) {
List<Location> levelOne = null;
try {
levelOne = getLevelOneByRoot(url, parentLocation.getLocalCode());
} catch (IOException e) {
log.error(" IOException pCode={}", parentLocation.getLocalCode(), e.getMessage(), url);
}
save(levelOne);
}
@Override
public boolean initLevelTwo(String url, Location location) {
try {
List<Location> secondLevelLocas = getLocationSecondLevel(url, location);
save(secondLevelLocas);
return true;
} catch (Exception e) {
return false;
}
}
/**
* 初始化省、直辖区、自治区
*
* @param url
* @return void
* @throws
* @method intiRootUrl
* @author WCNGS@QQ.COM
* @version
* @date 2018/6/30 23:29
* @see
*/
@Override
public boolean intiRootUrl(String url) {
try {
List<Location> rootLocations = getLocationRoot(url, "0");
save(rootLocations);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
public List<Location> getLocationRoot(String url, String pCode) {
List<Location> locas = new ArrayList<Location>(35);
try {
Elements eleProv = getElementsByConnection(url, "provincetr");
for (Element e : eleProv) {
Elements eleHerf = e.getElementsByTag("td").select("a[href]");
if (null == eleHerf || eleHerf.size() == 0) {
continue;
}
for (Element target : eleHerf) {
String urls = target.attributes().asList().get(0).getValue();
Location location = Location.builder().id(IdClazzUtils.getId(Location.class))
.localCode("0").url(urls).lv(0).localName(target.text())
.localCode(urls.substring(0, 2)).build();
locas.add(location);
}
}
} catch (IOException e) {
log.error(" IOException pCode={}", pCode, e.getMessage(), url);
}
return locas;
}
/**
* 方法实现说明
*
* @param url
* @param location
* @return void
* @throws
* @method thridLevelResolve
* @author WCNGS@QQ.COM
* @version
* @date 2018/7/1 9:50
* @see
*/
@Override
public void initLevelThrid(String url, Location location) {
this.initLevelThrid(url, location, "Y");
}
/**
* 方法实现说明
*
* @param url
* @param location
* @param flag
* @return void
* @throws
* @method thridLevelResolve
* @author WCNGS@QQ.COM
* @version
* @date 2018/7/1 16:24
* @see
*/
@Override
public void initLevelThrid(String url, Location location, String flag) {
try {
if (StringUtils.isEmpty(location.getUrl())) {
return;
}
List<Location> thridLevelLocas = getLocation(url, new String[]{"towntr", "href"}, location.getLocalCode(), 3, flag);
location.setFlag(flag);
locationService.updateByPrimaryKey(location);
save(thridLevelLocas);
} catch (Exception e) {
e.printStackTrace();
}
}
public void save(List<Location> locations) {
//结果为空,抛出异常
if (null == locations || locations.isEmpty()) {
log.error(" target saved is null!");
return;
}
locationService.insertBatchByOn(locations);
}
@Override
public void initLevelFour(String url, List<Location> thridLevelLocas) {
for (Location le : thridLevelLocas) {
List<Location> locations = new ArrayList<Location>(12);
String suffix = new StringBuilder().append(url).append(AreaCodeStringUtils.getUrlStrByLocationCode(le.getLocalCode(), 3)).append(le.getUrl()).toString();
Elements es = null;
try {
es = getElementsByConnection(suffix, "villagetr");
Location tempLocation = null;
for (Element e : es) {
tempLocation = new Location(e.child(0).text(), e.child(2).text(), le.getLocalCode(), null, 4);
tempLocation.setId(IdClazzUtils.getId(Location.class));
locations.add(tempLocation);
}
le.setFlag("Y");
locationService.updateByPrimaryKey(le);
save(locations);
} catch (IOException e) {
log.error(" IOException code={},msg={},url={}", le.getLocalCode(), e.getMessage(), suffix);
int times = AreaCodeStringUtils.getSecond(3);
try {
TimeUnit.SECONDS.sleep(times);
} catch (InterruptedException interruptedException) {
log.error("msg={} ", interruptedException.getMessage());
}
continue;
} catch (Exception e) {
log.error("Exception code={},msg={}", le.getLocalCode(), e.getMessage());
continue;
}
}
}
/**
* @param url
* @param location
* @return
* @Title: getLocationSecondLevel
* @Description: TODO(这里用一句话描述这个方法的作用)
* @return: List<Location>
*/
public List<Location> getLocationSecondLevel(String url, Location location) {
List<Location> locas = null;
try {
locas = new ArrayList<Location>(90);
//URL地址截取
//标识位
boolean flag = false;
Elements es = getElementsByConnection(url, "countytr");
if (null == es) {
log.error(url + " 不能解析!");
return null;
}
Location tempLocation = null;
for (Element e : es) {
//针对市辖区 这种无URL的做特殊处理
if (!flag) {
tempLocation = new Location(e.child(0).text(), e.child(1).text(), location.getLocalCode(), null, 2);
tempLocation.setId(IdClazzUtils.getId(Location.class));
locas.add(tempLocation);
//标识位置为TURE
flag = true;
continue;
}
es = e.getElementsByAttribute("href");
if (es.size() == 0) {
tempLocation = new Location(e.child(0).text(), e.child(1).text(), location.getLocalCode(), "", 2);
tempLocation.setId(IdClazzUtils.getId(Location.class));
locas.add(tempLocation);
continue;
}
List<Attribute> attrs = es.get(0).attributes().asList();
tempLocation = new Location(es.get(0).text(), es.get(1).text(), location.getLocalCode(), attrs.get(0).getValue(), 2);
tempLocation.setId(IdClazzUtils.getId(Location.class));
locas.add(tempLocation);
}
} catch (Exception e) {
e.printStackTrace();
}
return locas;
}
/**
* @param url
* @param pCode
* @return
* @Title: getLocationOneLevel
* @Description: 1、获取第一级地市信息
* 2、第二级区县信息
* @return: List<Location>
*/
public List<Location> getLevelOneByRoot(String url, String pCode) throws IOException {
List<Location> locas = new ArrayList<Location>(20);
Elements eles = getElementsByConnection(url, "citytr");
if (null == eles) {
log.error(url + " 不能解析!");
return null;
}
Location location = null;
for (Element e : eles) {
eles = e.getElementsByAttribute("href");
List<Attribute> attrs = eles.get(0).attributes().asList();
location = new Location(eles.get(0).text(), eles.get(1).text(), pCode, attrs.get(0).getValue(), 1);
location.setId(IdClazzUtils.getId(Location.class));
locas.add(location);
}
return locas;
}
public List<Location> getLocation(String url, String[] cssClazz, String parentCode, Integer lv, String flag) throws IOException {
List<Location> locas = new ArrayList<Location>(20);
Elements eles = getElementsByConnection(url, cssClazz[0]);
if (null == eles) {
log.error(url + " 不能解析!");
return null;
}
Location location = null;
for (Element e : eles) {
eles = e.getElementsByAttribute(cssClazz[1]);
List<Attribute> attrs = eles.get(0).attributes().asList();
location = new Location(eles.get(0).text(), eles.get(1).text(), parentCode, attrs.get(0).getValue(), lv, flag);
location.setId(IdClazzUtils.getId(Location.class));
locas.add(location);
}
return locas;
}
/**
* 案例
* <tr class='towntr'>
* <td><a href='02/340102001.html'>340102001000</a></td>
* <td><a href='02/340102001.html'>明光路街道</a></td>
* </tr>
*
* @param url
* @param cssClazz
* @param parentURLCode
* @return List<Location>
* @Title: getLocation
* @Description: TODO(这里用一句话描述这个方法的作用)
*/
public List<Location> getLocation(String url, String[] cssClazz, String parentCode, Integer lv) {
return getLocation(url, cssClazz, parentCode, lv);
}
/**
* 方法实现说明
*
* @param url
* @param clazzName
* @return org.jsoup.select.Elements
* @throws
* @method getElementss
* @author WCNGS@QQ.COM
* @version
* @date 2018/7/2 11:28
* @see
*/
public Elements getElementsByConnection(String url, String clazzName) throws IOException {
try {
/** CloseableHttpClient httpclient = HttpClients.createDefault(); **/
//设置CookieSpecs.STANDARD的cookie解析模式,下面为源码,对应解析格式我给出了备注
CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultRequestConfig(RequestConfig.custom()
.setCookieSpec(CookieSpecs.STANDARD).build())
.build();
HttpGet httpget = new HttpGet(url);
httpget.setHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0");
RequestConfig config = RequestConfig.custom()
//.setProxy(proxy)
//设置连接超时 ✔
// 设置连接超时时间 10秒钟
.setConnectTimeout(10000)
// 设置读取超时时间10秒钟
.setSocketTimeout(10000)
.build();
httpget.setConfig(config);
// 执行get请求
CloseableHttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
// 获取返回实体
String content = EntityUtils.toString(entity, "GBK");
// ============================= 【Jsoup】 ====================================
Document doc = Jsoup.parse(content);
return doc.getElementsByClass(clazzName);
} catch (ConnectTimeoutException e) {
log.error(" ConnectTimeoutException URL={},clazzName={},errMsg={}", url, clazzName, e.getMessage());
}
return null;
}
/**
* @param locations
* @return java.lang.String
* @throws
* @Description
* @date 2020/9/9 14:52
*/
public String appengUrl(List<Location> locations) {
Iterator<Location> it = locations.iterator();
String url = "";
StringBuilder sb = new StringBuilder();
while (it.hasNext()) {
Location cation = it.next();
String str = cation.getUrl();
if (cation.getLv() == 3) {
sb.append(str);
} else {
int i = cation.getUrl().indexOf(Constant.SLASH);
sb.append(str.substring(0, i)).append(Constant.SLASH).append(sb);
}
}
return url;
}
}
在单元测试中,自上而下,按个运行测试方法即可。
同时在运行中,可能会由于服务器拒绝连接,造成无法解析出来地址,这没关系,代码中已经容错这些,继续执行即可!
package xyz.wongs.drunkard.task;
import com.github.pagehelper.PageInfo;
import lombok.extern.slf4j.Slf4j;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import xyz.wongs.drunkard.base.BaseTest;
import xyz.wongs.drunkard.war3.domain.entity.Location;
import xyz.wongs.drunkard.war3.domain.service.LocationService;
import xyz.wongs.drunkard.war3.web.util.AreaCodeStringUtils;
import xyz.wongs.drunkard.war3.web.area.task.ProcessService;
import java.util.Iterator;
import java.util.List;
import java.util.Random;
import java.util.concurrent.TimeUnit;
/**
* @author WCNGS@QQ.COM
* @ClassName ProcessServiceImplTest
* @Description
* @Github <a>https://github.com/rothschil</a>
* @date 2020/9/9 15:26
* @Version 1.0.0
*/
@Slf4j
public class ProcessServiceTest extends BaseTest {
private static final String URL = "http://www.stats.gov.cn/tjsj/tjbz/tjyqhdmhcxhfdm/2020/";
private final static Logger logger = LoggerFactory.getLogger(ProcessServiceTest.class);
@Autowired
@Qualifier("processService")
private ProcessService processService;
@Autowired
private LocationService locationService;
/**
* 获取所有省,作为Root根节点
*
* @return
* @throws
* @Description
* @date 2020/4/30 0:41
*/
@Test
public void initRoot() {
processService.intiRootUrl(URL);
}
/**
* 解析所有省、直辖的城市
*
* @return void
* @throws
* @Description
* @date 2020/9/4 22:03
*/
@Test
public void intLevelOne() throws Exception {
city(1);
}
public void city(int pageNum) {
PageInfo<Location> pageInfo = locationService.getLocationsByLv(0, pageNum, 30);
if (pageInfo.getPages() == 0 || pageInfo.getPageNum() > pageInfo.getPages()) {
return;
}
List<Location> locations = pageInfo.getList();
Iterator<Location> iter = locations.iterator();
while (iter.hasNext()) {
Location location = iter.next();
String uls = URL + location.getUrl();
processService.initLevelOne(uls, location);
location.setFlag("Y");
locationService.updateByPrimaryKey(location);
}
city(pageNum + 1);
}
/**
* 根据地市,解析并初始化区县
*
* @return void
* @throws
* @Description
* @date 2020/9/5 10:21
*/
@Test
public void intLevelTwo() throws Exception {
exet(1);
}
public void exet(int pageNum) {
PageInfo<Location> pageInfo = locationService.getLocationsByLv(1, pageNum, 30);
if (pageInfo.getPages() == 0 || pageInfo.getPageNum() > pageInfo.getPages()) {
return;
}
List<Location> locations = pageInfo.getList();
Iterator<Location> iter = locations.iterator();
while (iter.hasNext()) {
Location location = iter.next();
String url2 = new StringBuilder().append(URL).append(location.getUrl()).toString();
processService.initLevelTwo(url2, location);
location.setFlag("Y");
locationService.updateByPrimaryKey(location);
}
exet(pageNum + 1);
}
/**
* 根据区县,解析并初始化乡镇 街道
*
* @return
* @throws
* @Description
* @date 2020/4/30 0:27
*/
@Test
public void intLevelThree() {
three(1);
}
public void three(int pageNum) {
PageInfo<Location> pageInfo = locationService.getLocationsByLv(2, pageNum, 100);
if (pageInfo.getPages() == 0 || pageInfo.getPageNum() > pageInfo.getPages()) {
return;
}
uot++;
List<Location> locations = pageInfo.getList();
Iterator<Location> iter = locations.iterator();
Location location = null;
while (iter.hasNext()) {
location = iter.next();
String url2 = new StringBuilder().append(URL).append(AreaCodeStringUtils.getUrlStrByLocationCode(location.getLocalCode(), 2)).append(location.getUrl()).toString();
processService.initLevelThrid(url2, location, "D");
try {
int times = AreaCodeStringUtils.getSecond(3);
TimeUnit.SECONDS.sleep(times);
} catch (InterruptedException e) {
log.error("msg={} ", e.getMessage());
}
}
if (uot == COT) {
return;
}
three(pageNum + 1);
}
private static int COT = 100;
private static int uot = 0;
/**
* 根据乡镇 街道,解析并初始化社区村
*
* @return
* @Description
* @throwsOperationImplicitParameterReader
* @date 2020/4/30 0:27
*/
@Test
public void intLevelFour() {
Location location = new Location();
location.setLv(3);
location.setFlag("D");
village(0, location);
}
public void village(int pageNum, Location location) {
PageInfo<Location> pageInfo = locationService.getLocationsByLvAndFlag(pageNum, 2, location);
log.error(pageInfo.toString());
if (pageInfo.getPages() == 0 || pageInfo.getPageNum() > pageInfo.getPages()) {
return;
}
uot++;
List<Location> locations = pageInfo.getList();
if (!locations.isEmpty()) {
processService.initLevelFour(URL, locations);
}
if (uot == COT) {
return;
}
village(pageNum + 1, location);
}
}
访问 http://localhost:9090/region/ip=109.27.45.12
这是我之前一个例子,用来解析IP地址,获取地域信息的。
觉得对你有帮助,请Star
前段时间因业务需要,客户提出分析外网访问的IP,即针对一些热点区域的IP访问想做到事后预警与分析。 因为我们服务是运行在相对隔离的资源环境中,无法直接去请求外网,于是想到用离线的方式来处理从网关发来的数据。找了下,看到个开源项目
使用起来相对成本较低,本着先用满足需求再说,虽然在性能上并不能满足海量IP的分析,还有提升的空间。
看作者的例子,较为简单,也不多说。先看下我的目录结构吧
war3-infi
+---src
| +---main
| | +---java
| | \---resources
| | +---generator
| | +---ipdb
| | | | +---ip2region.db
| | +---mapper
| | +---application.yml
ip2region.db
这是需要下载,下载地址,这里提供Csv、Txt、Db文件三种格式,根据需要自行选择。
ipdb
这是我放db库文件的路径,当然可以自定义,只需要在application.yml
中配置即可。
application.yml
这个就不多说啦。
server:
port: 9090
spring:
redis:
database: 0
host: 127.0.0.1
port: 6379
ip2region:
external: false
index-block-size: 4096
total-header-size: 8192
location: classpath:ipdb/ip2region.db
例子如下,RegionAddress
、DataBlock
两种结果返回封装。
package xyz.wongs.drunkard.war3.web.controller;
import com.github.hiwepy.ip2region.spring.boot.IP2regionTemplate;
import com.github.hiwepy.ip2region.spring.boot.ext.RegionAddress;
import lombok.extern.slf4j.Slf4j;
import org.nutz.plugins.ip2region.DataBlock;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import xyz.wongs.drunkard.base.aop.annotion.ApplicationLog;
import xyz.wongs.drunkard.base.message.annoation.ResponseResult;
import xyz.wongs.drunkard.base.message.exception.DrunkardException;
import xyz.wongs.drunkard.framework.limit.RequestLimit;
import java.io.IOException;
/**
* @ClassName IndexController
* @Description
* @author WCNGS@QQ.COM
* @Github <a>https://github.com/rothschil</a>
* @date 20/11/18 11:00
* @Version 1.0.0
*/
@Slf4j
@RestController
@ResponseResult
public class IndexController {
@Autowired
IP2regionTemplate template;
/** 根据输入IP地址,返回解析后的地址
* @Description
* @param ip
* @return xyz.wongs.drunkard.base.message.response.ResponseResult
* @throws
* @date 2020/8/17 18:26
*/
@GetMapping(value = "/convert/{ip}")
public DataBlock convertDataBlock(@PathVariable String ip){
DataBlock dataBlock = null;
try {
dataBlock = template.binarySearch(ip);
} catch (IOException e) {
e.printStackTrace();
}
return dataBlock;
}
/** 根据输入IP地址,返回解析后的地址
* @Description
* @param ip
* @return xyz.wongs.drunkard.base.message.response.ResponseResult
* @throws
* @date 2020/8/17 18:26
*/
@RequestLimit(maxCount=3)
@GetMapping(value = "/region/{ip}")
public RegionAddress convert(@PathVariable String ip){
RegionAddress regionAddress = null;
try {
regionAddress = template.getRegionAddress(ip);
} catch (IOException e) {
e.printStackTrace();
}
return regionAddress;
}
@GetMapping(value = "/region/ip={ip}")
public RegionAddress caseInsensitive(@PathVariable String ip){
RegionAddress regionAddress = null;
try {
regionAddress = template.getRegionAddress(ip);
} catch (IOException e) {
e.printStackTrace();
}
return regionAddress;
}
}
访问 http://localhost:9090/region/ip=109.27.45.12
这是我之前一个例子,用来解析IP地址,获取地域信息的。
我们在开发前端
和后端
进行交互服务过程中,受制于前后端的工作职责明确,在交互协议的定义上理解也较为不同,造成一个项目服务中重复定义交互内容以及编码上重复编写,不利于项目维护。所以基于此,将后端
按照约定请求URL路径,并传入相关参数,后端
服务器接收请求,进行业务处理,返回数据给前端,进行再次封装,供前端以及外部调用。
通常情况下我们后端
返回给前端
都会采用 JSON
的定义,具体如下:
{
// 返回状态码
code:integer,
// 返回信息描述
message:string,
// 返回值
data:object
}
在状态码的定义上,在满足业务需求的基础上,避免凌乱,一般业界同行做法就是参考HTTP请求返回的状态码。 具体如 百度 - HTTP状态码。
这里我贴出我将我项目中的常用的罗列出来,供大家参考。
package xyz.wongs.drunkard.base.message.enums;
/**
* @ClassName
* @Description
* 1000~1999 区间表示参数错误
* 2000~2999 区间表示用户错误
* 3000~3999 区间表示接口异常
* @author WCNGS@QQ.COM
* @Github <a>https://github.com/rothschil</a>
* @date 2020/8/2 13:31
* @Version 1.0.0
*/
public enum ResultCode {
/** 成功 **/
SUCCESS(0,"成功"),
/** 失败 **/
FAILURE(-1,"失败"),
EXCEPTION(201, "未知异常"),
RUNTIME_EXCEPTION(202, "运行时异常"),
NULL_POINTER_EXCEPTION(203, "空指针异常"),
CLASS_CAST_EXCEPTION(204, "类型转换异常"),
IO_EXCEPTION(205, "IO异常"),
SYSTEM_EXCEPTION(210, "系统异常"),
NOT_FOUND(404, "Not Found"),
/**
* 1000~1999 区间表示参数错误
*/
PARAMS_IS_INVALID(1001,"参数无效"),
PARAMS_IS_BANK(1002,"参数为空"),
PARAMS_TYPE_BIND_ERROR(1003,"参数类型错误"),
PARAMS_NOT_COMPLETE(1004,"参数缺失"),
/**
* 2000~2999 区间表示用户错误
*/
USER_NOT_LOGGED_IN(2001,"用户未登录,访问路径需要验证"),
USER_NOT_LOGIN_ERROR(2002,"用户不存在或密码错误"),
USER_ACCOUNT_FORBIDDEN(2003,"用户被禁用"),
USER_NOT_EXIST(2004,"用户不存在"),
USER_HAS_EXISTED(2005,"用户已存在"),
USER_IS_EXPIRED(2006,"用户账号已过期"),
USER_FIRST_LANDING(2007, "首次登录"),
USER_TOKEN_EXPIRED(2008,"Token过期"),
USER_TOKEN_GENERTATION_FAIL(2009,"生成Token失败"),
USER_SIGN_VERIFI_NOT_COMPLIANT(2010,"签名校验不合规"),
USER_PASSWORD_RESET_FAILED(2011, "重置密码失败"),
USER_UNKONWN_INDENTITY(2012, "未知身份"),
MANY_USER_LOGINS(2111,"多用户在线"),
TOO_MANY_PASSWD_ENTER(2112, "密码输入次数过多"),
VERIFICATION_CODE_INCORECT(2202,"图形验证码不正确"),
VERIFICATION_CODE_FAIL(2203,"图形验证码生产失败"),
/**
* 3000~3999 区间表示接口异常
*/
API_EXCEPTION(3000, "接口异常"),
API_NOT_FOUND_EXCEPTION(3002, "接口不存在"),
API_REQ_MORE_THAN_SET(3003, "接口访问过于频繁,请稍后再试"),
API_IDEMPOTENT_EXCEPTION(3004, "接口不可以重复提交,请稍后再试"),
API_PARAM_EXCEPTION(3005, "参数异常"),
API_PARAM_MISSING_EXCEPTION(3006, "缺少参数"),
API_METHOD_NOT_SUPPORTED_EXCEPTION(3007, "不支持的Method类型"),
API_METHOD_PARAM_TYPE_EXCEPTIION(3008, "参数类型不匹配"),
ARRAY_EXCEPTION(11001, "数组异常"),
ARRAY_OUT_OF_BOUNDS_EXCEPTION(11002, "数组越界异常"),
JSON_SERIALIZE_EXCEPTION(30000, "序列化数据异常"),
JSON_DESERIALIZE_EXCEPTION(30001, "反序列化数据异常"),
READ_RESOURSE_EXCEPTION(31002, "读取资源异常"),
READ_RESOURSE_NOT_FOUND_EXCEPTION(31003, "资源不存在异常"),
DATA_EXCEPTION(32004, "数据异常"),
DATA_NOT_FOUND_EXCEPTION(32005, "未找到符合条件的数据异常"),
DATA_CALCULATION_EXCEPTION(32006, "数据计算异常"),
DATA_COMPRESS_EXCEPTION(32007, "数据压缩异常"),
DATA_DE_COMPRESS_EXCEPTION(32008, "数据解压缩异常"),
DATA_PARSE_EXCEPTION(32009, "数据转换异常"),
ENCODING_EXCEPTION(33006, "编码异常"),
ENCODING_UNSUPPORTED_EXCEPTION(33006, "编码不支持异常"),
DATE_PARSE_EXCEPTION(34001, "日期转换异常"),
MAILE_SEND_EXCEPTION(35001, "邮件发送异常");
/**
*
*/
private Integer code;
/**
*
*/
private String msg;
ResultCode(Integer code, String msg) {
this.code = code;
this.msg = msg;
}
public Integer getCode() {
return code;
}
public void setCode(Integer code) {
this.code = code;
}
public String getMsg() {
return msg;
}
public void setMsg(String msg) {
this.msg = msg;
}
}
这个不解释啦, 就是编码的文字的意义,说清楚就行,没必要太较真,自行脑补。
这就是业务具体数据啦,根据具体业务,内容也不同,这一章节也没必要说。
这里小结下,我们除了要有需要定义的内容有两块:返回的JSON消息体(这里区分正常响应返回、异常响应返回),还需要一套状态码详细定义;再有我们这里做的是WEB,既然做通用,怎能少拦截器。
摆脱了繁琐的文字,下面开始张罗着贴实现代码啦。
结合我们定义的状态码,我们返回的消息体主要实现一个 Serializable
,不要问我为什么。
package xyz.wongs.drunkard.base.message.response;
import lombok.Data;
import xyz.wongs.drunkard.base.message.enums.ResultCode;
import java.io.Serializable;
/**
* @ClassName
* @Description
* @author WCNGS@QQ.COM
* @Github <a>https://github.com/rothschil</a>
* @date 2020/8/2 13:48
* @Version 1.0.0
*/
@Data
public class Result implements Serializable {
private static final long serialVersionUID = -4505655308965878999L;
private Integer code;
private String message;
private Object data;
private Result() {
}
public Result(ResultCode resultCode, Object data) {
this.code = resultCode.getCode();
this.message = resultCode.getMsg();
this.data = data;
}
private void setResultCode(ResultCode resultCode) {
this.code = resultCode.getCode();
this.message = resultCode.getMsg();
}
/** 返回成功
* @Description
* @param
* @return xyz.wongs.drunkard.base.message.response.R
* @throws
* @date 20/11/13 17:15
*/
public static Result success() {
Result result = new Result();
result.setResultCode(ResultCode.SUCCESS);
return result;
}
/** 返回成功
* @Description
* @param
* @return xyz.wongs.drunkard.base.message.response.R
* @throws
* @date 20/11/13 17:15
*/
public static Result success(Object data) {
Result result = new Result();
result.setResultCode(ResultCode.SUCCESS);
result.setData(data);
return result;
}
/** 返回失败
* @Description
* @param
* @return xyz.wongs.drunkard.base.message.response.R
* @throws
* @date 20/11/13 17:15
*/
public static Result fail(Integer code, String message) {
Result result = new Result();
result.setCode(code);
result.setMessage(message);
return result;
}
/** 返回失败
* @Description
* @param
* @return xyz.wongs.drunkard.base.message.response.R
* @throws
* @date 20/11/13 17:15
*/
public static Result fail(ResultCode resultCode) {
Result result = new Result();
result.setResultCode(resultCode);
return result;
}
}
package xyz.wongs.drunkard.base.message.response;
import lombok.Data;
import xyz.wongs.drunkard.base.message.enums.ResultCode;
import java.io.Serializable;
/**
* @author WCNGS@QQ.COM
* @ClassName ErrorResult
* @Description 异常错误的返回信息实体
* @Github <a>https://github.com/rothschil</a>
* @date 20/11/18 10:42
* @Version 1.0.0
*/
@Data
public class ErrorResult implements Serializable {
private static final long serialVersionUID = -4505655308965878999L;
/**
* 错误编码
**/
private Integer code;
/**
* 消息描述
**/
private String msg;
/**
* 错误
**/
private String exception;
public static ErrorResult fail(ResultCode resultCode, Throwable e, String message) {
ErrorResult errorResult = ErrorResult.fail(resultCode, e);
errorResult.setMsg(message);
return errorResult;
}
public static ErrorResult fail(ResultCode resultCode, Throwable e) {
ErrorResult errorResult = new ErrorResult();
errorResult.setCode(resultCode.getCode());
errorResult.setMsg(resultCode.getMsg());
errorResult.setException(e.getClass().getName());
return errorResult;
}
public static ErrorResult fail(Integer code, String message) {
ErrorResult errorResult = new ErrorResult();
errorResult.setCode(code);
errorResult.setMsg(message);
return errorResult;
}
}
这样两个消息体就写完啦。
我们这里需要做的就是利用拦截器拦截请求,检查判断是否此请求返回的值需要包装。核心就是判断一个注解annoation
是否存在方法或类中。
为了演示的完整,我将代码贴完整。
/**
* @ClassName ResponseResult
* @Description
* @author WCNGS@QQ.COM
* @Github <a>https://github.com/rothschil</a>
* @date 20/10/30 21:57
* @Version 1.0.0
*/
@Target({ElementType.TYPE,ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface ResponseResult {
}
package xyz.wongs.drunkard.base.interceptor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import org.springframework.web.method.HandlerMethod;
import org.springframework.web.servlet.HandlerInterceptor;
import xyz.wongs.drunkard.base.message.annoation.ResponseResult;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.lang.reflect.Method;
/**
* @author WCNGS@QQ.COM
* @ClassName ResponseResultInterceptor
* @Description 请求的拦截器
* @Github <a>https://github.com/rothschil</a>
* @date 20/10/30 22:08
* @Version 1.0.0
*/
@Slf4j
@Component
public class ResponseResultInterceptor implements HandlerInterceptor {
private static final String RESPONSE_RESULT_ANN = "RESPONSE-RESULT-ANN";
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
if (handler instanceof HandlerMethod) {
final HandlerMethod handlerMethod = (HandlerMethod) handler;
final Class<?> clazz = handlerMethod.getBeanType();
final Method method = handlerMethod.getMethod();
if (clazz.isAnnotationPresent(ResponseResult.class)) {
request.setAttribute(RESPONSE_RESULT_ANN, clazz.getAnnotation(ResponseResult.class));
} else if (method.isAnnotationPresent(ResponseResult.class)) {
request.setAttribute(RESPONSE_RESULT_ANN, method.getAnnotation(ResponseResult.class));
}
}
return true;
}
}
着十几行代码的核心处理逻辑,就是获取此请求Annoation注解,是否需要返回值包装,并设置一个属性标记,交由下一处理ResponseResultHandler
来具体封装返回值。
细心的人会发现这里只处置正常成功的内容返回,对于异常的内容并未处置。关于异常处置我理解统一放在一起来编写,这样代码结构性会更好。由此引出下一章节,全局异常
。
package xyz.wongs.drunkard.base.handler;
import lombok.extern.slf4j.Slf4j;
import org.springframework.core.MethodParameter;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.server.ServerHttpRequest;
import org.springframework.http.server.ServerHttpResponse;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
import org.springframework.web.servlet.mvc.method.annotation.ResponseBodyAdvice;
import xyz.wongs.drunkard.base.message.annoation.ResponseResult;
import xyz.wongs.drunkard.base.message.response.Result;
import javax.servlet.http.HttpServletRequest;
/**
* @ClassName ResponseResultHandler
* @Description 消息返回体
* @author WCNGS@QQ.COM
* @Github <a>https://github.com/rothschil</a>
* @date 20/11/10 09:28
* @Version 1.0.0
*/
@Slf4j
@ControllerAdvice(basePackages = "xyz.wongs.drunkard")
public class ResponseResultHandler implements ResponseBodyAdvice<Object> {
private static final String RESPONSE_RESULT_ANN = "RESPONSE-RESULT-ANN";
/**
* @Description 判断是否要执行 beforeBodyWrite 方法,true为执行,false不执行,有注解标记的时候处理返回值
* @param returnType
* @param converterType
* @return boolean
* @throws
* @date 20/11/13 10:50
*/
@Override
public boolean supports(MethodParameter returnType, Class<? extends HttpMessageConverter<?>> converterType) {
ServletRequestAttributes sra =(ServletRequestAttributes)RequestContextHolder.getRequestAttributes();
HttpServletRequest request = sra.getRequest();
ResponseResult responseResult = (ResponseResult)request.getAttribute(RESPONSE_RESULT_ANN);
return responseResult==null?false:true;
}
@Override
public Object beforeBodyWrite(Object body, MethodParameter returnType, MediaType selectContentType, Class<? extends HttpMessageConverter<?>> selectConverterType, ServerHttpRequest request, ServerHttpResponse response) {
log.error(" ENTER MSG .... Excu");
if(body instanceof Result){
return (Result) body;
}
return Result.success(body);
}
}
这里所有的异常都使用到 ErrorResult
类。
package xyz.wongs.drunkard.base.message.exception;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpStatus;
import org.springframework.validation.BindException;
import org.springframework.web.bind.MethodArgumentNotValidException;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.ResponseStatus;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import xyz.wongs.drunkard.base.message.enums.ResultCode;
import xyz.wongs.drunkard.base.message.response.ErrorResult;
import javax.servlet.http.HttpServletRequest;
import javax.validation.ConstraintViolationException;
/**
* @author WCNGS@QQ.COM
* @ClassName GlobalExceptionHandler
* @Description 全局异常处理Handler
* @Github <a>https://github.com/rothschil</a>
* @date 2019/9/23 15:03
* @Version 1.0.0
*/
@RestControllerAdvice
@Slf4j
public class GlobalExceptionHandler {
/**
* 参数校验不通过
*
* @param ex
* @return xyz.wongs.drunkard.base.message.response.ErrorResult
* @throws
* @author WCNGS@QQ.COM
* @See
* @date 2019/9/23 17:53
* @since
*/
@ExceptionHandler(value = ConstraintViolationException.class)
@ResponseBody
public ErrorResult handleConstraintViolationException(ConstraintViolationException ex) {
log.error("ConstraintViolationException msg:{}", ex.getMessage());
return ErrorResult.fail(ResultCode.PARAMS_IS_INVALID, ex);
}
/**
* 自定义异常
*
* @param request
* @param ex
* @return xyz.wongs.drunkard.base.message.response.ErrorResult
* @throws
* @author WCNGS@QQ.COM
* @See
* @date 2019/9/23 17:53
* @since
*/
@org.springframework.web.bind.annotation.ExceptionHandler(DrunkardException.class)
@ResponseBody
public ErrorResult handleWeathertopException(HttpServletRequest request, DrunkardException ex) {
log.error("WeathertopRuntimeException code:{},msg:{}", ex.getCode(), ex.getMessage());
return ErrorResult.fail(ex.getCode(), ex.getMessage());
}
/**
* @param e
* @param request
* @return xyz.wongs.drunkard.base.message.response.ErrorResult
* @throws
* @Description 拦截抛出的异常,@ResponseStatus:用来改变响应状态码
* @date 20/11/13 11:14
*/
@ResponseStatus(HttpStatus.INTERNAL_SERVER_ERROR)
@ExceptionHandler(Throwable.class)
public ErrorResult handlerThrowable(Throwable e, HttpServletRequest request) {
log.error("发生未知异常!原因是: ", e);
ErrorResult error = ErrorResult.fail(ResultCode.RUNTIME_EXCEPTION, e);
return error;
}
/**
* @param e
* @param request
* @return xyz.wongs.drunkard.base.message.response.ErrorResult
* @throws
* @Description 参数校验异常
* @date 20/11/13 11:14
*/
@ExceptionHandler(BindException.class)
public ErrorResult handleBindExcpetion(BindException e, HttpServletRequest request) {
log.error("发生参数校验异常!原因是:", e);
ErrorResult error = ErrorResult.fail(ResultCode.API_PARAM_EXCEPTION, e, e.getAllErrors().get(0).getDefaultMessage());
return error;
}
@ExceptionHandler(MethodArgumentNotValidException.class)
public ErrorResult handleMethodArgumentNotValidException(MethodArgumentNotValidException e, HttpServletRequest request) {
log.error("发生参数校验异常!原因是:", e);
ErrorResult error = ErrorResult.fail(ResultCode.API_PARAM_EXCEPTION, e, e.getBindingResult().getAllErrors().get(0).getDefaultMessage());
return error;
}
}
以上虽然将所有代码贴出,这列为凑完整,顺道将写个例子来,写个 Controller
package xyz.wongs.drunkard.war3.web.controller;
import com.github.hiwepy.ip2region.spring.boot.IP2regionTemplate;
import com.github.hiwepy.ip2region.spring.boot.ext.RegionAddress;
import lombok.extern.slf4j.Slf4j;
import org.nutz.plugins.ip2region.DataBlock;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import xyz.wongs.drunkard.base.aop.annotion.ApplicationLog;
import xyz.wongs.drunkard.base.message.annoation.ResponseResult;
import xyz.wongs.drunkard.base.message.exception.DrunkardException;
import xyz.wongs.drunkard.framework.limit.RequestLimit;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
/**
* @ClassName IndexController
* @Description
* @author WCNGS@QQ.COM
* @Github <a>https://github.com/rothschil</a>
* @date 20/11/18 11:00
* @Version 1.0.0
*/
@Slf4j
@RestController
@ResponseResult
public class IndexController {
@RequestLimit(maxCount=3,second=20)
@ApplicationLog
@GetMapping("/test")
public Map<String, Object> test() {
HashMap<String, Object> data = new HashMap<>(3);
data.put("info", "测试成功");
return data;
}
@ApplicationLog
@GetMapping("/fail")
public Integer error() {
// 查询结果数
int res = 0;
if( res == 0 ) {
throw new DrunkardException("没有数据");
}
return res;
}
@Autowired
IP2regionTemplate template;
/** 根据输入IP地址,返回解析后的地址
* @Description
* @param ip
* @return xyz.wongs.drunkard.base.message.response.ResponseResult
* @throws
* @date 2020/8/17 18:26
*/
@GetMapping(value = "/convert/{ip}")
public DataBlock convertDataBlock(@PathVariable String ip){
DataBlock dataBlock = null;
try {
dataBlock = template.binarySearch(ip);
} catch (IOException e) {
e.printStackTrace();
}
return dataBlock;
}
/** 根据输入IP地址,返回解析后的地址
* @Description
* @param ip
* @return xyz.wongs.drunkard.base.message.response.ResponseResult
* @throws
* @date 2020/8/17 18:26
*/
@RequestLimit(maxCount=3)
@GetMapping(value = "/region/{ip}")
public RegionAddress convert(@PathVariable String ip){
RegionAddress regionAddress = null;
try {
regionAddress = template.getRegionAddress(ip);
} catch (IOException e) {
e.printStackTrace();
}
return regionAddress;
}
@GetMapping(value = "/region/ip={ip}")
public RegionAddress caseInsensitive(@PathVariable String ip){
RegionAddress regionAddress = null;
try {
regionAddress = template.getRegionAddress(ip);
} catch (IOException e) {
e.printStackTrace();
}
return regionAddress;
}
}
访问 http://localhost:9090/region/ip=109.27.45.12
这是我之前一个例子,用来解析IP地址,获取地域信息的。
Spring Validation
对 hibernate validatio
n 进行了二次封装,可以让我们更加方便地使用数据校验功能。这边我们通过 Spring Boot 来引用校验功能。
若使用的 Spring Boot 版本小于 2.3.x,spring-boot-starter-web
会自动引入 hibernate-validator
的依赖。
如果 Spring Boot 版本大于 2.3.x,则需要手动引入依赖,我这版本就是 大于 2.3.x,所以需要手工引入
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
<version>6.0.1.Final</version>
</dependency>
注解 | 释义 |
---|---|
@Null | 被注释的元素必须为 null |
@NotNull | 被注释的元素必须不为 null |
@AssertTrue | 被注释的元素必须为 true |
@AssertFalse | 被注释的元素必须为 false |
@Min(value) | 被注释的元素必须是一个数字,其值必须大于等于指定的最小值 |
@Max(value) | 被注释的元素必须是一个数字,其值必须小于等于指定的最大值 |
@DecimalMin(value) | 被注释的元素必须是一个数字,其值必须大于等于指定的最小值 |
@DecimalMax(value) | 被注释的元素必须是一个数字,其值必须小于等于指定的最大值 |
@Size(max, min) | 被注释的元素的大小必须在指定的范围内,元素必须为集合,代表集合个数 |
@Digits (integer, fraction) | 被注释的元素必须是一个数字,其值必须在可接受的范围内 |
@Past | 被注释的元素必须是一个过去的日期 |
@Future | 被注释的元素必须是一个将来的日期 |
被注释的元素必须是电子邮箱地址 | |
@Length(min=, max=) | 被注释的字符串的大小必须在指定的范围内,必须为数组或者字符串,若微数组则表示为数组长度,字符串则表示为字符串长度 |
@NotEmpty | 被注释的字符串的必须非空 |
@Range(min=, max=) | 被注释的元素必须在合适的范围内 |
@Pattern(regexp = ) | 正则表达式校验 |
@Valid | 对象级联校验,即校验对象中对象的属性 |
开发过程中常见问题的解答。
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。