# log-collection

**Repository Path**: Java_in_action/log-collection

## Basic Information

- **Project Name**: log-collection
- **Description**: 微服务生成traceId跟踪日志解决方案
- **Primary Language**: Java
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 5
- **Created**: 2024-09-09
- **Last Updated**: 2024-09-09

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# 查看日志场景
1. 接口通过网关，访问服务1
2. 接口通过网关，访问服务1，服务1访问服务2
3. 定时任务，访问服务1

# 实现逻辑过程
1. HTTP接口请求经过网关时，利用过滤器，将生成的traceId加到到RequestHeader中
2. 通过网关请求到服务中，利用MVC拦截器取出Header中的traceId,并且将traceId值使用Log中MDC类写入到日志中。
3. 服务1，通过Feign请求其他服务之前，取出MDC类中的traceId赋值到RequestHeader中，被请求服务使用2中的方式取出traceId并记录到日志中。
4. 服务器安装filebeat（或其他日志收集软件）收集日志，发送给ElasticSearch。
5. 通过Kiabana查看日志内容。通过网关或其他方式定位到存在问题的请求中的traceId，通过traceId查看请求内的所有日志

# 相关代码
### Zuul网关部分代码
#### TracePreFilter.java 过滤器
生成traceId，将traceId加入到RequestHeader中，带入到下游请求中。
``` java 
@Component
public class TracePreFilter extends ZuulFilter {

    @Override
    public String filterType() {
        return "pre";
    }

    @Override
    public int filterOrder() {
        return -1;
    }

    @Override
    public boolean shouldFilter() {
        return true;
    }

    @Override
    public Object run() throws ZuulException {
        String traceIdVal = UUID.randomUUID().toString().replaceAll("-","");
        MDC.put(LogCollectionConstants.traceId, traceIdVal);
        RequestContext ctx = RequestContext.getCurrentContext();
        ctx.addZuulRequestHeader(LogCollectionConstants.traceId, traceIdVal);
        return null;
    }
}
```

#### LogFilter.java 过滤器
记录请求的详细信息，请求参数，返回值，时长等信息
``` java 
@Component
@Slf4j
public class LogFilter extends ZuulFilter {

    public static final String START_TIME_KEY = "start_time";

    @Override
    public String filterType() {
        return FilterConstants.POST_TYPE;
    }

    @Override
    public int filterOrder() {
        return 0;
    }

    @Override
    public boolean shouldFilter() {
        return true;
    }

    @Override
    public Object run() {
        try {
            HttpServletRequest request = RequestContext.getCurrentContext().getRequest();
            RequestContext.getCurrentContext().set(START_TIME_KEY, System.currentTimeMillis());
            HttpEntity httpEntity = new HttpEntity();

            httpEntity.setMethod(request.getMethod());
            httpEntity.setUrl(request.getRequestURI());
            httpEntity.setIp(request.getRemoteAddr());
            HashMap<String, Object> parameter = showParams(request);
            httpEntity.setParameter(JSON.toJSONString(parameter));
            httpEntity.setUserAgent(request.getHeader("user-agent"));
            String body = "";
            InputStream stream = RequestContext.getCurrentContext().getResponseDataStream();
            byte[] bytes = StreamUtils.copyToByteArray(stream);
            body = new String(bytes, StandardCharsets.UTF_8);
            httpEntity.setResult(body);
            long startTime = (long) RequestContext.getCurrentContext().get(START_TIME_KEY);
            httpEntity.setLaunchTime(new Date(startTime));
            httpEntity.setDuration(System.currentTimeMillis() - startTime);
            httpEntity.setTraceId(RequestContext.getCurrentContext().getZuulRequestHeaders().get(LogCollectionConstants.traceId));
            log.info("接口统计 {}",JSON.toJSONString(httpEntity));
            RequestContext.getCurrentContext().setResponseBody(body);
        } catch (Exception e) {
            log.error("日志统计失败", e);
            return true;
        }
        return true;
    }

    public static HashMap<String, Object> showParams(HttpServletRequest request) {
        HashMap<String, Object> map = new HashMap<>();
        Enumeration paramNames = request.getParameterNames();
        while (paramNames.hasMoreElements()) {
            String paramName = (String) paramNames.nextElement();
            String[] paramValues = request.getParameterValues(paramName);
            if (paramValues.length > 0) {
                String paramValue = paramValues[0];
                if (paramValue.length() != 0) {
                    map.put(paramName, paramValue);
                }
            }
        }
        return map;
    }
}
```

### 服务代码
#### WebMvcConfig.java 过滤器
注册过滤器 ，将上游请求中traceId值取出。使用MDC类将内容记录到日志中
``` java 
@Configuration
public class WebMvcConfig extends WebMvcConfigurationSupport {
    @Override
    protected void addInterceptors(InterceptorRegistry registry) {
        registry.addInterceptor(new TraceInterceptor()).addPathPatterns("/**");
        super.addInterceptors(registry);
    }
}
``` 
``` java
public class TraceInterceptor implements HandlerInterceptor {

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
        String traceIdVal = request.getHeader(LogCollectionConstants.traceId);
        if (StringUtils.isNotEmpty(traceIdVal)) {
            MDC.put(LogCollectionConstants.traceId, traceIdVal);
        }
        return true;
    }
}
```
#### HystrixConfig.java 熔断配置
``` java 
@Configuration
public class HystrixConfig extends LogHystrixConfig {
    public HystrixConfig(){
        super();
    }
}
``` 
``` java
public class LogHystrixConfig {
    public static final Logger log = LoggerFactory.getLogger(LogHystrixConfig.class);

    public LogHystrixConfig(){
        try {
            HystrixConcurrencyStrategy target = new MdcHystrixConcurrencyStrategy();
            HystrixConcurrencyStrategy strategy = HystrixPlugins.getInstance().getConcurrencyStrategy();
            if (strategy instanceof MdcHystrixConcurrencyStrategy) {
                return;
            }
            HystrixCommandExecutionHook commandExecutionHook = HystrixPlugins
                    .getInstance().getCommandExecutionHook();
            HystrixEventNotifier eventNotifier = HystrixPlugins.getInstance()
                    .getEventNotifier();
            HystrixMetricsPublisher metricsPublisher = HystrixPlugins.getInstance()
                    .getMetricsPublisher();
            HystrixPropertiesStrategy propertiesStrategy = HystrixPlugins.getInstance()
                    .getPropertiesStrategy();

            HystrixPlugins.reset();
            HystrixPlugins.getInstance().registerConcurrencyStrategy(target);
            HystrixPlugins.getInstance()
                    .registerCommandExecutionHook(commandExecutionHook);
            HystrixPlugins.getInstance().registerEventNotifier(eventNotifier);
            HystrixPlugins.getInstance().registerMetricsPublisher(metricsPublisher);
            HystrixPlugins.getInstance().registerPropertiesStrategy(propertiesStrategy);
        }
        catch (Exception e) {
            log.error("Failed to register Sleuth Hystrix Concurrency Strategy", e);
        }
    }
}
```
#### FeignInterceptorConfig.java Feign配置
``` java
@Configuration
public class FeignInterceptorConfig extends LogFeignInterceptorConfig implements RequestInterceptor {

    @Override
    public void apply(RequestTemplate requestTemplate) {
        requestTemplate.header(LogCollectionConstants.traceId, super.getTraceId());
    }
}
``` 
``` java
public class LogFeignInterceptorConfig {

    public String getTraceId() {
        return MDC.get(LogCollectionConstants.traceId);
    }
}
```
#### 定时任务AOP
定时任务在开始执行之前，给MDC类 赋值traceId
``` java 
@Aspect
@Component
@Slf4j
public class ScheduledAspect {

    @Pointcut("@annotation(org.springframework.scheduling.annotation.Scheduled)")
    public void proxyAspect() {
    }

    @Before("proxyAspect()")
    public void before(JoinPoint joinPoint) throws Throwable {
        String traceId= LogHelper.getTraceId();
        MDC.put(LogCollectionConstants.traceId, traceId);
    }
}
```
#### logback.xml 日志配置文件
使用{traceId}写入MDC中的traceId的值
``` xml
    <property name="CONSOLE_LOG_PATTERN" value="${CONSOLE_LOG_PATTERN:-%clr(%d{${LOG_DATEFORMAT_PATTERN:-yyyy-MM-dd HH:mm:ss.SSS}}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }) [%X{traceId}] %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>
    <property name="FILE_LOG_PATTERN" value="${FILE_LOG_PATTERN:-%d{${LOG_DATEFORMAT_PATTERN:-yyyy-MM-dd HH:mm:ss.SSS}} ${LOG_LEVEL_PATTERN:-%5p} ${PID:- }  [%X{traceId}] --- [%t] : %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>

```

# 测试效果图
#### 1、接口通过网关，访问服务1

在Service1中写入测试接口
``` java
@RestController
@Slf4j
public class UserController {

    @GetMapping("/user/getlist")
    public List<String> getlist() {
        log.info("测试接口user");
        List<String> list = asList("user1", "user2", "user3");
        log.warn("测测测");
        return list;
    }

}
```
通过网关请求接口效果如下
![](./images/2021-02-08-11-01-08.png)

网关中打印日志 通过日志找到traceId为40417dd85d224eda8d67925dce335b6b
![](./images/2021-02-08-11-01-51.png)
Service1中的日志
![](./images/2021-02-08-11-17-37.png)

#### 2、接口通过网关，访问服务1，服务1访问服务2
测试代码
Service1中
``` java
@RestController
@Slf4j
public class UserController {
    @Autowired
    private DemoApiClient demoApiClient;

    @GetMapping("/user/getlist2")
    public List<String> getlist2() {
        log.info("测试接口getlist2");
        List<String> result = demoApiClient.getlist2();
        log.info("请求Service2中接口成功,返回数据");
        return result;
    }
}
```
Service2中
``` java 

@RestController
@Slf4j
public class DemoController {
    @GetMapping("/demo/getlist2")
    public List<String> getlist2() {
        log.info("通过了其他的服务请求过来");
        List<String> list = asList("haha", "hehe", "youyou");
        log.info("请求完了");
        return list;
    }
}
``` 
通过网关查到traceId为f87e568753004b7e830e76497936fb08
![](./images/2021-02-08-11-22-01.png)
Service1中打印日志
![](./images/2021-02-08-11-22-54.png)
Service2中打印日志
![](./images/2021-02-08-11-23-14.png)


#### 3、定时任务，访问服务1
在Service1中加入测试代码
``` java
@Component
@Slf4j
public class TestJob {

    @Autowired
    private DemoApiClient demoApiClient;

    @Scheduled(fixedRate = 2000)
    public void execute() {
        List<String> result = demoApiClient.getlist2();
        log.info("定时任务执行 打印获取数据结果 {} ", result);

    }
}
```
在任务中找到一条执行记录中的traceId：56fda9d194e241f9a32e6c1ecf61ee58
![](./images/2021-02-08-11-47-34.png)

在Service2中查看打印日志
![](./images/2021-02-08-11-48-39.png)

博客地址:https://www.cnblogs.com/jiage/p/14388275.html