Compare commits

...

3 Commits

Author SHA1 Message Date
66d351afe4 feat(steady): 新增数据校验功能并优化稳态趋势查询
- 在 AddLedgerLineMapper.xml 中添加 lineInterval 字段映射
- 在 AddLedgerLinePathVO 中添加 lineInterval 属性用于存储统计间隔
- 为稳态趋势查询服务添加详细的执行日志记录和性能监控
- 重构 InfluxDB 查询组件,添加诊断信息构建方法和异常处理
- 限制谐波次数最大展示数量从 6 个调整为 3 个
- 新增数据校验相关组件、控制器和服务实现
- 实现数据连续性检查和缺失数据统计功能
- 添加数据校验查询参数和返回结果的数据结构定义
- 完善相关单元测试确保功能正确性
2026-05-27 08:04:49 +08:00
e5369fef5a docs: 规划deploy Linux远程运维实现 2026-05-21 21:13:17 +08:00
ba8bc43377 docs: 设计deploy Linux远程运维方案 2026-05-21 16:44:42 +08:00
26 changed files with 3696 additions and 7 deletions

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,514 @@
# Linux 服务器部署运维设计
## 1. 背景
`system-ops/deploy` 当前只提供系统部署菜单的基础入口:
- `GET /deploy/overview`
- `DeployController`
- `DeployService`
- `DeployOverviewVO`
本次需求是在 `deploy` 模块中补充 Linux 服务器远程运维能力。用户可以维护 Linux 服务器连接配置,基于 SSH/SFTP 连接服务器,完成远程文件上传、下载和基础命令终端操作。命令终端目标体验接近 Xshell 的基础能力。
当前仓库没有前端代码,本设计只定义页面布局、接口契约、后端模块拆分、数据存储和验证方式,不实现真实前端页面。
## 2. 范围确认
本期只支持 Linux 服务器。
本期包含:
- Linux SSH 连接配置的新增、编辑、删除、查询。
- SSH 连接测试。
- SFTP 文件列表、上传、下载、删除、新建目录。
- SSH Shell 基础命令交互。
- 前端单页运维工作台布局设计。
- 连接配置使用文件方式存储,不新建数据库表。
本期不包含:
- Windows 服务器。
- FTP 协议。
- 数据库存储连接配置。
- 部署任务编排。
- 命令审批、命令黑名单、命令历史。
- 批量文件压缩下载。
- 数据库专用客户端封装。
- Maven 编译、打包、测试。
说明:需求中提到的 “FPT” 本期按 Linux 服务器常用能力理解为 SFTP。SFTP 复用 SSH 账号、密码和端口,比单独 FTP 更适合本期场景。
## 3. 总体方案
推荐采用 “SSH/SFTP + WebSocket 终端” 方案:
- 服务器连接配置保存到本地 JSON 文件。
- 后端通过 SSH 建立 Linux 连接。
- 文件操作通过 SFTP 通道完成。
- 终端操作通过 SSH Shell 通道完成。
- 前端通过 WebSocket 与后端交换终端输入输出。
该方案可以复用同一份服务器连接配置,不需要引入 Windows 远程协议,也能满足类 Xshell 的基础交互需求。
## 4. 前端页面布局
页面路径建议沿用当前菜单路径:
```text
/systemOps/deploy
```
页面采用三块工作区:
- 左侧:服务器列表。
- 中间:远程文件管理。
- 右侧:连接详情和快捷操作。
- 底部SSH 终端区。
推荐布局:
```text
┌──────────────────────────────────────────────────────────────┐
│ 顶部工具栏:新增连接 测试连接 刷新 当前连接状态 │
├──────────────┬──────────────────────────────┬────────────────┤
│ 服务器列表 │ 远程文件管理 │ 连接详情/操作 │
│ │ │ │
│ Linux-测试 │ 路径栏:/opt/app │ 主机/IP │
│ Linux-生产 │ 上传 下载 新建目录 删除 刷新 │ 用户名 │
│ │ │ 端口 │
│ │ 文件表格 │ 测试连接 │
│ │ │ 打开终端 │
├──────────────┴──────────────────────────────┴────────────────┤
│ 终端 TabsLinux-测试 │
│ $ pwd │
│ /opt/app │
└──────────────────────────────────────────────────────────────┘
```
### 4.1 服务器列表
左侧服务器列表用于选择当前操作目标。
展示字段:
| 字段 | 说明 |
|---|---|
| 名称 | 服务器显示名称 |
| 主机地址 | IP 或域名 |
| SSH 端口 | 默认 22 |
| 连接状态 | 未测试、连接成功、连接失败 |
交互:
- 支持按名称、主机地址搜索。
- 点击服务器后加载连接详情,并将文件管理区切换到该服务器。
- 列表项提供编辑、删除、测试连接入口。
- 删除连接前必须二次确认。
### 4.2 连接配置弹窗
新增和编辑使用同一个弹窗。
字段:
| 字段 | 是否必填 | 说明 |
|---|---|---|
| 名称 | 是 | 页面展示名称 |
| 主机地址 | 是 | Linux 服务器 IP 或域名 |
| SSH 端口 | 是 | 默认 22范围 1-65535 |
| 用户名 | 是 | SSH 登录用户 |
| 密码 | 新增必填 | 编辑时留空表示不修改 |
| 备注 | 否 | 环境说明 |
按钮:
- 测试连接。
- 保存。
- 取消。
密码规则:
- 新增连接时密码必填。
- 编辑连接时密码不回显。
- 编辑时密码为空表示沿用原密码。
- 查询列表和详情接口均不返回密码。
### 4.3 远程文件管理
中间文件管理区基于当前选中的服务器工作。
顶部路径栏:
- 展示当前远程目录,例如 `/opt/app`
- 支持返回上级目录。
- 支持点击面包屑跳转到上级路径。
工具栏:
- 上传。
- 下载。
- 新建目录。
- 删除。
- 刷新。
文件表格字段:
| 字段 | 说明 |
|---|---|
| 名称 | 文件或目录名称 |
| 类型 | 文件、目录、软链接 |
| 大小 | 文件大小,目录可为空 |
| 权限 | Linux 权限字符串 |
| 修改时间 | 远程文件修改时间 |
交互规则:
- 双击目录进入下级目录。
- 下载只支持普通文件。
- 删除文件或目录前必须二次确认。
- 本期支持单文件上传和单文件下载。
- 上传目标目录为当前路径。
- 下载目录、批量压缩下载不在本期范围。
### 4.4 SSH 终端区
底部终端区用于执行 Linux 命令。
交互规则:
- 点击“打开终端”后创建 SSH Shell 会话。
- 前端输入通过 WebSocket 实时发送给后端。
- 后端将 Shell 输出通过 WebSocket 实时推送给前端。
- 本期建议限制为每台服务器最多一个终端会话。
- 关闭终端 Tab 时通知后端释放 SSH 会话。
- 终端断开后显示状态,不自动重连。
用户可以在终端中自行执行数据库命令,例如:
```bash
mysql -uroot -p
psql -h 127.0.0.1 -U postgres
redis-cli
```
后端不解析数据库命令,也不保存命令历史。
## 5. 后端结构设计
`system-ops/deploy` 模块内按职责新增类,保留现有 `DeployController``/deploy/overview`
建议结构:
```text
system-ops/deploy/src/main/java/com/njcn/gather/systemops/deploy/
├── config/
├── controller/
├── pojo/param/
├── pojo/vo/
├── pojo/dto/
├── repository/
├── service/
├── service/impl/
└── websocket/
```
职责拆分:
| 类 | 职责 |
|---|---|
| `DeployServerController` | 连接配置查询、新增、编辑、删除、测试连接 |
| `DeployFileController` | SFTP 文件列表、上传、下载、删除、新建目录 |
| `DeployTerminalWebSocketHandler` | SSH 终端 WebSocket 输入输出转发 |
| `DeployServerConfigService` | 连接配置业务校验和编排 |
| `DeployServerConfigRepository` | JSON 文件读写 |
| `DeploySftpService` | SFTP 文件操作 |
| `DeploySshTerminalService` | SSH Shell 会话创建、输入、输出、关闭 |
| `DeployCryptoService` | 密码加密和解密 |
| `DeployProperties` | deploy 配置项绑定 |
## 6. 连接配置存储
连接配置不入库,使用 JSON 文件落盘。存储目录通过配置指定。
建议配置:
```yaml
deploy:
storage-dir: ${log.homeDir}/deploy
terminal-idle-timeout-minutes: 30
```
`deploy.crypto-key` 不建议在默认 `application.yml` 中配置明文值。后续实现时可通过环境覆盖或外部配置提供,业务代码只读取配置,不写死密钥。
落盘文件:
```text
D:\logs\deploy\deploy-server-connections.json
```
JSON 结构:
```json
{
"servers": [
{
"id": "uuid",
"name": "测试服务器",
"host": "192.168.1.10",
"sshPort": 22,
"username": "root",
"password": "加密密文",
"description": "测试环境",
"createdTime": "2026-05-21 14:00:00",
"updatedTime": "2026-05-21 14:00:00"
}
]
}
```
写文件规则:
- 启动时如果文件不存在,自动创建空配置文件。
- 读写方法集中在 `DeployServerConfigRepository`
- 写入时先写临时文件,再替换正式文件,避免进程中断导致 JSON 损坏。
- 保存和删除操作需要加进程内锁,避免并发写入互相覆盖。
密码规则:
- 密码必须加密后落盘。
- 接口返回不包含密码。
- 日志不打印密码。
- 优先复用项目已有加密能力;如没有合适工具,则在 `deploy` 内封装 AES 加解密组件。
- 加密密钥通过配置提供,不在业务代码中硬编码。
## 7. 接口设计
接口风格沿用当前仓库常见写法:查询和变更优先使用 `POST`,返回 `HttpResult<T>`
### 7.1 连接配置接口
| 方法 | 路径 | 说明 |
|---|---|---|
| `POST` | `/deploy/server/list` | 查询服务器连接配置列表 |
| `POST` | `/deploy/server/add` | 新增服务器连接配置 |
| `POST` | `/deploy/server/update` | 修改服务器连接配置 |
| `POST` | `/deploy/server/delete` | 删除服务器连接配置 |
| `POST` | `/deploy/server/test` | 测试 SSH 连接 |
列表返回字段:
| 字段 | 说明 |
|---|---|
| `id` | 连接 ID |
| `name` | 服务器名称 |
| `host` | 主机地址 |
| `sshPort` | SSH 端口 |
| `username` | 用户名 |
| `description` | 备注 |
| `createdTime` | 创建时间 |
| `updatedTime` | 更新时间 |
新增参数:
| 字段 | 是否必填 |
|---|---|
| `name` | 是 |
| `host` | 是 |
| `sshPort` | 是 |
| `username` | 是 |
| `password` | 是 |
| `description` | 否 |
编辑参数:
| 字段 | 是否必填 | 说明 |
|---|---|---|
| `id` | 是 | 连接 ID |
| `name` | 是 | 服务器名称 |
| `host` | 是 | 主机地址 |
| `sshPort` | 是 | SSH 端口 |
| `username` | 是 | 用户名 |
| `password` | 否 | 为空表示不修改 |
| `description` | 否 | 备注 |
### 7.2 文件接口
| 方法 | 路径 | 说明 |
|---|---|---|
| `POST` | `/deploy/file/list` | 查询远程目录文件列表 |
| `POST` | `/deploy/file/mkdir` | 新建远程目录 |
| `POST` | `/deploy/file/delete` | 删除远程文件或目录 |
| `POST` | `/deploy/file/upload` | 上传本地文件到远程目录 |
| `POST` | `/deploy/file/download` | 下载远程普通文件 |
文件列表参数:
| 字段 | 是否必填 | 说明 |
|---|---|---|
| `serverId` | 是 | 服务器连接 ID |
| `path` | 是 | 远程目录路径 |
文件列表返回字段:
| 字段 | 说明 |
|---|---|
| `name` | 文件名 |
| `path` | 完整路径 |
| `type` | `FILE``DIRECTORY``LINK` |
| `size` | 文件大小 |
| `permissions` | 权限字符串 |
| `modifiedTime` | 修改时间 |
下载接口直接写入 `HttpServletResponse`。下载文件名沿用远程文件名,不追加日期;仓库“导出或生成文件追加日期”的规则适用于后端生成或导出文件,本功能是下载远程已有文件,不改变原文件名。
### 7.3 终端 WebSocket
终端连接:
```text
WebSocket /deploy/terminal?serverId={serverId}
```
前端发送输入:
```json
{
"type": "input",
"data": "ls -la\n"
}
```
前端发送窗口大小:
```json
{
"type": "resize",
"cols": 120,
"rows": 30
}
```
后端输出:
```json
{
"type": "output",
"data": "total 20\r\n..."
}
```
后端状态:
```json
{
"type": "status",
"status": "CONNECTED"
}
```
异常消息:
```json
{
"type": "error",
"message": "SSH连接失败"
}
```
## 8. 参数校验
后端至少补充以下校验:
- 服务器名称不能为空。
- 主机地址不能为空。
- SSH 端口范围为 `1-65535`
- 用户名不能为空。
- 新增连接时密码不能为空。
- 编辑连接时 `id` 必须存在。
- 删除连接时 `id` 必须存在。
- 同一主机、端口、用户名组合不建议重复保存。
- 文件路径不能为空。
- 文件上传目标必须是远程目录。
- 下载目标必须是远程普通文件。
- 删除路径不能为空,不能删除空路径或根目录 `/`
- 新建目录名称不能为空,不能包含路径分隔符。
## 9. 安全与资源控制
安全规则:
- 密码不明文落盘。
- 接口返回不包含密码。
- 日志不打印密码、终端输入内容、文件内容。
- 终端不保存命令历史。
- 文件路径需要做基础规范化,避免空路径、非法路径和目录穿越。
- 下载只允许下载普通文件。
资源规则:
- SSH 连接测试设置连接超时,例如 5 秒。
- SFTP 操作每次请求创建短连接,操作完成后释放。
- 终端会话保持长连接,关闭 WebSocket 后释放 SSH Session 和 Channel。
- 终端会话设置空闲超时,默认 30 分钟。
- 本期每台服务器最多保留一个终端会话。
## 10. 依赖建议
后续实现 SSH/SFTP 时建议优先选择 Java 8 可用、项目易接入的 SSH 客户端库,例如 JSch 或 sshj。
选择标准:
- 支持 SSH 密码登录。
- 支持 SFTP 文件操作。
- 支持 Shell Channel。
- 能在 Spring Boot 2.3 和 Java 8 下稳定使用。
最终依赖需要写入 `system-ops/deploy/pom.xml`,不影响其他模块。
## 11. 错误处理
连接测试需要区分常见错误:
| 场景 | 返回说明 |
|---|---|
| 主机不可达 | 连接服务器失败 |
| 端口不通 | SSH端口连接失败 |
| 账号或密码错误 | SSH认证失败 |
| SFTP 打开失败 | 文件通道打开失败 |
| 终端打开失败 | Shell通道打开失败 |
接口层仍使用项目现有 `HttpResult``CommonResponseEnum` 风格。具体错误文案由 Service 返回给 Controller不新增全局异常体系。
## 12. 验证方式
默认不执行 Maven 编译、打包、测试命令。后续实现完成后按以下方式验证:
- 检查 `deploy` 新增代码只位于 `system-ops/deploy`
- 新增连接后,接口返回和 JSON 文件内容一致。
- 编辑连接时密码留空不会覆盖原密码。
- 删除连接后JSON 文件同步移除对应记录。
- 查询接口不返回密码。
- JSON 文件中密码不是明文。
- 测试连接能识别成功、主机不可达、端口不通、账号密码错误。
- 文件列表能展示远程目录内容。
- 上传文件后远程目录可见。
- 下载普通文件内容与远程文件一致。
- 删除文件或目录后远程路径不存在。
- 新建目录后远程路径存在。
- 终端能打开 Linux Shell执行 `pwd``ls -la``mysql --version` 等基础命令。
- 关闭终端后,后端 SSH 会话被释放。
## 13. 后续扩展
后续如需求增加,可以在当前方案基础上扩展:
- SSH 私钥登录。
- 多终端 Tab。
- 命令审计和历史记录。
- 命令黑名单或审批。
- 部署脚本编排。
- 文件批量上传和批量下载。
- Windows WinRM 或 PowerShell Remoting。
这些能力不进入本期实现,避免当前 `deploy` 模块从基础入口一次扩张为完整运维平台。

View File

@@ -0,0 +1,75 @@
package com.njcn.gather.steady.checksquare.component;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareSegmentVO;
import org.springframework.stereotype.Component;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.List;
import java.util.Set;
/**
* 数据校验连续性计算组件。
*/
@Component
public class SteadyChecksquareCalculator {
public static final String STATUS_NORMAL = "NORMAL";
public static final String STATUS_MISSING = "MISSING";
private static final DateTimeFormatter OUTPUT_TIME_FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
public List<SteadyChecksquareSegmentVO> buildSegments(List<LocalDateTime> slots, Set<LocalDateTime> actualSlots,
int intervalMinutes) {
List<SteadyChecksquareSegmentVO> result = new ArrayList<SteadyChecksquareSegmentVO>();
if (slots == null || slots.isEmpty()) {
return result;
}
String currentStatus = resolveStatus(slots.get(0), actualSlots);
LocalDateTime segmentStart = slots.get(0);
LocalDateTime previousSlot = slots.get(0);
int pointCount = 1;
for (int i = 1; i < slots.size(); i++) {
LocalDateTime slot = slots.get(i);
String status = resolveStatus(slot, actualSlots);
if (!currentStatus.equals(status)) {
result.add(buildSegment(segmentStart, previousSlot, currentStatus, pointCount, intervalMinutes));
segmentStart = slot;
pointCount = 0;
currentStatus = status;
}
previousSlot = slot;
pointCount++;
}
result.add(buildSegment(segmentStart, previousSlot, currentStatus, pointCount, intervalMinutes));
return result;
}
public int maxContinuousMissingMinutes(List<SteadyChecksquareSegmentVO> segments) {
int result = 0;
if (segments == null) {
return result;
}
for (SteadyChecksquareSegmentVO segment : segments) {
if (segment != null && STATUS_MISSING.equals(segment.getStatus()) && segment.getDurationMinutes() != null) {
result = Math.max(result, segment.getDurationMinutes());
}
}
return result;
}
private SteadyChecksquareSegmentVO buildSegment(LocalDateTime startTime, LocalDateTime endTime, String status,
int pointCount, int intervalMinutes) {
SteadyChecksquareSegmentVO segment = new SteadyChecksquareSegmentVO();
segment.setStartTime(OUTPUT_TIME_FORMATTER.format(startTime));
segment.setEndTime(OUTPUT_TIME_FORMATTER.format(endTime));
segment.setStatus(status);
segment.setMissingPointCount(STATUS_MISSING.equals(status) ? pointCount : 0);
segment.setDurationMinutes(pointCount * intervalMinutes);
return segment;
}
private String resolveStatus(LocalDateTime slot, Set<LocalDateTime> actualSlots) {
return actualSlots != null && actualSlots.contains(slot) ? STATUS_NORMAL : STATUS_MISSING;
}
}

View File

@@ -0,0 +1,201 @@
package com.njcn.gather.steady.checksquare.component;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.njcn.common.pojo.enums.response.CommonResponseEnum;
import com.njcn.common.pojo.exception.BusinessException;
import com.njcn.gather.steady.datavie.config.SteadyInfluxDbProperties;
import com.njcn.gather.steady.datavie.pojo.bo.SteadyTrendResolvedFieldBO;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URLEncoder;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.time.LocalDateTime;
import java.time.OffsetDateTime;
import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.util.HashSet;
import java.util.Set;
/**
* 数据校验 InfluxDB 查询组件。
*/
@Slf4j
@Component
@RequiredArgsConstructor
public class SteadyChecksquareInfluxQueryComponent {
private static final DateTimeFormatter INFLUX_TIME_FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss'Z'");
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
private final SteadyInfluxDbProperties properties;
public Set<LocalDateTime> queryExistingSlots(SteadyTrendResolvedFieldBO field, LocalDateTime startTime,
LocalDateTime endTime, int intervalMinutes) {
validateConfig();
String query = buildChecksquareQuery(field, startTime, endTime);
long startMillis = System.currentTimeMillis();
log.info("数据校验 InfluxDB 查询开始measurement={}field={}lineId={}phase={}statType={}query={}",
field.getMeasurement(), field.getField(), field.getLineId(), field.getPhase(), field.getStatType(), query);
try {
String body = executeQuery(query);
Set<LocalDateTime> slots = parseExistingSlots(body, intervalMinutes);
log.info("数据校验 InfluxDB 查询结束slotCount={}costMs={}", slots.size(), System.currentTimeMillis() - startMillis);
return slots;
} catch (RuntimeException ex) {
log.warn("数据校验 InfluxDB 查询异常costMs={}error={}", System.currentTimeMillis() - startMillis, ex.getMessage());
throw ex;
}
}
public String buildChecksquareQuery(SteadyTrendResolvedFieldBO field, LocalDateTime startTime, LocalDateTime endTime) {
StringBuilder sql = new StringBuilder();
sql.append("SELECT \"").append(field.getField()).append("\" AS \"value\"");
sql.append(" FROM \"").append(field.getMeasurement()).append("\"");
sql.append(" WHERE time >= '").append(INFLUX_TIME_FORMATTER.format(startTime)).append("'");
sql.append(" AND time <= '").append(INFLUX_TIME_FORMATTER.format(endTime)).append("'");
sql.append(" AND \"line_id\" = '").append(escapeTagValue(field.getLineId())).append("'");
sql.append(" AND \"phasic_type\" = '").append(escapeTagValue(field.getPhase())).append("'");
if (hasValueTypeTag(field.getMeasurement())) {
sql.append(" AND \"value_type\" = '").append(resolveValueType(field.getStatType())).append("'");
}
sql.append(" ORDER BY time ASC");
return sql.toString();
}
private Set<LocalDateTime> parseExistingSlots(String body, int intervalMinutes) {
try {
JsonNode root = OBJECT_MAPPER.readTree(body);
JsonNode values = root.path("results").path(0).path("series").path(0).path("values");
Set<LocalDateTime> result = new HashSet<LocalDateTime>();
if (!values.isArray()) {
return result;
}
for (JsonNode value : values) {
if (value.size() < 2 || value.get(1).isNull()) {
continue;
}
LocalDateTime time = parseInfluxTime(value.get(0).asText());
if (time != null) {
result.add(alignToPreviousSlot(time, intervalMinutes));
}
}
return result;
} catch (IOException ex) {
throw fail("InfluxDB 返回结果解析失败:" + ex.getMessage());
}
}
private LocalDateTime alignToPreviousSlot(LocalDateTime time, int intervalMinutes) {
LocalDateTime minuteFloor = time.withSecond(0).withNano(0);
int minuteOfDay = minuteFloor.getHour() * 60 + minuteFloor.getMinute();
int remainder = minuteOfDay % intervalMinutes;
return minuteFloor.minusMinutes(remainder);
}
private LocalDateTime parseInfluxTime(String value) {
try {
return OffsetDateTime.parse(value).withOffsetSameInstant(ZoneOffset.UTC).toLocalDateTime();
} catch (RuntimeException ex) {
return null;
}
}
private String executeQuery(String query) {
HttpURLConnection connection = null;
try {
URL url = new URL(buildQueryUrl(query));
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setConnectTimeout(properties.getConnectTimeoutMs());
connection.setReadTimeout(properties.getReadTimeoutMs());
int status = connection.getResponseCode();
InputStream stream = status >= 200 && status < 300 ? connection.getInputStream() : connection.getErrorStream();
String body = readBody(stream);
if (status < 200 || status >= 300) {
throw fail("InfluxDB 查询失败:" + body);
}
return body;
} catch (IOException ex) {
throw fail("InfluxDB 查询异常:" + ex.getMessage());
} finally {
if (connection != null) {
connection.disconnect();
}
}
}
private String buildQueryUrl(String query) throws IOException {
StringBuilder url = new StringBuilder(trimRightSlash(properties.getUrl())).append("/query?");
url.append("db=").append(encode(properties.getDatabase()));
if (properties.getUsername() != null && !properties.getUsername().trim().isEmpty()) {
url.append("&u=").append(encode(properties.getUsername().trim()));
}
if (properties.getPassword() != null && !properties.getPassword().trim().isEmpty()) {
url.append("&p=").append(encode(properties.getPassword()));
}
url.append("&q=").append(encode(query));
return url.toString();
}
private void validateConfig() {
if (properties.getUrl() == null || properties.getUrl().trim().isEmpty()) {
throw fail("InfluxDB 地址未配置");
}
if (properties.getDatabase() == null || properties.getDatabase().trim().isEmpty()) {
throw fail("InfluxDB database 未配置");
}
}
private String readBody(InputStream stream) throws IOException {
if (stream == null) {
return "";
}
BufferedReader reader = new BufferedReader(new InputStreamReader(stream, StandardCharsets.UTF_8));
StringBuilder body = new StringBuilder();
String line;
while ((line = reader.readLine()) != null) {
body.append(line);
}
return body.toString();
}
private String escapeTagValue(String value) {
return value == null ? "" : value.replace("\\", "\\\\").replace("'", "\\'");
}
private String resolveValueType(String statType) {
if (statType == null || statType.trim().isEmpty()) {
return "AVG";
}
return statType.trim().toUpperCase();
}
private boolean hasValueTypeTag(String measurement) {
return !"data_flicker".equals(measurement) && !"data_fluc".equals(measurement) && !"data_plt".equals(measurement);
}
private String trimRightSlash(String value) {
String text = value.trim();
while (text.endsWith("/")) {
text = text.substring(0, text.length() - 1);
}
return text;
}
private String encode(String value) throws IOException {
return URLEncoder.encode(value, StandardCharsets.UTF_8.name());
}
private BusinessException fail(String message) {
return new BusinessException(CommonResponseEnum.FAIL, message);
}
}

View File

@@ -0,0 +1,43 @@
package com.njcn.gather.steady.checksquare.controller;
import com.njcn.common.pojo.annotation.OperateInfo;
import com.njcn.common.pojo.enums.common.LogEnum;
import com.njcn.common.pojo.enums.response.CommonResponseEnum;
import com.njcn.common.pojo.response.HttpResult;
import com.njcn.common.utils.LogUtil;
import com.njcn.gather.steady.checksquare.pojo.param.SteadyChecksquareQueryParam;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareQueryVO;
import com.njcn.gather.steady.checksquare.service.SteadyChecksquareService;
import com.njcn.web.controller.BaseController;
import com.njcn.web.utils.HttpResultUtil;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiOperation;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* 数据校验接口。
*/
@Slf4j
@Api(tags = "数据校验")
@RestController
@RequestMapping("/steady/data-view/checksquare")
@RequiredArgsConstructor
public class SteadyChecksquareController extends BaseController {
private final SteadyChecksquareService checksquareService;
@OperateInfo(info = LogEnum.BUSINESS_COMMON)
@ApiOperation("查询数据校验结果")
@PostMapping("/query")
public HttpResult<SteadyChecksquareQueryVO> query(@RequestBody SteadyChecksquareQueryParam param) {
String methodDescribe = getMethodDescribe("query");
LogUtil.njcnDebug(log, "{}开始查询数据校验结果param={}", methodDescribe, param);
SteadyChecksquareQueryVO result = checksquareService.query(param);
return HttpResultUtil.assembleCommonResponseResult(CommonResponseEnum.SUCCESS, result, methodDescribe);
}
}

View File

@@ -0,0 +1,33 @@
package com.njcn.gather.steady.checksquare.pojo.param;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.Data;
import java.io.Serializable;
import java.util.List;
/**
* 数据校验查询参数。
*/
@Data
@ApiModel("数据校验查询参数")
public class SteadyChecksquareQueryParam implements Serializable {
private static final long serialVersionUID = 1L;
@ApiModelProperty("监测点 ID")
private String lineId;
@ApiModelProperty("指标编码")
private List<String> indicatorCodes;
@ApiModelProperty("开始时间,格式 yyyy-MM-dd HH:mm:ss")
private String timeStart;
@ApiModelProperty("结束时间,格式 yyyy-MM-dd HH:mm:ss")
private String timeEnd;
@ApiModelProperty("谐波次数,谐波指标按请求次数查询")
private List<Integer> harmonicOrders;
}

View File

@@ -0,0 +1,62 @@
package com.njcn.gather.steady.checksquare.pojo.vo;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.Data;
import java.io.Serializable;
import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.List;
/**
* 数据校验总览项。
*/
@Data
@ApiModel("数据校验总览项")
public class SteadyChecksquareItemVO implements Serializable {
private static final long serialVersionUID = 1L;
@ApiModelProperty("校验项唯一键")
private String itemKey;
@ApiModelProperty("指标编码")
private String indicatorCode;
@ApiModelProperty("指标名称")
private String indicatorName;
@ApiModelProperty("谐波次数")
private Integer harmonicOrder;
@ApiModelProperty("当前校验项统计间隔,单位分钟")
private Integer intervalMinutes;
@ApiModelProperty("时间范围内是否存在任意数据")
private Boolean hasData;
@ApiModelProperty("期望点数")
private Integer expectedPointCount;
@ApiModelProperty("实际点数")
private Integer actualPointCount;
@ApiModelProperty("缺失点数")
private Integer missingPointCount;
@ApiModelProperty("缺失率")
private BigDecimal missingRate;
@ApiModelProperty("缺失率文本")
private String missingRateText;
@ApiModelProperty("最大连续缺失时长,单位分钟")
private Integer maxContinuousMissingMinutes;
@ApiModelProperty("统计类型摘要")
private List<SteadyChecksquareStatSummaryVO> statSummaries = new ArrayList<SteadyChecksquareStatSummaryVO>();
@ApiModelProperty("统计类型明细")
private List<SteadyChecksquareStatDetailVO> statDetails = new ArrayList<SteadyChecksquareStatDetailVO>();
}

View File

@@ -0,0 +1,37 @@
package com.njcn.gather.steady.checksquare.pojo.vo;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.Data;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
/**
* 数据校验查询结果。
*/
@Data
@ApiModel("数据校验查询结果")
public class SteadyChecksquareQueryVO implements Serializable {
private static final long serialVersionUID = 1L;
@ApiModelProperty("监测点 ID")
private String lineId;
@ApiModelProperty("监测点名称")
private String lineName;
@ApiModelProperty("开始时间")
private String timeStart;
@ApiModelProperty("结束时间")
private String timeEnd;
@ApiModelProperty("统计间隔,单位分钟")
private Integer intervalMinutes;
@ApiModelProperty("校验项")
private List<SteadyChecksquareItemVO> items = new ArrayList<SteadyChecksquareItemVO>();
}

View File

@@ -0,0 +1,32 @@
package com.njcn.gather.steady.checksquare.pojo.vo;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.Data;
import java.io.Serializable;
/**
* 数据校验连续性区间。
*/
@Data
@ApiModel("数据校验连续性区间")
public class SteadyChecksquareSegmentVO implements Serializable {
private static final long serialVersionUID = 1L;
@ApiModelProperty("开始时间")
private String startTime;
@ApiModelProperty("结束时间")
private String endTime;
@ApiModelProperty("状态NORMAL/MISSING")
private String status;
@ApiModelProperty("缺失点数")
private Integer missingPointCount;
@ApiModelProperty("持续时长,单位分钟")
private Integer durationMinutes;
}

View File

@@ -0,0 +1,28 @@
package com.njcn.gather.steady.checksquare.pojo.vo;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.Data;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
/**
* 数据校验统计类型明细。
*/
@Data
@ApiModel("数据校验统计类型明细")
public class SteadyChecksquareStatDetailVO implements Serializable {
private static final long serialVersionUID = 1L;
@ApiModelProperty("统计类型")
private String statType;
@ApiModelProperty("是否支持")
private Boolean supported;
@ApiModelProperty("连续性区间")
private List<SteadyChecksquareSegmentVO> segments = new ArrayList<SteadyChecksquareSegmentVO>();
}

View File

@@ -0,0 +1,45 @@
package com.njcn.gather.steady.checksquare.pojo.vo;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.Data;
import java.io.Serializable;
import java.math.BigDecimal;
/**
* 数据校验统计类型摘要。
*/
@Data
@ApiModel("数据校验统计类型摘要")
public class SteadyChecksquareStatSummaryVO implements Serializable {
private static final long serialVersionUID = 1L;
@ApiModelProperty("统计类型")
private String statType;
@ApiModelProperty("是否支持")
private Boolean supported;
@ApiModelProperty("是否存在数据")
private Boolean hasData;
@ApiModelProperty("期望点数")
private Integer expectedPointCount;
@ApiModelProperty("实际点数")
private Integer actualPointCount;
@ApiModelProperty("缺失点数")
private Integer missingPointCount;
@ApiModelProperty("缺失率")
private BigDecimal missingRate;
@ApiModelProperty("缺失率文本")
private String missingRateText;
@ApiModelProperty("最大连续缺失时长,单位分钟")
private Integer maxContinuousMissingMinutes;
}

View File

@@ -0,0 +1,12 @@
package com.njcn.gather.steady.checksquare.service;
import com.njcn.gather.steady.checksquare.pojo.param.SteadyChecksquareQueryParam;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareQueryVO;
/**
* 数据校验服务。
*/
public interface SteadyChecksquareService {
SteadyChecksquareQueryVO query(SteadyChecksquareQueryParam param);
}

View File

@@ -0,0 +1,349 @@
package com.njcn.gather.steady.checksquare.service.impl;
import com.njcn.common.pojo.enums.response.CommonResponseEnum;
import com.njcn.common.pojo.exception.BusinessException;
import com.njcn.gather.steady.checksquare.component.SteadyChecksquareCalculator;
import com.njcn.gather.steady.checksquare.component.SteadyChecksquareInfluxQueryComponent;
import com.njcn.gather.steady.checksquare.pojo.param.SteadyChecksquareQueryParam;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareItemVO;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareQueryVO;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareSegmentVO;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareStatDetailVO;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareStatSummaryVO;
import com.njcn.gather.steady.checksquare.service.SteadyChecksquareService;
import com.njcn.gather.steady.datavie.component.SteadyTrendIndicatorCatalog;
import com.njcn.gather.steady.datavie.pojo.bo.SteadyTrendIndicatorDefinitionBO;
import com.njcn.gather.steady.datavie.pojo.bo.SteadyTrendResolvedFieldBO;
import com.njcn.gather.steady.datavie.pojo.bo.SteadyTrendSeriesFieldBO;
import com.njcn.gather.tool.adddata.component.AddDataTimeSlotCalculator;
import com.njcn.gather.tool.addledger.pojo.constant.AddLedgerConst;
import com.njcn.gather.tool.addledger.pojo.vo.AddLedgerLinePathVO;
import com.njcn.gather.tool.addledger.service.AddLedgerService;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeParseException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* 数据校验服务实现。
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class SteadyChecksquareServiceImpl implements SteadyChecksquareService {
private static final DateTimeFormatter TIME_FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
private static final String EMPTY_TEXT = "-";
private static final int FLICKER_SHORT_INTERVAL_MINUTES = 10;
private static final int FLICKER_LONG_INTERVAL_MINUTES = 120;
private final SteadyTrendIndicatorCatalog indicatorCatalog;
private final SteadyChecksquareInfluxQueryComponent influxQueryComponent;
private final SteadyChecksquareCalculator calculator;
private final AddDataTimeSlotCalculator timeSlotCalculator;
private final AddLedgerService addLedgerService;
@Override
public SteadyChecksquareQueryVO query(SteadyChecksquareQueryParam param) {
validateParam(param);
String lineId = trimToNull(param.getLineId());
LocalDateTime startTime = parseRequiredTime(param.getTimeStart(), "开始时间不能为空");
LocalDateTime endTime = parseRequiredTime(param.getTimeEnd(), "结束时间不能为空");
if (startTime.isAfter(endTime)) {
throw fail("开始时间不能大于结束时间");
}
AddLedgerLinePathVO linePath = requireLinePath(lineId);
int intervalMinutes = resolveIntervalMinutes(linePath);
SteadyChecksquareQueryVO result = new SteadyChecksquareQueryVO();
result.setLineId(lineId);
result.setLineName(trimToNull(linePath.getLineName()) == null ? EMPTY_TEXT : linePath.getLineName());
result.setTimeStart(param.getTimeStart());
result.setTimeEnd(param.getTimeEnd());
result.setIntervalMinutes(intervalMinutes);
long startMillis = System.currentTimeMillis();
List<String> indicatorCodes = normalizeTextList(param.getIndicatorCodes());
List<Integer> harmonicOrders = normalizeHarmonicOrders(param.getHarmonicOrders());
log.info("数据校验查询开始lineId={}indicatorCount={}timeStart={}timeEnd={}intervalMinutes={}",
lineId, indicatorCodes.size(), startTime, endTime, intervalMinutes);
for (String indicatorCode : indicatorCodes) {
SteadyTrendIndicatorDefinitionBO indicator = requireIndicator(indicatorCode);
int itemIntervalMinutes = resolveIndicatorIntervalMinutes(indicator, intervalMinutes);
List<LocalDateTime> itemSlots = timeSlotCalculator.buildTimeSlots(startTime, endTime, itemIntervalMinutes);
result.getItems().addAll(buildIndicatorItems(lineId, indicator, harmonicOrders, startTime, endTime, itemSlots, itemIntervalMinutes));
}
log.info("数据校验查询结束lineId={}itemCount={}costMs={}", lineId, result.getItems().size(), System.currentTimeMillis() - startMillis);
return result;
}
private List<SteadyChecksquareItemVO> buildIndicatorItems(String lineId, SteadyTrendIndicatorDefinitionBO indicator,
List<Integer> harmonicOrders,
LocalDateTime startTime, LocalDateTime endTime,
List<LocalDateTime> slots, int intervalMinutes) {
List<SteadyChecksquareItemVO> result = new ArrayList<SteadyChecksquareItemVO>();
if (Boolean.TRUE.equals(indicator.getHarmonic())) {
for (Integer order : requireValidHarmonicOrders(indicator, harmonicOrders)) {
result.add(buildItem(lineId, indicator, order, startTime, endTime, slots, intervalMinutes));
}
return result;
}
result.add(buildItem(lineId, indicator, null, startTime, endTime, slots, intervalMinutes));
return result;
}
private SteadyChecksquareItemVO buildItem(String lineId, SteadyTrendIndicatorDefinitionBO indicator, Integer harmonicOrder,
LocalDateTime startTime, LocalDateTime endTime,
List<LocalDateTime> slots, int intervalMinutes) {
SteadyChecksquareItemVO item = new SteadyChecksquareItemVO();
item.setItemKey(buildItemKey(lineId, indicator, harmonicOrder));
item.setIndicatorCode(indicator.getIndicatorCode());
item.setIndicatorName(indicator.getName());
item.setHarmonicOrder(harmonicOrder);
item.setIntervalMinutes(intervalMinutes);
int totalExpected = 0;
int totalActual = 0;
int maxContinuousMissingMinutes = 0;
boolean hasData = false;
for (String statType : indicator.getSupportStats()) {
Set<LocalDateTime> actualSlots = queryMergedActualSlots(lineId, indicator, harmonicOrder, statType, startTime, endTime, intervalMinutes);
Set<LocalDateTime> effectiveActualSlots = retainExpectedSlots(slots, actualSlots);
List<SteadyChecksquareSegmentVO> segments = calculator.buildSegments(slots, effectiveActualSlots, intervalMinutes);
SteadyChecksquareStatSummaryVO summary = buildSummary(statType, slots.size(), effectiveActualSlots.size(), segments);
SteadyChecksquareStatDetailVO detail = buildDetail(statType, segments);
item.getStatSummaries().add(summary);
item.getStatDetails().add(detail);
totalExpected += summary.getExpectedPointCount();
totalActual += summary.getActualPointCount();
maxContinuousMissingMinutes = Math.max(maxContinuousMissingMinutes, summary.getMaxContinuousMissingMinutes());
hasData = hasData || Boolean.TRUE.equals(summary.getHasData());
}
item.setHasData(hasData);
item.setExpectedPointCount(totalExpected);
item.setActualPointCount(totalActual);
item.setMissingPointCount(Math.max(0, totalExpected - totalActual));
item.setMissingRate(calculateRate(item.getMissingPointCount(), totalExpected));
item.setMissingRateText(formatRateText(item.getMissingRate()));
item.setMaxContinuousMissingMinutes(maxContinuousMissingMinutes);
return item;
}
private Set<LocalDateTime> queryMergedActualSlots(String lineId, SteadyTrendIndicatorDefinitionBO indicator, Integer harmonicOrder,
String statType, LocalDateTime startTime, LocalDateTime endTime,
int intervalMinutes) {
Set<LocalDateTime> result = new HashSet<LocalDateTime>();
for (String phase : indicator.getPhaseCodes()) {
SteadyTrendResolvedFieldBO field = buildResolvedField(lineId, indicator, harmonicOrder, phase, statType);
result.addAll(influxQueryComponent.queryExistingSlots(field, startTime, endTime, intervalMinutes));
}
return result;
}
private Set<LocalDateTime> retainExpectedSlots(List<LocalDateTime> slots, Set<LocalDateTime> actualSlots) {
Set<LocalDateTime> result = new HashSet<LocalDateTime>();
if (slots == null || actualSlots == null || actualSlots.isEmpty()) {
return result;
}
for (LocalDateTime slot : slots) {
if (actualSlots.contains(slot)) {
result.add(slot);
}
}
return result;
}
private SteadyTrendResolvedFieldBO buildResolvedField(String lineId, SteadyTrendIndicatorDefinitionBO indicator,
Integer harmonicOrder, String phase, String statType) {
SteadyTrendResolvedFieldBO field = new SteadyTrendResolvedFieldBO();
field.setMeasurement(indicator.getTableName());
field.setField(resolveField(indicator, harmonicOrder));
field.setLineId(lineId);
field.setIndicatorCode(indicator.getIndicatorCode());
field.setIndicatorName(indicator.getName());
field.setPhase(phase);
field.setStatType(statType);
field.setUnit(indicator.getUnit());
return field;
}
private String resolveField(SteadyTrendIndicatorDefinitionBO indicator, Integer harmonicOrder) {
if (Boolean.TRUE.equals(indicator.getHarmonic())) {
return indicator.getHarmonicFieldPrefix() + "_" + harmonicOrder;
}
List<SteadyTrendSeriesFieldBO> fields = indicator.getSeriesFields();
if (fields == null || fields.isEmpty()) {
throw fail("稳态指标不支持:" + indicator.getIndicatorCode());
}
return fields.get(0).getField();
}
private SteadyChecksquareStatSummaryVO buildSummary(String statType, int expectedCount, int actualCount,
List<SteadyChecksquareSegmentVO> segments) {
SteadyChecksquareStatSummaryVO summary = new SteadyChecksquareStatSummaryVO();
summary.setStatType(statType);
summary.setSupported(true);
summary.setHasData(actualCount > 0);
summary.setExpectedPointCount(expectedCount);
summary.setActualPointCount(actualCount);
summary.setMissingPointCount(Math.max(0, expectedCount - actualCount));
summary.setMissingRate(calculateRate(summary.getMissingPointCount(), expectedCount));
summary.setMissingRateText(formatRateText(summary.getMissingRate()));
summary.setMaxContinuousMissingMinutes(calculator.maxContinuousMissingMinutes(segments));
return summary;
}
private SteadyChecksquareStatDetailVO buildDetail(String statType, List<SteadyChecksquareSegmentVO> segments) {
SteadyChecksquareStatDetailVO detail = new SteadyChecksquareStatDetailVO();
detail.setStatType(statType);
detail.setSupported(true);
detail.setSegments(segments);
return detail;
}
private String buildItemKey(String lineId, SteadyTrendIndicatorDefinitionBO indicator, Integer harmonicOrder) {
if (harmonicOrder == null) {
return lineId + "|" + indicator.getIndicatorCode();
}
return lineId + "|" + indicator.getIndicatorCode() + "|" + harmonicOrder;
}
private void validateParam(SteadyChecksquareQueryParam param) {
if (param == null) {
throw fail("数据校验参数不能为空");
}
if (trimToNull(param.getLineId()) == null) {
throw fail("监测点 ID 不能为空");
}
if (normalizeTextList(param.getIndicatorCodes()).isEmpty()) {
throw fail("指标不能为空");
}
parseRequiredTime(param.getTimeStart(), "开始时间不能为空");
parseRequiredTime(param.getTimeEnd(), "结束时间不能为空");
}
private LocalDateTime parseRequiredTime(String time, String emptyMessage) {
String text = trimToNull(time);
if (text == null) {
throw fail(emptyMessage);
}
try {
return LocalDateTime.parse(text, TIME_FORMATTER);
} catch (DateTimeParseException ex) {
throw fail("时间格式不正确,仅支持 yyyy-MM-dd HH:mm:ss");
}
}
private AddLedgerLinePathVO requireLinePath(String lineId) {
Map<String, AddLedgerLinePathVO> linePathMap = addLedgerService.listLinePathByLineIds(Collections.singletonList(lineId));
AddLedgerLinePathVO linePath = linePathMap.get(lineId);
if (linePath == null) {
throw fail("监测点不存在或不可用");
}
return linePath;
}
private int resolveIntervalMinutes(AddLedgerLinePathVO linePath) {
Integer interval = linePath.getLineInterval();
if (interval == null || interval <= 0) {
return AddLedgerConst.LINE_INTERVAL_DEFAULT;
}
return interval;
}
private int resolveIndicatorIntervalMinutes(SteadyTrendIndicatorDefinitionBO indicator, int lineIntervalMinutes) {
String indicatorCode = indicator == null ? null : indicator.getIndicatorCode();
if ("FLUC".equals(indicatorCode) || "PST".equals(indicatorCode)) {
return FLICKER_SHORT_INTERVAL_MINUTES;
}
if ("PLT".equals(indicatorCode)) {
return FLICKER_LONG_INTERVAL_MINUTES;
}
return lineIntervalMinutes;
}
private SteadyTrendIndicatorDefinitionBO requireIndicator(String indicatorCode) {
SteadyTrendIndicatorDefinitionBO indicator = indicatorCatalog.getIndicator(indicatorCode);
if (indicator == null) {
throw fail("稳态指标不支持:" + indicatorCode);
}
return indicator;
}
private BigDecimal calculateRate(int missingCount, int expectedCount) {
if (expectedCount <= 0) {
return BigDecimal.ZERO.setScale(6, RoundingMode.HALF_UP);
}
return new BigDecimal(missingCount).divide(new BigDecimal(expectedCount), 6, RoundingMode.HALF_UP);
}
private String formatRateText(BigDecimal rate) {
if (rate == null) {
return null;
}
return rate.multiply(new BigDecimal("100")).setScale(2, RoundingMode.HALF_UP).toPlainString() + "%";
}
private List<String> normalizeTextList(List<String> values) {
if (values == null || values.isEmpty()) {
return new ArrayList<String>();
}
Set<String> result = new LinkedHashSet<String>();
for (String value : values) {
String text = trimToNull(value);
if (text != null) {
result.add(text);
}
}
return new ArrayList<String>(result);
}
private List<Integer> normalizeHarmonicOrders(List<Integer> values) {
if (values == null || values.isEmpty()) {
return new ArrayList<Integer>();
}
List<Integer> result = new ArrayList<Integer>();
for (Integer value : values) {
if (value != null && !result.contains(value)) {
result.add(value);
}
}
return result;
}
private List<Integer> requireValidHarmonicOrders(SteadyTrendIndicatorDefinitionBO indicator, List<Integer> harmonicOrders) {
if (harmonicOrders == null || harmonicOrders.isEmpty()) {
throw fail("谐波次数不能为空");
}
for (Integer order : harmonicOrders) {
if (order < indicator.getHarmonicOrderStart() || order > indicator.getHarmonicOrderEnd()) {
throw fail("谐波次数只能在 " + indicator.getHarmonicOrderStart() + "" + indicator.getHarmonicOrderEnd() + " 之间");
}
}
return harmonicOrders;
}
private String trimToNull(String value) {
if (value == null) {
return null;
}
String trimmed = value.trim();
return trimmed.isEmpty() ? null : trimmed;
}
private BusinessException fail(String message) {
return new BusinessException(CommonResponseEnum.FAIL, message);
}
}

View File

@@ -8,6 +8,7 @@ import com.njcn.gather.steady.datavie.config.SteadyInfluxDbProperties;
import com.njcn.gather.steady.datavie.pojo.bo.SteadyTrendResolvedFieldBO;
import com.njcn.gather.steady.datavie.pojo.vo.SteadyTrendPointVO;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import java.io.BufferedReader;
@@ -29,6 +30,7 @@ import java.util.List;
/**
* 稳态趋势 InfluxDB 查询组件。
*/
@Slf4j
@Component
@RequiredArgsConstructor
public class SteadyInfluxQueryComponent {
@@ -43,8 +45,32 @@ public class SteadyInfluxQueryComponent {
LocalDateTime endTime, Integer qualityFlag) {
validateConfig();
String query = buildTrendQuery(field, startTime, endTime, qualityFlag);
String body = executeQuery(query);
return parseTrendPoints(body);
String diagnostic = buildTrendQueryDiagnostic(field, startTime, endTime, qualityFlag);
long startMillis = System.currentTimeMillis();
log.info("稳态趋势 InfluxDB 查询开始,{}query={}", diagnostic, query);
try {
String body = executeQuery(query);
List<SteadyTrendPointVO> points = parseTrendPoints(body);
log.info("稳态趋势 InfluxDB 查询结束,{}pointCount={}costMs={}", diagnostic, points.size(), System.currentTimeMillis() - startMillis);
return points;
} catch (RuntimeException ex) {
log.warn("稳态趋势 InfluxDB 查询异常,{}costMs={}error={}", diagnostic, System.currentTimeMillis() - startMillis, ex.getMessage());
throw ex;
}
}
String buildTrendQueryDiagnostic(SteadyTrendResolvedFieldBO field, LocalDateTime startTime, LocalDateTime endTime,
Integer qualityFlag) {
StringBuilder diagnostic = new StringBuilder();
diagnostic.append("measurement=").append(field.getMeasurement());
diagnostic.append(", field=").append(field.getField());
diagnostic.append(", lineId=").append(field.getLineId());
diagnostic.append(", phase=").append(field.getPhase());
diagnostic.append(", statType=").append(resolveValueType(field.getStatType()));
diagnostic.append(", qualityFlag=").append(qualityFlag);
diagnostic.append(", timeStart=").append(startTime);
diagnostic.append(", timeEnd=").append(endTime);
return diagnostic.toString();
}
public String buildTrendQuery(SteadyTrendResolvedFieldBO field, LocalDateTime startTime, LocalDateTime endTime,

View File

@@ -26,7 +26,7 @@ public class SteadyTrendFieldResolver {
private static final int MAX_LINE_COUNT = 8;
private static final int MAX_INDICATOR_COUNT = 8;
private static final int MAX_SERIES_COUNT = 24;
private static final int MAX_HARMONIC_ORDER_COUNT = 6;
private static final int MAX_HARMONIC_ORDER_COUNT = 3;
private static final DateTimeFormatter TIME_FORMATTER = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
private final SteadyTrendIndicatorCatalog indicatorCatalog;
@@ -89,7 +89,7 @@ public class SteadyTrendFieldResolver {
throw fail("谐波次数不能为空");
}
if (orders.size() > MAX_HARMONIC_ORDER_COUNT) {
throw fail("谐波次数最多选择 6 ");
throw fail("谐波次数不允许一次展示超过3");
}
List<SteadyTrendResolvedFieldBO> result = new ArrayList<SteadyTrendResolvedFieldBO>();
for (Integer order : orders) {

View File

@@ -32,6 +32,6 @@ public class SteadyTrendQueryParam {
@ApiModelProperty("质量标识")
private Integer qualityFlag;
@ApiModelProperty("谐波次数,谐波指标必填,最多 6")
@ApiModelProperty("谐波次数,谐波指标必填,默认最多展示 3")
private List<Integer> harmonicOrders = new ArrayList<Integer>();
}

View File

@@ -13,6 +13,7 @@ import com.njcn.gather.steady.datavie.service.SteadyDataViewTrendService;
import com.njcn.gather.tool.addledger.pojo.vo.AddLedgerLinePathVO;
import com.njcn.gather.tool.addledger.service.AddLedgerService;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import java.math.BigDecimal;
@@ -28,6 +29,7 @@ import java.util.Map;
/**
* 稳态趋势查询服务实现。
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class SteadyDataViewTrendServiceImpl implements SteadyDataViewTrendService {
@@ -70,11 +72,14 @@ public class SteadyDataViewTrendServiceImpl implements SteadyDataViewTrendServic
result.setSampled(false);
result.setLoadableDays(resolveLoadableDays(startTime, endTime));
int displayPointCount = 0;
long startMillis = System.currentTimeMillis();
log.info("稳态趋势查询开始seriesCount={}timeStart={}timeEnd={}qualityFlag={}", fields.size(), startTime, endTime, param.getQualityFlag());
for (SteadyTrendResolvedFieldBO field : fields) {
List<SteadyTrendPointVO> points = influxQueryComponent.queryTrendPoints(field, startTime, endTime, param.getQualityFlag());
displayPointCount += points.size();
result.getSeries().add(buildSeries(field, points));
}
log.info("稳态趋势查询结束seriesCount={}displayPointCount={}costMs={}", fields.size(), displayPointCount, System.currentTimeMillis() - startMillis);
/*
* 当前 Influx 查询按曲线独立执行,未额外发 count 查询sourcePointCount 保持与实际返回点数一致。
* 后续如需要精确原始点数,可单独增加 count(field) 查询。

View File

@@ -0,0 +1,38 @@
package com.njcn.gather.steady.checksquare.component;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareSegmentVO;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import java.time.LocalDateTime;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
/**
* 数据校验缺失区间计算测试。
*/
class SteadyChecksquareCalculatorTest {
@Test
void shouldMergeContinuousMissingSlots() {
SteadyChecksquareCalculator calculator = new SteadyChecksquareCalculator();
List<LocalDateTime> slots = Arrays.asList(
LocalDateTime.of(2026, 5, 1, 0, 0),
LocalDateTime.of(2026, 5, 1, 0, 1),
LocalDateTime.of(2026, 5, 1, 0, 2),
LocalDateTime.of(2026, 5, 1, 0, 3),
LocalDateTime.of(2026, 5, 1, 0, 4)
);
List<SteadyChecksquareSegmentVO> segments = calculator.buildSegments(slots,
new HashSet<LocalDateTime>(Arrays.asList(slots.get(0), slots.get(3))), 1);
Assertions.assertEquals(4, segments.size());
Assertions.assertEquals("MISSING", segments.get(1).getStatus());
Assertions.assertEquals("2026-05-01 00:01:00", segments.get(1).getStartTime());
Assertions.assertEquals("2026-05-01 00:02:00", segments.get(1).getEndTime());
Assertions.assertEquals(Integer.valueOf(2), segments.get(1).getMissingPointCount());
Assertions.assertEquals(Integer.valueOf(2), segments.get(1).getDurationMinutes());
}
}

View File

@@ -0,0 +1,36 @@
package com.njcn.gather.steady.checksquare.component;
import com.njcn.gather.steady.datavie.config.SteadyInfluxDbProperties;
import com.njcn.gather.steady.datavie.pojo.bo.SteadyTrendResolvedFieldBO;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import java.time.LocalDateTime;
/**
* 数据校验 InfluxQL 构造契约测试。
*/
class SteadyChecksquareInfluxQueryComponentTest {
@Test
void shouldBuildChecksquareQueryWithoutQualityFlag() {
SteadyChecksquareInfluxQueryComponent component = new SteadyChecksquareInfluxQueryComponent(new SteadyInfluxDbProperties());
SteadyTrendResolvedFieldBO field = new SteadyTrendResolvedFieldBO();
field.setMeasurement("data_v");
field.setField("rms");
field.setLineId("line-001");
field.setPhase("A");
field.setStatType("AVG");
String query = component.buildChecksquareQuery(field,
LocalDateTime.of(2026, 5, 1, 0, 0, 0),
LocalDateTime.of(2026, 5, 1, 23, 59, 59));
Assertions.assertTrue(query.contains("SELECT \"rms\" AS \"value\""));
Assertions.assertTrue(query.contains("\"line_id\" = 'line-001'"));
Assertions.assertTrue(query.contains("\"phasic_type\" = 'A'"));
Assertions.assertTrue(query.contains("\"value_type\" = 'AVG'"));
Assertions.assertFalse(query.contains("quality_flag"));
Assertions.assertFalse(query.contains("GROUP BY time"));
}
}

View File

@@ -0,0 +1,24 @@
package com.njcn.gather.steady.checksquare.controller;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import java.lang.reflect.Method;
/**
* 数据校验接口契约测试。
*/
class SteadyChecksquareControllerTest {
@Test
void shouldExposeChecksquareQueryEndpointInSeparateController() throws Exception {
RequestMapping requestMapping = SteadyChecksquareController.class.getAnnotation(RequestMapping.class);
Assertions.assertArrayEquals(new String[]{"/steady/data-view/checksquare"}, requestMapping.value());
Method method = SteadyChecksquareController.class.getDeclaredMethod("query", com.njcn.gather.steady.checksquare.pojo.param.SteadyChecksquareQueryParam.class);
PostMapping postMapping = method.getAnnotation(PostMapping.class);
Assertions.assertArrayEquals(new String[]{"/query"}, postMapping.value());
}
}

View File

@@ -0,0 +1,32 @@
package com.njcn.gather.steady.checksquare.pojo.param;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import java.lang.reflect.Field;
/**
* 数据校验查询参数契约测试。
*/
class SteadyChecksquareQueryParamTest {
@Test
void shouldOnlyExposeChecksquareQueryFields() {
Assertions.assertNotNull(field("lineId"));
Assertions.assertNotNull(field("indicatorCodes"));
Assertions.assertNotNull(field("timeStart"));
Assertions.assertNotNull(field("timeEnd"));
Assertions.assertNull(field("qualityFlag"));
Assertions.assertNull(field("statTypes"));
Assertions.assertNull(field("phases"));
Assertions.assertNotNull(field("harmonicOrders"));
}
private Field field(String name) {
try {
return SteadyChecksquareQueryParam.class.getDeclaredField(name);
} catch (NoSuchFieldException ex) {
return null;
}
}
}

View File

@@ -0,0 +1,130 @@
package com.njcn.gather.steady.checksquare.service.impl;
import com.njcn.gather.steady.checksquare.component.SteadyChecksquareCalculator;
import com.njcn.gather.steady.checksquare.component.SteadyChecksquareInfluxQueryComponent;
import com.njcn.gather.steady.checksquare.pojo.param.SteadyChecksquareQueryParam;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareItemVO;
import com.njcn.gather.steady.checksquare.pojo.vo.SteadyChecksquareQueryVO;
import com.njcn.gather.steady.datavie.component.SteadyTrendIndicatorCatalog;
import com.njcn.gather.tool.adddata.component.AddDataTimeSlotCalculator;
import com.njcn.gather.tool.addledger.pojo.vo.AddLedgerLinePathVO;
import com.njcn.gather.tool.addledger.service.AddLedgerService;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import java.time.LocalDateTime;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
/**
* 数据校验服务测试。
*/
class SteadyChecksquareServiceImplTest {
@Test
void shouldUseFixedFlickerIntervalsPerIndicator() {
SteadyChecksquareInfluxQueryComponent influxQueryComponent = mock(SteadyChecksquareInfluxQueryComponent.class);
AddLedgerService addLedgerService = mock(AddLedgerService.class);
SteadyChecksquareServiceImpl service = new SteadyChecksquareServiceImpl(new SteadyTrendIndicatorCatalog(),
influxQueryComponent, new SteadyChecksquareCalculator(), new AddDataTimeSlotCalculator(), addLedgerService);
AddLedgerLinePathVO linePath = new AddLedgerLinePathVO();
linePath.setLineId("line-001");
linePath.setLineName("进线一");
linePath.setLineInterval(1);
when(addLedgerService.listLinePathByLineIds(eq(Collections.singletonList("line-001"))))
.thenReturn(Collections.singletonMap("line-001", linePath));
when(influxQueryComponent.queryExistingSlots(any(), any(LocalDateTime.class), any(LocalDateTime.class), eq(10)))
.thenReturn(new HashSet<LocalDateTime>(Arrays.asList(
LocalDateTime.of(2026, 5, 1, 0, 0),
LocalDateTime.of(2026, 5, 1, 0, 10))));
when(influxQueryComponent.queryExistingSlots(any(), any(LocalDateTime.class), any(LocalDateTime.class), eq(120)))
.thenReturn(new HashSet<LocalDateTime>(Collections.singletonList(
LocalDateTime.of(2026, 5, 1, 0, 0))));
SteadyChecksquareQueryParam param = new SteadyChecksquareQueryParam();
param.setLineId("line-001");
param.setIndicatorCodes(Arrays.asList("FLUC", "PST", "PLT"));
param.setTimeStart("2026-05-01 00:00:00");
param.setTimeEnd("2026-05-01 02:00:00");
SteadyChecksquareQueryVO result = service.query(param);
Assertions.assertEquals(Integer.valueOf(1), result.getIntervalMinutes());
Assertions.assertEquals(3, result.getItems().size());
assertItemInterval(result.getItems().get(0), "FLUC", 10, 13);
assertItemInterval(result.getItems().get(1), "PST", 10, 13);
assertItemInterval(result.getItems().get(2), "PLT", 120, 2);
}
@Test
void shouldOnlyQueryRequestedHarmonicOrders() {
SteadyChecksquareInfluxQueryComponent influxQueryComponent = mock(SteadyChecksquareInfluxQueryComponent.class);
AddLedgerService addLedgerService = mock(AddLedgerService.class);
SteadyChecksquareServiceImpl service = new SteadyChecksquareServiceImpl(new SteadyTrendIndicatorCatalog(),
influxQueryComponent, new SteadyChecksquareCalculator(), new AddDataTimeSlotCalculator(), addLedgerService);
AddLedgerLinePathVO linePath = new AddLedgerLinePathVO();
linePath.setLineId("line-001");
linePath.setLineName("进线一");
linePath.setLineInterval(1);
when(addLedgerService.listLinePathByLineIds(eq(Collections.singletonList("line-001"))))
.thenReturn(Collections.singletonMap("line-001", linePath));
when(influxQueryComponent.queryExistingSlots(any(), any(LocalDateTime.class), any(LocalDateTime.class), eq(1)))
.thenReturn(new HashSet<LocalDateTime>(Collections.singletonList(
LocalDateTime.of(2026, 5, 1, 0, 0))));
SteadyChecksquareQueryParam param = new SteadyChecksquareQueryParam();
param.setLineId("line-001");
param.setIndicatorCodes(Collections.singletonList("V_HARMONIC"));
param.setHarmonicOrders(Collections.singletonList(5));
param.setTimeStart("2026-05-01 00:00:00");
param.setTimeEnd("2026-05-01 00:01:00");
SteadyChecksquareQueryVO result = service.query(param);
Assertions.assertEquals(1, result.getItems().size());
Assertions.assertEquals(Integer.valueOf(5), result.getItems().get(0).getHarmonicOrder());
}
@Test
void shouldKeepRequestedHarmonicOrdersDistinctAndOrdered() {
SteadyChecksquareInfluxQueryComponent influxQueryComponent = mock(SteadyChecksquareInfluxQueryComponent.class);
AddLedgerService addLedgerService = mock(AddLedgerService.class);
SteadyChecksquareServiceImpl service = new SteadyChecksquareServiceImpl(new SteadyTrendIndicatorCatalog(),
influxQueryComponent, new SteadyChecksquareCalculator(), new AddDataTimeSlotCalculator(), addLedgerService);
AddLedgerLinePathVO linePath = new AddLedgerLinePathVO();
linePath.setLineId("line-001");
linePath.setLineName("进线一");
linePath.setLineInterval(1);
when(addLedgerService.listLinePathByLineIds(eq(Collections.singletonList("line-001"))))
.thenReturn(Collections.singletonMap("line-001", linePath));
when(influxQueryComponent.queryExistingSlots(any(), any(LocalDateTime.class), any(LocalDateTime.class), eq(1)))
.thenReturn(new HashSet<LocalDateTime>());
SteadyChecksquareQueryParam param = new SteadyChecksquareQueryParam();
param.setLineId("line-001");
param.setIndicatorCodes(Collections.singletonList("V_HARMONIC"));
param.setHarmonicOrders(Arrays.asList(7, 5, 7));
param.setTimeStart("2026-05-01 00:00:00");
param.setTimeEnd("2026-05-01 00:01:00");
SteadyChecksquareQueryVO result = service.query(param);
List<SteadyChecksquareItemVO> items = result.getItems();
Assertions.assertEquals(2, items.size());
Assertions.assertEquals(Integer.valueOf(7), items.get(0).getHarmonicOrder());
Assertions.assertEquals(Integer.valueOf(5), items.get(1).getHarmonicOrder());
}
private void assertItemInterval(SteadyChecksquareItemVO item, String indicatorCode, int intervalMinutes, int expectedPointCount) {
Assertions.assertEquals(indicatorCode, item.getIndicatorCode());
Assertions.assertEquals(Integer.valueOf(intervalMinutes), item.getIntervalMinutes());
Assertions.assertEquals(Integer.valueOf(expectedPointCount), item.getExpectedPointCount());
}
}

View File

@@ -73,6 +73,31 @@ class SteadyInfluxQueryComponentTest {
Assertions.assertTrue(query.contains("\"value_type\" = 'AVG'"));
}
@Test
void shouldBuildDiagnosticTextForTrendQuery() {
SteadyInfluxQueryComponent component = new SteadyInfluxQueryComponent(new SteadyInfluxDbProperties());
SteadyTrendResolvedFieldBO field = new SteadyTrendResolvedFieldBO();
field.setMeasurement("data_harmpower_p");
field.setField("p_3");
field.setLineId("f828bc42132841c2aeebc6859f5a9b7c");
field.setPhase("A");
field.setStatType("AVG");
String diagnostic = component.buildTrendQueryDiagnostic(field,
LocalDateTime.of(2026, 5, 1, 0, 0, 0),
LocalDateTime.of(2026, 5, 31, 23, 59, 59),
0);
Assertions.assertTrue(diagnostic.contains("measurement=data_harmpower_p"));
Assertions.assertTrue(diagnostic.contains("field=p_3"));
Assertions.assertTrue(diagnostic.contains("lineId=f828bc42132841c2aeebc6859f5a9b7c"));
Assertions.assertTrue(diagnostic.contains("phase=A"));
Assertions.assertTrue(diagnostic.contains("statType=AVG"));
Assertions.assertTrue(diagnostic.contains("qualityFlag=0"));
Assertions.assertTrue(diagnostic.contains("timeStart=2026-05-01T00:00"));
Assertions.assertTrue(diagnostic.contains("timeEnd=2026-05-31T23:59:59"));
}
@Test
void shouldSkipValueTypeWhenMeasurementHasNoValueTypeTag() {
SteadyInfluxQueryComponent component = new SteadyInfluxQueryComponent(new SteadyInfluxDbProperties());

View File

@@ -109,6 +109,21 @@ class SteadyTrendFieldResolverTest {
Assertions.assertTrue(exception.getMessage().contains("谐波次数不能为空"));
}
@Test
void shouldRejectHarmonicTrendWithMoreThanThreeOrders() {
SteadyTrendQueryParam param = new SteadyTrendQueryParam();
param.setLineIds(Arrays.asList("line-001"));
param.setIndicatorCodes(Arrays.asList("V_HARMONIC"));
param.setStatTypes(Arrays.asList("AVG"));
param.setHarmonicOrders(Arrays.asList(3, 5, 7, 11));
param.setTimeStart("2026-05-01 00:00:00");
param.setTimeEnd("2026-05-01 01:00:00");
BusinessException exception = Assertions.assertThrows(BusinessException.class, () -> resolver.resolveFields(param));
Assertions.assertTrue(exception.getMessage().contains("谐波次数不允许一次展示超过3个"));
}
@Test
void shouldResolveSelectedHarmonicOrdersForAllCatalogPhases() {
SteadyTrendQueryParam param = new SteadyTrendQueryParam();

View File

@@ -34,7 +34,8 @@
equipment.name AS equipmentName,
equipment.mac AS equipmentMac,
line.line_id AS lineId,
line.name AS lineName
line.name AS lineName,
line.line_interval AS lineInterval
FROM cs_line line
INNER JOIN cs_equipment_delivery equipment ON equipment.id = line.device_id
INNER JOIN cs_project project ON project.id = equipment.associated_project
@@ -58,7 +59,8 @@
equipment.name AS equipmentName,
equipment.mac AS equipmentMac,
line.line_id AS lineId,
line.name AS lineName
line.name AS lineName,
line.line_interval AS lineInterval
FROM cs_line line
INNER JOIN cs_equipment_delivery equipment ON equipment.id = line.device_id
INNER JOIN cs_project project ON project.id = equipment.associated_project

View File

@@ -29,4 +29,7 @@ public class AddLedgerLinePathVO implements Serializable {
private String lineId;
private String lineName;
/** 监测点统计间隔,单位分钟。 */
private Integer lineInterval;
}