400-139-9200
Kibana OSS 7.10.2 连接 INFINI Easysearch
2024-12-30
大家都知道 ELK 生态里的各软件之间是有版本验证的。那如果想用 Kibana 连接 INFINI Easysearch，要怎么办呢？
今天我们来实现一下。
首先直接请求 Easysearch API，确认返回版本信息。
# curl -ku admin:a9ce40464e259c7d1e58 https://10.0.0.1:9203
{
"name" : "node-1",
"cluster_name" : "infini-logs-tmp",
"cluster_uuid" : "R_u1UaVQQo-wS1MFGOho3A",
"version" : {
"distribution" : "easysearch",
"number" : "1.9.1",
"distributor" : "INFINI Labs",
"build_hash" : "da95c34b95dd934779d7ae5c6c1bccd2f1c25954",
"build_date" : "2024-12-20T21:01:36.991581Z",
"build_snapshot" : false,
"lucene_version" : "8.11.2",
"minimum_wire_lucene_version" : "7.7.0",
"minimum_lucene_index_compatibility_version" : "7.7.0"
},
"tagline" : "You Know, For Easy Search!"
}
INFINI Easysearch 有提供版本兼容参数。
elasticsearch.api_compatibility: true
elasticsearch.api_compatibility_version: "7.10.2"
开启后再进行 API 访问，返回的版本信息已修改。
# curl -ku admin:a9ce40464e259c7d1e58 https://10.0.0.1:9203
{
"name" : "node-1",
"cluster_name" : "infini-logs-tmp",
"cluster_uuid" : "R_u1UaVQQo-wS1MFGOho3A",
"version" : {
"number" : "7.10.2",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "da95c34b95dd934779d7ae5c6c1bccd2f1c25954",
"build_date" : "2024-12-20T21:01:36.991581Z",
"build_snapshot" : false,
"lucene_version" : "8.7.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0"
},
"tagline" : "You Know, for Search"
}
配置 Kibana
elasticsearch.hosts: ["https://10.0.0.1:9203"]
elasticsearch.username: "admin"
elasticsearch.password: "a9ce40464e259c7d1e58"
使用 Kibana 进行连接时，还会出现版本错误信息。
# ./bin/kibana --allow-root
log [01:28:35.150] [info][plugins-service] Plugin "visTypeXy" is disabled.
log [01:28:35.315] [info][plugins-system] Setting up [40] plugins: [usageCollection,telemetryCollectionManager,telemetry,kibanaUsageCollection,newsfeed,securityOss,mapsLegacy,kibanaLegacy,share,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeVega,visTypeTimelion,timelion,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,esUiShared,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,bfetch]
log [01:28:35.553] [info][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
log [01:28:35.934] [error][savedobjects-service] This version of Kibana (v7.10.2) is incompatible with the following Elasticsearch nodes in your cluster: v1.9.1 @ 172.80.0.1:9203 (172.80.0.1)
分析错误信息，Kibana 应该请求了“/_nodes” API 接口，执行请求，看到确实有相应的版本信息。
# curl -ku admin:a9ce40464e259c7d1e58 https://10.0.0.1:9203/_nodes\?pretty
{
"_nodes" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"cluster_name" : "infini-logs-tmp",
"nodes" : {
"ZZtIpBZpQxWsy3yRC0FKcg" : {
"name" : "node-1",
"transport_address" : "172.80.0.1:9303",
"host" : "172.80.0.1",
"ip" : "172.80.0.1",
"version" : "1.9.1",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "da95c34b95dd934779d7ae5c6c1bccd2f1c25954",
"total_indexing_buffer" : 107374182,
"roles" : [
"data",
"ingest",
"master",
"remote_cluster_client"
],
.......
现在想办法把返回的版本信息进行调整，看可行不？
这里可以通过 INFINI Gateway 来进行返回数据的调整，配置（gateway.yml）如下：
entry:
- name: proxy_es
enabled: true
router: my_router
max_concurrency: 10000
network:
binding: 0.0.0.0:9002
router:
- name: my_router
default_flow: default_flow
rules:
- method:
- "GET"
pattern:
- "/_nodes"
flow:
- overwrite_flow
flow:
- name: default_flow
filter:
- http: ##another general option to proxy requests
schema: "https" #https or http
max_idle_conn_duration: "900s"
skip_failure_host: false
hosts:
- "10.0.0.1:9203"
- name: overwrite_flow
filter:
- http: ##another general option to proxy requests
schema: "https" #https or http
max_idle_conn_duration: "900s"
skip_failure_host: false
hosts:
- "10.0.0.1:9203"
- set_context:
context:
_ctx.response.body_json.nodes.ZZtIpBZpQxWsy3yRC0FKcg.version: '"7.10.2"'
启动 Gateway。
# ./gateway-linux-amd64
___ _ _____ __ __ __ _
/ _ \ /_\ /__ \/__\/ / /\ \ \/_\ /\_/\
/ /_\///_\\ / /\/_\ \ \/ \/ //_\\\_ _/
/ /_\\/ _ \/ / //__ \ /\ / _ \/ \
\____/\_/ \_/\/ \__/ \/ \/\_/ \_/\_/
[GATEWAY] A light-weight, powerful and high-performance search gateway.
[GATEWAY] 1.0.0_SNAPSHOT#001, 2024-09-02 08:53:26, 2024-12-31 10:10:10, 250cefeb7c16438f1bf4e6416b7f3eeeec7d9aef
[12-29 09:30:37] [INF] [app.go:285] initializing gateway, pid: 45373
[12-29 09:30:37] [INF] [app.go:286] using config: /opt/demo-environment/gateway_es_tmp/gateway.yml
[12-29 09:30:37] [INF] [instance.go:78] workspace: /opt/demo-environment/gateway_es_tmp/data/gateway/nodes/ctnsiut3q95hgcqen710
[12-29 09:30:37] [INF] [module.go:136] started module: badger
[12-29 09:30:37] [INF] [api.go:196] local ips: 192.168.3.181, 10.0.0.1, 172.80.2.1, 172.80.3.1, 169.254.25.10, 10.233.39.174, 10.233.9.199, 10.233.0.1, 10.233.0.3, 10.233.62.114, 10.233.23.74, 10.233.27.40, 10.233.23.200, 10.233.3.49, 10.233.23.21, 10.233.19.207, 10.233.41.58, 10.233.28.97, 10.233.17.129, 10.233.14.179, 10.233.24.184, 10.233.14.180, 10.233.112.0, 172.80.0.1, 172.34.0.1, 192.168.122.1
[12-29 09:30:37] [INF] [api.go:360] api listen at: http://0.0.0.0:2904
[12-29 09:30:37] [INF] [module.go:136] started module: api
[12-29 09:30:37] [INF] [module.go:136] started module: disk_queue
[12-29 09:30:37] [INF] [module.go:136] started module: elasticsearch
[12-29 09:30:37] [INF] [module.go:136] started module: kafka_queue
[12-29 09:30:37] [INF] [module.go:136] started module: queue
[12-29 09:30:37] [INF] [module.go:136] started module: redis
[12-29 09:30:37] [INF] [module.go:136] started module: s3
[12-29 09:30:37] [INF] [module.go:136] started module: simple_stats
[12-29 09:30:37] [INF] [module.go:136] started module: task
[12-29 09:30:37] [INF] [module.go:136] started module: pipeline
[12-29 09:30:37] [INF] [module.go:155] started plugin: floating_ip
[12-29 09:30:37] [INF] [module.go:155] started plugin: force_merge
[12-29 09:30:37] [INF] [module.go:155] started plugin: metrics
[12-29 09:30:37] [INF] [module.go:155] started plugin: statsd
[12-29 09:30:37] [INF] [entry.go:398] entry [proxy_es] listen at: http://0.0.0.0:9002
[12-29 09:30:37] [INF] [module.go:155] started plugin: gateway
[12-29 09:30:37] [INF] [module.go:161] all modules are started
[12-29 09:30:37] [INF] [app.go:511] gateway is up and running now.
通过 Gateway 访问 API 确认版本信息。
# curl -ku admin:a9ce40464e259c7d1e58 http://10.0.0.1:9002
{
"name" : "node-1",
"cluster_name" : "infini-logs-tmp",
"cluster_uuid" : "R_u1UaVQQo-wS1MFGOho3A",
"version" : {
"number" : "7.10.2",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "da95c34b95dd934779d7ae5c6c1bccd2f1c25954",
"build_date" : "2024-12-20T21:01:36.991581Z",
"build_snapshot" : false,
"lucene_version" : "8.7.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0"
},
"tagline" : "You Know, for Search"
}
# curl -ku admin:a9ce40464e259c7d1e58 http://10.0.0.1:9002/_nodes\?pretty
{
"_nodes" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"cluster_name" : "infini-logs-tmp",
"nodes" : {
"ZZtIpBZpQxWsy3yRC0FKcg" : {
"name" : "node-1",
"transport_address" : "172.80.0.1:9303",
"host" : "172.80.0.1",
"ip" : "172.80.0.1",
"version" : "7.10.2",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "da95c34b95dd934779d7ae5c6c1bccd2f1c25954",
"total_indexing_buffer" : 107374182,
"roles" : [
"data",
"ingest",
"master",
"remote_cluster_client"
],
......
修改 Kibana 配置文件（config/kibana.yml），通过 Gateway 访问集群。
elasticsearch.hosts: ["http://10.0.0.1:9002"]
启动 Kibana 成功，无报错。
# ./bin/kibana --allow-root
log [01:31:42.733] [info][plugins-service] Plugin "visTypeXy" is disabled.
log [01:31:42.865] [info][plugins-system] Setting up [40] plugins: [usageCollection,telemetryCollectionManager,telemetry,kibanaUsageCollection,securityOss,newsfeed,mapsLegacy,kibanaLegacy,share,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeVega,visTypeTimelion,timelion,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,esUiShared,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,bfetch]
log [01:31:43.098] [info][savedobjects-service] Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations...
log [01:31:43.151] [info][savedobjects-service] Starting saved objects migrations
log [01:31:43.228] [info][plugins-system] Starting [40] plugins: [usageCollection,telemetryCollectionManager,telemetry,kibanaUsageCollection,securityOss,newsfeed,mapsLegacy,kibanaLegacy,share,legacyExport,embeddable,expressions,data,home,console,apmOss,management,indexPatternManagement,advancedSettings,savedObjects,dashboard,visualizations,visTypeVega,visTypeTimelion,timelion,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,esUiShared,charts,visTypeVislib,visTypeTimeseries,visTypeTagcloud,visTypeMetric,discover,savedObjectsManagement,bfetch]
log [01:31:43.477] [info][listening] Server running at http://0.0.0.0:5602
log [01:31:43.516] [info][server][Kibana][http] http server running at http://0.0.0.0:5602
刘奎君
追求极致，无限可能。
Easysearch
x
Gateway
x
Console
x
