--- title: "Elasticsearch VS Easysearch 性能测试" date: 2024-12-19 lastmod: 2024-12-19 description: "对比测试显示，Easysearch在索引性能和磁盘压缩效率上显著优于Elasticsearch：吞吐量提升40%-70%，磁盘压缩效率提高2.5-3倍，尤其适合大规模分片与海量数据场景。" tags: ["Easysearch", "Elasticsearch", "ES"] summary: "压测环境 # 虚拟机配置 # 使用阿里云上规格： ecs.u1-c1m4.4xlarge，PL2: 单盘 IOPS 性能上限 10 万 (适用的云盘容量范围：461GiB - 64TiB) vCPU 内存 (GiB) 磁盘(GB) 带宽（Gbit/s） 数量 16 64 500 5000 24 Easysearch 配置 # 7 节点集群，版本：1.9.0 实例名 内网 IP 软件 vCPU JVM 磁盘 i-2zegn56cijnzklcn2410 172.22.75.144 Easysearch 16 31G 500GB i-2zegn56cijnzklcn240u 172.23.15.97 Easysearch 16 31G 500GB i-2zegn56cijnzklcn240i 172." --- ## 压测环境 ### 虚拟机配置 使用阿里云上规格：[ecs.u1-c1m4.4xlarge](https://help.aliyun.com/document_detail/25378.html#u1)，PL2: 单盘 IOPS 性能上限 10 万 (适用的云盘容量范围：461GiB - 64TiB) | vCPU | 内存 (GiB) | 磁盘(GB) | 带宽（Gbit/s） | 数量 | | --- | --- | --- | --- | --- | | 16 | 64 | 500 | 5000 | 24 | ### Easysearch 配置 7 节点集群，版本：1.9.0 | 实例名 | 内网 IP | 软件 | vCPU | JVM | 磁盘 | | --- | --- | --- | --- | --- | --- | | i-2zegn56cijnzklcn2410 | 172.22.75.144 | Easysearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240u | 172.23.15.97 | Easysearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240i | 172.25.230.228 | Easysearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240y | 172.22.75.142 | Easysearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240x | 172.22.75.143 | Easysearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240z | 172.24.250.252 | Easysearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240r | 172.24.250.254 | Easysearch | 16 | 31G | 500GB | ### Elasticsearch 配置 7 节点集群，版本：7.10.2 | 实例名称 | 内网 IP | 软件 | vCPU | JVM | 磁盘 | | --- | --- | --- | --- | --- | --- | | i-2zegn56cijnzklcn240m | 172.24.250.251 | Elasticsearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240p | 172.22.75.145 | Elasticsearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240o | 172.17.67.246 | Elasticsearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240t | 172.22.75.139 | Elasticsearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240q | 172.22.75.140 | Elasticsearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240v | 172.24.250.253 | Elasticsearch | 16 | 31G | 500GB | | i-2zegn56cijnzklcn240l | 172.24.250.250 | Elasticsearch | 16 | 31G | 500GB | ### 监控集群配置 单节点 Easysearch 集群，版本：1.9.0 | 实例名 | 内网 IP | 软件 | vCPU | 内存 | 磁盘 | | --- | --- | --- | --- | --- | --- | | i-2zegn56cijnzklcn240f | 172.25.230.226 | 监控集群：Console | 16 | 64G | 500GB | | i-2zegn56cijnzklcn240j | 172.23.15.98 | 监控集群：Easysearch | 16 | 64G | 500GB | ### 压测 loadgen 配置 loadgen 版本：1.25.0 4 台压 Easysearch，4 台压 Elasticsearch。 | 实例名 | 内网 IP | 软件 | vCPU | 内存 | 磁盘 | | --- | --- | --- | --- | --- | --- | | i-2zegn56cijnzklcn240n | 172.17.67.245 | Loadgen - 压 Easysearch | 16 | 64G | 500GB | | i-2zegn56cijnzklcn2411 | 172.22.75.141 | Loadgen - 压 Easysearch | 16 | 64G | 500GB | | i-2zegn56cijnzklcn240k | 172.25.230.227 | Loadgen - 压 Easysearch | 16 | 64G | 500GB | | i-2zegn56cijnzklcn240e | 172.22.75.138 | Loadgen - 压 Easysearch | 16 | 64G | 500GB | | i-2zegn56cijnzklcn240h | 172.24.250.255 | Loadgen - 压 Elasticsearch | 16 | 64G | 500GB | | i-2zegn56cijnzklcn240w | 172.24.251.0 | Loadgen - 压 Elasticsearch | 16 | 64G | 500GB | | i-2zegn56cijnzklcn240g | 172.24.250.248 | Loadgen - 压 Elasticsearch | 16 | 64G | 500GB | | i-2zegn56cijnzklcn240s | 172.24.250.249 | Loadgen - 压 Elasticsearch | 16 | 64G | 500GB | ### 压测索引 Mapping {{< expand "展开查看 Mapping" "..." >}} ```plain PUT nginx { "mappings": { "properties": { "method": { "type": "keyword" }, "bandwidth": { "type": "integer" }, "service_name": { "type": "keyword" }, "ip": { "type": "ip" }, "memory_usage": { "type": "integer" }, "upstream_time": { "type": "float" }, "url": { "type": "keyword" }, "response_size": { "type": "integer" }, "request_time": { "type": "float" }, "request_body_size": { "type": "integer" }, "error_code": { "type": "keyword" }, "metrics": { "properties": { "queue_size": { "type": "integer" }, "memory_usage": { "type": "integer" }, "thread_count": { "type": "integer" }, "cpu_usage": { "type": "integer" }, "active_connections": { "type": "integer" } } }, "cpu_usage": { "type": "integer" }, "user_agent": { "type": "keyword" }, "connections": { "type": "integer" }, "timestamp": { "type": "date", "format": "yyyy-MM-dd'T'HH:mm:ss.SSS" }, "status": { "type": "integer" } } }, "settings": { "number_of_shards": 7, "number_of_replicas": 0, "refresh_interval": "30s" } } ``` {{< /expand >}} ### 压测方法 每 4 个 loadgen 使用批量写入接口 bulk 轮询压测同一集群的 7 个节点，每个请求写入 10000 个文档。 具体请求如下： ```plain requests: - request: #prepare some docs method: POST runtime_variables: # batch_no: uuid runtime_body_line_variables: # routing_no: uuid # url: $[[env.ES_ENDPOINT]]/_bulk url: $[[ip]]/_bulk body_repeat_times: 10000 basic_auth: username: "$[[env.ES_USERNAME]]" password: "$[[env.ES_PASSWORD]]" body: | {"index": {"_index": "nginx", "_type": "_doc", "_id": "$[[uuid]]"}} $[[message]] ``` ### 压测数据样本 ```plain {"method":"DELETE","bandwidth":1955,"service_name":"cart-service","ip":"120.204.26.240","memory_usage":1463,"upstream_time":"1.418","url":"/health","response_size":421,"request_time":"0.503","request_body_size":1737,"error_code":"SYSTEM_ERROR","metrics":{"queue_size":769,"memory_usage":1183,"thread_count":65,"cpu_usage":68,"active_connections":837},"cpu_usage":70,"user_agent":"Mozilla/5.0 (iPad; CPU OS 14_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1","connections":54,"timestamp":"2024-11-16T14:25:21.423","status":500} {"method":"OPTIONS","bandwidth":10761,"service_name":"product-service","ip":"223.99.83.60","memory_usage":567,"upstream_time":"0.907","url":"/static/js/app.js","response_size":679,"request_time":"1.287","request_body_size":1233,"error_code":"NOT_FOUND","metrics":{"queue_size":565,"memory_usage":1440,"thread_count":148,"cpu_usage":39,"active_connections":1591},"cpu_usage":87,"user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1","connections":354,"timestamp":"2024-11-16T05:37:28.423","status":502} {"method":"HEAD","bandwidth":10257,"service_name":"recommendation-service","ip":"183.60.242.143","memory_usage":1244,"upstream_time":"0.194","url":"/api/v1/recommendations","response_size":427,"request_time":"1.449","request_body_size":1536,"error_code":"UNAUTHORIZED","metrics":{"queue_size":848,"memory_usage":866,"thread_count":86,"cpu_usage":29,"active_connections":3846},"cpu_usage":71,"user_agent":"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)","connections":500,"timestamp":"2024-11-16T15:14:30.424","status":403} ``` ## 压测索引 1 主分片 0 副本 ### Elastic 吞吐 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-1.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-2.jpg" "" %}} ### Elastic 线程及队列 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-3.jpg" "" %}} ### 资源消耗 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-4.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-5.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-6.jpg" "" %}} ### Easysearch 吞吐 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-7.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-8.jpg" "" %}} ### Easysearch 线程及队列 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-9.jpg" "" %}} ### 资源消耗 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-10.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-11.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-12.jpg" "" %}} ### 对比 | 软件 | 平均集群吞吐 | 平均单节点吞吐 | 最大队列 | 磁盘消耗 | | --- | --- | --- | --- | --- | | Elasticsearch | 5w | 5w | 811 | 10G | | Easysearch | 7w | 7w | 427 | 4G | ## 压测索引 1 主分片 1 副本 ### Elastic 吞吐 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-13.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-14.jpg" "" %}} ### Elastic 线程及队列 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-15.jpg" "" %}} ### 资源消耗 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-16.jpg" "" %}} ### Easysearch 吞吐 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-17.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-18.jpg" "" %}} ### Easysearch 线程及队列 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-19.jpg" "" %}} ### 资源消耗 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-20.jpg" "" %}} ### 对比 | 软件 | 平均集群吞吐 | 平均单节点吞吐 | 最大队列 | 磁盘消耗(~3000万文档） | | --- | --- | --- | --- | --- | | Elasticsearch | 10w | 5w | 791 | 22G | | Easysearch | 14w | 7w | 421 | 7G | ## 压测索引 7 主分片 ### Elastic 吞吐 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-21.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-22.jpg" "" %}} ### Elastic 线程及队列 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-23.jpg" "" %}} ### 资源消耗 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-24.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-25.jpg" "" %}} 网络 单节点平均接收 26MB/s，对应带宽：1456 Mb/s {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-26.jpg" "" %}} 5千万文档，总存储 105 GB，单节点 15 GB {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-27.jpg" "" %}} ### Easysearch 吞吐 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-28.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-29.jpg" "" %}} ### Easysearch 线程及队列 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-30.jpg" "" %}} ### 资源消耗 {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-31.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-32.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-33.jpg" "" %}} {{% load-img "/img/blog/2024/elasticsearch-vs-easysearch-stress-testing/elasticsearch-vs-easysearch-34.jpg" "" %}} ### 对比 | 软件 | 平均集群吞吐 | 平均单节点吞吐 | 最大队列 | 磁盘消耗 | | --- | --- | --- | --- | --- | | Elasticsearch | 35w | 5w | 2449 | 105G | | Easysearch | 60w | 8.5w | 1172 | 36G | ## 总结 通过对不同场景的压测结果进行对比分析，得出以下结论： + Easysearch 相比 Elasticsearch 的索引性能显著提升 Easysearch 集群的吞吐性能提升了 40% - 70%，且随着分片数量的增加，性能提升效果更为显著。 + Easysearch 相比 Elasticsearch 的磁盘压缩效率大幅提高 Easysearch 集群的磁盘压缩效率提升了 2.5 - 3 倍，并且随着数据量的增加，压缩效果愈发明显。 此测试结果表明，Easysearch 在日志处理场景中具有更高的性能与存储效率优势，尤其适用于大规模分片与海量数据的使用场景。 如有任何问题，请随时联系我，期待与您交流！ {{% load-img "/img/blog/banner/about_yangf.png" "" %}}