A Slot-based Virtual Node Method of Consistent Hashing for Optimization of Index Reconfiguration on Distributed Search
スポンサーリンク
概要
- 論文の詳細を見る
In the distributed search system, the method of mapping and managing documents on segmented indexes is significant to realize load balancing of distributed search process and efficient cluster reconfiguration. Consistent hashing is the advanced method of data mapping which minimizes the network traffic and redundant index data processing in the index reconfiguration. However, if the cluster consists of several thousands of nodes, it requires huge memory resources. Furthermore, it takes a long time to execute the index reconfiguration because of the overhead of many index splitting processes. In this paper, we propose a new method called slot-based virtual node method of consistent hashing to solve the above issues. As the multiple nodes are added or removed, our new method plots or reallocates the virtual nodes on the hash ring space to realize "a bunch of" data migration as far as possible to optimize the index reconfiguration. Slot-based virtual node management saves memory consumption for the mapping information. We actually evaluate memory consumptions of both conventional and our proposed methods to bring out the resource-saving effect. Besides, we estimate lapse times of index reconfiguration processes based on the data processing models to verify the effective reduction of time in our method.
- 2012-10-01
著者
-
KOMODA Norihisa
Osaka University
-
Onoyama Takashi
Hitachi Solutions Ltd.
-
Kirihata Yasuhiro
Hitachi Solutions Ltd.
-
ONOYAMA Takashi
Hitachi Solutions, Ltd.
関連論文
- Effectiveness Evaluation of Pre-review of Software Developed by Novice Engineers in China
- Optimized assignment of deduplication backup methods using integer programming
- Evaluation of Knowledge Cards Classification Method with Translation Dictionary
- A Knowledge Cards Classification Method With Conversion Loss Correction For Incomplete Translation Dictionary
- A Support Method of Model Description Error Detection on a Programming Environment for Multi Agent Simulation
- An Accuracy Improvement Method of Simulation on Qualitative and Quantitative Hybrid Model with Uncertain Landmarks
- E-mail Log Analysis in Offshore Software Development
- A Knowledge Cards Classification Method with Conversion Loss Correction for Incomplete Translation Dictionary
- WriteShield: A Pseudo Thin Client for Prevention of Information Leakage (特集 多様な情報社会に適応するシステム技術)
- A Slot-based Virtual Node Method of Consistent Hashing for Optimization of Index Reconfiguration on Distributed Search
- Improvement of performance and reduction of deduplication backup system using multiple layered architecture
- An Anomaly Detection Method on Web-based System by Certification of Structural Change on Autoregressive Model