StableSearch: A Searchable File Content Metadata System for the Gfarm File System

概要

論文の詳細を見る
Distributed file systems are allowing scientists, corporations, and users to access and share data on a scale previously unimaginable. The utility of these systems cannot be understated, but there are still many problems to be solved, not the least of which is helping users quickly search the petabytes of stored data for the files they are interested in. This problem has largely been solved on the desktop, for example Apple's Spotlight search, but these approaches are not scalable to thousands of nodes. Obviously there is a need for a search tool that can quickly search these files without negatively impacting the running system or giving a user access to metadata for data they don't have access to. To this end, we have designed StableSearch, a tool for searching Gfarm[1] distributed file systems. We have shown that our tool is capable of locating data significantly faster than grep while having a minimal impact on the performance of the running system.
2011-03-08