Redis Fast File System Search - search

Question

I am new to Redis, and
I would like to iterate all files and folders on a given computer and save it in Redis DB,
so I can search for files or folders by their name later.
I wonder how I should store the data in Redis and how can I make the search as fast as possible?
Thanks

Answer 1

Your requirement can divided as:
Iterate and save in Redis.
Get from Redis and search.
The choice of data type depends on usage of data.
Redis provides us "KEY/VALUE" relationship.
Taking some sample data:
File Name Location
----------------------------
Sys.log /root/tmp
info.txt /var/log
redis.log /var/log/redis/redis.log
abc.log /app/task
abc.log /home/test
Note there can be n files with same name at different location. This means we cannot use normal key/value with file names as keys.
One relationship which should be kept in mind is Parent-child. A directory(parent) will have files(child) or other directories.
Also there should be a way to distinguish between files and directories.
Solution:
Iterate file system and store in REDIS
(Directory design)
Create Redis Sets of directories with their content as file names and other directories which too have their own lists.
And every enter in the list should have a prefix which can be used to identify whether the entry is a file or directory. If it is a directory then you can use it to further search for more files.
This gives us a capability to use the sets for printing all their children.
127.0.0.1:6379> SADD "/var/log" "File:info.txt"
(integer) 1
127.0.0.1:6379> SMEMBERS "/var/log"
1) "File:info.txt"
127.0.0.1:6379> SADD "/var/log" "Dir:redis"
(integer) 1
127.0.0.1:6379> SMEMBERS "/var/log"
1) "Dir:redis"
2) "File:info.txt"
content of redis
127.0.0.1:6379> SADD "redis" "redis.log"
(integer) 1
127.0.0.1:6379> SADD "redis" "error.log"
(integer) 1
127.0.0.1:6379> SMEMBERS redis
1) "redis.log"
2) "error.log"
Search REDIS
( Input a file name print all possible locations where it is present.)
While iterating file system when we are creating sets for directories we parallel create a hashmap which stores .
Contents of a list for a file will show where all it is present.
127.0.0.1:6379> lpush "info.txt" "/var/log"
(integer) 1
127.0.0.1:6379> lpush "info.txt" "/tmp"
(integer) 2
127.0.0.1:6379> lrange "info.txt" 0 -1
1) "/tmp"
2) "/var/log"
Note: for fast experience with REDIS try and execute group of commands at one go .i.e use multi or use eval (lua scripts).
Hope this gives you a start in your design.