?
In the Redis configuration file redis.conf file, the size parameters of maxmemory are configured as follows:
?
?
If the actual storage exceeds the size of the configuration parameters of Redis, there is an elimination strategy in Redis to eliminate the keys that need to be eliminated and sort out a clean memory for the new key values.
?
Redis provides six elimination strategies, of which noeviction is the default. The elimination strategies in these six strategies are as follows:
?
?
?
LRU (least recently used) means the least recently used key, that is, the least recently accessed key. The algorithm deletes data according to the historical access records of data.
?
Its core idea is that if a key value is rarely used recently, it will be rarely accessed in the future.
?
In fact, the LRU implemented by Redis is not a real LRU algorithm, that is, nominally we use LRU algorithm to eliminate the key, but in fact, the eliminated key is not necessarily the longest useless.
?
Redis uses an approximate LRU algorithm, which eliminates keys by random collection, randomly selects five keys at a time, and then eliminates the least recently used keys.
?
The five keys here are only default numbers, and the specific numbers can also be configured in the configuration file, as shown in the following figure:
?
?
When the approximate LRU algorithm is large, it will be closer to the real LRU algorithm. It can be understood that the greater the value, the more complete the data, and the closer the eliminated data is to the least recently used data.
?
Then in order to realize LRU algorithm by time, Redis must add an extra memory space for each key to store the time of each key, the size is 3 bytes.
?
In Redis 3.0, the approximate LRU algorithm is optimized, and the memory of a candidate pool with the size of 16 will be maintained in Redis.
?
When sampling data randomly for the first time, put the data into the candidate pool, and the data in the candidate pool are sorted by time.
?
When the data is selected for the second time, only the data that has been in the candidate pool for less than the minimum time will be put into the candidate pool.
?
When the candidate pool is full at a certain moment, the key with the longest time will be squeezed out of the candidate pool. During elimination, the key with the shortest access time is directly selected from the candidate pool for elimination.
?
The purpose of this is to select the key value that seems to be accessed least recently, so as to eliminate the key value correctly, because the minimum time in the randomly selected sample may not be the real minimum time.
?
However, LRU algorithm has a disadvantage: if a key value has not been accessed before, but has been accessed recently, it will be considered as hot data and will not be eliminated.
?
However, some data have been frequently accessed before, but not recently, which may lead to the rejection of these data, thus leading to misjudgment and rejection of hot data.
?
So in Redis 4.0, besides LRU algorithm, a new LFU algorithm is added, so what is LFU algorithm?
?
?
LFU(Least frequency Used) refers to the keywords that have been frequently used recently, that is, the keywords that have been frequently accessed in the recent period of time, and it is based on the frequency of the times that have been accessed in the recent period of time.
?
Its core idea is: according to the frequency of key's recent visit, the key with less visits should be eliminated first, and vice versa.
?
LFU algorithm reflects the popularity of a key and will not be considered as hot data because LRU algorithm is accessed once in a while.
?
The LFU algorithm supports variable -LFU strategy and allkeys-lfu strategy.
?
?
There are three delete operations in Redis. This strategy is:
?
?
There are two ways to adhere to Redis: RDB and AOF.
?
In RDB, a copy of data at a certain point in memory is obtained in the form of a snapshot. When creating an RDB file, you can create an RDB file by using the save and bgsave commands.
?
Neither of these commands will save the expired key to the RDB file, so that the expired key can be deleted.
?
When loading the RDB file when starting Redis, the master server will not load the expired key, while the slave server will load the expired key.
?
In AOF mode, Redis provides optimization measures for rewriting, and the commands executed are REWRITEAOF and BGREWRITEAOF. Neither of these commands will write expired keys to the AOF file, and they can also delete expired keys.
?
?
RDB is a snapshot storage persistence method, which specifically saves the memory data of Redis at a certain moment to a file on the hard disk. By default, the saved file name is dump.rdb. When the Redis server is started, the data of the dump.rdb file will be reloaded into the memory to recover the data.
?
Turn on RBD persistent mode.
?
The way to start rdb persistence is simple. The client can send a save or bgsave command to the Redis server, let the server generate an RDB file, or specify the conditions for triggering RDB through the server configuration file.
?
?
The save command is a synchronous operation.
?
?
?
When the client sends a save command to the server for persistence, the server will block the requests of other clients after saving the command until the data synchronization is completed.
?
?
Unlike save command, bgsave command is an asynchronous operation.
?
?
?
?
?
?
?
When the client sends a bgsave command from the service, the main process of the Redis server will fork a child process to solve the data synchronization problem. After saving the data to the rdb file, the subprocess will exit.
?
Therefore, compared with the save command, the Redis server uses a sub-thread to write IO when processing bgsave, and the main process can still receive other requests, but the forks sub-process is synchronous, so the forks sub-process cannot receive other requests, which means that if a fork sub-process takes too long (usually very fast), the bgsave command will still block the requests of other customers.
?
?
In addition to sending commands through the client, there is another way, that is, saving the conditions that specify triggering RDB persistence in the Redis configuration file, such as starting RDB data synchronization when at least several write operations are reached within a few seconds.
?
For example, we can specify the following options in the configuration file redis.conf:
?
?
Then load the configuration file when the server starts.
?
?
This way of triggering rdb through the server configuration file is similar to the bgsave command. When the trigger condition is reached, the child process of forks will synchronize the data. But it is best not to trigger RDB persistence in this way, because it is easy to write RDB files frequently if the triggering time is too short, which will affect the performance of the server, and it will cause data loss if the time is set too long.
?
?
This paper introduces three ways for the server to generate rdb files, whether it is the main process or the sub-process, and the flow is as follows:
?
?
?
?
?
Another persistent way of Redis: AOF (file only).
?
Unlike RDB, which stores snapshots at a certain moment, aof persistently records every write command from the client to the server and saves these writes to the end of the aof file with suffix in Redis protocol. When the Redis server is restarted, it will load and run the AOF file command to recover the data.
?
?
By default, Redis does not enable AOF persistence. We can enable it in the configuration file and configure it in more detail, such as the following redis.conf file:
?
?
?
In the above configuration file, we can specify the write strategy by appendfsync option, which has three options.
?
?
?
When every write operation of the client is saved in the aof file, this strategy is safe, but every write request has IO operation, so it is also slow.
?
?
The default write strategy of Appendfsync is to write the aof file once every second, so the data of 1 may be lost at most.
?
?
Redis server is not responsible for writing to aof, but the operating system handles when to write to aof files. Faster, but also the most unsafe option, not recommended.
?
?
AOF appends every write operation of the client to the end of the aof file, such as executing the incr command of a key multiple times. At this point, aof saves every command in the aof file, and the aof file will become very large.
?
?
The aof file is too large, and it will be very slow to load the aof file to recover the data. To solve this problem, Redis supports rewriting aof files. By rewriting the aof, a minimum command set for restoring the current data can be generated, such as many commands in the above example, which can be rewritten as:
?
?
?
You can set whether to turn on rewriting through the option no-appendfsync-on-rewrite in the redis.conf configuration file. This method will be rewritten every time fsync, which will affect the performance of the server, so the default value is no, which is not recommended.
?
?
The client sends bgrewriteaof command to the server, or it can let the server rewrite aof.
?
?
Aof rewriting is also an asynchronous operation, that is, if you want to write an AOF file, the Redis main process will be handled by a forks subprocess, as shown below:
?
?
?
?
?
?
When writing the aof log file, if the Redis server is shut down, the aof log file will have a format error. When the Redis server is restarted, the Redis server will refuse to load this aof file. You can repair aof and restore data by the following steps.
?
?
?
AOF only appends log files, so it has little impact on server performance, is faster than RDB, and consumes less memory.
?
?
?
We can compare RDB and AOF from several aspects. In application, we should choose RDB or AOF according to our actual needs. In fact, if you want the data to be safe enough, you can open it in both ways, but the simultaneous IO operation in both persistent ways will seriously affect the performance of the server, so sometimes we have to make a choice.
?
?
When both RDB and AOF are open, Redis will give priority to using AOF logs to recover data, because the files saved by AOF are more complete than RDB files.