There is a need to improve access speed for a file in a storage system that is provided with multiple storage units having different access speeds.
The storage system (10) includes an access frequency acquiring part (14) that divides a past constant accessed data acquisition period into multiple unit periods, and obtains an access frequency for file stored in any of the multiple storage units (30 and 31), a similarity degree calculating part (16) that obtains, for each file, a similarity degree between the access frequency to the file for each unit period, and the access frequency to every other file for each unit period, a grouping process part (17) that groups one or more other files having a similarity degree equal to or higher than a predetermined value, in an identical group, and a file relocation part (18) that places a file belonging to the identical group into a storage unit (30) having a high access speed, when the file stored in the storage units (30 and 31) is accessed and there exists any other file within the identical group to which the file belongs.
|
6. A storage system provided with multiple storage units each having a different access speed, comprising:
a number of access to data item during a unit period acquiring process that divides a past constant accessed data acquisition period into multiple unit periods, and obtains a number of access to data item during the unit period for each data item stored in any of the multiple storage units;
an access degree computing process that obtains, with respect to each unit period and each data item, an access degree that is a ratio of the number of access to the data item during the unit period to the number of access to the data item during the past constant accessed data acquisition period, in order to show a tendency of the number of access to the data item during the past constant accessed data acquisition period;
a grouping degree computing process that obtains, with respect to each unit period, a grouping degree which is a ratio of the number of data items which is accessed at least once during the unit period to the number of all data items, in order to show a tendency of the number of access to a plurality of data items during the unit period;
a calibrated access degree computing process that multiplies, with respect to each unit period and each data item, the grouping degree during a unit period obtained in the grouping degree computing process by the access degree for each corresponding unit period and each data item obtained in the access degree computing process;
a similarity degree computing process that multiplies the calibrated access degree for each unit period of a particular data item, by the calibrated access degree during a corresponding unit period of the other data item, and sums a result of the multiplication for each unit period, as a similarity degree between the particular data item and the other data item, in order to show similarity of access tendency between data items during a past constant period;
a grouping process that groups the particular data item and one or more other data items having the similarity degree equal to or higher than a predetermined value with regard to the particular data item in an identical group, as a group of data items indicating a similar tendency of access during the past constant accessed data acquisition period; and
a data migrating process that migrates one or more data items belonging to the identical group into a storage unit having a high access speed among the multiple storage units, when at least one of data items belonging to the identical group and stored in a storage unit having a low access speed is accessed.
1. A data relocation method in a storage system provided with multiple storage units having different access speeds, the method comprising:
a number of access to data item during a unit period acquiring process that divides a past constant accessed data acquisition period into multiple unit periods, and obtains a number of access to data item during the unit period for each data item stored in any of the multiple storage units;
an access degree computing process that obtains, with respect to each unit period and each data item, an access degree that is a ratio of the number of access to the data item during the unit period to the number of access to the data item during the past constant accessed data acquisition period, in order to show a tendency of the number of access to the data item during the past constant accessed data acquisition period;
a grouping degree computing process that obtains, with respect to each unit period, a grouping degree which is a ratio of the number of data items which is accessed at least once during the unit period to the number of all data items, in order to show a tendency of the number of access to a plurality of data items during the unit period;
a calibrated access degree computing process that multiplies, with respect to each unit period and each data item, the grouping degree during a unit period obtained in the grouping degree computing process by the access degree for each corresponding unit period and each data item obtained in the access degree computing process;
a similarity degree computing process that multiplies the calibrated access degree for each unit period of a particular data item, by the calibrated access degree during a corresponding unit period of the other data item, and sums a result of the multiplication for each unit period, as a similarity degree between the particular data item and the other data item, in order to show similarity of access tendency between data items during a past constant period;
a grouping process that groups the particular data item and one or more other data items having the similarity degree equal to or higher than a predetermined value with regard to the particular data item in an identical group, as a group of data items indicating a similar tendency of access during the past constant accessed data acquisition period; and
a data migrating process that migrates one or more data items belonging to the identical group into a storage unit having a high access speed among the multiple storage units, when at least one of data items belonging to the identical group and stored in a storage unit having a low access speed is accessed.
5. A recording medium that records a data relocation program of a storage system provided with multiple storage units having different access speeds, which allows a computer to execute,
a number of access to data item during a unit period acquiring process that divides a past constant accessed data acquisition period into multiple unit periods, and obtains a number of access to data item during the unit period for each data item stored in any of the multiple storage units;
an access degree computing process that obtains, with respect to each unit period and each data item, an access degree that is a ratio of the number of access to the data item during the unit period to the number of access to the data item during the past constant accessed data acquisition period, in order to show a tendency of the number of access to the data item during the past constant accessed data acquisition period;
a grouping degree computing process that obtains, with respect to each unit period, a grouping degree which is a ratio of the number of data items which is accessed at least once during the unit period to the number of all data items, in order to show a tendency of the number of access to a plurality of data items during the unit period;
a calibrated access degree computing process that multiplies, with respect to each unit period and each data item, the grouping degree during a unit period obtained in the grouping degree computing process by the access degree for each corresponding unit period and each data item obtained in the access degree computing process;
a similarity degree computing process that multiplies the calibrated access degree for each unit period of a particular data item, by the calibrated access degree during a corresponding unit period of the other data item, and sums a result of the multiplication for each unit period, as a similarity degree between the particular data item and the other data item, in order to show similarity of access tendency between data items during a past constant period;
a grouping process that groups the particular data item and one or more other data items having the similarity degree equal to or higher than a predetermined value with regard to the particular data item in an identical group, as a group of data items indicating a similar tendency of access during the past constant accessed data acquisition period; and
a data migrating process that migrates one or more data items belonging to the identical group into a storage unit having a high access speed among the multiple storage units, when at least one of data items belonging to the identical group and stored in a storage unit having a low access speed is accessed.
2. The data relocation method according to
the similarity degree computing process obtains the similarity degree between the particular data item and every other data item, with respect to each data item,
the grouping process groups, in the identical group, one or more other data items whose similarly degree are equal to or higher than a predetermined value, with respect to each data item, and
when any among each of the data items is accessed, the data migrating process migrates the data item belonging to the identical group into the storage unit having the high access speed, if the identical group includes any data item.
3. The data relocation method according to
when any among each of the data items is accessed, this data item is assumed as the particular data item.
4. The data relocation method according to
the number of access to data item during a unit period acquiring process obtains a read number of accesses and a write number of accesses with respect to each data item stored in any of the multiple storage units, and
the similarity degree computing process obtains the similarity degree between the read and write number of accesses to the particular data item for each unit period, and the write number of accesses to every other data item for each unit period.
|
|||||||||||||||||||||||||||||||
This application claims the priority based on the Japanese Patent Application No. 2006-237626 filed on Sep. 1, 2006. The entire contents of which are incorporated herein by reference for all purpose.
The present invention relates to a storage system that is provided with multiple storage units having different access speeds, a data relocation method thereof, and a data relocation program.
In recent years, data volume handled by computers is dramatically increasing, and there is a growing need for a high-capacity storage system.
As a high-capacity storage system, there is a hierarchical storage system that combines a high-speed low-capacity storage unit having short required access time, but low capacity, and a low-speed high-capacity storage unit having long required access time but with high capacity. In this hierarchical storage system, an appropriate data arrangement is significant, placing highly accessed data in the high-speed, low-capacity storage unit, and infrequently accessed data in the low-speed, high-capacity storage unit, so that a user's time required for accessing can be reduced.
Considering the above discussed point, Japanese Patent Laid-open Publication No. H05-12077 (hereinafter, referred to as “Patent Document 1”), for instance, discloses the following technique.
Patent Document 1 discloses a technique that obtains access frequency based on an access history of data that is stored in multiple storage units each having different performance and capacity, and places frequently accessed data into a storage unit having a high accessing speed, and infrequently accessed data into a storage unit having a low accessing speed.
However, in this technique, there is a problem in that when a particular data item is accessed, there is a high probability that other data item having relevance to this particular data is accessed. Even when the particular data item is accessed, if this other data item is accessed only infrequently, it takes a long time for accessing, since the other data item is placed in a low-speed storage unit.
In order to solve the above problem, Japanese Patent Laid-open Publication No. H08-263335 (hereinafter, referred to as “Patent Document 2”) discloses a technique as follows: When the other data item is accessed within a certain period of time after the particular data item is accessed, it is assumed that the particular data item and the other data item are belonging to an identical group, and when any of the data within the group is accessed, the other data within the group are placed in the high-speed storage unit.
The technique disclosed by Patent Document 2 is assuredly able to solve the technical problems included in the art described in Patent Document 1. However, there are further problems such as the following: For example, if just after accessing a particular data item, other data item that has no relevance to this particular data is accidentally accessed, this other data item is treated as though it belongs to an identical group; and even though a data item having a high relevance to this particular data exists, this data item is not treated as belonging to the identical group, if this data is not accessed after the particular data is accessed, just before placement of data. Due to these problems, appropriate grouping is not performed, resulting in the time for accessing data being long.
In view of such problems of the conventional art as described above, an object of the present invention is to provide a storage system that optimizes grouping of data and reduces time for accessing data, a data relocation method thereof, and a data relocation program.
The data relocation method to solve the problems above, in a storage system provided with multiple storage units each having a different access speed, executes, an access frequency acquiring process that divides a past constant accessed data acquisition period into multiple unit periods, and obtains an access frequency as to each data item stored in any of the multiple storage units, a similarity degree calculating process that obtains similarity degrees between an access frequency for each unit period of a particular data item among each of the data items and the access frequency for each unit period of every other data item, a grouping process that groups the particular data item and one or more other data items in an identical group, one or more other data items having the similarity degree equal to or higher than a predetermined value with regard to the particular data item, and a data placing process that places one or more data items belonging to the identical group into a storage unit having a high access speed among the multiple storage units, when the particular data item is accessed and there exists any data item within the identical group to which the particular data item belongs.
Here, the similarity degree calculating process may obtain the similarity degree between the data item and every other data item, with respect to each data item, the grouping process may group one or more other data items whose similarity degree are equal to or higher than a predetermined value, in the identical group, with respect to each data item, and when any among each of the data items is accessed, the data placing process may place the data item belonging to the identical group into the storage unit having the high access speed, if the identical group includes any data item.
Furthermore, when any of each data item is accessed, this data item may be assumed to be the particular data item.
In order to solve the problems as described above, a computer readable data relocation program, that is recorded in a recording medium, is a data relocation program of a storage system provided with multiple storage units having different access speeds, allows a computer to execute, an access frequency acquisition step that divides a past constant accessed data acquisition period, into multiple unit periods, and obtains an access frequency as to each data item stored in any of the multiple storage units, a similarity degree calculating step that obtains similarity degrees between an access frequency for each unit period of a particular data item among each of the data items, and the access frequency for each unit period of every other data item, a grouping step that groups the particular data item and one or more other data items in an identical group, one or more other data items having the similarity degree equal to or higher than a predetermined value with regard to the particular data item, and a data placing step that places one or more data items belonging to the identical group into a storage unit having a high access speed among the multiple storage units, when the particular data item is accessed and there exists any data item within the identical group to which the particular data item belongs.
The storage system to solve the problems as described above is provided with multiple storage units having different access speeds, and a computer, and the aforementioned program is installed in the computer.
In the present invention, if a particular data item is accessed and other data item having a high accessing relevancy to this particular data item exists, this other data is placed in a high-speed storage unit even though the access frequency of the other data is low. Therefore, it is possible to reduce the access time. Further in the present invention, an access frequency for each data item for each unit period within the accessed data acquisition period is obtained, and a similarity degree between the access frequency of the particular data item for each unit period, and the access frequency of the other data item for each unit period is also obtained. Therefore, it is possible to group the data so that the data is not affected by chance. Therefore, also from this point of view, the access time can be reduced. For instance, in the present invention, even though other data item having no relevancy to a particular data item is accidentally accessed after the particular data item is accessed, it is possible to avoid treating this other data as belonging to an identical group. Furthermore, when there is other data item having high relevancy to the particular data, and even though this other data is not accessed just before placing the data, after the particular data is accessed, it is possible to treat this other data as belonging to the identical group. In particular, the present invention introduces the concept of degree of grouping, and eliminates chance as far as possible in the course of obtaining the similarity degree as an index for grouping. Therefore, very relevant grouping can be performed.
Hereinafter, various embodiments of the storage system relating to the present invention will be explained with reference to the accompanying drawings.
The first embodiment of the storage system relating to the present invention will be explained with reference to
As shown in
The processor 11 in the storage system 10 includes, as its functions, a file information acquisition part 12 that acquires information of a file stored in each of the storage units 30 and 31, an access date and time management part 13 that manages the date and time when the file stored in each of the storage units 30 and 31 is accessed, an access frequency calculation part 14 that obtains an access frequency to the file during a past period for acquiring accessed data, an access degree calculation part 15 that obtains a file access degree as described below, a file similarity degree calculation part 16 that obtains a similarity degree between the access degree of one file and the access degree of every other file with respect to each file, a grouping process part 17 that categorizes, into one group, multiple files having a high degree of similarity with one another, a file relocation part 18 that places, when one file in the group is accessed, the other file within an identical group into the high-speed low-capacity storage unit 30, and an access processing part 19 that performs access processing for the files stored in each of the storage units 30 and 31.
The program memory 28 stores various programs including the data relocation program 29 that the processor 11 executes. The data relocation program 29 is a program to implement, as functions of the processor 11, the access frequency calculation part 14, the access degree calculation part 15, the file similarity degree calculation part 16, the grouping process part 17, and the file relocation part 18.
The data memory 20 is provided with a file information table 21 that stores information of a file, which is acquired by the file information acquisition part 12, an access date and time table 22 that stores access date and time when the file is accessed, which is acquired by the access date and time managing part 13, an access frequency table 23 that stores an access frequency with respect to each file, which is obtained by the access frequency calculation part 14, a calibrated access degree table 24 that stores a calibrated access degree described below, which is obtained by the access degree calculation part 15, a file similarity degree table 25 that stores a similarity degree with respect to each file, which is obtained by the file similarity degree calculation part 16, and a file group table 26 that stores a file ID with respect to each group.
In the present embodiment, it is assumed that the storage system 10 includes each of the storage units 30 and 31 together with the processor 11 and the like, in the same enclosure. However, they are not necessarily accommodated in the same enclosure. In addition, in the present embodiment, the storage system 10 and the client terminal 2 are individual devices. However, there is no problem if the client terminal 2 contains the storage system 10, or the storage system 10 contains the client terminal 2. Here, the high-speed storage unit 30 has a low capacity and the low-speed storage unit 31 has a high capacity. However, the capacity size, low or high, does not directly relate to the present invention, and the present invention is applicable as long as there are multiple storage units each having a different access speed.
As mentioned above, the file information table 21 is a table that stores information of each file acquired by the file information acquisition part 12. As shown in
The access date and time table 22 is a table that stores a date and time when each of the files is accessed, which is acquired by the access date and time managing part 13. As shown in
The access frequency table 23 is a table that stores the number of times of access, i.e., access frequency, with respect to each file during the accessed data acquisition period, which is obtained by the access frequency calculation part 14. The accessed data acquisition period is a period necessary for data relocation described below, and it is predetermined by a user. Here, as shown in
The calibrated access degree table 24 is a table to store a calibrated access degree that is obtained by the access degree calculation part 15 by using the access frequency, for each unit period with respect to each file ID. Definitions of the calibrated access degree, the access degree to obtain the calibrated access degree, and a calculation method thereof will be described in detail below. Basically, both the access degree and the calibrated access degree are values indicating a degree of access frequency of the file during a unit period, relative to the total access frequency of the file during the accessed data acquisition period. Therefore, if the access degree or the calibrated access degree is high for a certain file during a constant unit period, this indicates that the number of times of access for this file is large during this unit period. This calibrated access degree table 24 includes a file ID field 24a to store the entire file IDs stored in the file ID field 23a of the access frequency table 23, and a calibrated access degree field 24b that stores a calibrated access degree for each unit period during the accessed data acquisition period with respect to each file ID. This particular example illustrates that “1” is stored in the file ID field 24a, and in the calibrated access degree fields 24b associated with this file ID “1”, the calibrated access degrees are respectively stored, i.e., “0” for “2005/10/1”, “0” for “2005/10/2”, “0.68” for “10/3”, “0.91” for “10/4”, “0.36” for “10/5”, “0.18” for “10/6”, and “0” for “10/7”.
The file similarity degree table 25 is a table where the file similarity degree calculation part 16 stores a similarity degree indicating a similarity degree between the access frequency for one file within the unit period and the access frequency for other files within the unit period, with respect to each file ID. Definitions of the similarity degree and a calculation method thereof will be described in detail below. The higher the value is, the higher the possibility is that the two files are accessed within an identical unit period. The file similarity degree table 25 includes a file ID field 25a to store all the file IDs stored in the file ID field 23a of the access frequency table 23, and a similarity degree field 25b that stores a similarity degree relative to another file, with respect to each file ID and the other file ID. This particular example illustrates that “1” is stored in the file ID field 25a, and, in the similarity degree fields 25b associated with the file ID “1”, the similarity degrees are respectively stored, i.e., a similarity degree “0.43” for a file having a file ID “2”, a similarity degree “1.15” for a file having a file ID “3”, a similarity degree “0.36” for a file having a file ID “4”, a similarity degree “0.66” for a file having a file ID “5”, a similarity degree “0” for a file having a file ID “6”, a similarity degree “0” for a file having a file ID “7”, a similarity degree “0.42” for a file having a file ID “8”, a similarity degree “0.30” for a file having a file ID “9”, and a similarity degree “0.53” for a file having a file ID “10”.
The file group table 26 is a table where the grouping process part 17 stores, with respect to each file ID, a second file ID that is assumed as belonging to a group that is the same as the file having the file ID. The group processing method will be described below. As shown in
Next, with reference to the flowchart shown in
Firstly, the access frequency calculation part 14 determines whether or not the latest unit period is finished (S10). If the latest unit period is not finished yet, that is, it is still within the unit period, the process in step 80 is carried out. If the latest unit period is already finished, the access frequency calculation part 14 sets a new unit period and a new accessed data acquisition period (S20).
By way of example, if the current date is Oct. 7, 2005, the accessed data acquisition period at this date is one week, from Sep. 30, 2005 to Oct. 6, 2005. In the case above, the access frequency calculation part 14 determines that the latest unit period is October 7 and determines whether or not this day is finished (S10). If this day is finished, the access frequency calculation part 14 sets this day, i.e., October 7, as a new unit period of the accessed data acquisition period, as well as changing the setting of the accessed data acquisition period to one week, from the Oct. 1, 2005 to Oct. 7, 2005. According to this change of settings, the access frequency calculation part 14 renews the date of each unit period in the access frequency table 23 (
Subsequently, the access frequency calculation part 14 refers to the access date and time table 22 and obtains the access frequency of each file with respect to each unit period within the accessed data acquisition period, and stores this access frequency in the access frequency table 23 (S30). In this example here, the access frequency calculation part 14 obtains only the access frequency during a unit period newly set in step 20, and as for the other unit periods during the accessed data acquisition period, the data during the unit period already calculated is used as it is. Consequently, as shown in
When the access frequency table 23 is updated by the access frequency calculation part 14, the access degree calculation part 15 obtains a calibrated access degree, and stores this value in the calibrated access degree table 24 (S40).
Here, with reference to the flowchart as shown in
The access degree calculation part 15 firstly determines whether or not the all the unit periods set in the access frequency table 23 are selected (S41). If all the unit periods are selected, the processing continues to step 45. If the entire unit periods are not selected, a non-selected unit period t is selected (S42), and the access degree calculation part 15 obtains the number of files that have been accessed during the unit period t, in other words, access frequency N(t) of all the files (S43). Subsequently, a grouping degree Y(t) (=N/N(t)) is calculated, which is a ratio between the number of all the files N stored in the file information table, and the access frequency N(t) to all the files within the unit period t (S44). For example, the number of all the files N is 10 (ten) having the file IDs “1” to “10”, and 5 (five) files are accessed during the unit period of October 1, which is the access frequency N (10/1) to all the files. Therefore, the grouping degree Y (10/1) of the unit period of October 1 is equal to two (=10/5). In addition, the grouping degree in the unit period of October 3 (10/3) is equal to 2.5 (=10/4). The grouping degree Y(t) is a value having a role of assigning weights to the access degree described below, and when the access frequency N(t) within the unit period t becomes larger, the value of the grouping degree Y(t) becomes smaller. The significance of the grouping degree will be explained later.
When the grouping degree Y(t) for a constant unit period t is calculated, the processing returns to step 41, and again it is determined whether or not all the unit periods have been selected. If all the unit periods are selected, it is determined whether or not all the files stored in the access frequency table 23 have been selected (S45). If all the files are selected, the process for calculating the calibrated access degree (S40) is terminated. If all the files have not been selected, file a, which has not been selected yet, is selected (S46). Then, the total access frequency C(a) to this file a during the accessed data acquisition period is obtained from the access frequency table 23 (S47). For example, the total access frequency C(1) to the file having the file ID “1” is “11” according to the access frequency table 23 (
Subsequently, the access frequency calculation part 14 determines whether or not all the unit periods within the accessed data acquisition period have already been selected (S48), and if they have already been selected, the processing returns to step 45. Otherwise, the unit period t that has not been selected yet is selected (S49). Then, the access frequency C to the file a, during the unit period t (a, t), is obtained from the access frequency table 23 (S50). Thereafter, the access degree X(a, t) (=C(a, t)/C(a)) is obtained, which is a ratio of the access frequency C(a, t) to the file a during the unit period t, relative to the total access frequency C(a) to the file a during the accessed data acquisition period (S51). For example, C(1, 10/3) is “3”, which is the access frequency to the file “1” during the unit period of October 3, according to the access frequency table 23 (
Next, the access degree X(a, t) to the file a during the unit period t is multiplied by the grouping degree Y(t) during the same unit period, which is obtained in step 44, and a calibrated access degree W(a, t) is calculated. Then, this calibrated access degree is stored in a corresponding field in the calibrated access degree table 24 (
When the calibrated access degree W(a, t) for the file a during the unit period t is obtained, the processing returns to step 48 again, and determines whether or not all the unit periods have been selected with regard to the file a. Here, if all the unit periods have already been selected with regard to the file a, and the calibrated access degree W(a, t) with respect to each unit period has already been obtained as to the file a, the processing returns to step 45, and again, it is determined whether or not all the files have been selected. At this timing, if all the files have already been selected, and a calibrated access degree for each of all the unit periods has been obtained with respect to each file, the process of calculating the calibrated access degree (S40) is terminated.
Here, an explanation will be made regarding the access degree X(a, t), the calibrated access degree W(a, t), and the grouping degree Y(t).
Basically, the access degree X(a, t) of the file a is a relative value of the frequency to access to the file a during a particular unit period t, which is relative to the access frequency to the file a during all the unit periods within the accessed data acquisition period. Therefore, if the access degree X(a, t) to the file a during the particular unit period t is high, it means that the access frequency to the file a during this particular unit period t is higher compared to the other unit periods. If another file b has a high access degree X(b, t) during this particular unit period t, both access frequencies to the file a and the file b are higher during this unit period t, relative to the other unit periods. Therefore, it is possible to determine that there is a high similarity between the access frequency to the file a and the access frequency to the file b, and when grouping files having a high similarity in access frequency, both the file a and file b are treated as belonging to the same group. It is to be noted here that if the access frequency to other multiple files during this unit period t is also extremely high, both the file a and the file b are accessed because there is a high possibility that both files are accidentally accessed during this unit period t, not because there is a certain relationship between the file a and the file b.
In the present embodiment, if the total access frequency N(t) to each file during a particular unit period t is high, the similarity between both files a and b is lowered. On the other hand, if the total access frequency to each file during the particular unit period t is low, a concept of grouping degree Y(t) (=N/N(t)) is introduced so as to enhance the similarity between the files a and b, and by using the grouping degree, the similarity between the files a and b is corrected. In other words, as described above, if the total access frequency N(t) to all the files during a particular unit period t becomes higher, the grouping degree Y(t) conversely becomes lower. Therefore, in the present embodiment, the access degree X(a, t) during a particular time is multiplied by the grouping degree Y(t) as a weighting, and the obtained value is assumed as a calibrated access degree W(a, t). The similarity of the access frequency with other files is considered, using this calibrated access degree W(a, t), to eliminate, as much as possible, chance increasing of similarity.
When the process of calculating the calibrated access degree (S40) is terminated, as shown in the flowchart of
As indicated by the flowchart in
Next, the file similarity degree calculation part 16 determines whether or not there exists in the calibrated access degree table 24, a file having an ID larger than the file ID of the file a (S63). If there is no file having a larger ID number, the processing returns to step 61. If there is a file having a larger ID number, all the larger IDs are extracted, and a set F of files having the larger IDs is formed (S64). Then, it is determined whether or not all the files within the set F have already been selected (S65). If all the files have already been selected, the process returns to S61. If all the files have not been selected yet, a non-selected file b within the set F is selected (S66).
When the file similarity degree calculation part 16 selects the non-selected file b within the set F, it is determined whether or not the entire unit periods within the accessed data acquisition period have been selected (S67). If all the unit periods have already been selected, the process returns to step 65, and if all the unit periods have not been selected yet, a non-selected unit period t within the accessed data acquisition period is selected (S68). Next, from the calibrated access degree table 24, the calibrated access degree W(a, t) of the file a and the calibrated access degree W(b, t) of the file b during this unit period t are extracted, and these are multiplied by each other, and an obtained value is set as a temporary similarity degree R(a, b) (S69). When the temporary similarity degree is obtained, the process returns to step 67. In step 68, a non-selected unit period is selected, and the calibrated access degree of the file a and the calibrated access degree of the file b during this unit period are multiplied by each other, and to this obtained value, the temporary similarity degree R(a, b) previously obtained is added (S69). Again, the process returns to step 67, and the step 67 to step 69 are repeated until selection of all the unit periods is completed. In other words, a value obtained by multiplying the calibrated access degree of the file a and the calibrated access degree of the file b with respect to each unit period within the accessed data acquisition period is summed, and an obtained total is set as a similarity degree R(a, b) between the file a and the file b. This similarity degree R(a, b) is stored in a corresponding field of the file similarity degree table 25 (
Similarity degree R(1, 2)=0×0.5(for October 1)+0×1.25(for October 2)+0.68×0.63(for October 3)+0.91×0(for October 4)+0.36×0(for October 5)+0.18×0(for October 6)+0×0(for October 7)=0.43
This similarity degree R(1, 2)=0.43 is stored in the field of similarity degree of the file “2” being associated with the file “1”, in the file similarity degree table 25 as shown in
When the similarity R(a, b) between the file a and file b is obtained, the process returns to step 65, and in step 66, a non-selected file c within the set F is selected. Then, a similarity degree R(a, c) between the file a and the file c is obtained in the process from step 67 to step 69. If non-selected file does not exist any more in the set F (S65), the process returns to step 61. Finally, when the similarity degree is obtained with regard to all combinations of files as to all the files stored in the calibrated access degree table 24, this process of calculating the file similarity degree (S60) is terminated.
When the process of calculating the file similarity degree (S60) is terminated, as shown in the flowchart of
When the process of the grouping process (S70) as described above is terminated, the file relocation part 18 and the access processing part 19 determine whether or not there is access to any of the files (S80) If there is access to any of the files, the access processing part 19 performs the access processing for this file. In addition, the file relocation part 18 refers to the file group table 26, and determines whether or not there is a file belonging to the same group with this accessed file (S90). If there is a file belonging to the same group with this file and the files in the group are placed in the low-speed high-capacity storage unit 31, those files within the group are placed in the high-speed low-capacity storage unit 30. If those files within the group are already placed in the high-speed low-capacity storage unit 30, the location of the files is kept the same (S100). In the relocation process, if the file being accessed is placed in the low-speed high-capacity storage unit 31, this file may also be placed in the high-speed low-capacity storage unit 30.
When this data relocation process (S100) is terminated, and furthermore when a negative judgment is made in the previous judgments (S80, 90), the process returns to step 10, and steps 80, 90, 100, and 10 are repeated until the latest unit period is over.
In the present embodiment as described above, if there is access to a certain file and there is a second file that has a high access relevance with this file, this second file is placed in the high-speed storage unit 30 even though the access frequency to this second file is low, and therefore, accessing time can be reduced. In the present embodiment, an access frequency with respect to each file for each unit period within the accessed data acquisition period is obtained, and a similarity degree between the access frequency for every unit period of a certain file and the access frequency for every unit period of another file is obtained. Therefore, it is possible to perform grouping that does not depend much on chance. Also from this point of view, it is possible to say that accessing time can be reduced. For example, in the present embodiment, even though a second file is accessed after the particular file is accessed, the second file not being relevant to a particular file, it is possible to avoid this second file being treated as belonging to the same group. Even if there is a second file highly relevant to the particular file, and this second file is not accessed after the particular file is accessed, just before the placement of the file, it is possible to treat this second file as belonging to the same group. Especially, in the present embodiment, a concept of grouping degree Y(t) is introduced and chance is eliminated as much as possible in the course of obtaining the similarity degree as an index of the grouping. Therefore, very relevant grouping can be performed.
In the description above, relocation of a file from the low-speed storage unit 31 to the high-speed storage unit 30 has been explained in detail. However, if only the relocation from the low-speed storage unit 31 to the high-speed storage unit 30 is performed, relocation of files in the high-speed storage unit 30 may become no longer possible at some future time. Therefore, it is required in the present embodiment that the relocation of a file from the high-speed storage unit 30 to the low-speed storage unit 31 be performed when a certain condition is satisfied. In order to implement the above processing, a method may be considered in which, for example, for a file having an access frequency equal to or less than a predetermined value, within the accessed data acquisition period, after the end of the file grouping process (S70) as shown in
Here, a load on the storage system relating to the relocation process as described above will be studied.
If the total number of files is assumed as N, and the number of unit periods included in the accessed data acquisition period is assumed as S, the number of arithmetical operations required for the processing is roughly calculated as follows.
In order to calculate C(a), the number of access times is counted for each unit period. Therefore, it is necessary that the number of arithmetical operation times be S. Since C(a, t) is divided by C(a) in order to calculate X(a, t), it is necessary that the number of arithmetical operation times be (S+1). In order to calculate X(a, t) of all the files for all the unit periods within the accessed data acquisition period, this calculation is performed for all the files and for all the unit periods. Therefore, it is necessary that the number of arithmetical operation times be (S+1)×S×N.
Since the information concerning whether a file is accessed or not is counted for each file, in order to calculate N(t), it is necessary that the number of arithmetical operation times be N. Since N(t) is divided by N in order to calculate Y(t), it is necessary that the number of arithmetical operation times be (N+1). In order to calculate Y(t) for all the unit periods within the accessed data acquisition period, this calculation is performed for all the unit periods. Therefore, it is necessary that the number of arithmetical operation times be (N+1)×S.
In order to calculate W(a, t) of all the files for all the unit periods within the accessed data acquisition period, X(a, t) is required to be divided by Y(t) for all the files and for all the unit periods. Therefore, it is necessary that the number of arithmetical operation times be (S+1)×S×N+(N+1)×S+S×N.
In order to calculate R(a, b) of combinations of all the files, multiplication and addition for all the unit periods are performed for N×(N−1)/2 times, which is the number of combinations of all the files. Therefore, it is necessary that the number of arithmetical operation times be (S+S)×N×(N−1)/2.
In order to reduce the number of calculation times, the similarity degree may be calculated only for an accessed file after the file access is performed, as described in the following second embodiment, without calculating in advance the similarity degree of each combination of all the files. In this case, to calculate the similarity degree of the file being accessed, the multiplication and addition for all the unit periods are required to be performed for (N−1) times, which is the number of files excluding the accessed file. Therefore, it is necessary that the number of arithmetical operation times be (S+S)×(N−1).
In the meantime, it is known that a Pentium (trademark)/2.8 GHz, which is a generally available CPU, has a computing performance of around 5.6 GFLOPS.
Here, it is assumed that by using a computer of 10 GFLOPS, the processing as described above is carried out under the condition that a total number of files is “one million”, designated acquisition period is “one year”, and an acquisition unit period is “one day”. In this case, a rough calculation as follows can be done: According to the above description, the computing number of times required for obtaining W(a, t) of all the files is equal to 134×109, and the time for computing is equal to 13.4 seconds. When the similarity degree of each combination of all the files is calculated in advance, the required number of computations is equal to 365067×109, and the time for computing is equal to 36506.7 seconds≈10 hours. When only the similarity degree is obtained as to an accessed file after the file accessing is performed, the required number of computations is equal to 0.73×109, and the time for computation is equal to 0.073 seconds.
According to the results as described above, the relocation method of the present embodiment is considered to be sufficiently feasible.
The second embodiment of the storage system according to the present invention will be explained with reference to
The storage system according to the present embodiment has a functional configuration that is basically the same as the functional configuration of the first embodiment as shown in
Specifically as shown in
The similarity degree calculation process (S60a) of the present embodiment does not need to obtain a similarity degree between each file and the other file for all the files. Therefore, as shown in the flowchart of
As thus described, unlike the first embodiment, in the present embodiment, the similarity degree calculating process (S60) and the grouping process (S70) are not carried out for all the files, and the similarity degree calculation process (S60a) and the grouping process (S70a) are carried out only for the file that has been accessed. Therefore, it is possible to reduce the load placed by the relocation process in the storage system.
Various Modifications
In the above examples, when a similarity degree between the files is obtained, an access degree and a calibrated access degree of this file are obtained according to the access frequency of the file, and by using this calibrated access degree, a similarity degree is obtained. However, there is another method that is capable of directly obtaining the similarity degree based on the access frequency, without acquiring the access degree and the calibrated access degree. By way of example, when a similarity degree between a file “1” and a file “2” is obtained, the access frequency of the file “1” and the access frequency of the file “2” are multiplied with respect to each unit period, and a value obtained by calculating the sum of those multiplied values may be assumed as the similarity degree between the file “1” and the file “2”. In other words, the similarity degree R(1, 2) between the file “1” and the file “2” may be obtained as in the following.
Similarity degree R(1, 2)=0×1(for October 1)+0×2(for October 2)+3×1(for October 3)+5×0(for October 4)+2×0(for October 5)+1×0(for October 6)+0×0(for October 7)=3
In the case above, the file similarity degree table is as shown in
If the similarity degree is obtained as thus described, the calculation becomes extremely simple. Therefore, a load placed by the data relocation process in the storage system can be reduced. However, in this modified example, the concept of grouping degree to eliminate chance as much as possible is not utilized. Therefore, there may be more possibility of chance in the grouping, relative to the first embodiment.
However, there may be cases in which too large a number of files exist, which there have a high possibility of being accessed next after the file a is accessed, and relocation of those files to the high-speed memory 30 is difficult. There are some countermeasures to deal with the situation above, such as a method to change grouping threshold, a method to select from the upper rank of access relevance degree, by the number of files or by the total size of file, and a method to relocate only a write file that has a relatively long access time, out of two accessing types, reading and writing.
Among the above-mentioned methods, the method to relocate only the write file requires adding information as an access attribute indicating whether or not the access is for reading or writing, to the access date and time table 22 shown in
Ww(b, t)=Xw(b, t)·Y(t)=[Cw(b, t)/C(b)]·Y(t)
Xw(b, t): Write access degree during unit period t
Cw(b, t): Write access frequency to file b during unit period t
In other words, the calibrated write access degree Ww(b, t) of the file b is obtained by multiplying the write access degree Xw(b, t) to the file b, by the grouping degree Y(t) during the unit period t.
In the above example, relocation using the file as a unit has been explained. Besides the file unit, relocation may be performed for any data unit, as long as the data unit allows the acquisition of access frequency and transferring between storage units, such as mail, a folder, a volume, a table of database, a record, or a field.
In the above example, two types of storage units 30 and 31, high-speed access and low-speed access speed, are provided. However, the present invention is not limited to this configuration, and it is applicable to cases where three or more types of access speeds are available. For such cases, if there is a relationship between the file a and the file b, and the file a is accessed, it is possible to consider a method that relocates the file b to the highest speed storage unit. As an alternative method, for example, two thresholds are set as thresholds of similarity degree for grouping. A combination of files having a similarity degree equal to or more than the first threshold (>second threshold) is configured as belonging to a group of high relevancy. A combination of files having the similarity degree less than the first threshold and equal to or more than the second threshold is configured as belonging to a group of medium relevancy. When one file in the group of high relevancy is accessed, the file within this group may be placed in the highest speed storage unit. When one file in the group of medium relevancy is accessed, the file within this group may be placed in the storage unit having the second higher speed.
| Patent | Priority | Assignee | Title |
| 8495013, | Dec 24 2010 | KT Corporation | Distributed storage system and method for storing objects based on locations |
| 8775870, | Dec 22 2010 | KT Corporation | Method and apparatus for recovering errors in a storage system |
| 8849756, | Apr 13 2011 | KT Corporation | Selecting data nodes in distributed storage system |
| 9052962, | Mar 31 2011 | KT CORPORTION | Distributed storage of data in a cloud storage system |
| 9158460, | Apr 25 2011 | KT Corporation | Selecting data nodes using multiple storage policies in cloud storage system |
| 9710474, | May 16 2008 | International Business Machines Corporation | Method and system for file relocation |
| 9888062, | Dec 24 2010 | KT Corporation | Distributed storage system including a plurality of proxy servers and method for managing objects |
| Patent | Priority | Assignee | Title |
| 5893139, | Jul 31 1995 | Kabushiki Kaisha Toshiba | Data storage device and storage method in which algorithms are provided for calculating access frequencies of data |
| 6032224, | Dec 03 1996 | EMC IP HOLDING COMPANY LLC | Hierarchical performance system for managing a plurality of storage units with different access speeds |
| 6640285, | Oct 26 2000 | EMC IP HOLDING COMPANY LLC | Method and apparatus for improving the efficiency of cache memories using stored activity measures |
| 6691136, | Sep 28 2000 | FAIR, ISSAC AND CORPORATION; Fair Isaac Corporation | Fast data retrieval based upon contiguous consolidation of records according to frequency of access |
| 6785767, | Dec 26 2000 | SK HYNIX NAND PRODUCT SOLUTIONS CORP | Hybrid mass storage system and method with two different types of storage medium |
| 6842876, | Apr 14 1998 | MAJANDRO LLC | Document cache replacement policy for automatically generating groups of documents based on similarity of content |
| 6948042, | Jan 21 2002 | Hitachi, LTD | Hierarchical storage apparatus and control apparatus thereof |
| 7356660, | Mar 27 2003 | Hitachi, Ltd. | Storage device |
| 7506012, | Feb 14 2005 | Hitachi, Ltd. | Method and computer system for allocating database data based on access frequency |
| 20050273638, | |||
| JP5012077, | |||
| JP8263335, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Apr 16 2007 | TAJIMA, SACHIE | Hitachi, LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 019307 | /0427 | |
| Apr 16 2007 | UEDA, RYOICHI | Hitachi, LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 019307 | /0427 | |
| May 07 2007 | Hitachi, Ltd. | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Dec 13 2011 | ASPN: Payor Number Assigned. |
| May 14 2014 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
| Jul 30 2018 | REM: Maintenance Fee Reminder Mailed. |
| Jan 21 2019 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
| Date | Maintenance Schedule |
| Dec 14 2013 | 4 years fee payment window open |
| Jun 14 2014 | 6 months grace period start (w surcharge) |
| Dec 14 2014 | patent expiry (for year 4) |
| Dec 14 2016 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Dec 14 2017 | 8 years fee payment window open |
| Jun 14 2018 | 6 months grace period start (w surcharge) |
| Dec 14 2018 | patent expiry (for year 8) |
| Dec 14 2020 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Dec 14 2021 | 12 years fee payment window open |
| Jun 14 2022 | 6 months grace period start (w surcharge) |
| Dec 14 2022 | patent expiry (for year 12) |
| Dec 14 2024 | 2 years to revive unintentionally abandoned end. (for year 12) |