A gateway system for distributing requests from multiple terminals to multiple first servers includes a second server device and a name resolution device wherein: an address generating function for generating an ip address from a character string is distributed to the name resolution device, and multiple specified ip addresses are registered in the second server devices; when the name resolution device receives a first request for resolving a name, the name resolution device extracts the access destination url from the first request, generates an ip address from the character string of the host name of the extracted url by utilizing the address generating function, and sends the generated ip address to the terminal; and when the terminal sends a second request setting the access destination of the url for the first server device, to the second server device where the ip address is registered, the second server device transfers that transmitted second request to the first server device based on the url of the access destination.
|
1. A gateway system to distribute requests from multiple terminals to multiple first server devices, comprising:
a plurality of second server devices;
a name resolution device that generates an ip address from a character string; and
a management device,
wherein address generating functions, which are hash functions that generate a hash value from the character string, generate the ip address from the character string are distributed to the name resolution device,
wherein a plurality of specified ip addresses are registered in each of the second server devices,
wherein, when a first request for resolving a name of an access destination url is received, the name resolution device:
extracts the access destination url from the received first request,
generates the ip address from the character string of a host name of the extracted url by utilizing the hash functions, and
sends the generated ip address to a terminal as a reply to the first request, and
wherein, when the terminal sends a second request setting the access destination of the url for one of the first server devices to one of the second server devices where the ip address sent as a reply to the first request is registered, the one second server device transfers the second request to the one first server device based on the access destination url,
wherein the management device:
distributes the address generating functions to the name resolution device,
divides all ip addresses generated by the name resolution device into a plurality of ip addresses, and
distributes the plurality of ip addresses to each of the second server devices, and
wherein the second server devices register the plurality of distributed ip addresses.
7. A control method implemented on a gateway system that distributes requests from a plurality of terminals to a plurality of first server devices; the gateway system including a plurality of second server devices, a management device, and a name resolution device,
an address generating function, which is a hash function that generates a hash value from a character string of an extracted url host name, to generate an ip address from the character string, which is distributed into the name resolution device, and
a plurality of specified ip addresses being registered into one of the second server devices,
the control method comprising:
the name resolution device extracting an access destination url from a received first request when the first request for resolving the name of an access destination url is received from one of the terminals;
the name resolution device generating the ip address from the character string of the extracted url host name by using the hash function;
the name resolution device sending the generated ip address to the one terminal as a reply to the first request;
the one second server device transferring a second request that was sent to one of the first server devices based on the access destination url, when the one terminal sends the second request setting the access destination url of the one first server device to the one second server device that registered the ip address that was sent;
the management device distributing an address resolution function to the name resolution device;
the management device dividing up all ip addresses generated by the name resolution device into a plurality of ip addresses;
the management device distributing all the plurality of ip addresses to each of the second server devices; and
the second server devices registering the plurality of distributed ip addresses.
5. A gateway system to distribute requests from a plurality of terminals to a plurality of first server devices, comprising:
a plurality of second server devices; and
a name resolution device,
wherein a plurality of specified ip addresses are registered in each of the second server devices,
wherein hash functions are distributed for generating a hash value from a character string,
wherein each of the second server devices:
extracts an access destination url from a received second request, after receiving the second request from one of the terminals for acquiring contents from one of the first server devices,
decides whether or not a hash value is contained in the extracted url,
deletes the hash value from the extracted url if the extracted url contains a hash value,
transfers the second request setting the url whose hash value was deleted, as an access destination to the one first server device,
generates a hash value from a character string of the extracted url by utilizing the distributed hash functions if the extracted url contains no hash value,
adds the generated hash value to a host name of the extracted url, and
instructs the one terminal to resend the second request setting the url containing the hash value as the access destination, and
wherein the name resolution device extracts the access destination url from the first request, when the one terminal sends a first request for resolving a name of the url containing the hash value to the name resolution device,
extracts the hash value from the extracted url,
combines a specified prefix with the extracted hash value to generate an ip address, and
sends the generated ip address to the one terminal as a reply to the first request, and
wherein the second request that was sent by the one terminal for setting the url containing the hash value as the access destination, is sent to one of the second server devices for registering the ip address sent as a reply to the first request.
2. The gateway system according to
wherein, after acquiring contents from the one first server device based on the second request for acquiring contents from the one first server device,
the one second server device:
links to the url where the contents are stored, and then stores the acquired contents, and
after receiving the second request sent from the terminal, decides whether or not to store the contents requested from the terminal, based on the url extracted from the second request that was received,
if the contents requested from the terminal are not stored, transfers the received second request to the one first server device,
acquires the contents requested from the one first server device, and
if the contents requested from the terminal are stored, transfers the stored contents to the terminal.
4. The gateway system according to
an address converter device,
wherein a plurality of specified ip addresses are registered in the address converter device,
wherein a plurality of specified key ip addresses are registered in the one second server device, and
wherein, when the terminal has sent a second request to the address converter device where the ip address that was sent is registered, the address converter device transfers the second request to the one second server device based on a corresponding relation between the plurality of registered specified ip addresses and a key ip address of the one second server device.
6. The gateway system according to
acquires the contents from the one first server device based on the second request,
generates a hash value from a character string of a link destination url by using the distributed hash functions when the acquired contents contain the link destination url,
adds the generated hash value to a host name of the link destination url,
stores the acquired contents and the link destination url, and
sends the acquired contents and the link destination url containing the hash value, to the one terminal as a reply to the second request.
8. The control method for the gateway system including the management device according to
the management device distributing hash functions for generating a hash value from the character string of the extracted url, to the one second server device;
the one second server device extracting the access destination url from the received second request, after receiving the second request sent from the one terminal;
the one second server device deciding whether or not hash value is contained in the extracted url;
the one second server device deleting the hash value from the extracted url if the extracted url contains the hash value;
the one second server device utilizing the distributed hash function to generate a hash value from the character string of the extracted url, when the extracted url contains no hash value;
the one second server device adding the generated hash value to the host name of the extracted url; and
the one second server device instructing the one terminal to resend the second request setting the url containing the hash value as the access destination.
9. The control method for the gateway system according to
the one second server device deciding after acquiring contents from the one first server device, whether or not the acquired contents contain a link destination url based on the second request for acquiring contents from the one first server device;
the one second server device generating a hash value from a character string of the link destination url by using a hash function, when the acquired contents contain the link destination url;
the one second server device adding the generated hash value to a host name of the link destination url; and
the one second server device storing the acquired contents and the link destination url containing the hash value, linking to the extracted url whose hash value was deleted.
10. The control method for the gateway system according to
the one second server device deciding whether the contents acquired from the one terminal are stored or not, based on the extracted url whose hash value was deleted, after the step for deleting the hash value;
the one second server device transferring the second request for setting the extracted url whose hash value was deleted as the access destination, to the one first server device, when the requested contents were not stored; and
the one second server device sending the stored contents and the link destination url containing the hash value to the one terminal when the requested contents were stored.
|
|||||||||||||||||||||||||||||||||
The present application claims priority from Japanese patent application JP 2009-185624 filed on Aug. 10, 2009, the content of which is hereby incorporated by reference into this application.
The present invention relates to technology for load balancing in gateway systems for client-server type network services.
Traffic processed by carrier gateways is vastly increasing due to boosted terminal performance and access networks that work at ever higher speeds. Carriers therefore attempt to increase the number of traffic lines to the Internet and to reduce traffic at the gateway by using information processing technology.
The content cache is one type of information processing technology for reducing traffic. In this technology, when a terminal has requested contents from a server device (such as a Web server) over the Internet, a gateway server device (such as a Web proxy) temporarily stores the contents acquired from the Web server; and when another terminal requests contents, the Web proxy acting as the Web server, sends back a reply to the stored contents.
The gateway can in this way reduce traffic with the Web servers. The gateway can also speed up the content acquisition process.
The gateway can accommodate a numerous terminals and may include multiple server devices for storing contents in large quantities. The gateway distributes the contents to these server devices. Distributing the contents to these multiple server devices is an extremely effective method when the server devices utilized by the gateway have only limited performance.
A load balancer device is usually installed for example between the terminal and the gateway server device in order to distribute the processing among multiple server devices. The load balancer device transfers the request (such as a content acquisition request) from a terminal to each server device. The process for distributing the load may vary depending on to what extent the request from the terminal is analyzed.
Load balancer devices can process a large volume of traffic if for example distributing loads based on low layer information such as the IP address of the transmit source. However the information needed for calculating the transfer destination from the terminal is minimal so the load balancer device might prove unable to correctly distribute the load.
The load balancer device can also distribute loads appropriately according to each application's characteristics when distributing loads based on information such as layer 7 information including HTTP requests. However, the load of the load balancer device itself becomes high so that the load balancer device might prove incapable of managing numerous servers.
The following load balancing technology capable of resolving the aforementioned problems have been proposed.
A technology was proposed for distributing requests to each server device based on a table for determining the transmit source (See for example, JP-A-Hei11(1999)-004261. The technology disclosed in JP-A-Hei11(1999)-004261 allows terminals to distribute requests to each server by utilizing a table distributed to the terminal for determining which server to transmit the request to. Moreover, the reply to the request from the terminal includes table update information and so also reduces the load imposed by distributing the table.
Another technology was also proposed for distributing server requests from terminals to each gateway based on the hash values generated from each server request (See for example, JP-T-2002-523838). In the technology disclosed in JP-T-2002-523838, the hash value generated from the request is utilized as the gateway index so requests with the same content (such as content acquisition requests for the same host) from each terminal are sent to the same gateway.
Still another technology was proposed for calculating the hash value based on contents requested from the terminal, and selecting an appropriate server device as the transfer destination for the request based on the hash value calculated by the load balancer device (See for example, JP-A-2004-030690). In the technology disclosed in JP-A-2004-030690, the load balancer device is capable of distributing the processing to each server device since the load balancer device transfers requests by utilizing a hash value calculated based on the content in the request from the terminal.
In the technology disclosed in JP-A-2004-030690, the load balancer device must calculate a hash value which increases the processing load on the load balancer device so the load balancer device might become a traffic bottleneck. In other words, the method utilizing a load balancer device concentrates the load in the load balancer device itself so that the load balancer device might become a single point of failure. The technology disclosed in JP-A-Hei11(1999)-004261 and JP-T-2002-523838 on the other hand does not require a load balancer device.
In the technology disclosed in JP-A-Hei11(1999)-004261 however, each terminal manages the transfer destination information (table) in the request. The problem therefore occurs that when the applicable server device (Web server) was changed, the table must then be updated which increases the load. Table search (retrieval) functions and update functions must also be added to the terminal. The patent document JP-A-Hei11(1999)-004261 therefore has the problem that high costs are required for adding new technology to systems already in use.
The technology disclosed in JP-T-2002-523838 has the same problem as disclosed for JP-A-Hei11(1999)-004261 in that table search functions and update functions must also be added to the terminal. In cases where the hash value is calculated by the terminal then a function must be added at the terminal to calculate the hash value.
In view of the above problems with the related art, the present invention has the object of providing a gateway system capable of distributing the load to multiple gateways without installing complicated load balancer devices at the stage prior to the gateway and without having to add complicated functions to the terminal. More specifically, the gateway system of this invention is capable of generating IP addresses based on values for identifying the request contents such as hash values generated from the destination host name or the URL, and then appropriately distributing content access to each gateway.
Representative aspects of the present invention are described next. Namely, a gateway system for distributing requests from multiple terminals to multiple first server devices includes multiple second server devices and a name resolution device, in which: unique aspects are address generating functions for generating an IP address from a character string distributed to the name resolution device, and multiple specified IP addresses that are registered in each of the second server devices; and when a first request for resolving the name of the access destination URL sent from the terminal is received, the name resolution device extracts the access destination URL from the first request that was received, generates an IP address from the character string of the host name of the extracted URL by using the address generating function, and sends the generated IP address to the terminal as the response to the first request; and when the terminal sends a second request setting the access destination of the URL for the first server device to the second server device where the IP address sent as the reply to the first request is registered, the second server device forwards that transmitted second request to the first server device based on the URL of the access destination.
The gateway system of the embodiments of this invention can distribute the load uniformly without using a load balancer device because the name resolution device in the gateway system generates an IP address for server devices based on hash values.
A brief summary of the present invention is described next.
The gateway system according to an exemplary embodiment of this invention includes multiple server devices (Web proxies), name resolver devices (DNS proxies) and management devices (management servers) and can be applied for example to content cache systems, etc. This invention is not limited to systems for caching contents but may also be applied for example to systems for distributing load between server devices.
The range of the assigned IP address is allotted to multiple Web proxies. The DNS proxy resolves the name by using address generating functions. Specifically, after receiving a request from a terminal, the DNS proxy calculates the hash value from the identifier in the request (such as a host name for the content acquisition destination), generates an IP address based on that calculated hash value, and sends a reply back the generated IP address to the terminal.
Based on the reply IP address, the terminal sends a request to the Web proxy handling that IP address, so the gateway system can distribute the load without having to use a load balancer device. The terminal can in this way acquire the transmit destination of the request by utilizing the DNS proxy processing. The terminal therefore does not require any new functions to utilize this gateway system. Moreover, the management server allots all IP addresses generated by the DNS proxy to a range covered by multiple IP addresses, and assigns that allotted IP address range to each of the Web proxies. The DNS proxy therefore does not have to retain information (assigned IP address table) showing the IP address range where each of the proxies are assigned.
When accessing the host (Web server) of that content acquisition address, the terminal also asks the DNS proxy about the IP address matching that host name. The DNS proxy sends back to the terminal an IP address generated based on the address generating function. The terminal then sends content acquisition requests and so on using that IP address as the transmit destination. This request then reaches one of the gateway's Web proxies by following the usual IP routing. The Web proxies of that gateway then cache the contents of the host for that assigned IP address.
The Web proxy may take advantage of its other functions to utilize information (path names) other than host name of the URL serving as the content acquisition address, as information sources for appropriately distributing the load.
More specifically, when a content acquisition request is received from the terminal, the Web proxy calculates the hash value from the full URL content of the acquisition destination shown in the content acquisition request, and adds that calculated hash value to the URL as the host section suffix. The Web proxy then returns the URL with the added hash value to the terminal.
The Web proxy acquires contents from the Web server based on the content acquisition request. If the acquired contents include a URL as a link destination, then the Web proxy calculates the hash value from that full URL and attaches that calculated hash value to the applicable URL. The Web proxy sends back the URL with attached hash value to the terminal as a reply to the content acquisition request. So if a hash value was generated in this way from information (full URL for the content acquisition address) contained in the request, then requests destined for the same Web server (content acquisition request with same host) can be distributed to Web proxies by user and by content.
The first embodiment of this invention is described next while referring to the drawings.
The gateway system of the present embodiment contains a DNS proxy 104, multiple Web proxies (Web proxy 1_105-1, Web proxy 2_105-2, . . . ) and a management server 108. Each device mutually connects via a local network 106.
The DNS proxy 104 and the multiple Web proxies (Web proxy 1_105-1 and others) connect by way of the wide area network 103 to the DNS server 101, multiple Web servers (Web server 1_102-1, Web server 2_102-2, . . . ) to provide services to the multiple terminals (terminal 107-1, terminal 2_107-2, . . . ) connected to the local network 106. For example, request messages from the terminal 107_1 to the Web server 1_102-1 and reply messages from Web server 1_102-1 to the terminal 107_1 are sent and received via any of the Web proxies.
If jointly describing all or any of multiple Web proxies, then Web proxy 105 is utilized as a general name for the multiple Web proxies in this description. Moreover if jointly describing any or all of multiple Web servers, then Web server 102 is utilized as a general term for multiple Web servers. If jointly describing any of multiple terminals then terminal 107 is utilized as a general term for multiple terminals in this description.
In the example for
The DNS server 101 utilizes a DNS protocol to execute normal name resolution tasks. The Web server 102 connected to the wide area network provides contents according to requests from the terminal 107. The term contents as used here, refers for example to information such as moving images, still images, text, and audio, etc.
The DNS proxy 104 executes name resolution processing when the terminal 107 sends a name resolution request. More specifically, when a host name shown in a name resolution request matches the host name specified by the management server 108, the DNS proxy 104 generates an IP address, and sends that generated IP address to the terminal 107 as a reply to the name resolution request.
The host name as referred to here is a character string denoting the name of the server device and for example is the “www.example.org” section in the URL “http://www.example.org/ex/1”. The IP address generated by the DNS proxy 104 here is an IP address utilized only in the gateway system of the present embodiment.
The DNS proxy 104 more specifically calculates the hash value from the host name using a pre-issued address generating function, and generates an IP address based on that calculated hash value. The address generating function is described in detail later on using
Based on the IP address that was sent, the terminal 107 transfers the content acquisition request to the Web proxy 105 assigned to that IP address.
If the host name does not match the host name specified by the management server 108, then the DNS proxy 104 transfers (or forwards) the name resolution request to the DNS server 101. The DNS server 101 sends the actual IP address of the Web server 102 corresponding to the host name to the terminal 107 in reply to the name resolution request.
The Web proxy 105 acquires the requested contents from the Web server 102 based on the content acquisition request sent from the terminal 107, and caches the acquired contents. The Web proxy 105 then sends the acquired contents to the terminal 107. A pre-assigned IP address range was allotted to the Web proxy 105 by the management server 108 and so the content acquisition request is sent to the assigned Web proxy 105 according to the IP address.
The management server 108 controls the IP addresses allotted to the Web proxies 105. The management server 108 specifies host names that were converted to IP addresses by the DNS proxy. The Web proxy 105 caches only the contents in Web server 102 that correspond to the specified host name.
The management server 108 contains an address assignment table 200 (See
The address assignment table 200 is a table showing the IP address range assigned to each Web proxy 105. The address assignment table 200 includes a server ID201, a start address 202 and an end address 203.
The server ID 201 is an identifier for identifying the server device (Web proxy 105). The start address 202 is a start address within the address range allotted to the Web proxy 105 specified by the server ID 201. The end address 203 is an end address within the address range. The IP address range assigned to a certain Web proxy 105 here need not be one continuous range and may be divided into a number of ranges provided that range does not overlap onto IP address ranges assigned to other Web proxies 105.
The management server 108 may utilize automatic settings to uniformly divide a pre-established address range and allot these uniformly subdivided address ranges to the Web proxy 105. Moreover, the administrator may set the address range assigned to each Web proxy according to the gateway system load status.
The address generating function 210 is a function for generating IP addresses from variable length data X such as character strings, and includes the hash function 215. When variable length data X is input, the address generating function 210 utilizes the hash function 215 to convert that variable length data X that was input for example into 64 bit hash values. The function then generates an IP address in IPv6 (Internet Protocol version 6) by joining the specified 64 bit IPv6 prefix with the converted 64 bit hash value, and outputs that generated IP address.
An algorithm such as SHA1 (Secure Hash Algorithm 1) may for example be utilized for the hash function. Besides SHA1, algorithms such as MD5 (Message Digest Algorithm 5) may be utilized.
In this embodiment, the host name (such as “www.example.org”) for the Web server 102 may be utilized for the variable length data X input to the address generating function 210.
The domain name table 220 is utilized to decide whether or not to convert the host name specified in the name resolution request to an IP address in the name resolution processing. This domain name table 220 includes for example a domain name (host name) 221 and conversion item 222.
The domain name (host name) 221 is a host name for Web server 102 that provides contents to the terminal 107. The conversion item 222 stores the information showing whether or not to convert the domain name (host name) shown in domain name 221 to an IP address.
The DNS proxy 104 executes the name resolution processing by using the address generating function 210 and the domain name table 220. The name resolution processing executed by the DNS proxy 104 is described in detail later on using
The management server 108 distributes the address generating function 210 and the domain name table 220 to the DNS proxy 104. The management table 108 distributes the address generating function 210 to the Web proxies 105. The management server 108 makes an assigned IP address table 600 (See
The DNS proxy 104 contains an address conversion processor unit 301, a memory 302, and a network interface 303. The DNS proxy 104 connects to the wide area network 103 and the local network 106.
The memory 302 stores the domain name table 220 and the address generating function 210 distributed from the management server 108.
The address conversion processor unit 301 performs processing such as address conversion using the address generating function 210 and the domain name table 220 stored in the memory 302. The processing executed by the address conversion processor unit 301 is described in detail later on using
The network interface 303 is an interface for connecting the DNS proxy 104 to the wide area network 103 and the local network 106.
In this process, first of all the address conversion processor unit 301 in the DNS proxy 104 receives a name resolution request (such as “AAAA? www.examle.org”) that was sent from the terminal 107 (S401). After receiving the name resolution request, the address conversion processor unit 301 searches the domain name table 220, and decides whether or not to convert the host name (such as “www.example.org”) shown in the name resolution request to an IP address (S402).
If decided in S402 not to convert the host name, then the DNS proxy 104 transfers the name resolution request sent from the terminal 107 to the DNS server 101 by way of the wide area network 103.
The DNS server 101 that provides a domain name service, executes the usual name resolution, and sends the actual IP address corresponding to the host name to the requesting party which is the DNS proxy 104, as a reply to the name resolution request. The DNS proxy 104 receives a reply to the name resolution request sent from the DNS server 101, and acquires the actual IP address corresponding to the host name (S403).
However, when decided in S402 that the host name can be converted, the address conversion processor unit 301 in the DNS proxy 104 utilizes the address generating function 210 to calculate the hash value (such as, “41de:3297:ec18:5fae”) from the host name (for example, “www.example.org”) shown by the name resolution request, and combines the calculated hash value with the specified IPv6 prefix (such as, “2001:db8::”) to generate the IP address (for example, “2001:db8::41de:3297:ec18:5fae”) (S404). Here, the generated IP address is the IP address utilized in the gateway system of the present embodiment, and corresponds to the Web proxy 105 that caches the Web server 102 contents.
Next, the address conversion processor unit 301 sends the actual IP address acquired from the DNS server 101 or the IP address generated by the address generating function 210 to the terminal 107 as a reply (for example, “AAAA 2001:db8::41de:3297:ec18:5fae”) to the name resolution request (S405).
The Web proxy 105 contains a Web cache process function unit 501, a memory 502, and a network interface 503. The Web proxy 105 connects to the wide area network 103 and the local network 106.
The memory 502 contains the assigned IP address table 600 and the contents table 610, and may also contain an address generating function 210 if needed.
The assigned IP address table 600 is described in detail later on using
The Web cache process function unit 501 utilizes the address generating function 210, the assigned IP address table 600, and the content table 610 stored in the memory 502 to acquire the requested contents from the Web server 102 based on the content acquisition request sent from the terminal 107, and to cache those acquired contents. The processing executed by the Web cache process function unit 501 is described in detail later on using
The network interface 503 is an interface for connecting the Web proxy 105 to the wide area network 103 and the local network 106. The Web proxy 105 registers the IP address set in the assigned IP address table 600 as an IP address for its own device into the network interface 503. The Web proxy 105 can in this way receive content acquisition requests sent from the terminal 107.
The Web proxy 105 in this embodiment includes a Web cache processing function. However the Web proxy 105 may contain for example a filter function or a content conversion function instead of the Web cache processing function. The gateway system of this embodiment can in this way also be applied not only to cache systems but also to systems providing other services.
The assigned IP address table 600 is a table for describing the range of the assigned IP address table 600 managed by one Web proxy 105. The assigned IP address table 600 includes a start address 601 and an end address 602.
The management server 108 may distribute the assigned IP address table 600 or the table 600 may be formed by the Web proxy 105 based on the IP address range notified by the management server 108.
The Web proxy 105 registers all IP addresses to which the applicable Web proxy is assigned into the network interface 503, and receives content acquisition requests having a registered IP address as their destination. The content acquisition processing executed by the Web proxy 105 is described later on using
After receiving an address resolution request for an IP address it manages, the Web proxy 105 informs the terminal 107 and adjacent routers of the MAC (Media Access Control) address set in its own device. The address resolution request as referred to here, is for example an ARP (Address Resolution Protocol) when using IPv4, and is a ND (Neighbor Discovery) protocol when using IPv6.
The content table 610 is a table for storing contents cached by the Web proxy 105. The content table 610 for example stores the URL611, destination IP address 612, content 613, and metadata 614.
The URL611 is the URL for the content acquisition destination. The destination IP address 612 is the destination IP address shown in the content acquisition request, and is the IP address generated from the host name of Web server 102 serving as the content acquisition destination by the DNS proxy 104.
The content 613 contains the contents acquired by the Web proxy 105 from the Web server 102. The metadata 614 is for example the acquired content size, date and time, and storage period, etc.
In this process, the Web proxy 105 first of all receives the content acquisition information request that was sent from the terminal 107 (S701). Next, the Web cache process function unit 501 in the Web proxy 105 searches the content table using the URL of the Web server 102 which is the content acquisition destination, as the search key (S702), and decides whether or not there is an entry matching the search key (S703).
When the Web proxy decides in S703 that there is an entry matching the search key, those requested contents are in a cache, so the Web cache process function unit 501 acquires the contents stored in the content 613 of the entry that matches the search key, and forms a reply to the content acquisition request that includes the requested contents (S704).
However, when the Web proxy 105 decides in S703 that there is no entry matching the search key, then the requested contents are not cached so the Web cache process function unit 501 transfers the content acquisition request sent from the terminal 107 to the Web server 102 (origin server) storing the applicable contents (S705).
The Web proxy 105 in this case executes the usual name resolution in order to acquire the IP address for Web server 102 which is the content acquisition destination. More specifically, the Web proxy 105 sends a name resolution request to the DNS server 101, and receives an actual IP address corresponding to the host name of Web server 102 (which is the content acquisition destination) as a reply from the DNS server 101 to the name resolution request. The Web proxy 105 can in this way transfer the content acquisition request to the Web server 102.
The Web proxy 105 adds the new entry to the content table 610, and respectively stores the URL and destination IP address for the content acquisition destination shown via the content acquisition request sent from the terminal 107, into the newly added URL 611 and IP address 612 entries.
Next, the Web cache process function unit 501 receives a reply to the content acquisition request sent from the Web server 102 (S706). After a reply is received to the content acquisition request, the Web cache process function unit 501 acquires the contents from that reply, and stores the acquired content and content information respectively in the new entries for the content 613 and metadata 614 that were newly added to content table 610. The Web cache process function unit 501 forms a reply to the content acquisition request that includes the acquired content (S707).
Finally, the Web cache process function unit 501 sends a reply to the content acquisition request made in step S704 or S707 (S708) and ends the processing (S709).
In another exemplary embodiment of this embodiment, the gateway system may include an address converter device at a pre-stage of the Web proxy 105. In this case, the management server 108 for example distributes the address assignment table 200 shown in
The address converter device includes a server ID (ID for Web proxy) listed in the address assignment table 200, and an address conversion table linked to the notified proxy IP address for Web proxy 105. The address converter device in this way rewrites the destination IP address in the content acquisition request sent from the terminal 107, onto the key IP address from the IP address that was generated based on the hash value. The address converter device then transfers the content acquisition request to the Web proxy 105 corresponding to the proxy IP address.
In another version of this embodiment, after acquiring contents from the Web server 102 in step S707, the Web proxy 105 may utilize the address generating function 210 distributed from the management server 108, to generate an IP address from the host name for Web server 102, and store the generated IP address into the destination IP address 612 for the new entry that was added to the content table 610.
A gateway system including an address converter device may be utilized when assigning a vast number of IP addresses to the Web proxy 105 is not practical in terms of cost effectiveness. This gateway system balances the load based on layer 4 information and so operation is faster and less expensive than gateway systems that balance the load based on information in the layer 7.
The content table 610 formed in the above processing easily specifies contents that were cached by the web proxies 105. These formed content tables 610 can also be used in the assignment change processing shown in
The Web server 102 shown in the example in
Firstly, the management server 108 distributes the address generating function 210 and the domain name table 220 to the DNS proxy 104 (S801). The management server 108 also distributes the address generating function 210 and the address assignment table 200 (or the assigned IP address table 600) to the Web proxy 105-1 and the Web proxy 105-2 (S802, S803).
Next, after receiving the assigned IP address table 600, the Web proxy 105 registers all IP addresses contained in the assigned IP address table 600 into the network interface 503 as IP addresses assigned to its own device, (S804, S805).
Next, when for example a URL (such as “http://www.example.org/ex/1”) that was input by the user is received (S806), the terminal 107 sends a content acquisition request for accessing this URL and acquiring the contents. In this case, the terminal 107 first of all sends for example a name resolution request (such as, “AAAA? www.example.org”) to the DNS proxy 104 in order to acquire an IP address corresponding to the host name (for example, “www.example.org”) (S807).
After receiving a name resolution request sent from the terminal 107, the DNS proxy 104 searches the applicable domain name table 220 and decides whether or not the host name (for example, “www.example.org”) shown by the received solution request can be converted. If decided that the applicable host name can be converted then the DNS proxy 104 uses the address generating function 210 to generate an IP address from the applicable host name (S808).
The DNS proxy 104 next sends the generated IP address (for example, “2001:db8::41de:3297:ec18:5fae”) to the terminal 107 as a reply to the name resolution request (S809).
After receiving a reply to the name resolution request, the terminal 107 acquires the IP address corresponding to the applicable host name from the received reply, and after setting the acquired IP address in the destination address for the content acquisition request, sends the content application request (S812). The TCP connection is now established.
More specifically, the content acquisition request is routed as an IP packet on IP routers along the path according to the destination IP address, and sent to one of the Web proxies 105 assigned to the applicable IP address. In this case, devices such as the terminal 107 and the IP routers along the path search devices where the applicable IP address is registered in order to execute the usual address resolution (ARP, ND, etc.) (S810). The device (for example, Web proxy 1_105-1) that received an address resolution request addressed to itself, notifies the requesting party (IP router along path, terminal 107, etc.) of its own device MAC address (S811). The TCP connection between for example the terminal 107 and the Web proxy 1_105-1 is established in this way, and the Web proxy 1_105-1 can receive content acquisition requests that are addressed to the applicable IP address.
Next, after receiving a content acquisition request from the terminal 107, the Web proxy 1_105-1 acquires the URL (for example, “http://www.example.org/ex/1”) for the content application request from the content application request (for example, “GET/ex/1 HTTP/1.0 Host: www.example.org - - -”) (S812).
The Web proxy 1_105-1 then searches the content table 610 using the acquired URL as the search key, and decides whether or not the requested contents have been cached. If decided here that the requested contents have not been cached (S813), then the Web proxy 1_105-1 forwards the content acquisition request sent from the terminal 107 to the Web server that stored the contents (S814).
After receiving the content acquisition request, the Web server 102 sends a reply containing the contents requested in the content acquisition request to the Web proxy 1_105-1 (S815).
After receiving a reply to the content acquisition request, the Web proxy 1_105-1 acquires the contents from the reply that was received, caches the applicable acquired contents (S816), and sends them to the terminal 107 as a reply to the content acquisition request (S817).
The above process allows the Web proxies 105 to cache the contents of the Web server 102 containing the host name linked to the assigned IP address. The gateway system of this embodiment in other words ensures that contents belonging to the same host name will be cached by the same Web proxy. Correct caching in ensured because IP addresses generated from host names based on the hash values are linked to any of the Web proxies 105.
The process for changing the assigned IP address range of the Web proxy 105 in the present embodiment is described next. In this process, the management server 108 can change the assigned IP address range of the Web proxies 105, when a certain number of the Web proxy units are scheduled to stop operation or when the load is improperly balanced.
An assignment change request 900 includes a movement source Web proxy IP address 901, a movement destination Web proxy IP address 902, and a movement target IP address range 903.
To change the IP address range assigned to the Web proxies 105, the management server 108 sends the assignment change request 900 to all Web proxies 105, or to movement source and movement destination Web proxies 105.
After a Web proxy 105 receives an assignment change request 900 sent from the management server 108 (S1001), the Web proxy 105 decides whether or not any of the movement source Web proxy IP address 901 and the movement destination Web proxy IP address 902, matches the IP address assigned to its own device (S1002).
If the Web proxy 105 decides in step S1002 that the movement source Web proxy IP address 901 matches the IP address of its own device then the Web proxy 105 that received the assignment change request 900 is the movement source Web proxy, so that movement source Web proxy extracts the entry from content table 610 corresponding to the IP address specified in the movement target IP address range 903 (S1003).
The movement source Web proxy then moves that extracted entry to the content table of the movement destination Web proxy specified by the movement destination Web proxy IP address 902 (S1004).
The movement source Web proxy in this case may delete for example data from the content 613 and the metadata 614 from the extracted entry from the content 613 as needed, and send just the URL611 and destination IP address 612 data to the movement destination Web proxy. The traffic between the Web proxies 105 can in this way be reduced in the assignment item request processing.
The movement source Web proxy next deletes entries matching the IP address specified by the movement target IP address range 903 (S1005). Moreover, the movement source Web proxy deletes the IP address range specified by the movement target IP address range 903, and deletes the applicable IP address registered in the network interface 503 (S1006).
However when the IP address 902 for the movement destination Web proxy was decided to match the IP address of the Web proxy's own device in step S1002, then that Web proxy 105 is the movement destination Web proxy so the movement destination Web proxy waits to receive movement target entries sent from the movement source Web proxy (S1007).
After receiving the movement target entries, the movement destination Web proxy adds the received entries to the content table 610 (S1008). The movement destination Web proxy also adds the IP address designated by the movement target IP address range 903 serving as the IP address movement range, to the assigned IP address table 600 (S1009). The movement destination Web proxy also stores (registers) the newly added IP address into the network interface 503.
The movement destination Web proxy also sends a neighbor advertisement packet or a Gratuitous ARP (S1010). The movement destination Web proxy can in this way notify the neighboring IP routers, terminal 107, Web proxy 105, and management server 108 where the address has been assigned. After receiving the neighbor advertisement packet or a Gratuitous ARP, the neighboring IP router, terminal 107 and so on once again send an address resolution request, and acquire the MAC address of the network interface where the new IP address was registered.
However when decided in step S1002 that the movement source Web proxy IP address 901 and the movement destination Web proxy IP address 802 do not match the IP addresses in the Web proxy's own device, then that assignment change request is not a message for its own device so the Web proxy 105 discards that message (S1011). The movement source and movement destination Web proxies can in this way correctly receive the assignment change request that was sent, even when the assignment change request was sent to all Web proxies 105 by multicast.
In the initial setting, the Web proxy 1_105-1 is assigned for example, a range of IP addresses from “2001:db8::4000:0000:0000:0000” to “2001:db8::4ff:ffff:ffff:ffff:” and caches the contents corresponding to the assigned IP address (S1101).
The Web proxy 2_105-2 is for example assigned a range of IP addresses from “2001:db8::3fff:ffff:ffff:ffff” to “2001:db8::4000:0000:0000:0000” and caches the contents corresponding to the assigned IP address (S1102).
The management server 108 next sends instruction for example to move the assigned contents of the Web proxy 1_105-1 that match the range from “2001:db8::4000:0000:0000:0000” to “2001:db8::4fff:ffff:ffff:ffff” to the Web proxy 2_105-2 to each of the Web proxies specified for movement (S1103).
After receiving instructions from the management server 108 to move the contents, the Web proxy 1_105-1, extracts the entry corresponding to the specified IP address range from the content table 610, and transfers that extracted entry to the Web proxy 2_105-2 which is the movement destination (S1104). The Web proxy 1_105-1 then deletes applicable entry from the content table 610. The Web proxy 1_105-1 then deletes the specified IP address from the assigned ID address table 600 and/or also deletes the specified IP address from the network interface 503 (S1105).
The Web proxy 2_105-2 on the other hand, adds the entry moved from the movement source Web proxy 1_105-1, to the content table 610. The Web proxy 2_105-2 then adds the range of the specified IP address to the assigned IP address table 600 or registers the added IP address range to the network interface 503 (S1106).
The Web proxy 2_105-2 next notifies the neighboring terminal 107, the IP router and the Web proxy 105 and so on of the change in the assigned IP address range by sending a neighbor advertisement packet or a Gratuitous ARP (S1107). The neighboring terminal 107, the IP router, and so on in this way resend an address resolution request addressed to the IP address shown in the neighbor advertisement packet or a Gratuitous ARP in order to acquire the MAC address corresponding to the applicable IP address.
The processing from here onwards from steps S1108 to S1116 is the same as the processing from steps S806 to S817 shown in
The gateway system in the first embodiment as described above, achieves uniform load balancing because the gateway system transfers content acquisition requests to the Web server, from the terminal to the appropriate Web proxy according to the IP address generated from the hash value. The storage capacity can also be extended on a scalable level. Moreover, no load balancer device is required so the cost of managing the gateway system can be reduced and the gateway system rendered more resistant to problems.
Further, if the load on the Web proxies becomes unbalanced, and there is a change in the IP address range assigned to the Web proxies (in other words, the cache contents were moved), then the gateway system exchanges only the information between the Web proxies, and does not exchange information between other devices such as the DNS proxy and terminals. The processing required for changing the assigned IP address can therefore be reduced.
The DNS proxy that performs name resolution does not control the Web proxy status. The DNS proxy can therefore be easily made redundant, and the gateway system can attain high reliability and high-speed operation.
The terminal utilizes the name resolution provided by the DNS proxy to access the Web proxy so there is no need to add a function to the terminal for accessing the Web proxy. The gateway system of this embodiment can therefore be easily utilized in environments where there are already numerous terminals.
The structure of the gateway system of the second embodiment is the same as the gateway structure of the first embodiment shown in
However a portion of the processing in the address converter processor unit of the DNS proxy of the second embodiment is different from the processing of the address converter processor unit of the first embodiment (S1404 and S1412 in
In the gateway system of the first embodiment, the DNS proxy 104 calculates the hash value from the host name for the content acquisition destination, and generates an IP address by using that calculated hash value. The content acquisition requests sent from the terminal 107 are therefore distributed into host name units.
Access requests sent to a Web server 102 (host) in the gateway system of the first embodiment are therefore received by a single Web proxy 105 so that if access is concentrated on the Web server 102, then load on the assigned Web proxy 105 becomes large, and an unbalanced load might possibly occur among the Web proxies 105.
Whereupon, in the gateway system of the second embodiment, the Web proxy 105 calculates the hash value from the entire URL at the content acquisition destination so that a URL embedding the calculated hash value as the host suffix is sent back to the terminal 107 as the content acquisition destination URL.
After receiving a name resolution request from the terminal 107, the DNS proxy 104 generates an IP address for the assigned Web proxy 105 from the host name containing the hash value. So content acquisition requests in the second embodiment are distributed to the Web proxies 105 based on the hash values generated from the entire URL.
In other words, the second embodiment avoids concentrating loads on a single Web proxy 105, because the contents are cached in multiple Web proxies 105 even if the contents have the same host name.
The second embodiment may be applied along with the first embodiment to cache systems, etc.
The rules for URL conversion in the second embodiment are described next.
In this embodiment, the Web proxy 105 utilizes the address generating function 210 (hash function 215) stored in the memory 502 to generate a hash value 1204 (for example, “820d92dcb08edcdd”) from the URL1201 (http://www.example.org/ex/2.html) including a host name and path name shown in the content acquisition request sent for example from the terminal 107.
The Web proxy 105 joins the section 1202 up to the URL host name (for example, http://www.example.org), and the period (.) 1205, and the hash value 1204 (for example, “820d92dcb08edcdd”), and the suffix 1206 of the host name (for example, “.hash.value”), and section 1203 from the URL host name onwards (for example, “/ex/4.html”). The URL1207 (for example, “http://www.example.org.820d92dcb08edcdd.hash.value/ex/2.html”) with an embedded hash value is generated in this way.
After receiving the content acquisition request sent from the terminal 107 (S1301), a Web proxy 105 acquires the host name for the content acquisition request from the packet header in the received content acquisition request (S1302). The Web proxy 105 next decides whether or not the suffix 1206 (for example, “.hash.value”) (or, hash value 1204) for the host name indicating a converted URL is contained in the acquired host name based on the URL conversion rules shown in
If the Web proxy 105 decides in step S1303 that the acquired host name does not contain a “.hash.value”, then the URL contained in the content acquisition request that was received is not a URL converted based on URL conversion rules shown in
The Web proxy 105 next sends back an HTTP reply for code 302 to the terminal 107 (S1305). A HTTP reply for code 302 as referred to here is an instruction the Web proxy 105 sends to the terminal 107 to resend a content acquisition request to the URL converted based on the URL conversion rules. After receiving the HTTP reply for code 302 the terminal 107 resends a content acquisition request to the URL containing the hash value shown in the received HTTP reply.
However, when the Web proxy 105 decides in step S1303 that the acquired host name does contain a “.hash.value”, then the URL contained in the received content acquisition request is an already converted URL, so the Web proxy 105 deletes the character string “.hash.value” for Suffix 1206 in the hash value 1204 and host name, from the host section of the URL acquired from the header in the packet of the received content acquisition request, and restores the URL to the URL prior to conversion (S1306).
The Web proxy 105 next searches the entries in the content table 610 using the restored URL as a search key (S1307). The Web proxy 105 decides whether or not there is an entry matching the search key (S1308).
When the Web proxy 105 decides in step S1308 that there is an entry matching the search key, the contents requested by the terminal 107 have been cached so the Web proxy 105 forms a reply to the content acquisition request that utilized the contents stored in this entry (S1309).
However when the Web proxy 105 decides in step S1308 that there is no entry matching the search key, the contents requested by the terminal 107 have not been cached so the Web proxy 105 transfers the content acquisition request to the Web server 102 (origin server) (S1310). The Web proxy 105 in this case adds the new entry to the content table 610, and stores each of the restored URL and IP addresses in the newly added entries for URL611 and the destination IP address 612.
The Web proxy 105 next receives the reply to the content acquisition request from the Web server 102, and acquires the contents contained in the reply (S1311). Next, when the acquired URL contains the link destination URL, the Web proxy 105 converts that link destination URL to a URL containing a hash value based on the URL conversion rules shown in
The Web proxy 105 next stores the acquired contents, and the URL containing the link destination hash value, into the content 613 for entries corresponding to the content table 610 (S1313).
Finally, the Web proxy 105 sends a reply to the content acquisition request to the terminal 107 (S1314) and ends the processing (S1315).
Prior to step S1401, the terminal 107 may send the name resolution request (for example, “AAAA? www.example.org”) to the DNS proxy 104 by way of step S807 as shown in
Next, after receiving the content acquisition request sent from the terminal 107, the Web proxy 1_105-1 acquires the URL for the content acquisition destination from the header of the received content acquisition request, and based on the URL conversion rules shown in
The DNS proxy 104 name resolution and address resolution processing are described next (S1404).
After receiving the HTTP reply for code 302, the terminal 107 once again sends a name resolution request to the DNS proxy 104 in order to acquire an IP address corresponding to a host name with a hash value (such as “www.example.org.820d92dcb08edcdd.hash.value”) (S1404).
Unlike the processing in the first embodiment, the DNS proxy 104 here may extract only the hash value (for example, “820d92dcb08edcdd”) from the host name containing the hash value (such as “www.example.org.820d92dcb08edcdd.hash.value”).
The DNS proxy 104 may then combine the IPv6 prefix utilizing the address generating function 210, with the extracted hash value to generate the IP address.
The DNS proxy 104 may use the address generating function 210 to calculate a new hash value from the host name (“www.example.org.820d92dcb08edcdd.hash.value”) containing a hash value, and generate an IP address based on the calculated hash value the same as in the first embodiment.
Further, in order to identify the device where the generated IP address is registered, the terminal 107 or the neighboring router sends an address resolution request, and acquires the MAC address of the Web proxy 105 assigned to the generated IP address.
The terminal 107 handles the generated IP address and for example sends a content acquisition request (for example,
“Get/ex/2 HTTP/1.0Host: www.example.org.820d92dcb08edcdd.hash.value . . . ”) to the Web proxy 2_105-2 (S1405).
Next, the Web proxy 2_105-2 acquires the URL containing a hash value from the header in the received content acquisition request, and deletes the hash value (for example, “820d92dcb08edcdd”) and the suffix (for example, “.hash value”) for the host name, from the URL (for example, “http://www.example.org.820d92dcb08edcdd.hash.value/ex/2”) containing the acquired hash value. The URL prior to conversion (the content acquisition destination URL requested in S1401 “http://www.example.org/ex/2”) is acquired in this way. The Web proxy 2_105-2 next searches the contents table 610 using the URL acquired prior to conversion as the search key, and decides whether or not the contents corresponding to the URL of the search key have been cached or not (S1406). If the contents corresponding to the URL have not been cached here then the Web proxy 2_105-2 transfers the content acquisition request to the Web server 102 (S1407).
The Web proxy 2_105-2 next receives the reply to the content acquisition request from the Web server 102 (S1408). The Web proxy 2_105-2 then acquires the contents contained that received reply, and stores the acquired contents in the entry for content table 610 (S1409). Here, if the link destination URL is contained in the acquired contents, then the Web proxy 2_105-2 converts the URL contained in the contents based on the URL conversion rule shown in
The Web proxy in other words converts the URL contained in contents sent to the terminal 107 based on URL conversion rules, to a URL containing the values beforehand. This prevents two requests; a content acquisition request containing no “.hash.value” and a content acquisition request containing a “.hash.value” from being issued when accessing contents in the terminal 107.
The Web proxy 2_105-2 next sends the acquired contents and the URL containing the hash value for the link destination of the contents, to the terminal 107 as a reply to the content acquisition request (S1410).
If the link destination URL of the contents that were sent was clicked by the user then a hash value was already embedded in that link destination URL so the processing from steps S1401 through S1404 can be omitted.
The second embodiment as described above distributes the cached contents to the Web proxies in lower order path units rather than host units so the gateway system can achieve even more efficient load balancing.
| Patent | Priority | Assignee | Title |
| 10205700, | Jan 31 2014 | Dell Products L.P. | Systems and methods for resolution of uniform resource locators in a local network |
| 9513970, | Mar 01 2013 | Cisco Technology, Inc. | Optimizing handling of virtual machine mobility in data center environments |
| Patent | Priority | Assignee | Title |
| 6108703, | Jul 14 1998 | Massachusetts Institute of Technology | Global hosting system |
| 6212565, | Aug 26 1998 | Oracle America, Inc | Apparatus and method for improving performance of proxy server arrays that use persistent connections |
| 6351775, | May 30 1997 | SAP SE | Loading balancing across servers in a computer network |
| 6553413, | Jul 14 1998 | Massachusetts Institute of Technology | Content delivery network using edge-of-network servers for providing content delivery to a set of participating content providers |
| 6823377, | Jan 28 2000 | International Business Machines Corporation | Arrangements and methods for latency-sensitive hashing for collaborative web caching |
| 20050038865, | |||
| 20060248195, | |||
| 20080235400, | |||
| 20090248786, | |||
| JP10222449, | |||
| JP114261, | |||
| JP2000207270, | |||
| JP2001273186, | |||
| JP2002520735, | |||
| JP2002523838, | |||
| JP200430690, | |||
| JP2004348494, | |||
| JP2011519203, | |||
| WO184800, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| Jun 15 2010 | TOUMURA, KUNIHIKO | Hitachi, LTD | ASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS | 025059 | /0736 | |
| Aug 04 2010 | Hitachi, Ltd. | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Feb 02 2017 | M1551: Payment of Maintenance Fee, 4th Year, Large Entity. |
| Apr 05 2021 | REM: Maintenance Fee Reminder Mailed. |
| Sep 20 2021 | EXP: Patent Expired for Failure to Pay Maintenance Fees. |
| Date | Maintenance Schedule |
| Aug 13 2016 | 4 years fee payment window open |
| Feb 13 2017 | 6 months grace period start (w surcharge) |
| Aug 13 2017 | patent expiry (for year 4) |
| Aug 13 2019 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Aug 13 2020 | 8 years fee payment window open |
| Feb 13 2021 | 6 months grace period start (w surcharge) |
| Aug 13 2021 | patent expiry (for year 8) |
| Aug 13 2023 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Aug 13 2024 | 12 years fee payment window open |
| Feb 13 2025 | 6 months grace period start (w surcharge) |
| Aug 13 2025 | patent expiry (for year 12) |
| Aug 13 2027 | 2 years to revive unintentionally abandoned end. (for year 12) |