The present invention relates to a process and a device for handling the execution of a job in an open data processing system as a function of the resources. The process comprises the steps of:
The present invention relates to a process and a device for handling the execution of a job in an open data processing system as a function of the resources. The process includes determining the start, the deference and the denial of the start of the job requested based on the comparison between the required and the available resources for the execution of the job.
|
0. 7. A device for handling the execution of a job in an open data processing comprising:
means for determining resources available in virtual memory, real memory, temporary file space, and central processing unit utilization time during the last time interval;
means for computing an amount of resources preallocated to other requests and not yet used;
means for comparing an amount of resources required for the execution of a job for which a request has been presented to the current amount of resources available, minus the total amount of resources preallocated to other requests;
and means for commanding, as a function of a result supplied by the comparing means, the start, the deferral and the denial of the start of the job requested.
1. A process for handling the execution of a job in an open data processing system as a function of the resources, characterized in that it comprises the following steps: comprising:
determining the resources available in virtual memory, real memory, temporary file space, and central processing unit utilization time during the last time interval;
computing the an amount of resources preallocated to other requests and not yet used;
comparing the an amount of resources required for the execution of a job for which the a request has been presented to the current amount of resources available minus the total amount of resources preallocated to other requests, in order to determine, as a function of the a result of this the comparison, the start, the deference or deferral and the denial of the start of the job requested.
0. 15. A computer readable medium having software instructions embodied thereon, the software instructions, when executed by a computer, cause the computer to perform a series of functions comprising:
determining resources available in virtual memory, real memory, temporary file space, and central processing unit utilization time during the last time interval;
computing an amount of resources preallocated to other requests and not yet used;
comparing an amount of resources required for execution of a job for which a request has been presented to the current amount of resources available minus the total amount of resources preallocated to other requests; and
determining, as a function of a result of the comparison, the start, the deferral and the denial of the start of the job requested.
0. 11. A process for handling the execution of a job in response to requests in an open data processing system as a function of system resources, comprising:
determining resources available in virtual memory, real memory, temporary file space, and central processing unit utilization time during the last time interval;
determining an amount of resources required for execution of a job for which a request has been presented;
computing an amount of resources preallocated to other requests and not yet used;
comparing the amount of resources required for execution of a job for which a request has been presented to the current amount of resources available minus the total amount of resources preallocated to other requests; and
determining, as a function of the result of this comparison, the start, the deferral and the denial of the start of the job requested.
2. The process according to
3. The process according to
0. 4. The process according to
0. 5. The device according to
0. 6. The device according to
0. 8. The device according to
0. 9. The device according to
0. 10. The device of
0. 12. The process according to
0. 13. The process according to
0. 14. The process according to
0. 16. The computer readable medium of
0. 17. The computer readable medium of
|
|||||||||||||||||||||||||||
The present invention relates to a device and a process for handling the execution of a job in a data processing system.
It applies to the field of data processing operations in an industrial environment and particularly to data processing systems of the “open” type running, for example, on software of the “UNIX” or “WINDOWS NT” type. The servers that run on this type of software are called “open” servers, and are highly valued at the present time.
The drawback of these servers is that they were developed for academic applications whose purposes were not the same as those for industrial purposes. Thus, in a “UNIX” operating system, the jobs, or more precisely each of the processes comprising the jobs are assigned an initial priority, either by the system or by the user who enters these jobs. In order for a job to be considered by the system to have a higher priority, the user must explicitly assign it an initial priority, or manually modify its priority using an administrative command. Thus, there is no control over or inter-correlation between the priority levels assigned to one job or another.
Likewise, in a standard “UNIX” system, the processes can start up without any limitation, even in heavily loaded systems, without any attention to the number of resources currently being used by the processes executed and without any attention to the number of resources the new process will require. This results in the following disadvantage a group of processes is executed, the critical processes as well as the non-critical processes, and the critical processes, which must be completed at a time desired by the user, have too few resources available to be completed at the desired time. At the same time, non-critical processes are using too many resources.
Another disadvantage in known systems is that in which there are not enough temporary files available and consequently, the write operations into the files fail. Sometimes, the application controls this error condition, sometimes it does not control it, and this can generate a false indication due to the fact that incomplete files are generated in one step of a job and processed in subsequent steps. Another detrimental characteristic is that in which there is not enough paging space and the system makes the arbitrary decision to kill the most recent process without considering its importance. This is unacceptable in an operating environment.
That is why the primary object of the invention is to propose a process for handling the execution of a job which makes it possible to eliminate these drawbacks.
This object is achieved as a result of the fact that the process for handling the execution of a job in an open data processing system (for example of the “UNIX” type) as a function of the resources is characterized in that it comprises the following steps:
According to another characteristic, the process includes a step comprised of defining the amount of resources required for the execution of a job by a specific command.
According to another characteristic, the process includes a step comprised of activating or deactivating the mechanism for determining whether there are enough system resources available.
Another object of the invention is to propose a device for handling the execution of a job.
In accordance with this object, the device for handling the execution of a job in an open operating system is characterized in that it comprises:
According to another characteristic, the device comprises a means for defining the amount of resources required for the execution of a job by a specific command.
According to another characteristic, the device comprises a means for activating or deactivating the mechanism for determining whether there are enough system resources available.
Other characteristics and advantages of the present invention will emerge more clearly with the reading of the description below, given in reference to the appended drawings, in which:
Prior to the description of an exemplary embodiment of the invention in UNIX, it is useful to recall the following definitions.
Hereinafter, the term “process” or “job” designates any execution of a program (and consequently, in particular, its environment in the system) at a given instant, the program itself constituting an inert object stored on a disk in the form of an ordinary executable file.
In “UNIX” systems, the existence of two types of processes is known:
The structure of the “UNIX” system represented in
To meet the needs of multitask applications in an open environment, two “api” program modules were developed around these standard elements in “UNIX” systems, one of which (JRS Job Reporting Service) provides reports on the execution of the jobs started by a user, while the other (LRM) makes it possible to display the proper execution or the occurrence of failures and thus to detect the causes. This first module will be called the “JRS” (Job Reporting Service) and the other will be called the “LRM” (Local Resource Management) module.
Added to this, for purposes of the invention, is a local resource management (LRM) daemon 3 which communicates with the kernel 2 and with an application program interface 34. This daemon 3 also communicates with an LRM configuration file 33 stored in the hard disk, and with an associated graphical user interface (LRM gui) 32. The application program interface 34 communicates with the daemon 3 and with a shared memory 31. This shared memory is also in communication with the graphical user interface 32 and the daemon 3. For purposes of pre-allocation, the system incorporates a “JRS” application program interface 44 for reporting the execution of jobs (Job Reporting Service). This application 44 communicates with a catalog and which stores the job report (jor) in a distinct file 5 of the hard disk. This application program interface 44, if it has been activated and the job report files have been stored, communicates with the local resource management daemon 3 so as to allow the pre-allocation and dynamic control of resources. The interface 44 comprises the lines of code that are necessary to allow the execution of the commands available to the user through this interface. The commands available through this interface 44 are the following:
With a system equipped in this way with the local resource management module, the non-critical processes should alter the continuation of the execution of the critical processes as little as possible. Each type of process has a specific importance. More or fewer resources should be allocated to it as a function of this importance and as a function of the load on the system. For this reason, the module “LRM” makes it possible to sort the processes by dimension. A dimension is a set of commonly executed processes which have the same importance from the point of view of the local resource manager. By default, five dimensions are provided: a first dimension “SYSTEM,” a second dimension “BATCH,” a third dimension “MISC,” a fourth dimension DB (data base) and a fifth dimension TP (transfer protocol). If the processes are started by the normal users directly in the background, then they belong to the dimension “MISC.” If the processes are started via a script, entered into the job management application without any declaration of explicit dimensions, then they belong to the third dimension “BATCH.” If the processes do not belong to any other dimension, then they belong to the dimension “SYSTEM.” The dimensions “DB” and “TP” are provided without any definitions of processes and can be defined by the user. With the exception of the dimension “SYSTEM,” it is possible for each dimension to have a relative weight, which is stored in a file. This relative weight is used to manage the priority of the processes comprising the dimension. The priority of all the processes belonging to a dimension will vary accordingly. The dimensions “SYSTEM” has no relative weight and the priorities of the processes belonging to this dimension are not modified by the local resource manager (LRM). The processes controlled by the dimensions will always have resources allocated to them as a function of their relative weight, even if the system is heavily loaded. At the moment of the startup of the local resource manager, or when a dimension is created without specifying a relative weight, by default this dimension will have a weight of −1, which means that it exists, but that it is not controlled. A dimension can also be activated or deactivated. If it is deactivated, the jobs entered that belong to it will not be started, but the jobs in the process of being executed in this dimension will continue to be executed.
When a process results from a job execution command “JRS,” it belongs to the dimension specified, if a dimension has been specified in the command “JRS.” When the process has an ancestor whose command name and user are associated with a dimension, the process belongs to this dimension. During the run time of a process, the relative weight will be used to manage and modify the priority of the processes comprising the dimension. A preallocated process either is not started by jrs or does not comprise any preallocation parameters.
As a result of the resource management application, the user as well as the resource manager will know for each resource at a given time, how many resources are actually being used and how many resources have been declared for each job. Thus, in the example represented in
The mechanism for preallocating virtual memory and temporary file space is implemented by a command “JRS,” which also constitutes a process that contributes to the proper operation of the system. When a job is entered, it can request via the command JRS a certain amount of resources. The resource manager then tries to ensure that there will be enough resources for the job. The jobs can be entered with options, a first of which is to supply a maximum elapsed time and a second of which is the expected total central processing unit time for the job. When a request for resources is examined, the following values are considered:
The current amount of resources “A” is a piece of information obtained periodically and updated periodically by the “daemon” LRM by verifying the system information tables that have been stored. The value of the resources already preallocated “R” is also determined by the daemon “LRM,” which gathers the current utilizations of resources by the jobs. “Ra” is a set value, supplied by the “daemon” LRM as being one of these arguments.
The request will be fulfilled if r<A−(R+Ra). Upon each request, the daemon performs this computation and fulfills or does not fulfill the request as a function of the result of this comparison. Once a request for resources corresponding to a job start has been fulfilled, the job is executed, taking into account the evolution of its priority.
The following appendices relate to the code parts that make it possible to compute the total available virtual memory and real memory. This computation is performed by the following code part.
. . . Computing Available and Total Virtual and Real Memory.
/*get_vmdata: Obtainment of Information on Available Real
and Virtual Memory */
get_vmdata(vm)
struct vmdata *vm
{
char
buff[1024];
int
bufsiz = 1024;
char
w1[20], w2[20], w3[20], w4[20];
while (1) {
int
v1, v2;
if(!init_access_to_commands_done) {
/*Initialization: Command vmstat is launched, and its output caught*/
init_access_to_commands_done = 1;
init_access_to_commands( );
fdvmcom = sag_popen_getline(“vmstat 1”, buff, &bufsiz, 2);
} else {
int cr;
/*A new line issued by command vmstat is read*/
cr = sag_popen_nextline(fdvmcom, buff, &bufsiz, 2);
if (cr <= 0) {
fdvmcom = sag_popen_getline(“vmstat 1”, buff,
*bufsiz, 2);
}
}
/*line is examined and information recovered*/
sscanf(buff, “%s %s %s %s”, w1, w2, w3, w4);
if (!strcmp(w1, “kthr”) && !strcmp(w2, “memory”))
continue;
if (!strcmp(w1, “-----”) && !strcmp(w2, “-----------”))
continue;
if (!strcmp(w1, “r”) && !strcmp(w2, “b”))
continue;
sscanf(w3, “%d”, &v1);
sscanf(w4, “%d”, &v2);
vm->totrealmem = vmconstant.totrealmem;
/*System Call psdanger returns Total Virtual Memory and Available
Virtual Memory */
vm->totvirtmem = psdanger (0) * 4;
vm->avrealmem = v2 * 4;
vm->avvirtmem = psdanger (−1) * 4;
return (0);
}
}
The process also comprises a step for computing the total and available temporary file space. This computing step is carried out by the following code part.
Computing Available and Total Temporary File Space
updtmpspace (avtmpspace, tottmpspace)
int
*avtmpspace;
int
*tottmpspace;
{
int
curnbtmpdev;
char
*ptdevlist;
int
i;
struct statfs
statfsbuf;
curnbtmpdev = enum_dev( );
if (curnbtmpdev > nbtmpdev){
nbtmpdev = curnbtmpdev;
if (tmpdevlist)
(unsigned char *) free(tmpdevlist);
tmpdevlist = (char*) malloc(nbtmpdev *200);
tmpdevno = (int *) malloc(nbtmpdev *sizeof(int));
}
if (curnbtmpdev > 0) {
get_dev(tmpdevlist, tmpdevno);
*avtmpspace − 0;
*tottmpspace = 0;
for (i = 0, ptdevlist = tmpdevlist; i < curnbtmpdev;
i++, ptdevlist += 200) {
int cr;
/*System call ststfs provides information on the File System*/
cr = statfs(ptdevlist, &statsbuf);
/*conversion into Kb: multiply by 4*/
*avtmpspace += 4 * statfsbuf.f_bfree;
*tottmpspace += 4 * statfsbuf.f_blocks;
}
}
}
Next, the central processing unit usage that occurred during the last time interval is determined, and this determination is made by means of the following code.
/*sysinfo and vminfo are AIX Kernel tables*/
/*The function* get_sysinfo_and_vminfo reads these
tables from the Kernel Memory*/
get_sysinfo_andvminfo (&sysinfo, &vminfo);
tpgspgouts = (double) (vminfo.pgspgouts −
ovminfo.pgspgouts)/(double) Irmdinterval;
trunque = (double) (sysinfo.runque −
oysinfo.runque)/(double) Irmdinterval;
tswpque = (double) (sysinfo.swpque-
oysinfo.swpque)/(double) Irminterval;
tuser = (double) (sysinfo.cpu[CPU_USER])/(double)
Irmdinterval;
tsystem = (double) (sysinfo.cpu[CPU_KERNEL]-
oysinfo.cpu[CPU_KERNEL])/(double) Irmdinterval;
tidle = (double) (sysinfo.cpu[CPU_IDLE]-
oysinfo.cpu[CPU_IDLE])/(double) Irmdinterval;
twait = (double) (sysinfo.cpu[CPU_WAIT]-
oysinfo.cpu[CPU_WAIT])/(double) Irmdinterval;
tsum = tuser + tsystem ÷ tidle + twait;
The process also comprises a step for determining the spaces preallocated but not yet used by the jobs. This step is carried out by the following code part.
Computing Prealloc space not used by jobs:
shmem->system.preallocvirtmem = 0;
shmem->system.prealloctmpspace = 0;
for (job = shmem->first_jb_started; job; job = job->next) {
if (job->requiredvirtmem && (job->requiredvirtmem >
job->currentvirtmem)) {
shmem->system.preallocvirtmem += (job->
requiredvirtmem − job->current virtmem);
}
if (job->requiredtmpspace && (job->required-
tmpspace > job->currentusedtmpspace))
shmem->system.prealloctmpspace += (job->
requiredtmpspace − job->currentusedtmpspace);
}
The evaluation is carried out according to the formula F <
(R + Ra) and is constituted by the following code part.
Checking Resources for a job:
LRMresourcescheck (LRMresources)
struct LRMresources *LRMresources;
{
struct timeval tv;
struct timezone tz;
struct systementry system;
gettimeofday (&tv, &tz);
shmgetsystem(&system);
if (LRMresources->requiredtmpspace >
(system.avtmpspace − system.prealloctmpspace −
system.reservedtmpspace))
35
return (0);
if (LRMresources->requiredvirtmem >
(system.avvirtmem − system.preallocvirtmem −
system.reservedvirtmem))
36
return (0);
if (LRMresources->expectedtermination) {
int
remains;
int
tl;
double
a, b;
remains-LRMresources->expectedtermination −
tv.tv_sec;
LRMresources->neededratecpu = 100.0 *
LRMresources->expectedcpu / remains;
/*
* test here with the current available cpu rate
/*
37
LRMdbgprintf(DBGALL, “Maximum Elapsed
time specified\n”);
LRMdbgprintf(DBGALL, “Remains: %d\n”,
remains);
LRMdbgprintf(DBGALL, “Needed Rate of CPU:
%f\n”, LRMresources->neededratecpu);
LRMdbgprintf(DBGALL, “Extrapolated Available
Rate of CPU: %d\n”, system.workingavcpu);
if (remains > 0 && LRMresources->neededratecpu <=
system.workingavcpu) {
LRMdbgprintf(DBGALL, “Rate of CPU is good
enough\n”); return (1);
} else {
LRMdbgprintf(DBGALL, “Rate of CPU is too
low\n”);
if (remains < 0 ∥ LRMresources->neededratecpu >
100.0* system.nbprocessors) {
LRMdbgprintf(DBGALL, “There will not
38
be enough free CPU to complete the job in time\n”);
return (−1);
}
return (0);
}
} else
return (1);
}
In this code part, the portion 35 performs the evaluation of the temporary space part and returns the value 0 if enough temporary space is available. Then, the process continues with an evaluation of the virtual memory part performed by the portion 36 which returns the value 0 is enough virtual memory space is available. Finally, the system performs an evaluation of the available CPU resources using the code part 38. This evaluation is done after having determined the remaining resources (REMAINS) and the necessary CPU rate (NEEDED RATE CPU). If the available CPU rate is enough, the system returns the value 1, which allows the execution of the request, whereas in the case where the system returns the value −1, the request for the execution of the job is deferred.
Thus, through the definition of the parameters necessary to the execution of a job pre-entered into the command JRS by the user, the latter can, using the mechanism described above, trigger the execution of a job in the system, when the necessary resources for its proper execution are available.
Any modification within reach of one skilled in the art is also part of the spirit of the invention. Thus, the process and the device described can be easily used and applied with another open system such as, for example, “WINDOWS NT”.
While the preferred forms and embodiments of the invention have been illustrated and described, it will be apparent to those of ordinary skill in the art that various changes and modifications may be made without deviating from the inventive concept and spirit of the invention as set forth above, and it is intended by the appended claims to define all such concepts which come within the full scope and true spirit of this invention.
APPENDIX 1
NAME: jrs command
jrs - submits a command to the resource manager and
generates an execution report in a file SYNOPTIQUE
jrs[-p <Number of virtual memory pages]
[-t <Amount of temporary file space]
[-q]
[-f]
[-l]
[-n]
[-o]
[-d <Dimension defined by user>]
[-c <Expected CPU time consumption>
-e <Maximum time elapsed]
[-C >Maximum CPU time>]
[-T <Maximum temporary file space]
[-P <Maximum number of pages of virtual memory]
[-r 0| 1| 2| 3]
[-v <environmentvariable>= <value>]. . .
[comand *args . . .]
Sitbon, Gerard, Urbain, Francois, Durand, Daniel Lucien
| Patent | Priority | Assignee | Title |
| 9015724, | Sep 23 2009 | International Business Machines Corporation | Job dispatching with scheduler record updates containing characteristics combinations of job characteristics |
| Patent | Priority | Assignee | Title |
| 5367637, | Mar 26 1992 | International Business Machines Corporation; INTERNATIONAL BUSINESS MACHINES CORPORATION A CORPORATION OF NEW YORK | Self-tuning virtual storage management for dedicated real-time computer system |
| 5550970, | Aug 31 1994 | International Business Machines Corporation | Method and system for allocating resources |
| 5787246, | May 27 1994 | Microsoft Technology Licensing, LLC | System for configuring devices for a computer system |
| 5826082, | Jul 01 1996 | Oracle America, Inc | Method for reserving resources |
| 5838968, | Mar 01 1996 | ATI Technologies Inc | System and method for dynamic resource management across tasks in real-time operating systems |
| 6763454, | May 27 1994 | Microsoft Technology Licensing, LLC | System for allocating resources in a computer system |
| EP676699, |
| Executed on | Assignor | Assignee | Conveyance | Frame | Reel | Doc |
| May 04 2004 | Bull S.A. | (assignment on the face of the patent) | / |
| Date | Maintenance Fee Events |
| Mar 15 2013 | M1553: Payment of Maintenance Fee, 12th Year, Large Entity. |
| Date | Maintenance Schedule |
| Sep 14 2013 | 4 years fee payment window open |
| Mar 14 2014 | 6 months grace period start (w surcharge) |
| Sep 14 2014 | patent expiry (for year 4) |
| Sep 14 2016 | 2 years to revive unintentionally abandoned end. (for year 4) |
| Sep 14 2017 | 8 years fee payment window open |
| Mar 14 2018 | 6 months grace period start (w surcharge) |
| Sep 14 2018 | patent expiry (for year 8) |
| Sep 14 2020 | 2 years to revive unintentionally abandoned end. (for year 8) |
| Sep 14 2021 | 12 years fee payment window open |
| Mar 14 2022 | 6 months grace period start (w surcharge) |
| Sep 14 2022 | patent expiry (for year 12) |
| Sep 14 2024 | 2 years to revive unintentionally abandoned end. (for year 12) |