Predicting clinical trial eligibility based on cohort trends

Abstract

Method and apparatus for predicting clinical trial eligibility for patients. Embodiments include determining a current value for each of a plurality of attributes of a first patient. Embodiments include identifying a cohort of patients that are clinically similar to the first patient, based on the plurality of attributes. Embodiments include analyzing data associated with the cohort of patients to determine an attribute trend for at least a first attribute the plurality of attributes. Embodiments include generating a predicted value for the first attribute, based on the current value of the first attribute and the attribute trend for the first attribute. Embodiments include identifying a plurality of clinical trials based on the first attribute. Embodiments include generating a probability that the first patient will be eligible for each of the plurality of clinical trials at a future time, based on the predicted value for the first attribute.

Description

BACKGROUND

The present disclosure relates to using machine learning techniques to deliver clinical trial recommendations, and more specifically, to predicting clinical trial eligibility based on cohort trends.

Clinical trials in medicine are research studies that are used to test and evaluate various medical treatments, drugs, or devices under development. Typically, clinical trials are defined as a treatment, drug, or device being developed, qualifying criteria (or inclusion criteria) defining the characteristics of patients who may be eligible to participate in a specified trial, and disqualifying criteria defining the characteristics of patients who are not eligible for participation in the trial. For example, the qualifying criteria may include the medical condition that the subject of the clinical trial is addressing, a stage of medical treatment that patients should be at, what previous treatments a patient may have received prior to entering the clinical trial, and the like. The disqualifying criteria defining the characteristics of patients who are not eligible to participate in a specified trial may include, for example, a stage of a disease beyond which a patient would be ineligible for inclusion in the trial, previous treatments that disqualify a patient from participating in the trial, and the like. However, qualifying and disqualifying criteria and other relevant information about clinical trials may sometimes be written in a non-standardized format that cannot easily be cross-compared.

At any given time, a patient may potentially be eligible for participation in a variety of clinical trials. Typically, to determine what clinical trial(s) a patient may be eligible for participation in, the patient's doctors and/or other clinical staff may review the patient's medical records and the qualifying and disqualifying criteria for a number of clinical trials to identify trials that may be of interest to the patient. However, the process of identifying trials that are potentially of interest for the patient may be a time consuming, manual process that requires doctors or other clinical staff to compare potentially voluminous patient records with at least the qualifying and disqualifying criteria for each clinical trial. Further, due to the number and wide variety of clinical trials that may be active at any time, manual searches for trials of interest may miss potentially relevant trials for a given patient. In some cases, manual analysis of potential clinical trials to enroll a patient in may rely on institutional procedures that prioritize clinical trials being run in certain institutions over potentially relevant clinical trials run in other institutions, which may result in potentially relevant clinical trials for a patient being overlooked or otherwise omitted from consideration.

Automated methods for analyzing patient records and clinical trial definitions may not be able to accurately match patients with the clinical trials that patients may be eligible to participate in for various reasons. For example, automated methods may not be able to accurately parse the intent of statements in a clinical trial definition. In another example, automated methods may not be able to understand the implications of a patient's medical records with respect to the ability to successfully complete a clinical trial. In still further examples, automated methods may be unable to determine or identify temporal relationships associated with eligibility or disqualifying criteria for a clinical trial. As such, there is a need in the art for improved techniques for automatically recommending clinical trials for patients.

SUMMARY

According to one embodiment of the present invention, a method includes determining a current value for each of a plurality of attributes of a first patient; identifying a cohort of patients that are clinically similar to the first patient, based on the plurality of attributes; analyzing data associated with the cohort of patients to determine an attribute trend for at least a first attribute the plurality of attributes; generating a predicted value for the first attribute, based on the current value of the first attribute and the attribute trend for the first attribute; identifying a plurality of clinical trials based on the first attribute; and generating a probability that the first patient will be eligible for each of the plurality of clinical trials at a future time, based on the predicted value for the first attribute. In certain embodiments, a computing system may be configured to perform methods described herein. In some embodiments, a computer-readable storage medium may store instructions that, when executed by one or more processors, cause the one or more processors to perform methods described herein.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 illustrates an example networked environment in which predictive cognitive models are used to deliver clinical trial recommendations based on patient medical records and clinical trial definitions, according to one embodiment.

FIG. 2 illustrates an example of training predictive models for predicting clinical trial eligibility based on cohort trends.

FIG. 3 illustrates an example of using trained predictive models to predict clinical trial eligibility based on cohort trends.

FIG. 4 illustrates example operations for predicting clinical trial eligibility based on cohort trends.

FIG. 5 illustrates an example system in which aspects of the present disclosure may be performed.

DETAILED DESCRIPTION

Embodiments presented herein involve predicting clinical trial eligibility based on cohort trends. One or more machine learning models may be trained using historical clinical trial data and historical patient data including historical patient attributes (e.g., over time) of a plurality of patients in order to predict future values of patient attributes based on cohort trends (e.g., changes in attributes over time for patients determined to be similar) and use the predicted future values in determining scores indicating probabilities that patients will be eligible for clinical trials in the future. In one example, trends in attributes of historical patients that participated in historical clinical trials are used to train the model(s) to predict future values of particular attributes for a patient based on current attribute values. In some embodiments, one or more models may also be trained to determine scores (e.g., confidence scores) indicating a probability that a patient will be eligible for clinical trials based on current and/or predicted future values of attributes of the patient. As such, techniques described herein allow for flexible and dynamic automated identification of clinical trials for which a patient may be eligible based on the patient's current and predicted future attributes.

FIG. 1 illustrates an example networked computing environment in which predictive models are used to predict clinical trial eligibility and recommend clinical trials for a patient based on patient attributes and clinical trial characteristics, according to an embodiment of the present disclosure. As illustrated, computing environment 100 includes a client device 120, a model trainer 130, an application server 140, a clinical trial data store 150, and a patient data store 160, connected via network 110.

Client device 120 generally is representative of a computing device on which a user can define and/or manage the training of predictive models used by trial recommendation engine 144 to recommend potentially relevant clinical trials for a patient and access application 142 on application server 140 to obtain a set of potentially relevant clinical trials for a patient. Client device 120 may be, for example, a laptop computer, a desktop computer, a thin client, a tablet computer, a mobile computing device, and the like.

Client device 120 includes an application 122 (e.g., a client side component of a client-server application) comprising a user interface 124, which allows a user of client device 120 to interact with a server side component, such as application 142 on application server 140. User interface 122 may, for instance, allow a user of client device 120 to initiate a search for recommended clinical trials that may be of interest to a patient by providing, to application server 140, identifying information of the patient (e.g., the patient name, date of birth, social security number, and/or the like) and/or the patient's medical records (e.g., entered in a standardized format or scanned copies of records that can be processed to determine patient attributes) in a request for one or more potentially relevant clinical trials to present to a patient.

Model trainer 130 generally uses information about patients that were previously enrolled in trials (e.g., from patient data store 160) and the criteria defined for those trials (e.g., from clinical trial data store 150) to train predictive models used in predicting future values of patient attributes, predicting clinical trial eligibility, and recommending clinical trials that are potentially relevant to a particular patient. As illustrated, model trainer 130 includes a vector generator 132 and a recommendation engine trainer 134. The training data may include a first set of labeled data used to train the predictive models and a second set of unlabeled data (e.g., for which the intended label is known) used to verify the accuracy of the trained predictive models and to refine the trained predictive models prior to deployment to application server 140.

Vector generator 132 generates a training data set for use by recommendation engine trainer 134 to train predictive models for predicting future patient attribute values based on current patient attribute values, determining probabilities of clinical trial eligibility, and recommending potentially relevant clinical trials to a user based on patient medical history (e.g., in the form of patient attributes). To generate the training data set, vector generator 132 can obtain information about previously completed clinical trials from clinical trial data store 150 and patient medical history data from patient data store 160. The information obtained from clinical trial data store 150 may include, for example, a roster of patients enrolled in a specific clinical trial and a definition of that clinical trial. The definition of the clinical trial may include eligibility and disqualifying criteria, patient requirements for participation in the trial, a trial enrollment deadline, and other information defining the clinical trial. The roster of patients may include information identifying each patient that vector generator 132 can use to obtain patient medical records from patient data store 160, information identifying whether each patient successfully completed the trial, and information related to the outcomes of trials for patients.

In certain embodiments, a training data set for a particular predictive model may include, for example, a plurality of training data instances that include patient attribute values at different times (e.g., at different intervals, such as monthly). In some implementations, a training data instance may be in the form of a vector comprising one or more features (e.g., “current” or beginning attribute values) associated with one or more labels (e.g., subsequent attribute values after a certain amount of time has passed). For example, historical patient medical records of a plurality of patients may be analyzed to determine values of different patient attributes of each patient over time (e.g., a patient's body mass index (BMI), whether or not the patient is taking certain medications, whether or not the patient has undergone certain procedures, and the like at different time intervals). Historical patient data may also include patient preferences, such as whether a patient has a preference for lower-risk care rather than more aggressive forms of treatment, and personal data about the patient, such as geographic location, age, gender, profession, and the like. Patient preferences may also be included as features in training data instances. As such, the training data may allow for the predictive model to be trained to predict future values of attributes based on cohort trends. For example, if the training data indicates that patients with certain preferences, a certain BMI, and who have previously undergone a certain procedure commonly begin taking a certain medication within a month, this may allow for future use of this medication to be predicted for a given patient with similar current attributes. In some embodiments, the predictive model is trained to output a confidence score along with each predicted future value of an attribute. These confidence scores may be used in determining a probability that a given patient is eligible to participate in a given clinical trial.

In certain embodiments, another training data set includes sets of patient attributes labeled with “eligible” or “ineligible” for a given clinical trial based on historical patient data of patients that were historically eligible to participate in certain clinical trials. This training data set may be used to train a predictive model to output a probability that a given patient is eligible to participate in a given clinical trial based on attributes of the patient, which may include future values of attributes that were predicted using another predictive model as described herein.

To generate the training data to be used by recommendation engine trainer 134, vector generator 132 can generate a first set of training data comprising feature data and label data used to train a predictive model and a second set of labeled feature data (e.g., provided as unlabeled data to the predictive model for testing purposes) that can be used to test the generated predictive model. For example, in an embodiment where machine learning models are used to predict future values of attributes for a patient, the first set of data may comprise a plurality of vectors, where the features in each vector include information from patient medical records, and the labels in each vector include a subsequent value of an attribute. The second set of data may comprise an unlabeled set of patient medical records, for which the subsequent value of an attribute is known but excluded from the data set for testing purposes.

In some embodiments, the first set of data may further include additional features that may be used to further refine predictions. These additional features may include, for example, information about a current stage of treatment that a patient is in, the specialty of the patient's clinicians, the institution that is treating the patient, and patient completion success for other trials that the patient may have participated in. For example, a predictive model may be trained, using the features, to output higher scores for particular labels when features of a given patient match these features in the training data.

Recommendation engine trainer 134 obtains the training data generated by vector generator 132 and, using supervised learning techniques, trains one or more predictive models for predicting future attribute values of patients, determining probabilities of clinical trial eligibility, and delivering recommendations of potentially relevant clinical trials for a patient. Training predictive models may involve using a training data set that includes training inputs that are associated with particular training outputs (e.g., labels). As such, training data comprising historical starting patient attributes associated with at least one historical subsequent attribute value is used to train one or more predictive models to output a predicted future attribute value based on one or more current attribute values of a given patient. For example, a trained machine learning model may accept a set of current patient attributes as an input and output a predicted future value of a certain patient attribute (e.g., the certain attribute may also be identified as an input). In some embodiments, the predictive model also outputs a confidence score with a predicted future attribute value. Recommendation engine trainer 134 may use different training data generated by vector generator 132 to train a predictive model to accept a set of patient attribute values as inputs (e.g., including predicted future attribute values, which may be accompanied by confidence scores) along with identifying information of a clinical trial and to output a probability that the patient is eligible for the clinical trial.

Predictive models described herein may be, for example, random forest models, and training a prediction model may involve building a “forest” of “trees” representing the training data, where input features are randomly permuted at each split (e.g., each division of the training data). In such embodiments, a prediction model is “trained” by constructing a plurality of decision trees based on sub-divisions of the training data that include random permutations of input features associated with labels.

It is noted that random forest techniques are only included as one example, and that other techniques may be employed for training prediction models. Other types of machine learning models, such as a Naïve Bayes model, logistic regression, decision tree, gradient-boosted tree, multilayer perceptron, and others may also be used as prediction models.

In certain embodiments a single prediction model is used, while in other embodiments a plurality (or ensemble) of prediction models may be used (e.g., for different institutions and/or for different stages of an overall process for recommending clinical trials). In some embodiments, additional attributes may be used to customize a generated prediction model and thus customize the results delivered by the trial recommendation engine 144.

Techniques described herein may be used as part of a larger process for recommending clinical trials for patients. For example, determining whether a patient is eligible for a plurality of clinical trials may be an initial step that is performed before using additional predictive models to recommend the most relevant and/or the most beneficial clinical trials for the patient. In one example, a patient may not currently meet a qualifying criterion for a clinical trial but a predictive model described herein may be able to predict that the patient will meet the criterion prior to the start of the clinical trial based on historical patient attributes (e.g., other patients with patient data similar to the patient's current patient data went on to meet the criterion within a certain time frame). In one example, a clinical trial requires a patient to have undergone a certain procedure. Historical patient data may indicate that patients with certain attributes (e.g., a medical condition for a certain amount of time) typically undergo the procedure within a particular time frame. This historical patient data may be used to train a predictive model to predict whether a patient will undergo the procedure within a certain time frame based on current attributes of the patient. The output of this predictive model for a given patient may be used to determine eligibility of the given patient for the clinical trial, and may also be provided along with other current attributes of the given patient as inputs to a predictive model for recommending clinical trials for the patient. For example, if the value of an attribute is predicted to change prior to the start of the clinical trial, the predicted future value of this attribute may be provided as an input to the predictive model rather than the current value of the attribute. As such, techniques described herein allow for predicting clinical trial eligibility and recommending clinical trials not only based on the patient's current attributes, but also based on attributes that the patient is predicted to have in the future.

Application server 140 includes an application 142 and a trial recommendation engine 144. Application 142 may be any type of application in which users can request recommendations of potentially relevant clinical trials for a patient by providing patient data (e.g., identifying information of a patient and/or medical condition information, treatment history, prior clinical trial participation history, and other relevant information) and, in some embodiments, user-defined filters in a search request executed by application 142. Client device 120 may instantiate or initiate a session of application 142 in response to a request for application content (e.g., a list of active clinical trials that are enrolling patients for participation) generated by a user of client device 120. In some embodiments, the instance of a session of application 142 may be instantiated by a user of client device 120 accessing a home page of an application 142 structured as a web application. In other embodiments, user interface 122 may instantiate the instance of application 142 by launching an executable file on client device that includes components that execute locally on client device 120 and use data provided by application 142.

During execution of application 142, a user of client device 120 may request a set of recommended clinical trials for a given patient by providing data related to that patient to application 142 in conjunction with a search request. In response, application 142 provides the received data to trial recommendation engine 144 for analysis. In some embodiments, the user may provide identifying information for a patient (e.g., name, date of birth, social security number, and/or other identifying information), and data for the patient may be retrieved (e.g., by trial recommendation engine 144) from patient data store 160 based on the identifying information. Application 142 may receive a list of potentially relevant clinical trials for the patient (e.g., ranked in an order of relevance) from trial recommendation engine 144, as discussed in further detail below, and display the list of potentially relevant clinical trials in user interface 122 of client device 120. In some embodiments, the list of potentially relevant clinical trials may include a predetermined number of potentially relevant clinical trials for the patient and may be sorted based on the rankings or scores associated with each clinical trial in the list of potentially relevant clinical trials. Application 142 may additionally allow a user of client device 120 to further refine the list of potentially relevant clinical trials using one or more user-defined filters.

Trial recommendation engine 144 uses the prediction model(s) generated by recommendation engine trainer 134, as discussed above, to examine patient data for a given patient and recommend potentially relevant clinical trials for the patient based, at least in part, on the patient's medical history. In some embodiments, trial recommendation engine 144 retrieves patient data, such as medical history, for a given patient from patient data store 160 based on identifying information of the patient (e.g., received from a user of client device 120). Patient data may include a plurality of different types of patient attributes, such as aspects of the given patient's medical history. Patient attributes may, in some embodiments, include scheduled events that have not yet occurred (e.g., medical procedures scheduled for future dates).

In some embodiments, trial recommendation engine 144 first determines a plurality of clinical trials for which the given patient is eligible based on qualifying criteria and disqualifying criteria for clinical trials (e.g., retrieved from clinical trial data store 150), such as evaluating the qualifying criteria and disqualifying criteria based on the patient attributes. In certain embodiments, trial recommendation engine 144 predicts future attributes of the given patient, such as by providing certain current attributes of the patient (e.g., retrieved from patient data store 160) as inputs to one or more predictive models that output predicted future attributes, which are accompanied in some embodiments by confidence scores. In certain embodiments, trial recommendation engine 144 uses the confidence scores to determine a probability that the patient is eligible for a given clinical trial. In some embodiments, trial recommendation engine 144 uses a predictive model to determine a probability that a patient is eligible for a given clinical trial, such as by providing current and predicted future attributes (e.g., along with confidence scores in some embodiments) as inputs to the predictive model along with identifying information of the given clinical trial, and receiving the probability that the patient is eligible for the given clinical trial as an output from the model.

Trial recommendation engine 144 provides patient attributes, which may include current and predicted future attributes, of the given patient as inputs to one or more predictive models for recommending clinical trials (with scores) for the given patient. In some embodiments, trial recommendation engine 144 also provides identifying information of the clinical trials for which the given patient is eligible as inputs so that the predictive model(s) only determine scores for those particular clinical trials. For example, in one embodiment a predictive model accepts patient attributes as inputs and outputs a score for every clinical trial for which the predictive model was trained, while in another embodiment a predictive model accepts both patient attributes and identifiers of clinical trials as inputs and outputs scores only for the clinical trials identified in the inputs.

For example, a predictive model may process the inputs and output a score for each of a plurality of clinical trials (e.g., identified by trial recommendation engine 144), each score indicating a strength of relevance of a respective clinical trial to the given patient. In certain embodiments, predictive models may be trained based on individual clinician preferences for a clinician that requested the recommended clinical trials, and outputs of the predictive models may be in accord with the clinician preferences (e.g., clinical trials of a type preferred by the clinician may be assigned higher scores according to the training of the model). Techniques described herein may involve fuzzy logic, as output scores are not only based on bright-line rules (e.g., whether or not a patient meets a criterion for a clinical trial), but are based on degrees of association (e.g., the patient may not meet a certain criterion or attribute that is historically associated with a clinical trial explicitly, but may be within a threshold distance and/or may be predicted to meet the criterion or attribute prior to the time of the clinical trial). Predictive models, therefore, are a means of implementing fuzzy logic. Fuzzy logic may also be employed in determining which clinical trials a patient is eligible for (e.g., as a first step before providing patient attributes and identifying information of the clinical trials for which the patient is eligible as inputs to a predictive model).

In some embodiments, trial recommendation engine 144 may rank the plurality of clinical trials based on the scores output by the predictive model(s). Trial recommendation engine 144 may further filter the clinical trials, such as based on user input (e.g., filters and/or preferences provided by the user via user interface 122). In some embodiments, trial recommendation engine 144 also applies certain bright-line rules in order to filter the clinical trials, such as to prevent recommending a clinical trial with a low score to a user when the clinical trial would in fact not be appropriate for the user at all. For example, if a clinical trial is only for HER-2 negative patients and the given patient is HER-2 positive, then the clinical trial may be filtered out of the list of clinical trials provided to the user.

Trial recommendation engine 144 provides a ranked list of clinical trials to application 142, which may provide the ranked list to user interface 122 for display to the user. In certain embodiments, trial recommendation engine 144 provides information to application 142 regarding attributes that were predicted so that this information can be provided to the user. For example, if trial recommendation engine 144 recommends a certain clinical trial based on a prediction that the given patient will undergo a certain procedure in the future, then the user may be informed that the clinical trial is only recommended if the prediction turns out to be accurate.

Clinical trial data store 150 may be a data storage entity, such as a database or repository, that stores data about clinical trials. For example, clinical trial data store 150 may store qualifying and disqualifying criteria for clinical trials, rosters of patients that participated in clinical trials, information about outcomes of clinical trials, and the like. For example, the data in clinical trial data store 150 may have been entered by one or more users (e.g., clinicians or professionals), or may have been mined from various sources, such as articles and reports related to clinical trials.

Patient data store 160 may be a data storage entity, such as a database or repository, that stores data about patients. For example, patient data store 160 may store personal data, patient attributes, medical records, patient preferences, and other data related to patients, which may have been provided by one or more users, such as patients and/or clinicians.

While model trainer 130, application server 140, clinical trial data store 150, and patient data store 160 are illustrated as separate components in FIG. 1, it should be recognized that model trainer 130, application server 140, clinical trial data store 150, and patient data store 160 may be implemented on any number of computing systems, either as one or more standalone systems or in a distributed environment.

FIG. 2 illustrates an example 200 of training predictive models for predicting clinical trial eligibility as described herein. Example 200 includes model trainer 130 (comprising vector generator 132 and recommendation engine trainer 134), clinical trial data store 150, patient data store 160, and application server 140 of FIG. 1.

Model trainer 130 retrieves historical clinical trial data 222 from clinical trial data store 150 and historical patient data 224 from patient data store 160. Historical clinical trial data 222 may include information regarding qualifying and disqualifying criteria for a plurality of clinical trials. Historical clinical trial data 222 may also include lists of patients (e.g., identified using information such as names, dates of birth, and social security numbers) that participated in historical clinical trials.

Historical patient data 224 may include attributes of patients that participated in historical clinical trials (e.g., retrieved based on identifying information of the patients included in historical clinical trial data 222), such as medical history of the patients, and may include attribute values over time.

Vector generator 132 uses historical clinical trial data 222 and historical patient data 224 to generate training data 220 for generating one or more trained models 210. For example, training data 220 may include a first training data set, each training data instance of the set associating beginning attribute values of a plurality of patient attributes with a subsequent value of at least one attribute. Another training data set in training data 220 may include associations between historical patient attribute values and historical eligibility or ineligibility for particular clinical trials. Another training data set may include training data instances associating historical patient attributes with characteristics of clinical trials.

Training data 220 is provided to recommendation engine trainer 134, which uses training data 220 to generate one or more trained models 210, such as using various supervised machine learning concepts. One or more of models 210 may be trained to output predicted future attribute values, in some cases with confidence scores, based on current attribute values (e.g., by identifying historical cohorts for a given patient based on similarities in attributes, and identifying trends in attribute values for the historical cohorts, which may be accomplished using training data). Other models may be trained to output probabilities that patients are eligible for clinical trials based on current and/or predicted future attribute values. Other models may be trained to output relevance scores for clinical trials based on input patient attributes. In some embodiments, one or more of models 210 are trained based on specific clinician preferences and are used to recommend clinical trials for patients when requested by certain clinicians. For example, training data 220 may further include data related to clinician preferences, and the clinician preferences may be used to adjust the weights of features within the models.

Model trainer 130 provides the one or more models 210 to application server 140, which loads the one or more models 210 into memory for use in predicting clinical trial eligibility and recommending clinical trials for patients.

FIG. 3 illustrates an example 300 of using trained predictive models to clinical trial eligibility for patients. Example 300 includes trial recommendation engine 144, patient data store 160, and clinical trial data store 150 of FIG. 1. Example 300 also includes predictive models 210a and 210b, which may be models 210, trained as illustrated in example 200 of FIG. 2. In example 300, predictive model 210a is trained to predict a future value of a given patient attribute based on current patient attributes identifying information (e.g., an identifier, such as a name) of the given patient attribute. Predictive model 210b is trained to output a probability that a given patient is eligible for a given clinical trial based on current and/or predicted future attributes of the patient and identifying information (e.g., an identifier, such as a name) of the given clinical trial.

Trial recommendation engine 144 sends a request 306 to patient data store 160 for patient data related to a patient, such as in response to a request from a user of a client device for recommended clinical trials for the patient. Patient data store 160 provides patient data 308 of the patient to trial recommendation engine 144 in response to request 306. Patient data 308 may, for instance, include patient attributes such as aspects of the patient's medical history (e.g., diagnoses, procedures, allergies, medications, family medical history, and the like).

Trial recommendation engine 144 provides current patient attributes (e.g., from patient data 308) and identifying information of a particular attribute 310 as inputs to predictive model 210a, which outputs one or more predicted future values of the particular attributes 312, which may include one or more predicted values of the particular attributes at one or more future points (e.g., at particular intervals).

Trial recommendation engine 144 then provides current and predicted future patient attributes along with identifying information of a clinical trial 314 to predictive model 210b, including current patient attributes determined from patient data 308, the predicted future value of the particular attribute 312, and identifying information of a clinical trials related to the particular attribute (e.g., an identifier, such as a name, of the clinical trial). Predictive model 210b outputs an eligibility probability for the clinical trial 316, which may be a value indicating a probability that the patient is eligible for the clinical trial identifier at 314.

Trial recommendation engine 144 may determine whether the patient is eligible for the clinical trial based on the probability output by predictive model 210b. For instance, if the probability is above a threshold, trial recommendation engine 144 may determine that the patient is eligible for the clinical trial.

While example 300 depicts predicting clinical trial eligibility for a single patient, it is noted that techniques described herein may also be used to predict eligibility for a plurality of clinical trials for a plurality of patients, such as in response to a request from a clinician.

FIG. 4 illustrates example operations 400 for predicting clinical trial eligibility for patients. For example, operations 400 may be performed by trial recommendation engine 144 of FIGS. 1 and 3.

At step 410, a current value for each of a plurality of attributes of a first patient is determined. For example, trial recommendation engine 144 of FIG. 1 may determine the current values for the plurality of attributes based on data retrieved from patient data store 160 of FIG. 1.

At step 420, a cohort of patients that are clinically similar to the first patient is identified based on the plurality of attributes. For example, trial recommendation engine 144 of FIG. 1 may provide the plurality of current values for the plurality of attributes along with identifying information of a first attribute as inputs to a predictive model, and the predictive model may determine the cohort of patients that are clinically similar to the first patient based on the cohort of patients having similar attribute values to the first patient (e.g., according to the training of the predictive model).

At step 430, data associated with the cohort of patients is analyzed to determine an attribute trend for at least a first attribute of the plurality of attributes. For example, the attribute trend for the first attribute (e.g., identified at step 420) may be identified by the predictive model according to its training based on attribute values over time for the cohort of patients. In one example, the trend comprises an increase in a prescribed dosage of a particular medication that commonly occurs when other attributes are present.

At step 440, a predicted value for the first attribute is generated based on the current value of the first attribute and the attribute trend for the first attribute. For example, the predictive model may output the predicted value for the first attribute by applying the attribute trend to the current value of the first attribute. In one example, the attribute trend comprises an increase in a prescribed dosage of a particular medication, and the predicted value is determined (e.g., using the predictive model) by applying the increase to the current prescribed dosage for the particular medication for the patient. For example, the attribute trend may comprise a direction and magnitude, and the predicted value may be generated by modifying the current value of the first attribute based on the direction and magnitude of the attribute trend.

At step 450, a plurality of clinical trials is identified based on the first attribute. For example, trial recommendation engine 144 of FIG. 1 may retrieve clinical trial data from clinical trial data store 150 of FIG. 1, and may identify the plurality of clinical trials by determining which clinical trials in the clinical trial data include qualifying or disqualifying criteria related to the first attribute.

At step 460, a probability that the first patient will be eligible for each of the plurality of clinical trials at a future time is generated based on the predicted value for the first attribute. For example, the predicted value for the first attribute may be provided (e.g., along with the plurality of attributes and identifying information of each of the plurality of clinical trials) as an input to a predictive model, which may output the probability.

The probability determined at step 460 may be used to determine whether the first patient is eligible to participate in each of the plurality of clinical trials, such as based on whether the probability for a given clinical trial is above a threshold. Once eligibility is determined, additional processing may be performed to determine the most relevant and/or most beneficial clinical trials for a given patient, which may be provided in a ranked list to a user via a user interface.

It is noted that while embodiments are described as involving predictive models using machine learning techniques, certain aspects of the present disclosure may be performed without the use of predictive models. For example, eligibility probabilities may be determined based on confidence scores associated with predicted future values of attributes output by a predictive model rather than by providing the predicted future values to a separate predictive model to determine eligibility probabilities.

FIG. 5 depicts a computing system 500 with which embodiments of the present disclosure may be implemented. For example, computing system 500 may represent application server 140 of FIG. 1.

System 500 includes a central processing unit (CPU) 502, one or more I/O device interfaces 504 (that may provide connections for various I/O devices 514, such as keyboards, displays, mouse devices, and the like) to the system 500, network interface 506 (e.g., a physical network interface card), memory 508, storage 510, and an interconnect 512. It is noted that one or more components of system 500 may alternatively be located remotely and accessed via a network, such as network 110. It is further contemplated that one or more components of system 500 may comprise physical or virtualized components.

CPU 502 may receive and execute instructions stored in memory 508. Similarly, the CPU 502 may receive and store data related to applications in memory 508. The interconnect 512 transmits programming instructions and application data, among the CPU 502, I/O device interface 504, network interface 506, memory 508, and storage 510. CPU 502 is included to be representative of a single CPU, multiple CPUs, a single CPU having multiple processing cores, and other arrangements.

Additionally, the memory 508 may represent a random access memory. Storage 510 may be a disk drive, solid state drive, or a collection of storage devices distributed across multiple storage systems. Although shown as a single unit, the storage 510 may be a combination of fixed and/or removable storage devices, such as fixed disc drives, removable memory cards or optical storage, network attached storage (NAS), or a storage area-network (SAN).

Storage 510 comprises clinical trial data store 540 and patient data store 550, which are representative of clinical trial data store 150 and patient data store 160 of FIG. 1. As shown, memory 508 includes application 520, trial recommendation engine 530, and model(s) 560, which are representative of application 142 and trial recommendation engine 144 of FIG. 1 and model(s) 210 of FIG. 2.

The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Reference is made herein to embodiments presented in this disclosure. However, the scope of the present disclosure is not limited to specific described embodiments. Instead, any combination of the following features and elements, whether related to different embodiments or not, is contemplated to implement and practice contemplated embodiments. Furthermore, although embodiments disclosed herein may achieve advantages over other possible solutions or over the prior art, whether or not a particular advantage is achieved by a given embodiment is not limiting of the scope of the present disclosure. Thus, the following aspects, features, embodiments and advantages are merely illustrative and are not considered elements or limitations of the appended claims except where explicitly recited in a claim(s). Likewise, reference to “the invention” shall not be construed as a generalization of any inventive subject matter disclosed herein and shall not be considered to be an element or limitation of the appended claims except where explicitly recited in a claim(s).

Aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.”

The present invention may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.

Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

Embodiments of the invention may be provided to end users through a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing may be defined as a computing capability that provides an abstraction between the computing resource and its underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. Thus, cloud computing allows a user to access virtual computing resources (e.g., storage, data, applications, and even complete virtualized computing systems) in “the cloud,” without regard for the underlying physical systems (or locations of those systems) used to provide the computing resources.

Typically, cloud computing resources are provided to a user on a pay-per-use basis, where users are charged only for the computing resources actually used (e.g. an amount of storage space consumed by a user or a number of virtualized systems instantiated by the user). A user can access any of the resources that reside in the cloud at any time, and from anywhere across the Internet. In context of the present invention, a user may access applications (e.g., application 142 and/or trial recommendation engine 144 of FIG. 1) or related data available in the cloud. Doing so allows a user to access this information from any computing system attached to a network connected to the cloud (e.g., the Internet).

While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.

Claims

1. A computer-implemented method comprising:

determining a current value for each of a plurality of attributes of a first patient;

providing one or more inputs to a machine learning model based on the current value for each of the plurality of attributes of the first patient, wherein the machine learning model has been trained through a supervised learning process comprising:

generating a training data set comprising historical starting values of a particular patient attribute and one or more historical values of additional patient attributes associated with labels indicating historical subsequent values of the particular patient attribute for a plurality of patients, wherein the particular patient attribute comprises one of:

a body mass index;

an indication of whether a given medication is currently being taken;

a medication dosage amount; or

an indication of whether a given procedure has been undergone; and

using the training data set to train the machine learning model to output predicted values with confidence scores for the particular patient attribute in response to inputs comprising a current value for the particular patient attribute and current values for the additional patient attributes;

determining, based on one or more outputs from the machine learning model in response to the one or more inputs, a predicted value of the particular patient attribute for the first patient and a confidence score for the predicted value;

identifying a plurality of clinical trials based on the particular patient attribute;

generating a probability that the first patient will be eligible for each of the plurality of clinical trials at a future time, based on the predicted value of the particular patient attribute for the first patient and the confidence score for the predicted value, using an additional machine learning model that has been trained for a particular clinician through a training process that comprises adjusting one or more weights associated with the additional machine learning model based on one or more preferences of the particular clinician; and

providing, in response to a request from the particular clinician, a ranked list of recommended clinical trials for the first patient based on the probability that the first patient will be eligible for each of the plurality of clinical trials at the future time, wherein a given clinical trial of the plurality of clinical trials is only included in the ranked list if a corresponding probability that the first patient will be eligible for the given clinical trial at the future time is above a threshold.

2. The computer-implemented method of claim 1, wherein determining the predicted value of the particular patient attribute for the first patient comprises modifying the current value of the particular patient attribute for the first patient based on a direction and a magnitude of an attribute trend for the first particular patient attribute for the first patient.

3. The computer-implemented method of claim 1, wherein identifying the plurality of clinical trials comprises selecting clinical trials that include the particular patient attribute in corresponding stated criteria.

4. The computer-implemented method of claim 3, the method further comprising:

selecting a first clinical trial of the plurality of clinical trials;

determining that the first patient is not currently eligible for the first clinical trial, based on the current value of the particular patient attribute for the first patient;

determining that the predicted value of the particular patient attribute for the first patient satisfies the corresponding stated criteria of the first clinical trial; and

providing an indication of the first clinical trial.

5. The computer-implemented method of claim 3, the method further comprising:

selecting a first clinical trial of the plurality of clinical trials;

determining that the first patient is currently eligible for the first clinical trial, based on the current value of the particular patient attribute for the first patient;

determining that the predicted value of the particular patient attribute for the first patient does not satisfy the corresponding stated criteria of the first clinical trial; and

providing an indication of the first clinical trial.

6. The computer-implemented method of claim 1, wherein each respective clinical trial of the plurality of clinical trials is associated with a respective future time that is determined based on one or more characteristics of the respective clinical trial.

7. A system, comprising one or more processors and a computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by the one or more processors to perform a method, the method comprising:

determining a current value for each of a plurality of attributes of a first patient;

providing one or more inputs to a machine learning model based on the current value for each of the plurality of attributes of the first patient, wherein the machine learning model has been trained through a supervised learning process comprising:

generating a training data set comprising historical starting values of a particular patient attribute and one or more historical values of additional patient attributes associated with labels indicating historical subsequent values of the particular patient attribute for a plurality of patients, wherein the particular patient attribute comprises one of:

a body mass index;

an indication of whether a given medication is currently being taken;

a medication dosage amount; or

an indication of whether a given procedure has been undergone; and

using the training data set to train the machine learning model to output predicted values with confidence scores for the particular patient attribute in response to inputs comprising a current value for the particular patient attribute and current values for the additional patient attributes;

determining, based on one or more outputs from the machine learning model in response to the one or more inputs, a predicted value of the particular patient attribute for the first patient and a confidence score for the predicted value;

identifying a plurality of clinical trials based on the particular patient attribute;

generating a probability that the first patient will be eligible for each of the plurality of clinical trials at a future time, based on the predicted value of the particular patient attribute for the first patient and the confidence score for the predicted value, using an additional machine learning model that has been trained for a particular clinician through a training process that comprises adjusting one or more weights associated with the additional machine learning model based on one or more preferences of the particular clinician; and

providing, in response to a request from the particular clinician, a ranked list of recommended clinical trials for the first patient based on the probability that the first patient will be eligible for each of the plurality of clinical trials at the future time, wherein a given clinical trial of the plurality of clinical trials is only included in the ranked list if a corresponding probability that the first patient will be eligible for the given clinical trial at the future time is above a threshold.

8. The system of claim 7, wherein determining the predicted value of the particular patient attribute for the first patient comprises modifying the current value of the particular patient attribute for the first patient based on a direction and a magnitude of an attribute trend for the particular patient attribute for the first patient.

9. The system of claim 7, wherein identifying the plurality of clinical trials comprises selecting clinical trials that include the particular patient attribute in corresponding stated criteria.

10. The system of claim 9, the method further comprising:

selecting a first clinical trial of the plurality of clinical trials;

determining that the first patient is not currently eligible for the first clinical trial, based on the current value of the particular patient attribute for the first patient;

determining that the predicted value of the particular patient attribute for the first patient satisfies the corresponding stated criteria of the first clinical trial; and

providing an indication of the first clinical trial.

11. The system of claim 9, the method further comprising:

selecting a first clinical trial of the plurality of clinical trials;

determining that the first patient is currently eligible for the first clinical trial, based on the current value of the particular patient attribute for the first patient;

determining that the predicted value of the particular patient attribute for the first patient does not satisfy the corresponding stated criteria of the first clinical trial; and

providing an indication of the first clinical trial.

12. The system of claim 7, wherein each respective clinical trial of the plurality of clinical trials is associated with a respective future time that is determined based on one or more characteristics of the respective clinical trial.

13. A computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more processors to perform a method, the method comprising:

determining a current value for each of a plurality of attributes of a first patient;

providing one or more inputs to a machine learning model based on the current value for each of the plurality of attributes of the first patient, wherein the machine learning model has been trained through a supervised learning process comprising:

generating a training data set comprising historical starting values of a particular patient attribute and one or more historical values of additional patient attributes associated with labels indicating historical subsequent values of the particular patient attribute for a plurality of patients, wherein the particular patient attribute comprises one of:

a body mass index;

an indication of whether a given medication is currently being taken;

a medication dosage amount; or

an indication of whether a given procedure has been undergone; and

using the training data set to train the machine learning model to output predicted values with confidence scores for the particular patient attribute in response to inputs comprising a current value for the particular patient attribute and current values for the additional patient attributes;

determining, based on one or more outputs from the machine learning model in response to the one or more inputs, a predicted value of the particular patient attribute for the first patient and a confidence score for the predicted value;

identifying a plurality of clinical trials based on the particular patient attribute;

generating a probability that the first patient will be eligible for each of the plurality of clinical trials at a future time, based on the predicted value of the particular patient attribute for the first patient and the confidence score for the predicted value, using an additional machine learning model that has been trained for a particular clinician through a training process that comprises adjusting one or more weights associated with the additional machine learning model based on one or more preferences of the particular clinician; and

providing, in response to a request from the particular clinician, a ranked list of recommended clinical trials for the first patient based on the probability that the first patient will be eligible for each of the plurality of clinical trials at the future time, wherein a given clinical trial of the plurality of clinical trials is only included in the ranked list if a corresponding probability that the first patient will be eligible for the given clinical trial at the future time is above a threshold.

14. The computer-readable storage medium of claim 13, wherein determining the predicted value of the particular patient attribute for the first patient comprises modifying the current value of the particular patient attribute for the first patient based on a direction and a magnitude of an attribute trend for the particular patient attribute for the first patient.

15. The computer-readable storage medium of claim 13, wherein identifying the plurality of clinical trials comprises selecting clinical trials that include the particular patient attribute in corresponding stated criteria.

16. The computer-readable storage medium of claim 15, the method further comprising:

selecting a first clinical trial of the plurality of clinical trials;

determining that the first patient is not currently eligible for the first clinical trial, based on the current value of the particular patient attribute for the first patient;

determining that the predicted value of the particular patient attribute for the first patient satisfies the corresponding stated criteria of the first clinical trial; and

providing an indication of the first clinical trial.

17. The computer-readable storage medium of claim 15, the method further comprising:

selecting a first clinical trial of the plurality of clinical trials;

determining that the first patient is currently eligible for the first clinical trial, based on the current value of the particular patient attribute for the first patient;

determining that the predicted value of the particular patient attribute for the first patient does not satisfy the corresponding stated criteria of the first clinical trial; and

providing an indication of the first clinical trial.

18. The computer-readable storage medium of claim 13, wherein each respective clinical trial of the plurality of clinical trials is associated with a respective future time that is determined based on one or more characteristics of the respective clinical trial.