"COMPUTER AIDED MEDICAL DATA EXCHANGE SYSTEM FOR MEDICAL DATA ANALYSIS"

> 0. In the case of single scalar values, temporal trends for a medical event can be compared with respect to known trends for normal and abnormal cases.
The report module provides the display and quantification capabilities, for the user to visualize and or quantify the results of temporal comparison. In practice, one would use'all the available data for the analysis. In the case of medical images, several different visualization methods can be employed. Results of temporal comparisons can be simultaneously displayed or overlaid on one another using a logical operator based on some pre-specified criterion. For quantitative comparison, color look-up tables can be used. The resultant data can also be coupled with an automated pattern recognition technique tp perform' farther qualitative and/or manual/automated quantitative analysis of the results.
ARTIFICIAL NEURAL NETWORK
A general diagrammatical representation of an artificial neural network is shown in Fig. 15 and designated by the reference numeral 202. Artificial neural networks consist of a number of units and connections between them, and can be implemented by hardware and/or software. The units of the neural network may generally be categorized into three types of different groups (layers), according to their functions, as illustrated in Fig. 15. A first layer, input layer 204, is assigned to accept a set of data representing an input pattern, a second layer, output layer 208, is assigned to provide a set of data representing an output pattern, and an arbitrary number of intermediate layers, hidden layers 206, convert the input pattern to the output pattern. Because the number of units in each layer is determined arbitrarily, the input layer and the output layer include sufficient numbers of units to represent the input patterns and output patterns, respectively, of a problem to be solved. Neural networks have been used to implement computational methods that learn to distinguish between objects or classes of events. The networks are first trained by presentation of known data about objects or classes of events, and then are applied to distinguish between unknown objects or classes of events.
Briefly, the principle of neural network 202 can be explained in the following manner. Normalized input data 210, which may be represented by numbers ranging from 0 to 1, are supplied to input units of the neural network. Next, the output data 212 are
provided from output units through.two successive nonlinear calculations (in a case of one hidden layer 206) in the hidden and output layers 208, 210. The calculation at each unit in the layer, excluding the input units, may include a weighted summation of all entry numbers, an addition of certain offset terms and a conversion into a number ranging from 0 to 1 typically using a sigmoid-shape function. In particular, as represented diagrammatically in Fig. 16, units 214, which may be labeled Oj to On, represent input or hidden units, Wj through Wn represent the weighting factors 216 assigned to each respective output from these input or hidden units, and T represents the summation of the outputs multiplied by the respective weighting factors. An output.218, or O is calculated using the sigmoid function 220 given where 6 represents an offset value for T. An example sigmoid function is given by the following expression: 1/[I + exp(-T+ 0)]. The weighting factors and offset values are internal parameters of the neural network 202, which are determined for a given set of input and output data.
Two different basic processes are involved in the neural network 202, namely, a training process and a testing process. The neural network is trained by the back-propagation algorithm using pairs of training input data and desired output data. The internal parameters of the neural network are adjusted to minimize the difference between the actual outputs of the neural network and the desired outputs. By iteration of this procedure in a random sequence for the same set of input and output data, the neural network learns 'a relationship between the training input data and the desired output data. Once trained sufficiently, the neural network can distinguish different input data according to its learning experience.
EXPERT SYSTEMS
One of the results of research in the area of artificial intelligence (AI) has been the development of techniques which allow the modeling of information at higher levels of abstraction. These techniques are embodied in languages or tools, which allow programs to be built to closely resemble human logic in their implementation and are
therefore easier to develop and maintain. These programs, which emulate human expertise in well-defined problem domains, are generally called expert systems.
The component of the expert system that applies the knowledge to the. problem is called the inference engine. Four basic control components may be generally identified in an inference engine, namely, matching (comparing current rules to given patterns), selection (choosing most appropriate rule), implementation (implementation of the best rule), and execution (executing resulting actions).
To build an expert system that solves problems in a given domain, a knowledge engineer, an expert in AI language and representation, starts by reading domain-related literature to become familiar with the issues and the terminology. With that as a foundation, the knowledge engineer then holds extensive interviews with one or more domain experts to "acquire" their knowledge. Finally, the knowledge engineer organizes the results of these interviews and translates them into software that a computer can use. The interviews typically take the most time and effort of any of these 'stages.
Rule-based programming is one of the most, commonly used techniques for developing expert systems. Other techniques include fuzzy expert systems, which use a collection of fuzzy membership functions and rules, rather than Boolean logic, to reason relationships between data. In rule-based programming paradigms, rules are used to represent heuristics, or "rules of thumb," which specify a set of actions to be performed for a given situation. A rule is generally composed of an "if portion and a "then" portion. The "if portion of a rule is a series of patterns which specify the facts (or data) which cause the rule to be applicable. The process of matching facts to patterns is generally called pattern matching. The expert system tool provides the inference engine, which automatically matches facts against patterns and selects the most appropriate rule. The "if portion of a rule can actually be thought of as the "whenever" portion of a rule, because pattern matching occurs whenever changes are made to facts. The "then" portion of a rule is the set of actions to be implemented when the rule is applicable. The actions of applicable rules are executed when the inference engine is instructed to begin execution. The inference engine selects a rule,
and then the actions of the selected rule are executed (which may affect the list of applicable rules by adding or removing facts). The inference engine then selects another rule and executes its actions. This process continues until no applicable rules remain.
INITIATION OF PROCESSING FUNCTIONS AND STRINGS
As used herein, the term "processing string" is intended to.relate broadly to computer-based activities performed to acquire, analyze, manipulate, enhance, generate or otherwise modify or derive data, within the integrated knowledge base or from data within the integrated knowledge base. The processing may include, but is hot limited to analysis of patient-specific clinical data. Processing strings may act upon such data, or upon entirely non-clinical data, but in general will act upon both. Thus, processing strings may include activities for acquisition of data (both for initiating acquisition and terminating acquisition, and for setting acquisition settings and protocols, or notification that acquisition is desired or desirable).
A user-initiated processing string, for example, might include launching of a computer-assisted detection routine to identify calcifications possibly visible within cardiac CT data. While this processing string proceeds, moreover, the system, based upon the requested routine and the data available from other resources, may automatically initiate a processing string which fetches cholesterol test results from the integrated knowledge base for analysis of possible relationships between the requested data analysis and the cholesterol test results. Conversely, when analysis of cholesterol test results is requested or initiated, the system may detect the utility in performing imaging that would assist in evaluating or diagnosing related conditions, and inform the user (or a different user) of the need or desirability to schedule acquisition of images that would form the basis for the complementary evaluation.
It should also be noted that the users that may initiate processing strings may include a wide range of persons with diverse needs and uses for the raw and processed data. These might include, for example, radiologists requesting data within and derived from images, insurers requesting information relating or supporting insurance claims,


nurses in need of patient history information, pharmacists accessing prescription data, and so forth. Users may also include the patient him or herself, accessing diagnostic information or their own records. Initiation based upon a change in data state may look to actual data itself, but may also rely on movement of data to or from a new workstation, uploading or downloading of data, and so forth. Finally, system-initiated processing strings may rely on simple timing (as at periodic intervals) or may rely on factors such as the relative level of a parameter or-resource. System-initiated processing strings may also be launched as new protocols or routines become available, as to search through existing data to determine whether the newly available processing might assist in identifying a condition therefore unrecognized.
As noted above, the data processing system 10, integrated knowledge base 12, and federated database 14 can all communicate with one another to provide access, translation, analysis and processing of various types of data from the diverse resources available. Figure 17 illustrates this feature of the present technique again, with emphasis upon the interface 8 provided for users, such as clinicians and physicians. The interface 8, while permitting access to the various resources of the system, including the data processing system, the integrated knowledge base, and the federated database, will generally allow for a wide range of interface types and systems. In particular, as designated diagrammatically by the reference numeral 222 in Fig. 17, the "unfederated" interface layer comprising the interface 8 may include a range of disparate and different interface components at single institutions, or at a wide range of different institutions widely geographically dispersed from one another. Moreover, the basic operating systems of the interfaces need not be the same, and the present technique contemplates that various types of interfaces may be united and configured in the unfederated interface layer separately, and nevertheless enable to communicate with one or more of the data processing system, the integrated knowledge base and the federated database. In particular, where an integrated knowledge base and a federated database are provided, these may accommodate the various types of interfaces in the layer, such as through the use of standardized protocols as noted above, including HTML, XML, and so forth. The interface layer may also permit automatic or use-
prompted queries of the integrated knowledge base, the data processing system, or the federated database. In particular, where appropriate, the users may not be aware of queries executed by programs implemented on workstations, such as by management of input or output of client data, filing of claims, prescription of data acquisition sequences, medications, and so forth.
The interface layer, and the programming included therein and in the data processing system may permit a wide range of processing functions to be executed based upon a range of triggering events. These events maybe initiated and carried out in conjunction with use requests, or may be initiated in various other manners. Figure 18 diagrammatically illustrates certain of the initiating and processing functions which may be performed in this manner.
As shown in Fig. 18, various initiating sources 224 may be considered for initiating the data acquisition, processing, and analysis on the data from the resources and knowledge base described above. The initiating sources 224 commence processing as indicated generally at reference numeral 22.6 in Fig. 18, in accordance with routines stored in one or more of the data processing system, integrated knowledge base, and federated database, or further more within the resources, including the controllable prescribable resources and the data resources. The particular processing may be stored, as noted above, and a single computer system comprised in the data processing system, or dispersed through various computer systems which cooperate with one another to perform the data processing and analysis. Following initiation of the processing, processing strings may be carried out as indicated generally at reference numeral 228 in Fig. 18. These processing strings may include a wide range of processing and analysis of functions, typically designed to provide a caregiver with enhanced insights into patient care, to process the data required for the patient care, including clinical and non-clinical data, to enhance function of an institution providing the care, to detect trends or relationships within the patient data, and to perform general discovery and mining of relationships for future use.
The present technique contemplates that a range of initiating sources 224 may commence the processing and analysis functions in accordance with .the routines executed by the system. In particular, for such initiating sources are illustrated in Fig. 18, including a user initiating source 230, an event or patient initiating source 232, a data state change source 234, and a system or automatic initiating source 236. Where a user, such as a clinician, physician, insurance company, clinic or hospital employee, management or staff user, and the like initiates a request that draws upon the integrated knowledge base or the various integrated resources described above, a processing string may begin that calls upon information either already stored within the integrated knowledge base or accessible by locating, accessing, and processing data within one or more of the various resources. In a typical setting, a user may initiate such processing at a workstation where a query or other function is performed. As noted above, the query may be obvious to the user, or may be inherent in the function performed on the workstation.
Another contemplated initiating source is the event or patient as indicated at reference numeral 232 in Fig. 18. In general, many medical interactions will begin with specific symptoms or medical events which trigger contact with a medical institution or practitioner. Upon logging such an event by a patient or clinician interfacing with the patient, a processing.string may begin which will include a range of interactive steps, such as access to patient records, updating of patient records, acquisition of details relating to symptoms,- and so forth as described more fully below. The event to 'patient initiated processing string, while used to perform heretofore unavailable and highly integrated processing in the present context, may be generally similar to the types of events which drive current medical service provision.
The data processing system 10 may generally monitor a wide range of data parameters, including the very state of the data (static or changing) to detect when new data becomes available. The new data may become available by updating patient records, accessing new information, uploading or downloading data to and from the various controllable and prescribable resources and data resources, and so forth. Where desired, the programs executed by the data processing system may initiate
processing based upon such changes in the state of data. By way of example, upon detecting that a patient record has been updated by a recent patient contact or the availability of clinical or non-clinical data, the processing string may determine whether subsequent actions, notifications, reports or examinations are in order. Similarly, the programs carried out by the data processing system may automatically initiate certain processing as indicated at reference numeral 236 in Fig. 18. Such system-initiated processing may be performed on a routine bases, such as predetermined time intervals or at the trigger of various system parameters, such as inventory levels, newly-available data or identification of relationships between data, and so forth.
A particularly powerful aspect of the highly integrated approach of the present technique resides in the fact that, regardless of the initiating source of the processing, various processing strings may result. As summarized generally in Fig. 18, for example, the processing strings 228, while generally aligned with various initiating sources in the figure, may result from other initiating sources and executed programs. For example, a user or context string 238 may include processing which accesses and returns processed information to respond precisely to a user-initiated processing event, or in conjunction with the particular context within which a user accesses the system. However, such processing strings may also result from event or patient initiated processing, data state changes, and system-initiated processing. Moreover, it should be noted that several types of specific strings may follow within the various categories. For example, the user or context string 238 may include specific query-based processing as indicated at reference numeral 240, designed to identify and return data which is responsive to specific queries posed by a user. Alternatively, user or environment-based strings 242 may result in which data accessed and returned is user-specific or environment-specific. Examples of such processing strings might include access and processing of data for analysis of interest to specific users, such as specific types of clinicians or physicians, financial institutions, and insurance companies.
As a further example of the various processing strings which may result from the initiating source processing, event strings 244 may include processing which.is specific to the medical event experienced by a patient, or to events experienced in the past or which may be possible in future. Thus, the event strings 244 may result from user initiation, event or patient initiation, data state change initiation, or system initiation. In a typical context, the event string may simply follow the process of a medical event or symptom being experienced by a patient to access information, process the information, and provide suggestions or diagnoses based upon the processing. As noted above, the suggestions may include the performance of additional processing or analysis, the acquisition of additional information, both automatically and with manual assistance, and so forth.
A general detection string 246 might also be initiated by the various initiating sources. In the present context; the general detection string 246 may include processing designed to identify relevant data or relationships which were not specifically requested by a user, event, patient, data state change or by the system. Such general detection strings may correlate new data in accordance with relationships identified by the data processing system or integrated knowledge base. Thus, even where a patient or user has not specifically requested detection of relationships or potential correlations, programs executed by the data processing system 10 may nevertheless execute comparisons and groupings to identify risks, potential treatments, financial management options and so forth under a general detection string.
Finally, a processing string designated in Fig. 18 as a system string 248 may be even more general in nature. The system string may be processing which is executed with the goal of discovering relationships between data available from the various resources. These new relationships may be indicative of new ways to diagnose or treat patients such as based upon recognizable trends or correlations, analysis of success or failure rates, statistical analyses of patient care results, and so forth. As in the previous examples, the system string may be initiated in various manners, including at the automatic initiation of the system, but also with changes in data state,

upon the occurrence of newly detected medical event or by initiation of the patient, or by a specific request of a user.
COMPUTER-ASSISTED PATIENT DATA CAPTURE AND PROCESSING
In accordance with one aspect of the present technique, enhanced processing of patient data is provided by coordinating data collection and processing directly from the patient with data stored in the integrated knowledge base 12. For the present purposes, it should be borne in mind that the integrated knowledge base 12 may be considered to include information within various resources themselves, or processed information resulting from analysis of such raw data. Moreover, in the present context the integrated knowledge base is considered to include data which may be stored in a variety of locations both within an institution and within a variety of institutions located in a single location or in quite disparate locations. The integrated knowledge base may, therefore, include a variety of coordinated data collection and repository sites. Exemplary, logical action classes and timeframes, with associated exemplary actions, are illustrated generally in Fig. 19.
Referring to Fig. 19,. the patient information which is included in the integrated knowledge base may result from any one or more of the types of modalities described above, and, more generally, of the various resource types. Moreover, as also described above, patient information may result from analysis of this type of data in conjunction with other generally available data in the data resources, such as different graphic information, proprietary or generally accessible databases, subscription databases, digitized reference materials, and so forth. However, the information is particularly useful when coordinated with a patient contact, such as a visit to a physician or facility. In the diagrammatical representation of Fig. 19, different distinct classes of action, designated generally at reference numeral 250, may be grouped logically, such as patient interactions, system interactions, and report or education-type actions. These action classes may be further considered, generally, as inputs, processing, and outputs of the overall system. Moreover, the action classes may be thought of as occurring by reference to a patient contact, such as an on-site visit. In this sense, the actions may be
generally classified as those taken prior to a visit or contact, as noted at reference numeral 252, those taken during a contact, as illustrated at reference numeral 254, and post-contact actions, as indicated at reference numeral 256.
It has been found, in the present technique, that by collection of certain patient information at these various stages of interaction, information from the integrated knowledge base may be extremely useful in providing enhanced diagnosis, analysis, patient care, and patient instruction. In particular, several typical • scenarios may be envisaged for the collection and processing of data prior to a patient contact or on-site visit.
As an example of the type of information which may be collected prior to a patient contact, sub-classes of actions may be performed, as indicated at reference numeral 258 in Fig. 19. By way of example, prior to a patient visit, a record for the patient contact or medical event (e.g. the reason for the visit) may be captured to begin a new or continuing record. Such initiation may begin by a patient phone call, information entered into a website or other interface,, instant messages, chat room messages, electronic messages, information input via a web camera, and so forth. The data relating to the record may be input either with human interaction or by automatic prompting or even through unstructured questionnaires. In such questionnaires, the patient may be prompted to input a chief complaint or symptoms, medical events, and the like, with prompting from voice, textual or graphical interfacing. In one exemplary embodiment, for example, the patient may also respond to graphical depictions of the human body, such as for selection of symptomatic region of the body.
Other information may be gathered prior to the patient contact, such as biometric information. Such information may be used for patient identification and/or authentication before data is entered into the patient record. Moreover, remote vital sign diagnostics may be acquired by patient input or by remote monitors, if available. Where data is collected by voice recording, speech recognition software or similar software engines may. identify key medical terms for later analysis. Also, where necessary, particularly in emergency situations, residential or business addresses, cellular telephone
locations, computer terminal locations, and the like can be accessed to identify the physical location of a patient. Moreover, patient insurance information can be queried, with input by the patient to the extent such information is known or available.
Based upon the patient interactions. 258, various system interactions 260 may be taken prior to the patient visit or contact. In particular, as the patient-specific data is acquired, data is .accessed from the integrated knowledge base (including the various resources) for analysis of the patient information. Thus, the data may be associated or analyzed to identify whether appointments for visits are in order, if not already arranged, and such appointments may be scheduled based upon the availability of resources and facilities, patient preferences and location, and so forth. Moreover, the urgency of such scheduled appointments may be assessed based upon the information input by the patient.
Among the various recommendations which may be made based upon the analysis, pre-visit imaging, laboratory examinations, and so forth may be recommended and scheduled to provide the most relevant information likely to be needed for efficient diagnosis and feedback during or immediately after the patient visit. Such recommendations may entail one or more of the various types of resources described above, and one or more of the modalities within each resource. The various information may also be correlated with information in the integrated knowledge base to provide indications of potential diagnoses or relevant questions and information that can be gathered during the patient visit. The entire set of data can then be uploaded to the integrated knowledge base to create or supplement a patient history database within the integrated knowledge base.
As a result of the uploading of data into the integrated knowledge base, various types of structured data may be stored for later access and processing. For example, the most relevant captured patient data may be stored, in a structured form, such as by classes or fields which can be searched and used to evaluate- potential recommendations for the procedures used prior to the medical visit, during the visit and after the visit. The data may be used, then for temporal analysis of changes in patient conditions, identification of trends, evaluation of symptoms recognized by the
patient, and general evaluation of conditions which may not even be recognized by the patient and which are not specifically being complained of. The data may also include, and be processed to recognize, potentially relevant evidence-based data, demographic risk assessments, .and results of comparisons and analyses of hypothesis for the existence or predisposition for medical events and conditions.
Following the system interaction, and resulting from the system interaction, various output-type functions may be performed by the system. For example, as noted at reference numeral 262 in Fig. 19, patient-specific recommendations may be communicated to the patient prior to the patient contact. These recommendations may include appointments for the contact or for other examinations or analyses, educational information relating to such procedures, protocols to be followed prior to the procedures (e.g. dietary recommendations, prescriptions, timing and duration of visits). Moreover, the patient information may be specifically tailored or adapted to the patient. In accordance with one aspect of the technique, for example, educational information may be conveyed to the patient in a specific language of preference based upon textual information available in the integrated knowledge base and the language of preference -indicated by the patient in the patient record. Such instructions may further include detailed data, such as driving or public transportation directions, contact information (telephone and facsimile numbers, website addresses, etc.). As noted above, actions may include ordering and scheduling of exams and data acquisition.
A further output action which may be taken by the system prior to and on-site visit might include reports or recommendations for clinicians and physicians. In particular, the reports may include output based upon the indications and designation of symptoms experienced by the patient, patient history information collect, and so forth. The report may also include electronic versions of images, computer-assisted processed (e.g. enhanced) images, and so forth. Moreover, such physician reports may include recommendations or prioritized lists of information or examinations which should be performed during the visit to refine or rule out specific diagnoses.
The process summarized in Fig. 19 continues with information which.is collected by patient interaction during a contact, such as an on-site visit, as indicated at reference numeral 264. In a present example, the information collected at the time of the contact might begin with biometric information which, again can be used for patient identification and authentication. The visit may thus begin with a check-in process in which the patient is either registered on-site or pre-registered off-site prior to a visit. Coordinated system interactions may be taken during this time, such as automatic access to the patient record established during the pre-visit phase. Additional information, similar to or supplementing the information collected prior to the visit may then be entered into the patient record. Patient conversation and inputs may be recorded manually or automatically during this interview process in preparation for a clinician or physician interview. As before, where voice data is collected, speech recognition engines may identify key medical terms or symptoms which can be associated with information in the integrated knowledge base to further enhance the diagnosis or treatment. Video data may similarly be collected to assess patient interaction, mental or physical state, and so forth. This entire check-in process may be partially or fully automated to make optimal use of institutional resources prior to actual interview with a clinician, nurse, or physician.
The on-visit may continue with an interview by a clinician or nurse. The patient conversation or interaction may again be recorded in audio or video formats, with complaints, symptoms and other key data being input into the integrated knowledge base, such as for identification of trends and temporal analysis of advancement of a condition or event. Again, and similarly, vital sign information may be updated, and the updated patient record may be evaluated for identification of trends and possible diagnoses, as well as or recommendations of additional medical procedures, as noted above.
The on-site visit typically continues with a physician or clinician interview. As noted above, during the on-site visit itself, analyses and correlations with information hi the integrated knowledge base may be performed with reports or recommendations being provided to the physician at the time of the interview. Again, the reports may provide
recommendations, such as rank-ordered proposals tor potential diagnoses, procedures, or simply information which can be gathered directly from the patient to enhance the diagnosis and treatment. The interview itself may, again, be recorded in whole or in part, and key medical terms recognized and stored in the patient's record for later use. Also during the on-site visit, reports, recommendations, educational material, and so forth may be generated for the patient or the patient care provider. Such information, again, may be customized for the patient and the patient condition, including explanations of the results of examinations, presentations of the follow-up procedures if any, and so forth. The materials may further include general health recommendations based upon the patient record, interaction during the contact and information from the integrated knowledge base, including general reference material. The material provided to the patient may include, without limitation, text, images, animations, graphics, and other reference material, raw or processed, structured video and/or audio recordings of questions and answers, general data on background, diagnoses, medical regimens, risks, referrals, and so forth. The form of such output may suit any desired format, including hard-copy printout, compact disk output, portable storage media, encrypted electronic messages, and so forth. As before, the communication may also be specifically adapted to the patient in a language of preference. The output may also include information on financial arrangements, including insurance data, claims data, and so forth.
The technique further allows for post-contact data collection and analysis. For example, following a patient visit, various patient interactions may be envisaged, as indicated generally at reference numeral 266 in Fig. 19. Such interactions may include general follow-up questions, symptom updates, remote vital sign capture, and the like, generally similar to information collected prior to the contact. Moreover, the post-contact patient interaction may include patient rating of an institution or care providers, assistance in filing or processing insurance claims, invoicing, and the like. Again, based upon such inputs, data is accessed, which may be patient-specific or more general in nature, from the integrated knowledge base to permit the information to the coordinated with patient records and all other available data to facilitate the
follow-up activities, and to generate any reports and feedback both for .the patient and for the care provider.
INTEGRATED KNOWLEDGE BASE INTERFACE
As noted above, the "unfederated" interface for the integrated knowledge base and, more -generally, for the processing system and resources, may be specifically adapted for a variety of users, environments, functions, and the like. Figure 20 generally illustrates an interface processing system which facilitates interactions with the integrated knowledge base. The system generally includes a series of input parameters or sources 270, which may be widely varied in nature, location, and utility. Based upon inputs from such sources, a logical parser 272, which may be generally part of the data processing system 10 described above, identifies interfaces and access of for interaction between users, hardware, and systems on one hand, and user workstations on the other, as well as access to the integrated knowledge base. The interface and access output functions, indicated generally at reference numeral 274, are then used to-pro vide customized interfaces and access to the integrated knowledge base depending upon the inputs received by the parser.
As summarized in Figure 20, input parameters or sources 270 may generally include parameters relating to users, including patients 4 and clinicians 6, as well as to any other users of the system, such as financial or insurance companies, researchers, and any other persons or institutions having the right to access the data. For user-initiated events, or any contact with the integrated knowledge base in which a user is involved, various access levels, functions, profiles, environments and the like may be considered in customizing the user interface and the level of access to the integrated knowledge base data and processing capabilities. By way of example, a radiologist reviewing an image or images at a review workstation, a technologist operating a CT scanner, or an administrator scheduling appointments or entering billing information may all be users to the system. The parameters or characteristics of the user which may be considered by the logical parser 272 may, as noted, vary greatly. In a present exemplary embodiment such characteristics include the function being performed by
the user, as noted at reference numeral 276, as well as a personal profile of a user as noted at reference numeral 278. The information relating to functions and personal profiles may, where appropriate, be subject to a manual override as indicated at reference numeral 280 in Figure 20. Moreover, all of the access by specific users may be filtered through various types of authentication as indicated in reference numeral 282.
In a typical scenario, a user may enter .an authentication module, such as on a workstation 304, illustrated in Figure 20, to enable secure access to the system. Where the function performed by the user is one of the criteria considered for interfacing and access, the user may be prompted to enter a current function, or the function may be recognized for the individual user profile, hi this matter, the same user may .have multiple functions in the system, such as in the case of thoracic radiologist at a hospital functioning as an interventionalist in one context and having additional functions as a mammographer at other periods, a manager at certain periods, and so forth. As a further example, a general practice nurse may function as a clinician at certain times, such as to input medical history information, and as an appointment scheduler at other times, and as a clerical person for input of billing, record data or insurance data at still other times. Each individual or institution, may customize one or more profiles containing personal preferences or information for each function. The. profile may. contain data about the user, and information describing the user interface preferences, if any, for different data access modes or functions.
Similarly, certain hardware or modality systems may have direct access to the integrated knowledge base, such as for uploading or downloading information useful in the analysis, processing, or data acquisition functions performed by the system. As illustrated in Figure 20, such hardware, denoted generally by reference numeral 284, may include imaging systems, patient input stations, general purpose of computers linked via websites, and so forth. The hardware may interface with the parser by similar designation of one or more functions 286, in a matter similar to that described above for the users. Similarly, parameters such as the environment of the hardware,
as indicated at reference numeral 288, may be considered. Such environments may provide an indication, for example, of where and how a system is used, such as .to differentiate specific functionalities of imaging systems used in emergency room settings from those used in other clinical applications, mobile settings, and so forth. As will be appreciated by those skilled in the art, such function and environment information may influence the type and amount of data-which can be accessed from or uploaded to the integrated knowledge base, and may be used, for example, in prioritization qr processing of information from the integrated knowledge base depending upon urgency of treatment, and so forth.
A general system input 290 is also illustrated in Figure 20, which may be considered by the logical parser. General system information may be relative to individual interfacing 'systems, including a system on which a user or piece of hardware interfaces with the knowledge base. By way of example, a system utilized by a user to interface with the knowledge base may, automatically or with user intervention, provide information relating to specific hardware devices, parameters, system capabilities, functions of the device, environments in which the devices are located or used, and so forth. Such information may indicate, for example, that a device is used as an image review workstation, such that different default interface characteristics may be employed in a radiology reading room and in an intensive care unit. Such interface characteristics may offer unique advantages, such as different presentation modes for similar data; customised resolution and bandwidth utilization, and so'forth.
Based upon the information provided to the logical parser 272, the parser determines appropriate user interface definitions, as well as definitions of access to the integrated knowledge base. Among the determinations made by the logical parser 272, may be allowable data state changes which can be initiated by the user, hardware or system, allowed methods and fields for data input and output, defined graphical or other (e.g. audio) presentation modes, and so forth. In providing such definition, the logical parser may draw upon specific levels or classifications of access, as well as upon specific pre-defined graphical interfaces or other fields, which are utilized in formulating the interfaces. In particular, for a given knowledge base request, the
logical parser 272 may utilize algorithms embedded within the knowledge base interface software, pre-defined sets of instructions from an interface manager, or self-learning algorithms, in addition to such pre-defined access and interface configurations. Where a user is allowed to manually override characteristic data or configurations, the logical parser may customize the interface or given application or function. For example, an individual user may utilize a review workstation 304 in an intensive care unit to review a trauma case, but utilizing default emergency room settings by overriding the intensive care unit settings. A wide variety of other definitional functions and overrides may be envisioned, all permitting standard and customized interfaces and access levels to the integrated knowledge base.
Among the functions defined by the logical parser are certain functions for defining the user interface, and other functions for defining access to the integrated knowledge base. As illustrated in Figure 20, such functions may include a definition of allowed input fields, as illustrated at reference numeral 292. Such fields may, in the context of a graphical user interface, be shown, not shown, or "grayed out" in a particular user interface, depending upon the factors discussed above. In addition, allowed input modes, as indicated at reference numeral 294, may be defined, again allowing various types of input, such as through the display or non-display of specific input pages, interactive web pages, and so forth. Similarity, specific graphical interfaces may be defined by the logical parser as indicated at reference numeral 296. It should be noted, that the various interface fields, modes, and presentations identified by the logical parser based upon the input information may be stored remotely, such as in the processing system or system data repository, or locally in a management system or within a workstation 304 itself.
The logical parser may also define specific levels of interaction or access which are permitted between users, systems, and hardware on one hand, and the integrated knowledge base on the other. Such access control may define both the accessing of information from the knowledge base, and the provision of information to the knowledge base. The access control may also define the permitted processing functions associated with the knowledge base via the data processing system, hi the
examples illustrated in Figure 20, such functions may include defining allowed data for read access, as indicated at reference numeral 298, defining allowed data for read-write access, as indicated at reference numeral 300, and defining allowed data for write access, as indicated at reference numeral 302.
As noted above* the interface processing system 268 permits various types of authentication to be performed, particularly for users attempting to gain access to the' integrated knowledge base. This authentication function may be achieved in a range of manners, including by password comparisons, voice recognition, biometrics, script or files contained within an interface device (e.g. a "cookie") or password file, and so 'forth. Because a wide range of diverse data may be included in the integrated knowledge base, authentication and security issues can be the focus of specific software and devices to carefully guard access and avoid tampering or unauthorized access. Thus, in addition to the use of standard user authentication protocols, data encryption techniques for knowledge communicated to and from the knowledge base may be employed, and associated infrastructure may be offered at input sides and output sides of the interface.
In general, a user may be responsible for setting the security or access level for data generated or administrated by that user, or other participates may be responsible for such security and access control. Thus, the system can be programmed to implement default access levels for different types of users or user functions, as noted above. Moreover, different privacy levels may be set by a user for different situations and for other users. Specifically, a patient or primary care physician may be in a best position to set access to his or her medical data, such that a specific set of physicians or institutions can access the information, depending upon their need. Access can also be broadened to include other physicians and institutions, such as in the event of accident or incapacitation of a patient. Moreover, access levels can be sorted by individual, situation, institution, and the like, with particular access levels being implemented in particular situations, such as in case of emergency, for clinical visits, during a transfer of control or oversight to an alternative physician during periods of a vacation, and so forth.
In general, the authentication and security procedures may be implemented through
software which may question a patient and implement defaults based upon the
•
responses. Thus, a patient may be prompted for classes of individuals, insurance companies, primary care physicians and specialists, kin, and the like, as well as for an indication of what level of access is to be provided to each class. Parsing and access to the information, as well as customization of the interfaces may then follow such designations.
Certain inherent advantages flow from the interface system described above. By way of example, an individual patient can become, effectively, a data or case manager granting access to information based upon the patient's desires and objectives. The mechanism can also be customized, and easily altered, for conformance with local, state and federal or other laws or regulations, particularity those relating to access to patient data. Such regulations may also relate to access to billing and financial information, access by employers, disability information, access to and for insurance claims, Medicare and Medicaid information, and so forth. Moreover, the technique offers automatic or easily adapted compliance with hospital information system data access regulations, such that data can be flagged to insure privacy based upon the user or access method. Finally, the technique provides for rapid and convenient setting, such as by the patient or a physician, of privacy levels for a broad range of users, such as by class, function, environment, and so forth.
MULTI-LEVEL SYSTEM ARCHITECTURE
As described generally above, the present techniques offer input, analysis, processing, output and general access to data at various levels, for various users, and for various needs. In particular, the system offers the capability of providing various levels of data access and processing, with all of the various levels generally being considered as contributing to, maintaining, or utilizing portions of the integrated knowledge base and functionality described herein. The various levels, rising from a patient or user level may include workstations, input devices, portions of the data processing system, and so forth which contribute the needed data and which extract needed data for the
functionality carried out at the corresponding level. Where levels in the system architecture can satisfy the users needs, such as within a specific institution, insurance company, department, region, and so forth, sharing and management of data may take place solely at such.levels. Where, however, additional functionality, is desired, the system architecture offers for linking the lower and any intermediate levels as necessary to accommodate such functionality.
Figures 21 and 22 generally illustrate exemplary architectures and management functions carried out in accordance with such multi-level architectures. Figure 21 illustrates the present data exchange system 2 as including a number of integrated levels and clusters of input and output stations or users. The users, which would typically be. patients 4 or clinicians 6 (including radiologists, nurses, physicians, management personnel, insurance companies, research institutions, and so forth) reside at fundamental or local level 306. As noted above, various functionalities may be carried out at such local levels, including tailoring of data input and output functions, access control, interface customization, and so forth. Within a local group or cluster level 308, then, such users may 'communicate with one another and with system elements of the type described above. That is, each local group or cluster 'level 308 may include any or all of the various resources discussed above, including both data resources and. controllable and prescribable resources. In a practical implementation, a local group or cluster level 308 may include, by way of example, departments within a particular institution, institutions affiliated in some way, institutions located in a specific geographical region, institutions linked by virtue of their practice area or specialization, and so forth. The Unking of the users and components at such local group or cluster levels, .then, permits specific functions to be carried out, to the extent possible, fairly locally and without the need to access remote data resources .or other local groups or clusters.
Similar remote groups or clusters may then be linked, and may be similar or generally similar internal structures, as indicated at reference numerals 310, 312, and 314 in Figure 21. It should be noted, however, that each of such clusters may vary widely in size, character, and even in its own network architecture, depending upon the needs
and functions of the users within the group or cluster. The various local groups and cluster levels, then, may be linked by one or more central clusters as indicated generally at reference numeral 318.
Although a "centralized/decentralized" system architecture is generally illustrated in Figure 21, it .should also be borne in mind that the functionality .of the multi-level system offered by aspects of the present technique may take on various analytical forms. That is, any or all of available network architectures, including centralized architectures, ring Structures, hierarchical structures, decentralized structures, centralized structures, and combinations of these may reside at the various levels in the overall system. Moreover, the various remote groups or clusters may, where desired, be linked to one another in alternative fashions without necessarily passing through a central group or cluster. Thus, preferential links between specific institutions or practitioners may be provided such that a "virtual cluster" is defined for the exchange of data and processing of data. Such links may be particularly useful where special relationships or repetitive operations are carried out between such users.
The functions described above, including the data acquisition, processing, analysis, and other functions may be carried out at specific workstations within the architecture of Figure 21, within local groups or clusters, or by use. of more expanded resources incorporating one or more remote group or cluster. Certain of these functions, according to the multi-level architecture scenario, are generally illustrated in Figure 22. As shown in Figure 22, certain functions may be carried out at local group or cluster levels 308, with generally similar functions being carried out at higher levels 318. Again, it should be noted that the same or similar functions may even be carried ' out at an individual terminal or workstation, and that further levels may be provided in the architecture.
As illustrated in Figure 22, users 4, 6 may be linked to the system and inputs and access filtered through a security/access control modules 320.- As noted above, such modules may employ various forms of security and access control, such as based upon passwords, voice recognition, biometrics, and more sophisticated techniques. In
general, the modules 320 will maintain a desired level of assurance that those linking to the network have rights to the specific data to be uploaded, downloaded, or processed. The modules 320 allow the users to gain access to a local knowledge base 322 which, from a general standpoint, may be considered to be part of the integrated knowledge base discussed above. It should also be noted that the local knowledge base 322 may also incorporate features of a federated database as discussed above wherein certain data may be pre-processed or translated for use by the programmed functionalities..
A validation or data management module-324 will typically be provided in some form to control access to and quality of data within the local knowledge base 322 and data from the other components of the overall system. That is, certain data, particularly that data which is used at a local level, may be preferential stored within the local knowledge base 322. However, where the overall system functionality requires, such data may be uploaded to higher levels, or to piers in other local groups or clusters. Similarly, data may be downloaded or processed from other remote sources. To maintain the validity and quality of such data, the validation and data management module 324 may carry out.specific functions, typically bi-directionally, as indicated in Figure 22. Such functions may include' those of the reconciliation modules as indicated at reference numeral 326, which can reconcile or validate certain data, such as based upon time of entry, source of the data; or any other validating criteria. Where such reconciliation or validation is not available, such as due to conflicting updates or inputs, such matters may be flagged to a user for reconciliation. A synchronizer module 328 'provides, similarly, for synchronizing records between the local knowledge base 322 and remote resources. Finally, a link-upload/download module . 330 provides for locating, accessing, and either storing up or downloading from other memories or repositories for the data from the local knowledge bases.
Generally similar functionality may be carried out, then, at other levels or within other relationships, as indicated generally by 318 in Figure 22. Thus, as between local groups or clusters, security and access control modules 332 may, in conjunction with modules 320, provide secure access to data from other users, groups, clusters or
levels. Moreover, cluster knowledge base 334 may be maintained which compliment, or even replicate some of the local knowledge base data. As with the local knowledge base 322, the cluster knowledge base 334 may be generally considered to be part of the overall integrated knowledge base. Other functions may be performed at such higher levels as well. Thus, as indicated at reference numeral 336, validation and data management modules may be implemented which, again, may be coordinated with the functionality of similar modules 324 at local levels. Such modules may, again, include reconciler modules 338, synchronizer modules 340 and link/upload/download modules 342 which facilitate exchange of data between groups or clusters.
The multi-level architecture described -above offers significant advantages and functionalities. First, data may be readily accessed by specific members of groups or clusters with specifically-tailored access control functions. That is, for suck functions as insurance billing, clinical analysis, and so forth, reduced levels of securities may be provided within a specific group or cluster. Access to data by other users in other groups or clusters, then, may be more regulated, such as by application of different security or access control mechanisms. Moreover, certain functionalities may be provided at very basic levels, such as at patient or clinician workstations, with additional access to data and processing capabilities being linked as necessary.
Moreover, it should be noted that in presently contemplated embodiments, the overall network topology tends to mirror the underlying data structure which in itself mirrors and facilitates computer-assisted data operation algorithms discussed below. That is, where functionality or data are related by specific relationships, processing needs, access needs, validation needs, and so forth, the establishment of groups or clusters may follow similar structures. That is, as noted above, "typical" access, use, needs, and functionalities may reside at more or less tight nodes or clusters, with more distant or infrequent structures or functionalities being more distributed.
The linking of various clusters or groups also permit functionalities to be carried out that were heretofore unavailable in existing systems. For example, analysis for trends, relationships and the like between data at various groups or cluster levels may be
facilitated which can aid in identifying traditionally unavailable information. By way of example, where a specific prevalence level of a disease state occurs at a specific institution, department within an institution, or a geographic region, existing systems tend to not recognize or belatedly recognize any relationship between such occurrence and similar occurrences in other locations. The present system, on the other hand, permits such data to.be operated upon, mined, analyzed, and associated so as to easily and quickly recognize the development of trends at various locations and even related by various data, such as quality of care, and so forth. Thus, coordinated access and analysis of peer information is available for identification of such disease states in overall population.
Similarly, resource management may be improved by the multi-level architecture offered by the present technique. In particular, trends, both past and anticipated in inventory use, insurance claims, human resource needs, and so forth may also be identified based upon the availability of data and processing resources at the various levels described above.
PATENT-ORIENTED MEDICAL DATA MANAGEMENT
The present technique offers further advantages in the ability of patients to be informed and even manage their own respective medical care. As noted above, the system can be integrated hi such a manner as to collect patient data prior to medical contacts, such as office visits. The system also can be employed to solicit additional information, where needed, for such interactions. Furthermore, the system can be adapted to allow specific individualized patient records to be maintained that may be controlled by the individual patient or a patient manager. Figure 23 generally represents aspects of the technique designed for creation and management of integrated patient records.
As shown in Figure 23, the arrangement of functionalities and modules may be referred to generally as a patient-management knowledge .base system 344, which at least partially includes features of the integrated knowledge base and other techniques described above. A patient 4 provides patient data, as indicated generally at reference
numeral 346 in Figure 23. The patient data may be provided in any suitable manner, such as via hard copies, analysis of tissue samples, input devices at institutions or clinics, or input devices which are individualized for the patient. Such input devices may include, for example, devices which are provided to, worn by, implanted in, or directly implemented by the patient as at the patient's home or place of employment. Thus,, the patient data 346 may be provided by mobile samplers (e.g. for blood analysis), sensing systems for physiological data (e.g. blood pressure, heart rate, etc.). The patient data may be stored locally, such as within the sensing device or within a patient computer or workstation. Similarly, the patient data may be provided either at the prompting of the patient or through system prompting, such as via accessible Internet web pages. Further, patient data may be extracted from external resources, including the resources of the integrated knowledge base as described more fully below. Thus, the patient data, in implementation, may be exchanged in a bi-directional fashion such that the patient may provide information to the record and access information from the record. Similarly, the patient may manage input to the record of data from outside resources as well as manage access to output of the record to outside resources.
The patient data is exchanged with other element of the system via a patient network interface 348. The patient network interface may be as simple as a web browser, or may include more sophisticated management tools that control access to, validation of, and exchange of data between the patient and the outside resources. The patient network interface may communicate with a variety of other components, such as directly with care providers as indicated at reference numeral 350. Such care providers may include primary care physicians, but may also include institutions and offices that store patient clinical data, and institutions that store non-clinical data such as insurance claims, financial resource data, and so forth. The patient network interface 348 may further communicate with reference data repository 352. Such reference data repositories were discussed above with general reference to the integrated knowledge base; The repositories 352 may be the same or other repositories, and may be useful by the patient network interface for certain processing functions carried out by the interface, such as comparison of patient data to known ranges or demographic information, integration

into patient-displayed interface pages of background and specific information relating to disease states, care, diagnoses and prognoses, and so forth. The patient network interface 348 where necessary, may further communicate with a translator or processing module as indicated generally at reference numeral 354. The translator and processing modules may completely or partially transfonn the accessed data or the patient data for analysis and storage. Again, the translator and processing, functions may be bidirectional such that they may translate and process both data originating from the patient and data.transferred to the patient from outside resources.
An integrated patient record module 356 is designed to generate an integrated patient record, as represented generally by reference numeral 362 in Figure 23. As used in the present context, the integrated patient record may include a wide range of information, both acquired directly from the patient, as well as acquired from institutions which provide care to the patient. The record may also include data derived from such data, such as resulting from analysis of raw patient data, image data, and the like both by automated techniques and by human care providers, where appropriate. Similarly, the integrated patient record may include information incorporated from reference data repositories 352. The integrated patient record module preferably stores some or all of the integrated patient record 362 in one or more data repository 358.
As noted above, the system 344 permits creation of an integrated patient record 362 which may include a wide range of patient data. In practice, the integrated patient record, or portions of the patient record, may be stored at various locations, such as at a patient location as indicated adjacent to the patient data block 346, at individual care providers (e.g. with a primary care physician) as indicated adjacent to block 350, or within a data repository 358 accessed by the integrated patient record module 356. It should also be noted that some or all of the functionality provided by the patient network interface 348, the translator and processing module 354 and the integrated patient record module 356 may be local or remote to the patient. That is, software for carrying out the creation and maintenance of the patient record may be stored direct at a patient terminal, or may be fully or partially provided remotely, such as through a subscription service. Similarly, the patient record repository 358 may be local or remote from the patient.
The integrated patient record module 356 also is preferably designed to communicate with the integrated knowledge base 12 via an integrated knowledge base interface 360. The interface 360 may conform to the general functionalities described above with respect to access, validation, tailoring for patient needs or uses, and so forth. The integrated knowledge base interface 360 permits the extraction of information from resources 18, which may be internal to specific institutions as indicated in Figure 23. The interface also permits data from the patient to be uploaded to such resources and institutions. As also noted in Figure 23, the integrated patient record 3S6, fully or in part, may be stored generally within the integrated knowledge base 12 to facilitate access by care providers, for example. The record may also be stored within individual institutions, such as within a hospital or clinic which has or will provide specific patient care.
The system functionality illustrated in Figure 23 offers significant advantages. By way of example, as noted above, the access to specific information and the creation of records may be controlled and regulated more directly by a patient That is, the system serves as an enabler for empowering the patient with respect to proactive management of medical records. Such interaction may take the form of patient-controlled access to portions of the patient record provided to specific care providers. Similarly, the system offers the potential for improving the education of the patient as regards to general questions as well as specific clinical and non-clinical issues. The system also provides a powerful tool for accessing patient data, including raw data, processed data, links, updates, and so forth which may be used by care providers for identifying and tracking patient conditions, scheduling patient care visits, and so forth. Such functions may-be provided by "push" or "pull" exchange techniques, such as on a timed basis, or through notifications, electronic messages, wireless messages, and so forth. Direct interaction with the patient may include, therefore, uploading of patient data, downloading of patient data, prescription reminders, office visit reminders, screening communications, and so forth. Moreover, the integration of the patient data with other functionality and data from other resources permits the integrated patient record to be created and stored periodically or in advance of specific needs by the patient or by an institution, or
compiled at the time of a specific query by linking to and accessing data for response to the query.
PREDICTIVE MODELING
The present technique, by virtue of the high degree of integration of the data storage, access and processing functions described aBbve, provides a powerful tool for development of predictive models, both clinical and non-clinical in nature. In particular, data can be drawn from the various resources in the integrated knowledge base or a federated data base, processed, and analyzed to improve patient care by virtue of predictive model development. The development of such predictive models can be fully or partially automated, and such modeling may serve to adapt certain computer-assisted functions of the types described above. .
Figures 24 and 25 generally illustrate aspects of predictive model development which may be implemented in accordance with aspects of the present technique. Figure 24 represents a predictive modeling system 364 that may be built upon or compliment the integrated knowledge base and network functions described above. The predictive modeling system 364 draws upon the resources 18, both data resources and controllable and prescribable resources, as well as upon any federated databases 14 provided in the system and upon the integrated knowledge base 12, which again may be centralized or distributed in nature. The system 364 relies upon software identified in Figure 24 as data mining -and analysis modules 366 designed to extract data from the various resources, knowledge bases and databases, and to identify relationships between the data useful in developing predictive models. The analysis performed by the data mining and analysis modules 366 may be initiated in any suitable manner, as indicated by the initiators block 368 in Figure 24, including any or all of the initiating events outlined above with reference to Figure 18. Once processing is initiated, the modules search for and identify data which may be linked to specific disease states, medical events, or to yet unidentified or unrecognized disease states or medical events. Moreover, the modules may similarly seek non-clinical data for development of similar models, such as for prediction of resource needs, resource allocation, insurance rates, financial planning, and
so forth. It should be noted that the data mining and analysis functions performed by the modules 366 may operate on "raw" data from the resources and databases (again both clinical and non-clinical), as well as on filtered, validated, reduced-dimension, and similarly processed data from any one of these resources. Moreover, initiation of such processing, or validation of data may be provided by an expert, such as a clinician represented at reference numeral 6 in Figure 24.
Based upon the mining an analysis performed by modules 366, a predictive model development module 370 further acts to convert the data and analysis into a representative model that can be used for diagnostic, planning, and other purposes. In the clinical context, a wide range of model types may be developed, particularly for refinement of computer-assisted processes referred to above. As noted above, these processes, referred to here in as CAX processes, permit powerful computer-assisted work flow such as for acquisition, processing, analysis, diagnostics, and so forth. The methodologies employed by the predictive model development module 370 may vary depending upon the application, the data available, and the desired output In presently contemplated embodiments, for example, the processing may be based upon regression analysis, decision trees, clustering algorithms, neural network structures, expert systems, and so forth. Moreover, the predictive model development module may target a specific disease state or medical condition or event, or may be non-condition specific. Where data is known to relate to a specific medical condition, for example, the model may consist in refinement of rules and procedures used to identify the likelihood of occurrence of such conditions based upon all available information from the resources and knowledge base. More generally, however, fee data mining and analysis functions, in conjunction with the model development algorithms, may provide for identification of disease states and relationships between these disease states and available data which were not previously recognized.
hi applications where the predictive model development module 370 is adapted for refinement of a computer-assisted process CAX, the model may identify or refine parameters useful in carrying out such processes. The output of the module 370 may therefore consist of one or more parameters identified as relating to a specific condition,
event or diagnosis. Outputs from the predictive model development module 370, typically in the form of data relationships, may then be further refined or mapped onto parameters available to and used by the CAX processes 85 illustrated in Figure 24. In a presently contemplated embodiment, therefore, a parameter refinement function 372 is provided wherein parameters utilized in the CAX processes 85 are identified, as indicated at reference numeral 374, and "best" or optimized values or ranges of the values are identified or as indicated at reference numeral 376. The parameters and their values or ranges are then supplied to the CAX process algorithms for future use in the specific process. As a general rule, the CAX processes produce some output as indicated at reference numeral 378. .
It should, be noted that various functions performed and described above in the
predictive modeling system 364 may be performed on one or more processing systems,
and based upon various input data. Thus; as mentioned above, the integrated knowledge
base and therefore the data available for predictive model development is inherently
expandable such that models may be developed differently or enhanced as improved or
additional information is available. It should also be noted that the various components
of the system illustrated in Figure 24 provide for highly interactive model development
That is, various modules and functions may influence one another to further improve
model development. •
By way of example, where a predictive model is developed by module 370 based upon specific data mining, the model development module may identify that additional or complimentary data would also be useful in improving the performance of the CAX processes. The model development module may then influence the data mining and analysis function based upon such insights. Similarly, the identification of parameters and parameter optimization carried out in the parameter refinement process can influence the predictive model development module. Furthermore, the results of the CAX process 85 can similarly affect the predictive model development module, such as for development or refinement of other CAX processes.
The latter possibility of interaction between the components and functions illustrated in Figure 24 is particularly powerful. In particular, it should be recognized that the predictive model development module 370 may, in some respects, itself serve as a CAX process 85, such as for recognizing relationships between available data and matching such relationships to potential disease -states, events, resource needs, financial considerations, and so forth. The process is not limited to any particular CAX process, however. Rather, although model development may focus on the diagnosis of a disease state; for example, the output of the CAX process (e.g. computer-assisted diagnosis or detection) may give rise to improvements in processing and modeling of desired processing of data. Similarly, the results of the CAX process in processing may lead to recognition of improvements in a model implemented for computer-assisted acquisition (CAA) of data. Other computer-assisted processes, including computer-assisted assessment (CAAx) of health or financial states, prognoses, prescriptions, therapy, and other decisions may similarly be impacted both by the predictive model development module, and by feedback from refined other processes.
As illustrated in Figure 24, certain steps involved in development of clinical and non-clinical predictive models may be subject to validation or input from elements of the system or from experts. Thus, the CAX output 378 would typically be reviewed by an expert 6. Similarly, CAX output which may influence the predictive model development module 370 is preferably subject to validation as indicated at block 380 in Figure 24. Such validation may be performed by the system itself (such as by crosschecking data or algorithm output, or by one or more experts). The output of the validation may then be linked to the resources, including the original resources themselves 18, and the integrated knowledge base 12. For. example, it may be useful to link or pre-process certain data, or flag certain data for use in the CAX processes implemented by the developed model.
In use, the developed or improved model will typically be available for remote processing or may be downloaded to systems, including computer systems, medical diagnostic imaging equipment, and so forth, which employ the model for improving data acquisition, processing, diagnosis, decision support, or any of the other functions served
by the CAX process. During such implementation, and as described above, the implementing system may access the integrated knowledge base, the federated database, or the originating resources themselves to extract the data needed for the CAX process.
Within the predictive model development module 370 several functions may.be resident and carried out either on a routine basis or as specifically programmed or initiated by a user or by the system. Figure 25 illustrates an example of certain of these processes carried out by the model development module. As shown in Figure 25, based upon data mined and analyzed (i.e. acquired or extracted from the resources), the module will typically identify relationships between, available data as indicated at block 382 of Figure 25. The relationships may be based upon known interactions between the data, or based upon identification algorithms as noted above (e.g. regression analysis, decision trees, clustering algorithms, neural networks, expert input, etc.). Moreover, it. should be noted that the relationship identification may be based on any available data. That is, the data may be most usefully employed in the system when considered separate from its type, modality, practice area, and so forth. By way. of example, clinical data may be employed from imaging systems and used in conjunction with demographic information and with histological information on a particular patient. The data may also incorporate non-patient specific (e.g. general population) data which may be further indicative of risk or likelihood of a particular disease state, and so forth. Based upon the identified relationships, rule identification is carried out as indicated at block 384. Such rules may include comparisons, Boolean relationships, regression equations, and so forth used to link the various items of data or input in the identified relationships.
Input refinement steps are carried out as indicated at'block 386 in which the relationships are linked to various data inputs which are available from the resources or database or knowledge base. As noted in Figure 25, such inputs 388 may be non-parametric, that is, relate to raw or processed data which is not specifically influenced by settings or parameters of the CAX process. Other input identification, as indicated at block 390, is targeted to parametric inputs which can be impacted by alteration of the CAX process. Based upon the input identification, the rule identification and the relationship identification, reconciliation and refinement of the model is possible as
indicated at block 392. Again, such reconciliation and refinement may include addition or deletion of certain inputs, placement of certain conditions on inclusion of inputs, weighting of some inputs, and so forth. Such reconciliation and refinement may be carried but by the system or with input from an expert as indicated at reference numeral 6 in Figure 25. The entire process, then, may be somewhat iterative as indicated by the return arrows in Figure 25, such that the reconciliation and refinement process may further impact identification of relationships, rules and inputs.
A wide range of models may be developed by the foregoing techniques. In a clinical context for example, different types of data as described above maybe accessible to the CAX algorithms, such as image data, demographic data, and non-patient specific data. By way of example, a model may be developed for diagnosing breast cancer in women residing in a specific region of a country during a specific period of years known to indicate an elevated risk of such conditions. Additional factors that may be considered where available, could be patient history as extracted from questionnaires completed by the patient (e.g. smoking habits, dietary habits, etc.).
As a further example, and illustrating the interaction between the various processes, a model for acquiring data or processing data may be influenced by a computer-assisted diagnosis (CADx) algorithm, hi one example, for example, the output from a therapy algorithm with highlighting of abdominal images derived from scanned data may be altered based upon a computer-assisted diagnosis. Therefore, the image data may be acquired or processed in relatively thin slices for a lower abdomen region where the therapy algorithm called for an appendectomy. The rest of the data may be processed in a normal way with thicker slices. Thus, not only can the CAX algorithms of different focus influence one another in development and refinement of the predictive models, but data of different types and from different modalities can be used to improve the models for identification and treatment of diseases, as well as for non-clinical purposes.
ALGORITHM AND PROFESSIONAL TRAINING
As noted above, a number of computer-assisted algorithms may be implemented in the present technique. Such algorithms, generally referred to herein as CAX algorithms,
may include processing and analysis of a number of types of data, such as medical diagnostic image data. The present techniques offer enhanced utility in refining such processes as described above, and for refining the processes through a learning or training process to enhance detection, segmentation, classification and other fiinctions carried out by such processes. The present techniques also offer the potential for providing feedback, such as for training purposes, of medical professionals at various levels, including radiologists, physicians, technicians, clinicians, nurses, and so forth. Figure 26 illustrates exemplary steps in such a training process both for an algorithm and for a medical professional.
Referring to Fig. 26, an algorithm and professional training process 394 is illustrated diagrammatically. The process may include separate, although interdependent modes, such as a professional training mode 396 and an algorithm training mode 398. In general, both modes may be programmed and functioned in one or more operating environments, with the actual functionality performed varying depending upon how the user is currently implementing the process.
In general, the process provides for interaction between computer-assisted algorithms, such as a CAD algorithm, and functions performed by a medical professional. The process will be explained herein in context of a CAD program used to detect and classify features in medical diagnostic image data. However, it should be borne in mind that similar processes can be implemented for other CAX algorithms, and on different types of medical diagnostic data, including data from different modalities and resource types.
The process 394 may be considered to begin at a step 400 where an expert or medical professional performs feature detection and classification. As will be recognized by those skilled in the art, such functions are typically performed as part of a diagnostic image reading process, beginning typically with a reconstructed image or a set of images in an examination sequence. The expert will typically draw the data from the integrated knowledge base 12 or from the various resources 18 and may draw upon additional data from such resources to support the "reading" process of feature detection and
classification. The expert then produces a dataset labeled Dl, and referred to in Fig. 26 by reference numeral 402, which may be an annotated medical diagnostic image in a particular application. Any suitable technique can be used for producing the dataset, such as conventional annotation, dictation, interactive marking, and similar techniques.
In parallel with the expert feature detection and classification functions, an algorithm, in the example a CAD algorithm, performs similar feature detection and classification functions at step 404. As noted above, various programs are available for such functions, typically drawing upon raw or processed image data, and identifying segmenting and classifying identified features in accordance with parametric settings. 'Such settings may include mathematically or logically-defined feature recognition steps, intensity or color-based feature detection, automated or semi-automated feature segmentation, and classification based upon comparisons of identified and segmented features with known characteristics of identified pathologies. As a result of step 404, a second dataset D2, referred to in Fig. 26 by reference numeral 406, is produced, which may be similarly annotated for display.
The expert-produced dataset 402 is subjected to verification by the same or a different computer algorithm at step 408. The algorithm verification step 408 is illustrated in broken lines in Fig. 26 due to the optional nature of this step when the system is operating in algorithm training mode. That is, the algorithm verification of the expert reading is preferred where feedback is provided to the expert as described below. Alternatively, the algorithm verification step may be implemented in all cases, such that a subsequently processed dataset includes both the reading by the expert and by the algorithm and the filtering of the expert-identified and classified features as produced by the algorithm verification step. In general, the algorithm verification step will serve to eliminate false positive readings as produced by the expert. It should also be noted that a particular algorithm and/or the parametric settings employed by the algorithm at step 408 may be different from those used in step 404. That is, the algorithm verification step may be performed by a different algorithm, or with different parametric settings, so as to provide a more or less stringent filter at step 408 than was applied for the algorithm feature detection and classification at step 404. Step 408 results in a further refined
dataset D3, referred to in Fig. 26 by reference numeral 410, which may constitute a reconstructed image, annotated to indicate, where desired, both the expert feature detection and classification results, and changes in such results as result of the algorithm verification.
Similarly, the dataset 406 resulting from the algorithm feature detection and classification is subjected to expert verification at step 412. As with step 408, step 412 may be an optional step, particularly where the system functions in professional training mode. That is, where feedback is intended to be provided to the medical professional or expert, the step may be eliminated so as to provide comparison of the algorithm feature detection and classification with that produced by the medical professional. It should also be noted that a particular expert and/or the decision thresholds employed by the expert at step 412 may be different from those used in step 400. The resulting dataset D4, referred to in Fig. 26 by reference numeral 414, again, may be reconstructed, when the data represents images, and may be annotated to indicate features identified by the algorithm and the changes made to such identification or classification by the expert or medical professional.
In a present implementation, the datasets 410 and 414 are joined in a union dataset 416, which may again comprise of one or more images displaying the origin of particular features detected and classified, along with changes made by either the algorithm or the expert during verification. Block 418 in Fig. 26 represents a reconciler which may be a medical professional (the same or a different medical professional than carrying out the feature detection and classification or verification), or the reconciler may include automated or semi-automated processing. The purpose of the reconciler 418 is to resolve conflicts between detection and classification by the algorithm and the expert, along with such conflicts that may result from modifications following the verification at steps 408 and 412.
Once the reconciler has acted upon the dataset DS, referred to in Fig. 26 by reference numeral 416, in an algorithm training mode 398, changes made by the expert verification at step 412 and by the reconciler 418 are analyzed as indicated at step 420.
The analysis may consist of comparing the changes made and determining why the changes were necessitated. As will be appreciated by those skilled in the art, CAX processing typically includes various settings which can be altered to change the feature identification, detection, segmentation, and classification that may have been performed.. The analysis performed at step 420, then, can be directed to identifying how such parametric inputs can be modified to permit the results of the verification and reconciliation to conform. It should be noted, however, the analysis performed at step 420 may not necessarily imply mat a change in the algorithm is needed to desired. That is, in certain situations it may be desirable that the algorithm not produce exactly the same results as the expert, in order to enhance the "second reader" or "independent first reader" nature of the algorithm functions. At step 422, then, validation of any possible changes to the algorithm are made, such as by an expert or a team of experts. Where the validation step results in a conclusion that a change in the algorithm may be in order, such modification may be implemented as indicated at step 424. While reference is made in the present process to parametric modification of such algorithms, it should also be noted that such modifications may include identification and consideration of other inputs, such as inputs available from the integrated knowledge base 12, as discussed above with reference to Fig. 24.
When operating in a professional training mode 396, similar analysis of the dataset 416 can be made as indicated at step 426 in Fig. 26'. Such analysis, again, may be intended to determine why changes in the expert reading were made by the algorithm in the verification 408, and how such performance can be brought into conformity. Based upon such analysis, at step 428 the results may be reported and instruction provided for the medical professional. It should be noted that such reporting and instruction may . simply provide feedback for the medical professional, such as to indicate changes that would have been made to the dataset 402 by algorithm verification. However, the reports or instruction may also provide useful didactic input, references to teaching materials, samples, image-based data retrieval, and so forth, such that the medical professional is apprised of relevant considerations for improvement of performance.
Following creation of the dataset 416, results may be reported and displayed in a conventional manner as indicated at step 430. Moreover, and optionally, other processes may be performed on the resulting data which may similarly provide assistance in refining either the CAX algorithm or teaching the medical professional. Such processes are illustrated in Fig. 26 at reference numerals 432 and 434.
It should be noted that the foregoing processes can be implemented as normal operating procedures, where desired, That is, complimentary algorithm and expert reading procedures, with complimentary algorithm and expert verification procedures, and with the use of a reconciler, may be employed.for regular handling of data for diagnostic and other purposes. In a professional training mode, however, a relatively "heavy" filter may be used at the algorithm verification step, such as to identify more positive reads as potential false positive reads for training purposes. A different or "lighter" filter may be used during normal operation and for the algorithm feature detection classification formed at step 404. In addition; the analysis'performed either at step 420 or at 426 may further rely upon the integrated knowledge base to identify trends, prognoses, and so forth based upon both patient-specific data, non-patient specific data, temporal data of both a patient-specific and non-patient specific nature, and so forth. It should also be noted that, as discussed above, various changes can be made to the CAX algorithms as a result of the training operations. Such changes may include changes hi processing, and may be "patient-specific", with such changes being stored for future analysis of data relating to the same patient. .That is, for example, for image data relating to a patient with certain anatomical characteristics (e.g. weight, bone mass, size, implants, prosthesis, etc.), the algorithm may be specifically tailored for the patient by altering parametric settings to enhance the utility of future application of the algorithm and future correction or suggestions made to expert readings based upon the determinations made by the algorithm. In addition, changes can also be made to the integrated knowledge base itself based upon the learning mode outcome, such as to adjust "normal ranges" within the data stored in the knowledge base.
IN VITRO CHARACTERISTIC IDENTIFICATION
As noted above, among the many resources and types of resources available for the present technique, certain resources will produce data or samples which may be subject to in vitro data acquisition and analysis. The present techniques offer a particularly useful tool in the processing of such data and samples for several reasons. First, the samples may be analyzed based upon input of data of multiple types of resources. Various computer-assisted processes, including data 'acquisition, content-based information retrieval, processing and analyzing of retrieved and/or acquired data, identification of characteristics, and classification of data based upon identified characteristics may be implemented. Moreover, temporal analysis may be performed to analyze characteristics of in vitro samples as they relate to previously-identified characteristics using known data, such as from the integrated knowledge base. The information retrieval processes may furthermore be based upon specific attributes of the in vitro sample, such as spatial attributes (e.g. size of specific components or characteristics), temporal attributes (e.g. change in features over time), or spectral attributes (e.g. energy level, intensity, color,'etc.). Such content, also identified, where possible, from information stored in the integrated knowledge base, may include biomarkers, images, relationship tables, standardized matrixes, and so forth. Thus, multiple attributes may be used to enhance the acquisition, processing and analysis of in vitro samples through reference to available data, particularly information in the integrated knowledge base.
Fig. 27 generally represents steps in processing of an in vitro sample in accordance with such improved techniques. The in viti-o characteristic identification process, generally represented by reference numeral 436 in Fig. 27, begins at step 438 where the in vitro diagnostics sample is acquired. As noted above, any suitable technique can be used for acquiring the sample, which may typically include body fluids, tissues, and so forth. At step 440 an analysis is performed on the acquired sample. The analysis is informed by input from the integrated knowledge base as indicated at block 442. The input may include data relating to other modalities, resource types, or temporal data relating to similar samples from the patient. The analysis performed at step 440 may include
certain comparisons with such data and may be somewhat preliminary in nature. Thus, without departing from the acquisition step in the overall process, the sample acquisition may be tailored to the needs of the process as indicated at step 444. Such tailoring may include acquisition of other samples, acquisitions of samples under specific conditions (e.g. later in time during an office visit, during patient activity or rest periods, from other regions of the body,, and so forth). Thus, the in vitro-diagnostics sample acquisition process may be improved by computer analysis that influences the acquisition of the sample itself.
Following acquisition of the sample, processing of the sample may be performed at step 446. The processing performed at step 446, rather than data processing, is typically sample processing to condition the sample for extraction of data either manually or in a semi-automated or fully-automated process. Following the processing at step 446, results of the processing are analyzed at step 448. As before, the analysis performed at step 448 may include consideration of data from the integrated knowledge base, including data from other modalities, resource types, and times. As with the analysis performed at- step 440, the analysis at step 448 may be preliminary in nature, or further analysis may be performed by tailoring the processing as indicated at step 452. Thus, prior to final analysis of an in vitro diagnostic sample, additional processing may be in order, such as slide preparation, analysis for the presence of various chemicals, tissues, pathogens, and so forth.
At step 454 results of the analysis are compared to known profiles, such as from the integrated knowledge base, to determine possible diagnoses. As before, the comparisons made at step 454 may be based upon data from different modalities, resource types and times. The comparisons may result 'in classification of certain data indicative of disease.states, medical events, and so forth as indicated at step 458. The comparison and classification may further indicate that a specific patient (or a population of patients) is undergoing certain trends that may be indicative of potential diagnoses, prognoses, and so forth. The results of the classification made at step 458 may be validated, such as by a medical professional, at step 460.
In general, for the present purposes, quantifiable signs, symptoms and/or analytes (e.g. chemicals, tissues, etc:) in biological specimens characteristic of a particular disease or predisposition for a disease state or condition may be referred to as "biomarkers" for the disease or condition. While reference has been made hereinto analysis and comparison in general, such biomarkers may include a wide range of features, including the spatial, temporal and spectral attributes mentioned above, but also including genetic markers (e.g. the presence or absence of specific genes), and so forth.
By way of example, in a typical application, a patient's tissue will be sampled and transmitted to a laboratory for analysis. The laboratory acquires the data with computer assistance using appropriate detectors, such as microscopes, fluorescent probes, micro arrays, and so forth. The data contents, such as biomarkers, image signals, and so forth are processed and analyzed. As noted above, the acquisition and processing steps themselves may influenced by the reference to other data, such as from the integrated knowledge base. Therefore, such data is retrieved from the knowledge base for assisting in the acquisition, analysis, comparison and classification steps.
The comparisons made in the process may be parametric in nature or non-parametric. That is, as noted above, parametric comparisons may be based upon measured quantities and parameters where characteristics are indexed or referenced in parameter space and comparisons are performed in terms of relative similarity of one dataset to another with respect to certain indices, such as a Euclidean distance measure between two feature set vectors. Such indices may include, in the example of microscopy, characteristic cell structures, colors, reagent, indices, and so forth. Other examples may include genetic composition, presence or absence of specific genes or gene sequences, and so forth.
Non-parametric comparisons include comparisons made without specific references to indices, such as for a particular patient over a period of time. Such comparisons may be based upon the data contents of one dataset that is compared for similarity to characteristics from the data contents of another dataset. As will be noted by those skilled in the art, one or both of such comparisons may be performed, and hi certain situations one of the comparisons may be preferred over the other. The parametric
approach is typically used when a comparison is to be made between a given specimen and a different specimen with known characteristics, such as based upon, information from the integrated knowledge base. For example, in addition to deriving textures and shape patterns of cells in a histopathology image, parameters may also be derived from demographic data, electrical diagnostic data, imaging diagnostic data, and concentrations of biqmarkers in biological fluid or a combination of these. Thus; the comparisons can be made based upon data from different modalities and different resource types, as noted above. Non-parametric comparisons may generally be made, again, for temporal comparison purposes. By way of example, a specimen may exhibit specific ion concentrations dynamically changing and temporal variations of data attributes (e.g. values, ratios of values, etc.) may need to be analyzed to arrive at a final clinical decision.
COMPUTER-ASSISTED DATA OPERATING ALGORITHMS
As noted above, the present technique provides for a high level of integration of operations in computer-assisted data operating algorithms. As also noted above, certain such algorithms have been developed and are in relatively limited use in various fields, such as for computer-assisted detection or diagnosis of disease, computer-assisted processing or acquisition of data, and so forth. In the present technique, however, an advanced level of integration and interoperability is afforded by interactions between algorithms both in their development, as discussed above with regards to model development, and in their use. Moreover, such algorithms may be envisaged for both clinical and non-clinical applications. Clinical applications include a range of data analysis, processing, acquisition, and other techniques as discussed in further detail below, while non-clinical applications may include various types of resource management, financial analysis, insurance claim processing, and so forth.
Fig. 28 provides an overview of interoperability between such algorithms, referred to generally in a present context as computer-assisted data operating algorithms or CAX. As noted above, CAX algorithms in the present context may be built upon algorithms presently in use, or may be modified or entirely constructed on the basis of the additional
data resources, integration of such data resources, or interoperability between such resources in the algorithms and between the algorithms themselves as discussed throughout the present description. In the overview of Fig. 28, for example, an overall CAX system 462 is illustrated as including a wide range of steps, processes or modules which may be included in a fully integrated system. As noted above, more limited implementations may also be envisaged in which some or a few only of such processes, functions or modules are present Moreover, in a presently contemplated embodiment, such CAX systems are implemented in the context of integrated knowledge basis such that information can be gleaned to permit adaptation and optimization of both the algorithms themselves and the data managed in the algorithms. Such development and optimization may be carried out, as noted above, through the model development modules described herein, and various aspects of .the individual CAX algorithms may be altered, including rules or processes implemented hi the algorithms, as well as various settings. More will be said about such aspects of the CAX algorithms below with regards to Fig. 29.
As summarized in Fig. 28, in general, the CAX algorithms begin at a step 464 in which data is acquired. As noted throughout the present discussion, the acquisition of data may take many forms, particularly depending upon the resource type and the resource modality providing the data. Thus, data may be input manually, such as from forms or conventional terminals, or data. may be acquired through laboratory reporting techniques, imaging systems, automatic or manual physiological parameter acquisition systems, and so forth. The data is typically stored in one or more memory devices as discussed above, some of which may be incorporated in the data acquisition systems themselves, such as in imaging systems, picture archiving systems, and so. forth.
At step 466 data of interest or utility for the functions carried out by the CAX algorithm is accessed. A series of operations may then be performed on the accessed data as indicated generally at reference numeral 468. Throughout such processing, and indeed at step 466, the integrated knowledge base 12, in full or in part, may be accessed to extract data, validate data, synchronize data, download data or upload data during the functioning of the CAX algorithm.
While many such computer-assisted data operating algorithms may be envisaged, at present, some ten such algorithms are anticipated for carrying out specific functions, again both clinical and non-clinical. Summarized in Fig. 28, therefore, are steps in algorithms for computer-assisted detection of features (CAD), and algorithms for computer aided diagnosis of medical conditions (CADx). Further, computer-assisted clinical decision algorithms (CADs) are implemented in which clinical decisions are automatically made based upon 'analysis and processing. Similarly, therapeutic or treatment decisions may be implemented through additional routines (CATx). Specific computer-assisted acquisition (CAA) and computer-assisted processing (CAP) algorithms may be implemented of type described in detail above. Further, computer-, assisted analysis (CAAn) algorithms may be implemented as discussed below. Computer-assisted prediction or prognosis (CAPx) algorithms are also envisaged in a medical context, as are prescription validation, recommendation or processing algorithms (CARx). Finally, computer-assisted assessment (CAAx) algorithms are envisaged for a range of conditions, both clinical and non-clinical.
Considering in further detail the data operating steps summarized in Fig. 28, at step 470 accessed data is generally processed, such as for digital filtering, conditioning of data, adaptation of dynamic ranges, association of data, and so forth. As will be appreciated both those skilled in the art, the particular processing carried out in step 470 will depend upon the type of data being analyzed in the type of analysis or functions being performed. It should be noted, however, that data may be processed from any of the resources discussed above, and indeed data from more than one modality or even type of resource may be processed, such as for complex analysis of the presence risk, or treatment of medical conditions, and so forth. At step 472, similarly, analysis of the data is performed. Again, such analysis will depend upon the nature of the data and the nature of the algorithm on which the analysis is performed.
Following such processing and analysis, at step 474 features of interest are segmented or circumscribed in a general manner. Again, in image data such feature segmentation may identify the limits of anatomies or pathologies, and so forth. More generally, however, the segmentation carried out at step 474 is intended to simply discern the limits of any
type of feature, including various relationships between data, extents of correlations, and so forth. Following such segmentation, features may be identified in the data as summarized at step 476. While such feature identification may be accomplished on imaging data to identify specific anatomies or pathologies, it should be borne in mind that the feature identification carried out at step 476 may be much broader in nature. That is, due to the wide range of data which may be integrated into the inventive system, the feature identification may include associations of data, such as clinical data from all types of modalities, non-clinical data, demographic data, arid so forth. In general, the feature identification may include any sort of recognition of correlations between the data that may be of interest for the processes carried out by the CAX algorithm. At step 478 such features are classified. Such classification will typically include comparison of profiles in the segmented feature with known profiles for known conditions. The classification may generally result from parameter settings, values, and so forth which match such profiles in a known population of datasets with a dataset under consideration. However, the classification may also be based upon non-parametric profile matching, such as through trend analysis for a particular patient or population of patients over time.
Based upon the processing carried out by the algorithm, a wide range of decisions may be made. As summarized in step 462, such decisions may include clinical decisions 480, therapeutic decisions 482, data acquisition decisions 484, data processing decisions 486, data analysis decisions 488, condition prediction or prognosis decisions 490, prescription recommendation or validation decisions 492, and assessment of conditions 494. As noted above, the high level of integration of the processing operations provided by the present technique, and the integration of data from a range of resources, permits any one of the categories of functions carried out by the CAX algorithm to be modified or optimized, both for non-patient specific reasons and for patient-specific reasons, as summarized .in Fig. 28. Thus, as a result of any one of the decisions made in the algorithm, modifications in the same or different CAX algorithms may be made as summarized at step 496. As also noted below, such modifications may include selection of a different algorithm type, modification, addition or removal of one or more functions
carried out by the algorithm, or modification of parameters and settings employed by the algorithm in carrying out the functions. Thus, in the flow diagram of Fig. 28, feedback may be had to any one of the steps summarized above including data acquisition, processing, analysis, feature identification, feature segmentation, feature classification, or any other function carried out within the CAX algorithms. In general, some form of reporting or display of results of the algorithms will be provided as summarized at step 498.
In general, in the present context, each decision submodule has a task (e.g., acquisition) and a purpose (e.g., cancer detection) associated with it. Depending upon the task and the intended purpose, decision rules are established. In one implementation, a domain expert can decide on the rules tp be used for a given task and purpose. In another implementation, a library of rules relating to all possible tasks and purposes can be determined by a panel of experts and used by the submodule. In another implementation, the library of rules can be accessed from the integrated knowledge base. In another implementation, new rules may be stored in integrated knowledge base, but are derived from other means prior to storage in the knowledge base. In a typical implementation, the combination of the current data and the rules are used to develop a summary of hypothesized decision options for the data. These options may lead to several outcomes, some of which may be desired and some undesired. To obtain the optimal outcome, a metric is established to provide scores for each of the outcomes. Resultant outcomes are thus evaluated, and the selected (i.e. optimal) outcome determines the function provided in the decision block.
As mentioned, the various CAX algorithms may be employed individually or with some level of interaction. Moreover, the algorithms may be employed in the present technique without modification, or some or a high level of adaptability may be offered by virtue of integration of additional data resources, and processing in the present system. Such adaptation may be performed in real time or after or prior to data acquisition events. Moreover, as noted above, triggering of execution or adaptation of CAX algorithms may be initiated by any range of initiation factors, such as scheduled timing, operator intervention, change of state of data, and so forth. In general, a number
of aspects of the CAX system or specific CAX algorithms may be altered. As summarized in Fig. 29, the present technique envisages at a substantially new and different approach to compiling, analyzing and altering such CAX algorithms for the adaptation and optimization provided.
Referring to Fig. -29, an overall CAX formulation, designated generally by the reference numeral 500, may be represented by separate functionalities or parameters [i][j][k]. These aspects of the CAX algorithms, in the present formulation, represent first the primary type of function performed by the algorithm, as denoted by the list 502 in Fig. 29, the functions carried out by the algorithm, as denoted by reference numeral 504 in Fig. 29 and the specific data attributes 506 employed in the algorithms. The algorithm designations 502 may follow general lines for functionality in the algorithms, although those skilled in the art will recognize that more than one such functionality may be employed, such as through subroutine, submodules, and the like. The [j] level of functionality in the algorithms may include a wide range of integrated or modular functions that are carried out in the various algorithms, some of which may be shared by a different algorithm. Noted in particular, in Fig. 29 are functions such as data access, feature identification, analysis, segmentation, classification, decision, comparison, prediction, validation, and reconciliation. Other functions may, of course, be employed as well. In general, in the present context such functionalities are implemented as submodules of the algorithms, and may generally be implemented as "tool kits" which are called upon by the algorithm and developed by programming, expert systems,, neural networks, and so forth as discussed above.
The [k] level of the CAX algorithm represents generally, variables or inputs that are used by the CAX algorithms for performing the functions specified at the [j] level. By way of example, in presently contemplated embodiments, items at the PC] level may include parameters, settings, values, ranges, patient-specific data, organ-specific data, condition-specific data, temporal data, and so forth. Such parameters and settings may be altered in the manner described above, such as for patient-specific implementation of the CAX algorithm or for more broadly-based changes as for a population of patients, institutions, and so forth. It should also be noted, that, as described above with respect
to modeling, alterations made in a CAX algorithm may include consideration of data which was not considered prior to a modification. That is, as new data or new relationships are identified, the CAX algorithm may be altered to accommodate consideration of the new data. As will be appreciated by those skilled in the art then, the high degree of integration of the present technique allows for new and useful relationships to be identified among and between data from a wide range of resources and such knowledge incorporated into the CAX algorithm to further enhance its performance. Where available, the data may then be extracted from the integrated knowledge base or a portion of the knowledge base to carry out the function when called upon by the CAX algorithm.
It should be noted that, while a single CAX algorithm may be implemented in accordance with the present technique, a variety of CAX algorithms may be implemented in parallel and in series for addressing a wide range of conditions. As summarized in Fig. 30, for example, a niulti-CAX implementation 508 may include a first type of algorithm 510, which may be any of the algorithms summarized above. Moreover, the selected type of algorithm may be implemented in parallel, such that multiple different or complementary functions may be executed. Each such algorithm will typically include fundamental operations such as noted at reference numeral 512. Such operations may generally resemble those of CAD algorithms, including steps such as feature segmentation 514, feature identification 516, and feature classification 518. Based upon such steps, decisions may be made, such as for specific recommendations for future actions, as indicated at step 520. As noted above, based upon such operations, the algorithm may be-modified, as noted at step 522. The modification is then implemented by returning to the system or method employed to generate or process the data, as noted at step 524. As noted above, the modifications may be made as various levels in the algorithms, such as levels [j] and [k] discussed above.
As also summarized in Fig. 30, a number of CAX algorithms of different type (i.e. CAX[ij) may be executed in parallel, such as to identify features of interest of different type, or from data of different type or modality. Such additional algorithms,
designated by reference numerals 526 and 528 may include any of the algorithm types discussed above. Similarly, CAX algorithms of the same or different type may be executed in series, as indicated at reference numerals 530 and 532.in Fig. 30. Such algorithms may, in fact, be selected based upon results of earlier-executed algorithms.
While all of the CAX algorithms discussed above may have application in addressing a range of clinical and non-clinical issues, a more complete discussion of certain of thdse is useful in understanding the types of .data operations performed by the modules or submodules involved.
COMPUTER-ASSISTED DIAGNOSIS (CADX):
Computer-assisted diagnosis modules aid in identifying and diagnosing specific conditions, typically in the area of medical imaging. However, in accordance with the present technique, such modules may incorporate a much wider range of data, both from imaging types and modalities, as well as from other types and modalities of resources. The following is a general description of an exemplary computer-assisted diagnosis module. As described above and shown in Fig. 28, CADx- consists of a computer-assisted detection (CAD) module and a feature classification block.
As described above, the medical practitioner derives information regarding a medical condition from a variety of sources. The present technique provides computer-assisted algorithms and techniques calling upon these sources' from multi-modal and multi-dimensional perspectives for the detection and classification of a range of medical conditions in clinically relevant areas including (but not limited to) oncology, radiology, pathology, neurology, cardiology, orthopedics, and surgery. The condition identification can be in the form of screening using the analysis of body fluids, and detection alone (e.g., to determine the presence or absence of suspicious candidate lesions) or in the form of diagnosis (e.g., for classification of detected lesions as either benign or malignant nodules). For the purposes of simplicity, one present embodiment will be explained in terms of a CADx module to diagnose benign or malignant lesions.
In the present context, a CADx module may have several parts, such as data sources, optimal feature selection, and classification, training, and display of results. Data sources, as discussed above, may typically include image acquisition system information, diagnostic image data sets, electrical diagnostic data, clinical laboratory diagnostic .data from body fluids, histological diagnostic data, and patient demographics/symptoms/history, such as smoking, history, sex, age, clinical symptoms.
Feature selection may, itself comprise different types of analysis and processing, such as segmentation and feature extraction. In the data, a region of interest can be defined to calculate features. The region of interest can be defined in several ways, such as by using the entire data "as is," or by using a part of the data, such as a candidate nodule region in the apical lung field. The segmentation of the region of interest can be performed either manually or automatically. The manual segmentation involves displaying the data and delineating the region, such as by a user interfacing with the system in a computer mouse. Automated segmentation algorithms can use prior knowledge, such as the shape and size of a nodule, to automatically delineate the area of interest. A semi-automated method which is the combination of the above two methods may also be used.
The feature extraction process involves performing computations on the data sources. For example, in image-based data and for a region of interest, statistics such as shape, size, density, curvature can be computed. On acquisition-based and patient-based data, the data themselves may serve as the features. Once the features are computed, a pre-trained classification algorithm can be used to classify the regions of interest as benign or malignant nodules. Bayesian classifiers, neural networks, rule-based methods, fuzzy logic or Other suitable techniques can be used for classification. It should be noted here that CADx operations may be performed once by incorporating features from all data, or can be performed in parallel. The parallel operation would involve performing CADx operations individually on sets of data and combining the results of some or all CADx operations (e.g., via AND, OR operations or a

combination of both). In addition, CADx operations to detect multiple disease states or medical conditions or events can be performed in series or parallel.
Prior to classification, such as, of nodules, in the example, using a CAD module, prior knowledge from training of the module may be performed. The training phase may involve the computation of several candidate features on known samples of benign and malignant nodules. A feature selection algorithm is then employed to sort through the candidate features and select only the useful ones, removing those that provide no information or redundant information. This decision is based on classification results with different combinations of candidate features. The feature 'selection algorithm is also used to reduce the dimensionality from a practical standpoint. Thus, ha the example of breast mass analysis, a feature set is derived that can optimally discriminate benign nodules from malignant nodules. This optimal feature set is extracted on the regions of interest in the CAD module. Optimal feature selection can be performed using a well-known distance measure techniques including divergence measure, Bhattacharya distance, Mahalanobis distance, and so forth.
The proposed method enables, for example, the use of multiple biomarkers for review by human or machine observers. CAD techniques may operate on some or all of the data, and display the results on each kind or set of data, or synthesize the results for display. This provides the benefit of improving CAD performance by simplifying the segmentation process, while not increasing the quantity or type of data to be reviewed.
Again following the lesion analysis example, following identification and classification of a suspicious candidate lesion, its location and characteristics may be displayed to the reviewer of the data, hi certain CADx applications this is done through the superposition of a marker (for example an arrow or circle) near or around the suspicious lesion. In other cases CAD and CADx afford the ability to display computer detected and diagnosed markers on any of multiple data sets, respectively. In this way, the reviewer may view a single data set upon which results from an array of CADx operations can be superimposed (defined by a unique segmentation (i.e. regions of interest), feature extraction, and classification procedures).
COMPUTER-ASSISTED ACQUISITION (CAA)
Computer-assisted acquisition processing modules may be implemented to acquire further data, again from one or more types of resources and one or more modalities within each type, to assist in enhanced understanding and diagnosis of patient conditions. The acquisition of data may entail one or more patient visits, or sessions (including, for example, remote sessions with the patient), in which additional data is acquired based upon determinations made automatically by .the data processing system 10. The information is preferably based upon data available in the integrated database 12, to provide heretofore unavailable levels of integration and acquisition of subsequent for additional data for use in diagnosis and analysis.
In accordance with one aspect of the present technique, for example, initial CAD processing may be used to guide additional data acquisition with or without additional human operator assistance. CT lung screening will serve as an example .of this interaction. Assuming first that original CT data is acquired with a 5 mm slice thickness. This is .a common practice for many clinical sites to achieve a proper balance between diagnostic accuracy, patient dose, and number of images to review. Once the CAD algorithm identifies a suspicious site, the computer may automatically direct the CT scanner (or recommend to the CT operator) to re-acquire a set of thin slices at the suspected location (e.g., 1 mm slice thickness). In addition, an increased X-ray flux can be used for better signal-to-noise. Because the location is well-defined, the additional dose to the patient is kept to a minimum. The thin slice image provides better spatial resolution and, therefore, improved diagnostic accuracy. Advantages of such interactions include improved image quality and the avoidance of patient rescheduling. It should be noted that most of the diagnostic process generally occurs long after the patient has left the CT scanner room. In conventional approaches, if the radiologist needs thinner slices, the patient has to be called back and re-scanned. Because scan landmarking is performed with a scout image, the subsequent localization of the feature of interest is often quite poor. As a result, a larger volume of the patient organ has to be re-scanned. This leads not only to lost time, but also an increased dose to the patient.
Although this example is for a single modality, the methodology can be applied across modalities, and even across types of resources as discussed above, and over time. For example, the initial CAD information generated with images acquired via a first modality may be used by the CAA algorithm to guide additional data acquisition via a modality B. A specific example of such interaction is the CAD detection of a suspicious nodule in chest x-ray guiding the acquisition of a thin slice helical chest CT exam.
COMPUTER-ASSISTED PROCESSING (CAP)
Computer-assisted processing modules permit enhanced analysis of data which is already available through one or more acquisition sessions. The processing may be based, again, one or more types of resources, and on one or more modalities within each type. As also noted above, while computer-jassisted processing modules have been applied in the past to single modalities, typically in the medical imaging context, the present technique contemplates the use of such modules in a much broader context by use of the various resources available and the integrated knowledge base.
As an example, CAD generated information may be used to further optimize the process of obtaining new images. Following data acquisition and initial image formation (or based-upon un-processed or partially processed data without image reconstruction), CAD modules may be used to perform the initial feature detection. Once potential pathology sites are identified and characterized, a new set of images may be generated by a CAA module based upon the findings. The new set of images may be generated to assist the human observer's detection/classification task, or to improve the performance of other CAX algorithms.
For illustration, a CT lung-screening example is considered, although the approach may be, of course, generalized to other imaging modalities, other resource types, and other pathologies. We assume initially that an image is reconstructed with a "Bone" (high-resolution) filter kernel and with a 40 cm reconstruction field of view (FOV). Once a suspicious lung nodule is identified, a CAP module may reconstruct a new set of images at the suspected location with the original scan data. For example, a first
images with a "Standard" (l°wer resolution kernel) filter- kernel may first be reconstructed. Although the Standard kernel produces poor spatial resolution, it has the property of maintaining accurate CT numbers. Combining such images with those produced via the Bone algorithm, a CAP algorithm can separate calcified nodules from the non-calcified nodules based on their CT number. Additionally, the CAP module may perform targeted-reconstruction at the suspected locations to provide improved spatial resolution, or to improve algorithm performance and/or to facilitate human observer analysis. By way of further example, for a present CT scanner, typical image size is 512x512 pixels. For a 40 cm reconstruction FOV, each pixel is roughly 0.8 mm along a side. From a Nyquist sampling point of view, this insufficient to support high spatial resolutions. When the CAP module re-generates the image, however, with a 10 cm FOV at a suspicious site, each pixel is roughly 0.2 mm along a side and, therefore, can support much higher spatial resolution. Because the additional reconstruction and processing is performed only at the isolated sites, instead of the entire volume, the amount of image processing, reconstructionj and storage becomes quite manageable. It should be noted that a simple example is presented here for the purpose of illustration. Other processing steps (such as image enhancement, local 3D modeling, image reformation, etc.) could also be performed with under the guidance of the CAP module, such as based on the initial CAD result and the results of further processing. The additional images can be used either to refine the original findings of CAD processing, as input to further CAX analyses, or may be presented to the radiologists.
COMPUTER-ASSISTED PROGNOSIS (CAPX)
Medical prognosis is an estimate of cure, complication, recurrence of disease, length of stay in health care facilities or survival for a patient or group of patients. The simplistic meaning of prognosis is a prediction of the future course and outcome of a disease and an indication of the likelihood of recovery from that disease.
Computational prognostic model may be used, in accordance with the present technique to predict the natural course of disease, or the expected outcome after
treatment. Prognosis forms an integral part of systems for treatment selection and treatment planning. Furthermore, prognostic models may play an important role in guiding diagnostic problem solving, e.g. by only requesting information concerning tests, of which the outcome affects knowledge of the prognosis.
In recent years several methods and techniques from the fields of artificial intelligence, decision theory and statistics have been introduced into models of the medical management of patients (diagnosis, treatment, follow-up);- in some of these models, assessment of the expected prognosis constitutes an integral part. Typically, recent prognostic methods rely on explicit patho-physiological models, which may be combined with traditional models of life expectancy. Examples of such domain models are causal disease models, and physiological models of regulatory mechanisms in the human body. Such model-based approaches have the potential to facilitate the development of knowledge-based systems, because the medical domain models can be partially obtained from the medical literature.
Various methods, have been suggested for the representations of such domain models ranging from quantitative and probabilistic approaches to symbolic and qualitative ones. Semantic concepts such as time, e.g. for modeling the progressive changes of regulatory mechanisms, have formed an important and challenging modeling issue. Moreover, automatic learning techniques of such models have been proposed. When model construction is hard, less explicit domain models have been studied such as the use of case-based representations and its combination with more explicit domain models.
COMPUTER-ASSISTED ASSESSMENT (CAAX)
Computer-assisted assessment modules may include algorithms for analyzing a wide range of conditions or situations. By way of example, such algorithms may be employed to evaluate the outcome of a medical procedure (e.g., surgery), the outcome of therapy due to an injury (e.g. spinal injury), conditions (e.g. pregnancy), situations (e.g. trauma), processes .(e.g. insurance, reimbursement, equipment utilization), and individuals (e.g. patients, students, medical professionals).
Certain exemplary steps in a CAAx algorithm are illustrated generally in Fig. 31. The algorithm 534 begins with input of key data at step 536. Depending upon the purpose of the algorithm, such data may include a designation or description of a situation, task, available results, intended person, requested information, and so forth. The data is used to identify a desired software tool, as indicated at step 538, which may take the form of a "wizard" used as an interface to lead a user through the assessment process. The interface may be at least partially based upon input from a professional or expert in the field of the operations executed by the algorithm or in the field of the data or assessment to be performed.
At step 540, more specific information may be evoked from one or more users, or automatically acquired or accessed from the various resources described above. Where the data is input by an individual, a customized interface may be.provided in a manner described above, such as via the unfederated interface layer 222, drawing upon information from the integrated knowledge base 12 and data resources 18. As noted above, such interfaces may be customized for the particular user, the function performed, the data to be provided or accessed, and so forth.
Based upon the information provided, assessment is performed, as indicated at step 542. Such assessment will generally vary widely based upon the condition, situation, or other issue being evaluated. In a presently contemplated implementation, a score is determined from the assessment, and a comparison is performed based upon the score at step 544. The comparison is then the basis of a recommendation for further action, or may simply serve as the basis for reported results of the assessment Moreover, results of the process may optionally be reconciled, where potential conflicts or judgments are in order, as indicated at step 546, including input from a human expert, where desired.
BUSINESS MODEL IMPLEMENTATION
The foregoing techniques permit implementation in a wide range of manners. For example, as noted repeatedly, the use of data and the interaction between data and modules may be implemented on a very small scale, including at a single workstation.
Higher levels of integration may be provided by network links between various types }f resources and workstations, and at various levels between network components as dso described above. It should also be noted that the present techniques may be implemented as overall business models within an industry or a portion of an industry.
Ihe business model implementation for the present techniques may include software installed on one or more memory devices or machine-readable media, such as disks, hard drives, flash memory, and so forth.-. A user may then employ the techniques individually, or by access to specific sites, links, services, databases; and so forth through a network. Similarly, a business model based upon the techniques may be developed such that the technique is offered on a pay-per-use, subscription, or any other suitable basis.
Such business models may be employed for any or all of the foregoing techniques, and may be offered on a "modular" basis. By way of example, institutions may subscribe or order services for evaluation of patient populations, scheduling of services and resources,. development of models for prediction of patient conditions, training purposes, and so forth. Individuals or institutions may subscribe or purchase similar services for maintenance of individual patient records, integration of records, and the like. Certain of the techniques may be offered in conjunction with other assets or services, such as imaging systems, workstations, management networks, and so forth.
As will be appreciated by those skilled in the art, the business models built upon the foregoing techniques may employ a wide range of support software and hardware, including servers, drivers, translators, and so forth which permit or facilitate interaction with databases, processing resources, and the data and controllable and prescribable resources described above. Supporting components which provide for security, verification, interfacing and synchronization of data maybe incorporated into such systems, or may be distributed among the systems and the various users or clients. Financial support modules, including modules which permit tracking and invoicing for services may be incorporated in a similar manner.
It is similarly contemplated that certain of the foregoing techniques may be implemented in sector-wide or industry-wide manners. Thus, high levels o1 integration may be enabled by appropriately standardizing or tagging data for access, exchange, uploading, downloading, translation, processing, and so forth.
While the invention may be susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and have been described in detail herein. However, it should be understood that the invention is not intended to be limited to the particular forms disclosed Rather, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the following appended claims.
We claim:

A computer aided medical data exchange system for medical data analysis incorporating in vitro test data (10), said system comprising:

- acquisition means (438) for acquiring an in vitro test sample;
- means for accessing data (442, 450) from an integrated knowledge base (12) including data derived from the plurality of controllable and prescribable data resources (40) for generating patient treatment data, said controllable and prescribable resources comprising at least a plurality of medical data acquisition systems;
- analyzing means (440) for analyzing the in vitro test sample and the accessed data via a computer-assisted data operating algorithm (22) to identify at least one required feature of the in vitro test sample; and
- modify acquisition means (444, 452) of an in vitro test sample based upon the identification

2. The system as claimed in claim 1, wherein said system comprises means for processing (446) an in vitro test sample based upon the identification as herein described

3. The system as claimed in claim 1, wherein said system comprises means for processing (446) data derived from an in vitro test sample and modifying based upon the comparison

4. The system as claimed in claim 1, wherein the at least one said required feature comprises a biomarker for a medical condition